LLMs at home? An evaluation on the feasibility of popularising On-device-ANI capable hardware in consumer grade devices
Yiding Wang · Advances in Engineering Innovation · 2024
Artificial Narrow Intelligences (ANI) are rapidly becoming an integral part of everyday consumer technology. With products like ChatGPT, Midjourney, and Stable Diffusion gaining widespread popularity, the demand for local hosting of neural networks has significantly increased. However, the typical 'always-online' nature of these services presents several limitations, including dependence on reliable internet connections, privacy concerns, and ongoing operational costs. This essay will explore potential hardware solutions to popularize on-device inferencing of ANI on consumer hardware and speculate on the future of the industry.