iGenEdge: Intelligent Generative AI Service Deployment for Edge-Connected IoT Devices
Faiza Akram, Asad Waqar Malik, Samee U. Khan · IEEE Internet Computing · 2025
Emerging IoT applications drive a growing demand for near-real-time computation. Given the limited resources of IoT devices, these systems rely on edge computing, but the advent of generative AI (GenAI) applications poses new deployment challenges as AI models are computationally intensive, and concurrent deployment can deprive other applications of edge resources. We propose an Intelligent Generative AI management framework (iGenEdge), designed for IoT edge devices, aims to dynamically provision resources to handle varying demands.iGenEdgeleverages collaborative edge computing to optimize GenAI deployment, ensuring efficient use of edge resources. We evaluated our framework using lightweight LLMs DistilBERT and CLIP ViT-B-32, on Raspberry Pi’s. Our study revealed 70% reduction in context-switching cost for high-computation inferences. However, task completion time increased by 38%, CPU usage escalated by ≈ 86%, and inline memory-based implementation reduced the swapping-overhead for the textual model by ≈ 96%.