Improving Agents in the Wild Via Agent Server Collaboration for Continual Semantic Understanding: A Framework
Nazmus Sakib Ahmed, Naushin Nower · IEEE Access · 2026
Autonomous agents deployed in dynamic real-world environments—such as scientific exploration, industrial monitoring, and wildlife observation—must continually adapt to evolving conditions and new inputs. However, traditional cloud-edge frameworks rely on static models where models are trained offline, making them ill-suited for such unpredictable scenarios. The existing distributed framework, which combines federated learning and transfer learning, enables faster responses in low-resource settings but struggles to handle highly dynamic edge environments characterized by frequent shifts in data distribution. This work proposes a novel framework for improving semantic understanding in autonomous agents through continual learning, enabled by agent-server collaboration and curriculum learning. The server orchestrates the learning process by constructing adaptive curricula using both historical and streaming data, training models on increasingly complex examples, and periodically distributing the updated models to agents operating at the edge. A key component of our approach is the introduction ofConcept Inconsistency (CI), a metric that quantifies semantic misalignment across classes based on model parameters. CI allows the server to group tasks into coherent learning stages without manual difficulty heuristics or full data set access. Additionally, a general-purpose vision-language model acts as a world model to label cases encountered by the agents, further improving the adaptation. We demonstrate the effectiveness of our approach through experiments on image classification tasks using convolutional neural networks (CNNs), showing consistent performance gains—up to 19%—especially in data-scarce and high-class-count scenarios. Our framework offers a scalable, efficient solution for enhancing continual learning in autonomous agents operating in the wild.