MorphCloud-LLM: Elastic Spot-Instance-Aware LLM Serving with Transparent Preemption Recovery and Speculative Decoding Continuity
Hassan A. Jari · Electronics · 2026
Serving large language models (LLMs) on cloud spot and preemptible instances reduces costs by 60 to 90 percent compared to on-demand pricing, but unpredictable instance preemptions cause request failures, KV-cache state loss, and degraded user experience. We present MorphCloud-LLM, an elastic LLM serving system designed to achieve the reliability properties of on-demand serving at spot-instance pricing. MorphCloud-LLM integrates three synergistic components: (1) an asynchronous incremental KV-cache checkpointing engine that streams only delta state to disaggregated persistent storage with less than 3% throughput overhead, enabling sub-second KV-cache delta streaming and reconstruction for KV-cache sizes up to 32 GB on replacement instances (total end-to-end migration latency: 1390 ms); (2) a gradient-boosted preemption prediction model trained on spot market telemetry that achieves 89% recall at a 30-s prediction horizon, providing sufficient lead time for proactive migration before forced eviction; and (3) a speculative decoding continuity engine that offloads draft model token generation to on-demand fallback nodes during migration windows, bounding the user-visible interruption to a sub-second buffering pause. MorphCloud-LLM is deployed and evaluated on AWS and GCP using LLaMA-70B and Mixtral-8x7B across 521 trace-injected preemption events, achieving up to 76% cost reduction under active-serving accounting (69.8% for LLaMA-70B; 67% including warm standby fallback capacity) with only 2.1% p99 latency overhead and zero dropped requests. Extensive ablation studies confirm the contribution of each component to overall system resilience. Note that preemption events are reproduced via a trace-driven simulation framework built on empirical AWS and GCP spot interruption traces rather than fully uncontrolled live production preemptions. Production generalizability under uncontrolled preemption—including simultaneous multi-node failures, network congestion, storage contention, and replacement-instance scarcity remains subject to future validation in sustained live deployments.