Hassan Jari

MorphCloud-LLM: Elastic Spot-Instance-Aware LLM Serving with Transparent Preemption Recovery and Speculative Decoding Continuity

Serving large language models (LLMs) on cloud spot and preemptible instances reduces costs by 60 to 90 percent compared to on-demand pricing, but unpredictable instance preemptions cause request failures, KV-cache state loss, and degraded user experience. We present MorphCloud-LL...