Zinuo Cai

Hermes \({}^{+}\) : Adaptive Memory-Efficient Pipeline Inference for Large Models on Edge Devices

The application of Transformer-based large models has achieved significant success in recent years. However, the exponential growth in the parameters of large models introduces formidable memory challenges for edge deployment. Prior works to address this challenge mainly focus on...

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

Large Language Models (LLMs) have enabled diverse AI applications. However, LLMs inference impose unprecedented computational and memory overhead, creating an inherent trade-off between latency Service Level Objectives (SLOs) and resource constraints. Serverless computing, with o...