Measuring Workload-Network Coupling in Local LLM Serving: Traffic Demand and Client Latency

Abstract

AI workloads can change network demand, and network conditions can affect AI service latency. We measure one narrow part of this relationship: a locally served Qwen2.5-3B model under different application-path profiles, request concurrency, and output caps. On one Windows host with an RTX 5070, we ran 20 synchronized request cohorts for each of 36 workload and path conditions. At a configured 40-ms delay and 10-Mbps downlink, increasing the output cap from 64 to 192 tokens raised median response bytes per request from 7.85 to 21.82 KB; four-client cohorts carried 31.42 and 87.32 KB, respectively. At a 192-token cap and 40-ms delay, reducing the shared downlink from 1 to 0.1 Mbps increased median completion from 0.969 to 1.914 s for one client and from 1.392 to 7.090 s for four. A follow-up on three other locally installed models with a corrected relay found that the median applied delay was within 2 ms of the configured value; where the cohorts could resolve it, the first-output increase still differed from the configured delay by up to 17.3 ms, consistent with variation in server time between conditions. The study measures workload effects on application bytes and configured path effects on client latency in this local API setup. It does not evaluate production-cloud redesign or physical fabrics.