85% lower median TTFT
Google Cloud releases llm-d 1.0 open-source inference stack for LLM serving fleets
Google Cloud and partners released llm-d 1.0, a stable Kubernetes-native layer that coordinates vLLM or SGLang model servers into one pool. A vendor-reported Ironwood TPU coding benchmark showed median time-to-first-token cut by more than 85% and 2.4x…