Introducing Kinference: An Open Source Inference Control Plane for vLLM and Kubernetes
The anchor launch: vLLM's own open-source control plane for multi-tenant serving on Kubernetes.
5 items · 4 sources · 5 days
Operational story trace
Follow in this browser to see new updates on your Live feed.
Latest change
Distributed inference work has moved from datacenter clusters to local hardware: an arXiv paper (Superfluid) on a local server shared by several agents, then a Show HN project (Shard) that splits inference across multiple computers.
This is a pattern across independent projects, not one event. It opened with vLLM's Kinference, an open-source control plane for serving vLLM on Kubernetes, plus a third-party benchmark of it against plain Kubernetes, then moved to an InfoQ podcast on llm-d.
Arc
The anchor launch: vLLM's own open-source control plane for multi-tenant serving on Kubernetes.
First third-party benchmark, from Buoyant, comparing Kinference with plain Kubernetes.
Practitioner background on llm-d as another route to distributed vLLM on Kubernetes.
Academic take on the local case: several agents sharing one locally served model.
Hobbyist project that pools multiple computers to run one model.
The anchor launch: vLLM's own open-source control plane for multi-tenant serving on Kubernetes.
First third-party benchmark, from Buoyant, comparing Kinference with plain Kubernetes.
Practitioner background on llm-d as another route to distributed vLLM on Kubernetes.
Academic take on the local case: several agents sharing one locally served model.
Hobbyist project that pools multiple computers to run one model.
What to watch — open questions
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.