{"slug":"inference-kubernetes","label":"Inference Kubernetes","item_count":3,"day_count":3,"source_count":3,"first_seen":"2026-11-04T00:00:00+00:00","last_updated":"2026-11-09T11:00:00+00:00","generated_at":"2026-11-10T00:13:24.787725+00:00","sources":["hackernews_ai","infoq_ai_ml","vllm_blog"],"days":[{"date":"2026-11-04","items":[{"title":"Introducing Kinference: An Open Source Inference Control Plane for vLLM and Kubernetes","url":"https://vllm.ai/blog/2026-11-04-introducing-kinference","source":"vllm_blog","type":"news","summary_1line":"Kinference is a new open source inference control plane for building scalable inference platforms with vLLM and Kubernetes, providing token rate limiting, tenant usage and cost tracking, intelligent load balancing, an...","sid":"2aca440b2c0823d7","published":"2026-11-04T00:00:00+00:00","editor_note":"First announcement: an open source control plane covering token rate limiting, tenant usage and cost tracking, and load balancing for vLLM."}]},{"date":"2026-11-05","items":[{"title":"Distributed Inference Benchmarks: [K]Inference vs. Plain Kubernetes","url":"https://www.buoyant.io/blog/distributed-inference-benchmarks-kinference-vs-plain-kubernetes","source":"hackernews_ai","type":"news","summary_1line":"Distributed Inference Benchmarks: [K]Inference vs. Plain Kubernetes","sid":"509ceae27e7b57e7","published":"2026-11-05T19:07:41+00:00","editor_note":"A benchmark post comparing Kinference with plain Kubernetes, shared on Hacker News."}]},{"date":"2026-11-09","items":[{"title":"Podcast: Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM","url":"https://www.infoq.com/podcasts/distributed-llm-inference-kubernetes/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=AI%2C+ML+%26+Data+Engineering","source":"infoq_ai_ml","type":"news","summary_1line":"In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and optimizing distributed AI inference using llm-d and vLLM on Kubernete...","sid":"1d4bb99a8acae991","published":"2026-11-09T11:00:00+00:00","editor_note":"Podcast with Edoardo Vacchi covering llm-d and vLLM on Kubernetes, widening the thread beyond Kinference."}]}],"editorial":{"tldr":"On Nov 4 the vLLM project introduced Kinference, an open source inference control plane for vLLM on Kubernetes. A benchmark post comparing it against plain Kubernetes followed a day later.","stale":false,"whats_new":"An InfoQ podcast (Nov 9) places llm-d and vLLM on Kubernetes in the wider distributed-inference conversation, alongside WebAssembly on the backend and TornadoVM for Java on GPUs.","why_it_matters":"Serving LLMs on Kubernetes is moving from generic pod scheduling toward inference-aware control planes (token rate limiting, tenant cost tracking, load balancing), which changes what a platform team builds versus adopts.","take_for_builders":"If you run vLLM on Kubernetes, read the Kinference benchmark methodology and rerun it on your own traffic mix before replacing your current routing and rate limiting.","status":{"state":"Emerging","tone":"rising","changed":"2026-11-09","detail":"Inference-aware Kubernetes tooling is getting launch, benchmark and practitioner coverage, but only one benchmark post exists so far."},"beats":[{"kicker":"LAUNCH","tone":"launch","headline":"Kinference debuts as an inference control plane for vLLM on Kubernetes","sids":["2aca440b2c0823d7"]},{"kicker":"EVIDENCE","tone":"rising","headline":"Buoyant benchmarks Kinference against plain Kubernetes","sids":["509ceae27e7b57e7"]},{"kicker":"NOW","tone":"now","headline":"Practitioner view: llm-d and vLLM on Kubernetes, plus Wasm and TornadoVM","sids":["1d4bb99a8acae991"]}],"open_questions":["Does an independent benchmark, not from the Kinference or Buoyant side, reproduce the comparison against plain Kubernetes?","How do Kinference and llm-d compare on features such as load balancing and tenant usage tracking?"],"generated_at":"2026-11-10T00:30:00+00:00"}}