Story
aws_ml_blog ยท Nov 17, 2026 ยท news
aws.amazon.comNov 17, 2026
original source linked
In brief
Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, a new strategy that routes each request to the instance holding the longest cached match for its prompt. Learn how it works, when to use it c...