X @Avi Chawla
Avi Chawlaยท2025-07-09 06:30
The fastest serving engine for LLMs is here (open-source)!LMCache is an LLM serving engine designed to reduce time-to-first-token and increase throughput, especially under long-context scenarios.It boosts vLLM with 7x faster access to 100x more KV caches.100% open-source! https://t.co/IfyZzdnq4z ...