X @Avi Chawla

How to think about model routing in production:Traditional routers classify by intent, and inside one task the intent keeps changing. Each call gets scored on its own, usually with no context that it's the middle of something.So if the routing layer switches the model midway, the new model has to build the whole cache again, at cold rates.Routing per call is the problem, not routing itself, so one common solution is to route once per task, not once per call.First LLM call routes normally, and that model get ...