FlexRouter is a routing framework for Large Language Models (LLMs) that optimizes for answer coverage by selecting models that complement each other, improving the probability of a correct response. It uses Determinantal Point Processes (DPPs) and a novel training objective to adaptively determine subset sizes without a predefined budget.

Researchers propose a market-aware routing approach for Large Language Models (LLMs) that considers not just the model's cost but also the quality, latency, and availability of different providers serving the model. They introduce a policy that routes requests to the cheapest provider that meets quality and health criteria, and a certification mechanism to ensure reliable routing. This work has implications for the development and operation of LLMs in production environments.

Researchers introduce two methods to improve the routing in sparse mixture-of-experts large language models by aligning routing affinities with token-level error. They achieve improved accuracy on multiple benchmarks, while preserving the native sparse execution budget and aggregation policy.