semantic-router
A programmable Mixture-of-Models router for heterogeneous LLM inference
About this project
Make Your Mixture-of-Models Programmable. Documentation Playground Blog Publications Hugging Face Slack About vLLM Semantic Router is a programmable routing layer for building Mixture-of-Models systems across heterogeneous LLM infrastructure. It evaluates request signals, user preferences, and application policies to select—or compose—the right model path for each request. Use it to improve quality, cost, latency, privacy, and safety without hard-coding routing logic into applications. Dimension Fragmented today With vLLM SR --- --- --- Models Models specialize in different work. Compose personalized model paths. Compute GPUs,…
Technologies
Project health
GitHub
Reviews
Built by
Maintain vllm-project/semantic-router? Claiming verifies admin access through your GitHub account and gives you control of this listing.
