semantic-router

Name: semantic-router
Author: vllm-project

5.0kstars768forks

How to install semantic-router?

vllm-project/semantic-router

$git clone https://github.com/vllm-project/semantic-router

Installs into the current project.

Summary

Building Mixture-of-Models: The Next-Generation Model Architecture for Heterogeneous LLM Inference

Files · 1

View on GitHub

README.md

1	<div align="center">
2
3	<img src="website/static/img/artworks/vllm-sr-logo.dark.png" alt="vLLM Semantic Router" width="50%"/>
4
5	<p><strong>Building Mixture-of-Models: The Next-Generation Model Architecture for Heterogeneous LLM Inference</strong></p>
6
7	<p>
8	<a href="https://vllm-sr.ai">Documentation</a> \|
9	<a href="https://app.vllm-sr.ai/playground">Playground</a> \|
10	<a href="https://vllm-sr.ai/blog/">Blog</a> \|
11	<a href="https://vllm-sr.ai/publications/">Publications</a> \|
12	<a href="https://huggingface.co/LLM-Semantic-Router">Hugging Face</a>
13	</p>
14
15	</div>
16
17	---
18
19	## About
20
21	We believe Mixture-of-Models is the next-generation model architecture for heterogeneous LLM inference.
22
23	That is why we built vLLM Semantic Router—to turn signals and preferences into executable model paths for every user, product, and workload.
24
25	\| Dimension \| Fragmented today \| With vLLM SR \|
26	\| --- \| --- \| --- \|
27	\| Models \| Models specialize in different work. \| Compose personalized model paths. \|
28	\| Compute \| GPUs, accelerators, edge, and cloud coexist. \| Route across heterogeneous compute. \|
29	\| Location \| Inference spans edge, private, and cloud. \| Keep data within its boundaries. \|
30	\| Preference \| "Best" changes by user and workload. \| Make every preference executable. \|
31
32	[Explore how it works →](https://vllm-sr.ai/docs/intro/)
33
34	## Getting Started
35
36	### Install
37
38	```bash
39	curl -fsSL https://vllm-sr.ai/install.sh \| bash
40	```
41
42	For platform notes, detailed setup options, and troubleshooting, see the [Installation Guide](https://vllm-sr.ai/docs/installation/).
43
44	<details>
45	<summary>Online playground credentials</summary>
46
47	- Username: `love@vllm-sr.ai`
48	- Password: `vllm-sr`
49
50	</details>
51
52	## Latest News
53
54	- [2026/06/29] New Blog: [Micro-Agent: Beat Frontier Models with Collaboration inside Model API](https://vllm.ai/blog/2026-06-29-micro-agent-frontier-models)
55	- [2026/06/16] New Blog: [Beyond One Model: Fusion in vLLM Semantic Router](https://vllm.ai/blog/2026-06-16-vllm-sr-fusion-api)
56	- [2026/06/05] v0.3 Released: [vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing](https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release)
57
58	<details>
59	<summary>Earlier announcements</summary>
60
61	- [2026/03/24] Vision Paper Released: [The Workload-Router-Pool Architecture for LLM Inference Optimization](https://vllm-sr.ai/vision-paper)
62	- [2026/03/10] v0.2 Released: [vLLM Semantic Router v0.2 Athena Release](https://vllm.ai/blog/v0.2-vllm-sr-athena-release)
63	- [2026/02/27] White Paper Released: [Signal Driven Decision Routing for Mixture-of-Modality Models](https://vllm-sr.ai/white-paper/)
64	- [2026/01/05] Iris v0.1 Released: [vLLM Semantic Router v0.1 Iris: The First Major Release](https://blog.vllm.ai/2026/01/05/vllm-sr-iris.html)
65	- [2025/12/16] Collaboration: [AMD × vLLM Semantic Router: Building the System Intelligence Together](https://blog.vllm.ai/2025/12/16/vllm-sr-amd.html)
66	- [2025/12/15] New Blog: [Token-Level Truth: Real-Time Hallucination Detection for Production LLMs](https://blog.vllm.ai/2025/12/14/halugate.html)
67	- [2025/11/19] New Blog: [Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale](https://blog.vllm.ai/2025/11/19/signal-decision.html)
68	- [2025/11/03] Paper Published: [Category-Aware Semantic Caching for Heterogeneous LLM Workloads](https://arxiv.org/abs/2510.26835)
69	- [2025/10/27] New Blog: [Scaling Semantic Routing with Extensible LoRA](https://blog.vllm.ai/2025/10/27/semantic-router-modular.html)
70	- [2025/10/12] Paper Accepted: [When to Reason: Semantic Router for vLLM](https://arxiv.org/abs/2510.08731)
71	- [2025/10/08] Collaboration: vLLM Semantic Router with [vLLM Production Stack](https://github.com/vllm-project/production-stack) Team.
72	- [2025/09/01] Released the project: [vLLM Semantic Router: Next Phase in LLM inference](https://blog.vllm.ai/2025/09/11/semantic-router.html).
73
74	</details>
75
76	More announcements are available on the [Blog](https://vllm-sr.ai/blog/) and **[Publications](https://vll

People who liked this also viewed

6 picks

Type

odoo-mcp-proAI connector for Odoo ERP | Connect Claude, ChatGPT, Cursor and other AI tools to Odoo via MCPMCP ServersJul 2026551k63
omnigentUniversal autonomous agent framework with ReAct loop, multi-provider LLM routing, reasoning graph, and MCP integration, domain-agnostic for building specialized AI agents.MCP ServersFeb 202683k30
arxiv-mcp-serverAn MCP server that enables interacting with the arXiv API using natural languageMCP ServersAug 202576k40
United States WeatherProvide real-time and forecast weather information for locations in the United States using natural language queries. Access current conditions, multi-day and hourly forecasts, weather alerts, and nearby weather stations without requiring aMCP Serversrecently56k0
mcp-server-demoDemo of MCP Server in Python using FastMCPMCP ServersAug 202556k10
mcp-serverlearn to make some mcp serversMCP ServersMay 202556k5

semantic-router

5.0kstars768forks

View on GitHub

How to install semantic-router?

vllm-project/semantic-router

$git clone https://github.com/vllm-project/semantic-router

Installs into the current project.

Summary

Building Mixture-of-Models: The Next-Generation Model Architecture for Heterogeneous LLM Inference

Files · 1

View on GitHub

README.md

1	<div align="center">
2
3	<img src="website/static/img/artworks/vllm-sr-logo.dark.png" alt="vLLM Semantic Router" width="50%"/>
4
5	<p><strong>Building Mixture-of-Models: The Next-Generation Model Architecture for Heterogeneous LLM Inference</strong></p>
6
7	<p>
8	<a href="https://vllm-sr.ai">Documentation</a> \|
9	<a href="https://app.vllm-sr.ai/playground">Playground</a> \|
10	<a href="https://vllm-sr.ai/blog/">Blog</a> \|
11	<a href="https://vllm-sr.ai/publications/">Publications</a> \|
12	<a href="https://huggingface.co/LLM-Semantic-Router">Hugging Face</a>
13	</p>
14
15	</div>
16
17	---
18
19	## About
20
21	We believe Mixture-of-Models is the next-generation model architecture for heterogeneous LLM inference.
22
23	That is why we built vLLM Semantic Router—to turn signals and preferences into executable model paths for every user, product, and workload.
24
25	\| Dimension \| Fragmented today \| With vLLM SR \|
26	\| --- \| --- \| --- \|
27	\| Models \| Models specialize in different work. \| Compose personalized model paths. \|
28	\| Compute \| GPUs, accelerators, edge, and cloud coexist. \| Route across heterogeneous compute. \|
29	\| Location \| Inference spans edge, private, and cloud. \| Keep data within its boundaries. \|
30	\| Preference \| "Best" changes by user and workload. \| Make every preference executable. \|
31
32	[Explore how it works →](https://vllm-sr.ai/docs/intro/)
33
34	## Getting Started
35
36	### Install
37
38	```bash
39	curl -fsSL https://vllm-sr.ai/install.sh \| bash
40	```
41
42	For platform notes, detailed setup options, and troubleshooting, see the [Installation Guide](https://vllm-sr.ai/docs/installation/).
43
44	<details>
45	<summary>Online playground credentials</summary>
46
47	- Username: `love@vllm-sr.ai`
48	- Password: `vllm-sr`
49
50	</details>
51
52	## Latest News
53
54	- [2026/06/29] New Blog: [Micro-Agent: Beat Frontier Models with Collaboration inside Model API](https://vllm.ai/blog/2026-06-29-micro-agent-frontier-models)
55	- [2026/06/16] New Blog: [Beyond One Model: Fusion in vLLM Semantic Router](https://vllm.ai/blog/2026-06-16-vllm-sr-fusion-api)
56	- [2026/06/05] v0.3 Released: [vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing](https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release)
57
58	<details>
59	<summary>Earlier announcements</summary>
60
61	- [2026/03/24] Vision Paper Released: [The Workload-Router-Pool Architecture for LLM Inference Optimization](https://vllm-sr.ai/vision-paper)
62	- [2026/03/10] v0.2 Released: [vLLM Semantic Router v0.2 Athena Release](https://vllm.ai/blog/v0.2-vllm-sr-athena-release)
63	- [2026/02/27] White Paper Released: [Signal Driven Decision Routing for Mixture-of-Modality Models](https://vllm-sr.ai/white-paper/)
64	- [2026/01/05] Iris v0.1 Released: [vLLM Semantic Router v0.1 Iris: The First Major Release](https://blog.vllm.ai/2026/01/05/vllm-sr-iris.html)
65	- [2025/12/16] Collaboration: [AMD × vLLM Semantic Router: Building the System Intelligence Together](https://blog.vllm.ai/2025/12/16/vllm-sr-amd.html)
66	- [2025/12/15] New Blog: [Token-Level Truth: Real-Time Hallucination Detection for Production LLMs](https://blog.vllm.ai/2025/12/14/halugate.html)
67	- [2025/11/19] New Blog: [Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale](https://blog.vllm.ai/2025/11/19/signal-decision.html)
68	- [2025/11/03] Paper Published: [Category-Aware Semantic Caching for Heterogeneous LLM Workloads](https://arxiv.org/abs/2510.26835)
69	- [2025/10/27] New Blog: [Scaling Semantic Routing with Extensible LoRA](https://blog.vllm.ai/2025/10/27/semantic-router-modular.html)
70	- [2025/10/12] Paper Accepted: [When to Reason: Semantic Router for vLLM](https://arxiv.org/abs/2510.08731)
71	- [2025/10/08] Collaboration: vLLM Semantic Router with [vLLM Production Stack](https://github.com/vllm-project/production-stack) Team.
72	- [2025/09/01] Released the project: [vLLM Semantic Router: Next Phase in LLM inference](https://blog.vllm.ai/2025/09/11/semantic-router.html).
73
74	</details>
75
76	More announcements are available on the [Blog](https://vllm-sr.ai/blog/) and **[Publications](https://vll

People who liked this also viewed

6 picks

Type