ai-gateway
byazure-samples· 1 MCP server
How to install ai-gateway?
azure-samples/ai-gateway
$git clone https://github.com/azure-samples/ai-gatewayInstalls into the current project.
›Prefer a prompt? Paste this to your agent
Install ai-gateway by running `git clone https://github.com/azure-samples/ai-gateway`, then use it for the current task and follow its documentation at https://github.com/azure-samples/ai-gateway.
README.md
| 1 | <!-- markdownlint-disable MD033 --> |
| 2 | |
| 3 | <div align="center"> |
| 4 | |
| 5 | # ✨ AI Gateway Labs |
| 6 | [](https://github.com/Azure-Samples/AI-Gateway) |
| 7 | [](https://github.com/Azure-Samples/AI-Gateway/stargazers) |
| 8 | [](https://codespaces.new/Azure-Samples/AI-Gateway) |
| 9 | |
| 10 | |
| 11 | [](http://aka.ms/ai-gateway/labs) |
| 12 | |
| 13 | ### Explore the enterprise-grade gateway for managing AI Models, Tools, and Agents |
| 14 | |
| 15 | <br/> |
| 16 | |
| 17 | [](http://aka.ms/ai-gateway/labs) |
| 18 | |
| 19 | [](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities) |
| 20 | |
| 21 | </div> |
| 22 | |
| 23 | > 📰 **New!** The [AI Gateway Dev Portal](https://aka.ms/ai-gateway/dev-portal) is now live — A starting point for building your own developer portal on top of Azure API Management AI Gateway's. Fork it, open it in VS Code with GitHub Copilot (or any coding agent), and shape it to fit your needs! |
| 24 | |
| 25 | ## Why AI Gateway? |
| 26 | |
| 27 | Building production-ready AI applications requires more than just calling model APIs. You need **security**, **reliability**, **observability**, and **cost control**—without slowing down innovation. |
| 28 | |
| 29 | [**AI Gateway**](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities) powered by [Azure API Management](https://learn.microsoft.com/en-us/azure/api-management/) provides: |
| 30 | |
| 31 | - 🔐 **Security** — OAuth 2.0, managed identities, content safety filtering |
| 32 | - ⚡ **Performance** — Load balancing, semantic caching, request routing |
| 33 | - 📊 **Observability** — Token metrics, built-in logging, tracing |
| 34 | - 💰 **Cost Control** — Rate limiting, quota management, FinOps framework |
| 35 | - 🔌 **Extensibility** — MCP protocol support, function calling, multi-model routing |
| 36 | |
| 37 | ## 📚 Explore the Labs |
| 38 | |
| 39 | > **🔗 Browse all 30+ labs at [aka.ms/ai-gateway/labs](http://aka.ms/ai-gateway/labs)** |
| 40 | |
| 41 | Each lab is a hands-on Jupyter notebook with step-by-step instructions, Bicep infrastructure templates, and APIM policies you can deploy to your Azure subscription. |
| 42 | |
| 43 | ## 🧠 AI Gateway for Models |
| 44 | |
| 45 | Manage and control access to Large Language Models with enterprise-grade policies. |
| 46 | |
| 47 | | Lab | Description | |
| 48 | |-----|-------------| |
| 49 | | [**Backend Pool Load Balancing**](labs/backend-pool-load-balancing/backend-pool-load-balancing.ipynb) | Distribute requests across multiple model endpoints | |
| 50 | | [**Token Rate Limiting**](labs/token-rate-limiting/token-rate-limiting.ipynb) | Control token consumption with rate limiting policies | |
| 51 | | [**Semantic Caching**](labs/semantic-caching/semantic-caching.ipynb) | Cache responses using vector similarity for faster, cheaper completions | |
| 52 | | [**Model Routing**](labs/model-routing/model-routing.ipynb) | Route requests to different backends based on model and version | |
| 53 | | [**FinOps Framework**](labs/finops-framework/finops-framework.ipynb) | Manage AI budgets with automated quota controls | |
| 54 | |
| 55 | ## 🔧 AI Gateway for Tools |
| 56 | |
| 57 | Enable secure tool access with MCP protocol and function calling capabilities. |
| 58 | |
| 59 | | Lab | Description | |
| 60 | |-----|-------------| |
| 61 | | [**Model Context Protocol (MCP)**](labs/model-context-protocol/model-context-protocol.ipynb) | Plug & play tools with OAuth credential management | |
| 62 | | [**MCP Client Authorization**](labs/mcp-client-authorization/mcp-client-authorization.ipynb) | Implement MCP with the client authorization flow | |
| 63 | | [**Function Calling**](labs/function-calling/function-calling.ipynb) | Use OpenAI function calling with Azure Functions backend | |
| 64 | | [**Realtime Audio + MCP**](labs/realtime-mcp-agents/realtime-mcp-agents.ipynb) | Combine realtime voice API with MCP tools | |
| 65 | |
| 66 | ## 🤖 AI Gateway for Agents |
| 67 | |
| 68 | Build and control agentic applications with orchestration frameworks. |
| 69 | |
| 70 | | Lab | Description | |
| 71 | |-----|-------------| |
| 72 | | |
Related
6 picksType
- langchain-mcp-adaptersMake Anthropic Model Context Protocol (MCP) tools compatible with LangChain and LangGraph agents.Jul 2026
- mcporterTypeScript runtime and CLI for connecting to configured Model Context Protocol servers.Jul 2026
- mcp-proxyA MCP server which proxies requests to a remote MCP server over streamable HTTP or SSE.Jul 2026
- mcp-atlassianThe Model Context Protocol (MCP) Atlassian integration is an open-source implementation that bridges Atlassian products (Jira and Confluence) with AI language models following Anthropic's MCP specification.Jul 2026
- open-webuiOpen WebUIJul 2026
- rulesyncUnified AI rules management CLI tool that generates configuration files for various AI development toolsJul 2026
