AI-Gateway
installs/mo · pypistarsforks
How to install AI-Gateway?
azure-samples/ai-gateway
$git clone https://github.com/Azure-Samples/AI-GatewayREADME.md
| 1 | <!-- markdownlint-disable MD033 --> |
| 2 | |
| 3 | <div align="center"> |
| 4 | |
| 5 | # ✨ AI Gateway Labs |
| 6 | [](https://github.com/Azure-Samples/AI-Gateway) |
| 7 | [](https://github.com/Azure-Samples/AI-Gateway/stargazers) |
| 8 | [](https://codespaces.new/Azure-Samples/AI-Gateway) |
| 9 | |
| 10 | |
| 11 | [](http://aka.ms/ai-gateway/labs) |
| 12 | |
| 13 | ### Explore the enterprise-grade gateway for managing AI Models, Tools, and Agents |
| 14 | |
| 15 | <br/> |
| 16 | |
| 17 | [](http://aka.ms/ai-gateway/labs) |
| 18 | |
| 19 | [](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities) |
| 20 | |
| 21 | </div> |
| 22 | |
| 23 | > 📰 **New!** The [AI Gateway Dev Portal](https://aka.ms/ai-gateway/dev-portal) is now live — A starting point for building your own developer portal on top of Azure API Management AI Gateway's. Fork it, open it in VS Code with GitHub Copilot (or any coding agent), and shape it to fit your needs! |
| 24 | |
| 25 | ## Why AI Gateway? |
| 26 | |
| 27 | Building production-ready AI applications requires more than just calling model APIs. You need **security**, **reliability**, **observability**, and **cost control**—without slowing down innovation. |
| 28 | |
| 29 | [**AI Gateway**](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities) powered by [Azure API Management](https://learn.microsoft.com/en-us/azure/api-management/) provides: |
| 30 | |
| 31 | - 🔐 **Security** — OAuth 2.0, managed identities, content safety filtering |
| 32 | - ⚡ **Performance** — Load balancing, semantic caching, request routing |
| 33 | - 📊 **Observability** — Token metrics, built-in logging, tracing |
| 34 | - 💰 **Cost Control** — Rate limiting, quota management, FinOps framework |
| 35 | - 🔌 **Extensibility** — MCP protocol support, function calling, multi-model routing |
| 36 | |
| 37 | ## 📚 Explore the Labs |
| 38 | |
| 39 | > **🔗 Browse all 30+ labs at [aka.ms/ai-gateway/labs](http://aka.ms/ai-gateway/labs)** |
| 40 | |
| 41 | Each lab is a hands-on Jupyter notebook with step-by-step instructions, Bicep infrastructure templates, and APIM policies you can deploy to your Azure subscription. |
| 42 | |
| 43 | ## 🧠 AI Gateway for Models |
| 44 | |
| 45 | Manage and control access to Large Language Models with enterprise-grade policies. |
| 46 | |
| 47 | | Lab | Description | |
| 48 | |-----|-------------| |
| 49 | | [**Backend Pool Load Balancing**](labs/backend-pool-load-balancing/backend-pool-load-balancing.ipynb) | Distribute requests across multiple model endpoints | |
| 50 | | [**Token Rate Limiting**](labs/token-rate-limiting/token-rate-limiting.ipynb) | Control token consumption with rate limiting policies | |
| 51 | | [**Semantic Caching**](labs/semantic-caching/semantic-caching.ipynb) | Cache responses using vector similarity for faster, cheaper completions | |
| 52 | | [**Model Routing**](labs/model-routing/model-routing.ipynb) | Route requests to different backends based on model and version | |
| 53 | | [**FinOps Framework**](labs/finops-framework/finops-framework.ipynb) | Manage AI budgets with automated quota controls | |
| 54 | |
| 55 | ## 🔧 AI Gateway for Tools |
| 56 | |
| 57 | Enable secure tool access with MCP protocol and function calling capabilities. |
| 58 | |
| 59 | | Lab | Description | |
| 60 | |-----|-------------| |
| 61 | | [**Model Context Protocol (MCP)**](labs/model-context-protocol/model-context-protocol.ipynb) | Plug & play tools with OAuth credential management | |
| 62 | | [**MCP Client Authorization**](labs/mcp-client-authorization/mcp-client-authorization.ipynb) | Implement MCP with the client authorization flow | |
| 63 | | [**Function Calling**](labs/function-calling/function-calling.ipynb) | Use OpenAI function calling with Azure Functions backend | |
| 64 | | [**Realtime Audio + MCP**](labs/realtime-mcp-agents/realtime-mcp-agents.ipynb) | Combine realtime voice API with MCP tools | |
| 65 | |
| 66 | ## 🤖 AI Gateway for Agents |
| 67 | |
| 68 | Build and control agentic applications with orchestration frameworks. |
| 69 | |
| 70 | | Lab | Description | |
| 71 | |-----|-------------| |
| 72 | | |
People who liked this also viewed
6 picksType
- fastmcp🚀 The fast, Pythonic way to build MCP servers and clients.MCP ServersJul 2026
- playwright-mcpPlaywright MCP serverMCP ServersJul 2026
- ext-appsOfficial repo for spec & SDK of MCP Apps protocol - standard for UIs embedded AI chatbots, served by MCP serversMCP ServersJul 2026
- langchain-mcp-adaptersLangChain 🔌 MCPMCP ServersJul 2026
- mcporterCall MCPs via TypeScript, masquerading as simple TypeScript API. Or package them as cli.MCP ServersJul 2026
- mcp-proxyA bridge between Streamable HTTP and stdio MCP transportsMCP ServersJul 2026
