azure-aigateway

Name: azure-aigateway
Author: microsoft

470kinstalls215forks

How to install azure-aigateway?

microsoft/azure-skills/azure-aigateway

$npx -y skills add microsoft/azure-skills --skill azure-aigateway

Installs into the current project.

Summary

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway,

Files · 1

View on GitHub

SKILL.md

1	# Azure AI Gateway
2
3	Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.
4
5	> To deploy APIM, use the azure-prepare skill. See [APIM deployment guide](https://learn.microsoft.com/azure/api-management/get-started-create-service-instance).
6
7	## When to Use This Skill
8
9	\| Category \| Triggers \|
10	\|----------\|----------\|
11	\| Model Governance \| "semantic caching", "token limits", "load balance AI", "track token usage" \|
12	\| Tool Governance \| "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" \|
13	\| Agent Governance \| "content safety", "jailbreak detection", "filter harmful content" \|
14	\| Configuration \| "add Azure OpenAI backend", "configure my model", "add AI Foundry model" \|
15	\| Testing \| "test AI gateway", "call OpenAI through gateway" \|
16
17	---
18
19	## Quick Reference
20
21	\| Policy \| Purpose \| Details \|
22	\|--------\|---------\|---------\|
23	\| `azure-openai-token-limit` \| Cost control \| [Model Policies](references/policies.md#token-rate-limiting) \|
24	\| `azure-openai-semantic-cache-lookup/store` \| 60-80% cost savings \| [Model Policies](references/policies.md#semantic-caching) \|
25	\| `azure-openai-emit-token-metric` \| Observability \| [Model Policies](references/policies.md#token-metrics) \|
26	\| `llm-content-safety` \| Safety & compliance \| [Agent Policies](references/policies.md#content-safety) \|
27	\| `rate-limit-by-key` \| MCP/tool protection \| [Tool Policies](references/policies.md#request-rate-limiting) \|
28
29	---
30
31	## Get Gateway Details
32
33	```bash
34	# Get gateway URL
35	az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv
36
37	# List backends (AI models)
38	az apim backend list --service-name <apim-name> --resource-group <rg> \
39	--query "[].{id:name, url:url}" -o table
40
41	# Get subscription key
42	az apim subscription keys list \
43	--service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>
44	```
45
46	---
47
48	## Test AI Endpoint
49
50	```bash
51	GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)
52
53	curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
54	-H "Content-Type: application/json" \
55	-H "Ocp-Apim-Subscription-Key: <key>" \
56	-d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'
57	```
58
59	---
60
61	## Common Tasks
62
63	### Add AI Backend
64
65	See [references/patterns.md](references/patterns.md#pattern-1-add-ai-model-backend) for full steps.
66
67	```bash
68	# Discover AI resources
69	az cognitiveservices account list --query "[?kind=='OpenAI']" -o table
70
71	# Create backend
72	az apim backend create --service-name <apim> --resource-group <rg> \
73	--backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"
74
75	# Grant access (managed identity)
76	az role assignment create --assignee <apim-principal-id> \
77	--role "Cognitive Services User" --scope <aoai-resource-id>
78	```
79
80	### Apply AI Governance Policy
81
82	Recommended policy order in `<inbound>`:
83
84	1. Authentication - Managed identity to backend
85	2. Semantic Cache Lookup - Check cache before calling AI
86	3. Token Limits - Cost control
87	4. Content Safety - Filter harmful content
88	5. Backend Selection - Load balancing
89	6. Metrics - Token usage tracking
90
91	See [references/policies.md](references/policies.md#combining-policies) for complete example.
92
93	---
94
95	## Troubleshooting
96
97	\| Issue \| Solution \|
98	\|-------\|----------\|
99	\| Token limit 429 \| Increase `tokens-per-minute` or add load balancing \|
100	\| No cache hits \| Lower `score-threshold` to 0.7 \|
101	\| Content false positives \| Increase category thresholds (5-6) \|
102	\| Backend auth 401 \| Grant APIM "Cognitive Services User" role \|
103
104	See [references/troubleshooting.md](references/troubleshooting.md) for details.
105
106	---
107
108	## References
109
110	- [Detailed Policies](references/policies.md) - Full policy examples
111	- [Configuration Patterns](references/patterns.md) - Step-by-step patterns
112	- [Troubleshooting](references/troubleshooting.md) - Common issues
113	- [AI-Gateway Samples](https://github.com/Azure-Samples/AI-Gateway)
114	- [GenAI Gateway Docs](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities)
115
116	## SDK Quick References
117
118	- Content Safety: [Python](references/sdk/azure-ai-contentsafety-py.md) \| [TypeScript](references/sdk/azure-ai-contentsafety-ts.md)
119	- API Management: [Python](references/sdk/azure-mgmt-apimanagement-py.md) \| [.NET](references/sdk/azure-m

People who liked this also viewed

6 picks

Type

azure-aigateway

470kinstalls215forks

View on GitHub

How to install azure-aigateway?

microsoft/azure-skills/azure-aigateway

$npx -y skills add microsoft/azure-skills --skill azure-aigateway

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Azure AI Gateway
2
3	Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.
4
5	> To deploy APIM, use the azure-prepare skill. See [APIM deployment guide](https://learn.microsoft.com/azure/api-management/get-started-create-service-instance).
6
7	## When to Use This Skill
8
9	\| Category \| Triggers \|
10	\|----------\|----------\|
11	\| Model Governance \| "semantic caching", "token limits", "load balance AI", "track token usage" \|
12	\| Tool Governance \| "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" \|
13	\| Agent Governance \| "content safety", "jailbreak detection", "filter harmful content" \|
14	\| Configuration \| "add Azure OpenAI backend", "configure my model", "add AI Foundry model" \|
15	\| Testing \| "test AI gateway", "call OpenAI through gateway" \|
16
17	---
18
19	## Quick Reference
20
21	\| Policy \| Purpose \| Details \|
22	\|--------\|---------\|---------\|
23	\| `azure-openai-token-limit` \| Cost control \| [Model Policies](references/policies.md#token-rate-limiting) \|
24	\| `azure-openai-semantic-cache-lookup/store` \| 60-80% cost savings \| [Model Policies](references/policies.md#semantic-caching) \|
25	\| `azure-openai-emit-token-metric` \| Observability \| [Model Policies](references/policies.md#token-metrics) \|
26	\| `llm-content-safety` \| Safety & compliance \| [Agent Policies](references/policies.md#content-safety) \|
27	\| `rate-limit-by-key` \| MCP/tool protection \| [Tool Policies](references/policies.md#request-rate-limiting) \|
28
29	---
30
31	## Get Gateway Details
32
33	```bash
34	# Get gateway URL
35	az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv
36
37	# List backends (AI models)
38	az apim backend list --service-name <apim-name> --resource-group <rg> \
39	--query "[].{id:name, url:url}" -o table
40
41	# Get subscription key
42	az apim subscription keys list \
43	--service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>
44	```
45
46	---
47
48	## Test AI Endpoint
49
50	```bash
51	GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)
52
53	curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
54	-H "Content-Type: application/json" \
55	-H "Ocp-Apim-Subscription-Key: <key>" \
56	-d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'
57	```
58
59	---
60
61	## Common Tasks
62
63	### Add AI Backend
64
65	See [references/patterns.md](references/patterns.md#pattern-1-add-ai-model-backend) for full steps.
66
67	```bash
68	# Discover AI resources
69	az cognitiveservices account list --query "[?kind=='OpenAI']" -o table
70
71	# Create backend
72	az apim backend create --service-name <apim> --resource-group <rg> \
73	--backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"
74
75	# Grant access (managed identity)
76	az role assignment create --assignee <apim-principal-id> \
77	--role "Cognitive Services User" --scope <aoai-resource-id>
78	```
79
80	### Apply AI Governance Policy
81
82	Recommended policy order in `<inbound>`:
83
84	1. Authentication - Managed identity to backend
85	2. Semantic Cache Lookup - Check cache before calling AI
86	3. Token Limits - Cost control
87	4. Content Safety - Filter harmful content
88	5. Backend Selection - Load balancing
89	6. Metrics - Token usage tracking
90
91	See [references/policies.md](references/policies.md#combining-policies) for complete example.
92
93	---
94
95	## Troubleshooting
96
97	\| Issue \| Solution \|
98	\|-------\|----------\|
99	\| Token limit 429 \| Increase `tokens-per-minute` or add load balancing \|
100	\| No cache hits \| Lower `score-threshold` to 0.7 \|
101	\| Content false positives \| Increase category thresholds (5-6) \|
102	\| Backend auth 401 \| Grant APIM "Cognitive Services User" role \|
103
104	See [references/troubleshooting.md](references/troubleshooting.md) for details.
105
106	---
107
108	## References
109
110	- [Detailed Policies](references/policies.md) - Full policy examples
111	- [Configuration Patterns](references/patterns.md) - Step-by-step patterns
112	- [Troubleshooting](references/troubleshooting.md) - Common issues
113	- [AI-Gateway Samples](https://github.com/Azure-Samples/AI-Gateway)
114	- [GenAI Gateway Docs](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities)
115
116	## SDK Quick References
117
118	- Content Safety: [Python](references/sdk/azure-ai-contentsafety-py.md) \| [TypeScript](references/sdk/azure-ai-contentsafety-ts.md)
119	- API Management: [Python](references/sdk/azure-mgmt-apimanagement-py.md) \| [.NET](references/sdk/azure-m

People who liked this also viewed

6 picks

Type