microsoft-foundry
bymicrosoft· 555 skills
Deploy, evaluate, fine-tune, and manage Foundry agents end-to-end with azd: hosted agent scaffold/run/deploy, prompt agent create, batch eval, continuous eval, prompt optimizer, Agent Optimizer scaffold, agent.yaml, dataset curation from traces, model fine-tuning (SFT/DPO/RFT). USE FOR: azd ai agent, azd provision/deploy, deploy agent, hosted agent, create agent, add tool to agent, invoke agent, evaluate agent, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, optimize agent instructions, agent optimizer, deploy model, Foundry project, RBAC, role assignment, permissions, quota, capacity, region, troubleshoot agent, deployment failure, AI Services, create Foundry resource, provision, knowledge index, customize deployment, onboard, availability, fine-tune, SFT, DPO, RFT, training-data, grader, distillation, fine-tuned model, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).
How to install microsoft-foundry?
microsoft/azure-skills/microsoft-foundry
$npx -y skills add microsoft/azure-skills --skill microsoft-foundryInstalls into the current project.
›Prefer a prompt? Paste this to your agent
Run `npx skills use "https://github.com/microsoft/azure-skills" --skill "microsoft/azure-skills/microsoft-foundry"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.
Use the skills in "https://github.com/microsoft/azure-skills" that are relevant to the current task. Run `npx skills add "https://github.com/microsoft/azure-skills"` and select the relevant skills, then follow their instructions.
SKILL.md
| 1 | # Microsoft Foundry Skill |
| 2 | |
| 3 | This skill helps developers work with Microsoft Foundry resources, covering model discovery and deployment, complete dev lifecycle of AI agent, evaluation workflows, and troubleshooting. |
| 4 | |
| 5 | ## Pre-Execution Requirements |
| 6 | |
| 7 | Before using Foundry MCP operations, call the Azure MCP `foundry` tool and inspect the available Foundry MCP tools and related parameters. Treat this as the discovery/help step for MCP-based workflows. |
| 8 | |
| 9 | ## Sub-Skills |
| 10 | |
| 11 | > **MANDATORY: Before executing ANY workflow-specific steps, you MUST read the corresponding sub-skill document.** Do not call workflow-specific MCP tools for a workflow without reading its skill document. This applies even if you already know the MCP tool parameters — the skill document contains required workflow steps, pre-checks, and validation logic that must be followed. This rule applies on every new user message that triggers a different workflow, even if the skill is already loaded. |
| 12 | |
| 13 | > **MANDATORY: Before executing ANY azd command, you MUST read [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) and strictly follow the shared rules defined in it, especially the `AZURE_DEV_USER_AGENT` setting rules.** |
| 14 | |
| 15 | This skill includes specialized sub-skills for specific workflows. **Use these instead of the main skill when they match your task:** |
| 16 | |
| 17 | | Sub-Skill | When to Use | Reference | |
| 18 | |-----------|-------------|-----------| |
| 19 | | **deploy** | Deploy hosted agents to Foundry, smoke-test a deployment, create or update prompt agents, and manage agent versions and multi-environment deploys. | [deploy](foundry-agent/deploy/deploy.md) | |
| 20 | | **cicd** | Set up a CI/CD deployment pipeline for a Foundry agent. | [cicd](foundry-agent/cicd/cicd.md) | |
| 21 | | **invoke** | Send messages to an agent, single or multi-turn conversations | [invoke](foundry-agent/invoke/invoke.md) | |
| 22 | | **routine** | Schedule or event-trigger Foundry agents with routines; use `azd` for CRUD, enable/disable, manual dispatch, and viewing past runs, or define routines in `azure.yaml`. | [routine](foundry-agent/routine/routine.md) | |
| 23 | | **invocations-ws** | Build, deploy, and connect to hosted agents that speak the `invocations_ws` duplex WebSocket protocol — voice agents, real-time streams, and signaling for out-of-band media transports. | [invocations-ws](foundry-agent/invocations-ws/invocations-ws.md) | |
| 24 | | **observe** | Evaluate agent quality, run batch evals, analyze failures, optimize prompts, improve agent instructions, compare versions, set up CI/CD monitoring, and enable continuous production evaluation | [observe](foundry-agent/observe/observe.md) | |
| 25 | | **trace** | Query traces, analyze latency/failures, correlate eval results to specific responses via App Insights `customEvents` | [trace](foundry-agent/trace/trace.md) | |
| 26 | | **troubleshoot** | View hosted agent logs, query telemetry, diagnose failures | [troubleshoot](foundry-agent/troubleshoot/troubleshoot.md) | |
| 27 | | **create (quick start)** | Create a new hosted Foundry agent from scratch end-to-end — scaffold, provision or use an existing Foundry project, deploy, and smoke-test. Do not use for any work on existing code. For anything not covered by the quickstart, use **create**. | [create/quick-start-hosted.md](foundry-agent/create/quick-start-hosted.md) | |
| 28 | | **create** | Use when the standard end-to-end happy path doesn't fit. Create a new Foundry agent, update code of an existing agent, continue development of an existing agent, wire connections at scaffold time, use advanced setup or A2A (Agent2Agent), or recover from a failed quickstart run. | [create](foundry-agent/create/create-hosted.md) | |
| 29 | | **agent-optimizer** | Make existing Python hosted-agent code optimization-ready, configure eval.yaml, run Agent Optimizer jobs, apply candidates locally, and deploy through azd after review. | [agent-optimizer](foundry-agent/agent-optimizer/agent-optimizer.md) | |
| 30 | | **eval-datasets** | Harvest production traces into evaluation datasets, manage dataset versions and splits, track evaluation metrics over time, detect regressions, and maintain full lineage from trace to deployment. Use for: create dataset from traces, dataset versioning, evaluation trending, regression detection, dataset comparison, eval lineage. | [eval-datasets](foundry-agent/eval-datasets/eval-datasets.md) | |
| 31 | | **project/create** | Creating a new Azure AI Foundry project for hosting agents and models. Use when onboarding to Foundry or setting up new infrastructure. | [project/create/create-foundry-project.md](project/create/create-foundry-project.md) | |
| 32 | | **resource/create** | Creating Azure AI Services multi-service resource (Foundry resource) using Azure CLI. Use when manually provisioning AI Services resources with granular control. | [resource/create/create-foundry-resource.md](resource/create/create-foundry-resource.md) | |
| 33 | | **private-network** | Answer questions about Foundry network isolation **and** deploy Foundry with VNet isolation (BYO VNet, Managed VNet, hybrid). Covers architecture concepts, template selection, deployment, and post-deployment validation. | [resource/private-network/private-network.md](resource/private-network/private-network.md) | |
| 34 | | **models/deploy-model** | Unified model deployment with intelligent routing. Handles quick preset deployments, fully customized deployments (version/SKU/capacity/RAI), and capacity discovery across regions. Routes to sub-skills: `preset` (quick deploy), `customize` (full control), `capacity` (find availability). | [models/deploy-model/SKILL.md](models/deploy-model/SKILL.md) | |
| 35 | | **quota** | Managing quotas and capacity for Microsoft Foundry resources. Use when checking quota usage, troubleshooting deployment failures due to insufficient quota, requesting quota increases, or planning capacity. | [quota/quota.md](quota/quota.md) | |
| 36 | | **rbac** | Managing RBAC permissions, role assignments, managed identities, and service principals for Microsoft Foundry resources. Use for access control, auditing permissions, and CI/CD setup. | [rbac/rbac.md](rbac/rbac.md) | |
| 37 | | **finetuning** | Fine-tune models on Azure AI Foundry — SFT distillation, DPO preference optimization, RFT with graders and tool calling. Dataset preparation, grader calibration, training, checkpoint selection, deployment, evaluation. Use for: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, large file upload. | [finetuning/SKILL.md](finetuning/SKILL.md) | |
| 38 | | **azd-guidance** | Provide shared azd knowledge and guidance for managing Foundry agents. Read this first for any workflows related to azd. | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) | |
| 39 | |
| 40 | > 💡 **Tip:** For a complete onboarding flow: `project/create` (public) or `private-network` (VNet isolation) → `models/deploy-model` → agent workflows (`create` → `deploy` → `invoke`). |
| 41 | |
| 42 | > 💡 **Fine-Tuning:** Use `finetuning` for all model customization — SFT distillation, DPO preference optimization, and RFT with graders. Includes quickstart, grader calibration, and training curve analysis. |
| 43 | |
| 44 | > 💡 **Model Deployment:** Use `models/deploy-model` for all deployment scenarios — it intelligently routes between quick preset deployment, customized deployment with full control, and capacity discovery across regions. |
| 45 | |
| 46 | > 💡 **Prompt Optimization:** For requests like "optimize my prompt" or "improve my agent instructions," load [observe](foundry-agent/observe/observe.md) and use the `prompt_optimize` MCP tool through that eval-driven workflow. |
| 47 | |
| 48 | ## Infrastructure Lifecycle |
| 49 | |
| 50 | Match user intent to the correct infrastructure workflow. |
| 51 | |
| 52 | | User Intent | Workflow | |
| 53 | |-------------|---------| |
| 54 | | "Create Foundry" / "Set up Foundry" (ambiguous) | Use `AskUserQuestion`: (a) just an AI Services resource, (b) a project with public access, or (c) a project with network isolation? Route: (a) → [resource/create](resource/create/create-foundry-resource.md), (b) → [project/create](project/create/create-foundry-project.md), (c) → [private-network](resource/private-network/private-network.md) | |
| 55 | | Set up Foundry with VNet isolation | [private-network](resource/private-network/private-network.md) | |
| 56 | | Create a Foundry project (public) | [project/create](project/create/create-foundry-project.md) | |
| 57 | | Create a bare Foundry resource | [resource/create](resource/create/create-foundry-resource.md) | |
| 58 | |
| 59 | ## Agent Development Lifecycle |
| 60 | |
| 61 | Match user intent to the correct agent workflow. Read each sub-skill in order before executing. |
| 62 | |
| 63 | | User Intent | Workflow (read in order) | |
| 64 | |-------------|------------------------| |
| 65 | | Create a new hosted agent end-to-end (scaffold + deploy + test) | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → [quick-start-hosted](foundry-agent/create/quick-start-hosted.md) (self-contained end-to-end) | |
| 66 | | Anything beyond the standard quickstart (existing code, migration, re-hosting, deployment customization, scaffold-time connections, A2A (Agent2Agent), recovery) | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → [create](foundry-agent/create/create-hosted.md) → [deploy](foundry-agent/deploy/deploy.md) → [invoke](foundry-agent/invoke/invoke.md) | |
| 67 | | Optimize existing Python hosted agent | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → [agent-optimizer](foundry-agent/agent-optimizer/agent-optimizer.md) → scaffold/review → eval.yaml → optimize → apply candidate → deploy → invoke | |
| 68 | | Deploy an agent (code already exists) | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → deploy (includes eval-suite setup) → invoke → observe (evaluate/optimize) | |
| 69 | | Update/redeploy an agent after code changes | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → deploy (includes eval-suite setup) → invoke → observe (evaluate/optimize) | |
| 70 | | Set up a CI/CD deployment pipeline for a hosted agent | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → cicd | |
| 71 | | Invoke/test/chat with an agent | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → invoke | |
| 72 | | Schedule/event-trigger an agent, or CRUD/enable/disable/dispatch a routine | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → routine | |
| 73 | | Optimize / improve agent prompt or instructions | observe (Step 4: Optimize) | |
| 74 | | Evaluate and optimize agent (full loop) | observe | |
| 75 | | Enable continuous evaluation monitoring | observe (Step 6: CI/CD & Monitoring) | |
| 76 | | Troubleshoot an agent issue | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → invoke → troubleshoot | |
| 77 | | Fix a broken agent (troubleshoot + redeploy) | [azd-guidance](foundry-agent/azd-guidance/azd-guidance.md) → invoke → troubleshoot → apply fixes → deploy → invoke | |
| 78 | |
| 79 | ## Agent: .foundry Workspace Standard |
| 80 | |
| 81 | Every agent source folder can keep Foundry-specific cache and overlay state under `.foundry/`: |
| 82 | |
| 83 | ```text |
| 84 | <agent-root>/ |
| 85 | .foundry/ |
| 86 | agent-metadata.yaml |
| 87 | agent-metadata.prod.yaml |
| 88 | suites/ |
| 89 | datasets/ |
| 90 | evaluators/ |
| 91 | results/ |
| 92 | ``` |
| 93 | |
| 94 | - In azd projects, derive deployment context (project endpoint, agent name/version, ACR, App Insights) from `azure.yaml` plus `azd env get-values`; do not duplicate those values in metadata when azd already provides them. |
| 95 | - `agent-metadata.yaml` is the preferred local/dev overlay for non-azd values, remote Foundry suite references, local cache paths, result summaries, and explicit overrides. Optional sidecar files such as `agent-metadata.prod.yaml` can hold a single prod or CI-targeted overlay without mixing multiple environments in one file. |
| 96 | - `suites/`, `datasets/`, and `evaluators/` are local cache folders. Reuse them when they are current, and ask before refreshing or overwriting them. |
| 97 | - See [Agent Metadata Contract](references/agent-metadata-contract.md) for the canonical schema and workflow rules. |
| 98 | |
| 99 | ## Agent: Setup References |
| 100 | |
| 101 | - [Standard Agent Setup](references/standard-agent-setup.md) — advanced setup for production workloads that need data-residency control (bring-your-own Cosmos DB / Storage / AI Search via a Foundry capability host). The default `azd ai agent` flow uses **Basic Agent Setup** and does **not** provision `capabilityHosts/agents` — do not flag its absence as a bug. For default post-provision state, see the "Expected env-var fingerprint" section in [foundry-agent/create/create-hosted.md](foundry-agent/create/create-hosted.md). |
| 102 | |
| 103 | ## Agent: Common Project Context Resolution |
| 104 | |
| 105 | Agent skills should run this step **only when they need configuration values they don't already have**. If a value (for example, agent root, environment, project endpoint, or agent name) is already known from the user's message or a previous skill in the same session, skip resolution for that value. |
| 106 | |
| 107 | ### Step 1: Discover Agent Roots and azd Context |
| 108 | |
| 109 | First check whether the workspace has `azure.yaml` with services using `host: azure.ai.agent`. |
| 110 | |
| 111 | - **One azd agent service** -> use that service's `project` folder as the agent root. |
| 112 | - **Multiple azd agent services** -> require the user to choose the target service/folder. |
| 113 | - **No azd agent service** -> search the workspace for `.foundry/` folders that contain `agent-metadata.yaml` or `agent-metadata.<env>.yaml`. |
| 114 | - **One match** -> use that agent root. |
| 115 | - **Multiple matches** -> require the user to choose the target agent folder. |
| 116 | - **No matches** -> for create/deploy workflows, seed a new `.foundry/` folder during setup; for all other workflows, stop and ask the user which agent source folder to initialize. |
| 117 | |
| 118 | After selecting an agent root, keep all local `.foundry` cache inspection, source inspection, evaluator suggestions, dataset suggestions, and prompt-optimization context inside that folder only. Do **not** scan sibling agent folders unless the user explicitly switches roots. |
| 119 | |
| 120 | ### Step 2: Resolve Environment and Deployment Context |
| 121 | |
| 122 | If `azure.yaml` is present, resolve the azd environment first: |
| 123 | |
| 124 | 1. Environment explicitly named by the user |
| 125 | 2. `AZURE_ENV_NAME` from `azd env get-values` |
| 126 | 3. azd default environment from `.azure/config.json` |
| 127 | 4. Environment already selected earlier in the session |
| 128 | |
| 129 | Run `azd env get-values` for the selected environment when project/deployment values are not already known. Prefer azd values for deployment context: |
| 130 | |
| 131 | | azd Variable | Resolves To | |
| 132 | |-------------|-------------| |
| 133 | | `AZURE_AI_PROJECT_ENDPOINT` or `AZURE_AIPROJECT_ENDPOINT` | Project endpoint | |
| 134 | | `AGENT_<SERVICE>_NAME` | Agent name for the selected azd service | |
| 135 | | `AGENT_<SERVICE>_VERSION` | Agent version for the selected azd service | |
| 136 | | `AZURE_CONTAINER_REGISTRY_NAME` or `AZURE_CONTAINER_REGISTRY_ENDPOINT` | ACR registry name / image URL prefix | |
| 137 | | `APPLICATIONINSIGHTS_CONNECTION_STRING` | App Insights connection string for trace workflows | |
| 138 | | `AZURE_SUBSCRIPTION_ID`, `AZURE_RESOURCE_GROUP`, `AZURE_AI_ACCOUNT_NAME`, `AZURE_AI_PROJECT_NAME` | Azure resource lookup and Playground links | |
| 139 | |
| 140 | When azd supplies these values, use them as the source of truth and do not copy them into `.foundry/agent-metadata*.yaml` on metadata writes. |
| 141 | |
| 142 | ### Step 3: Select Metadata Overlay and Resolve Environment |
| 143 | |
| 144 | Inside the selected agent root, choose the metadata file in this order: |
| 145 | 1. Metadata filename or path explicitly provided by the user or workflow |
| 146 | 2. If an explicit environment is already known and `.foundry/agent-metadata.<env>.yaml` exists, use that file |
| 147 | 3. `.foundry/agent-metadata.yaml` |
| 148 | 4. If multiple metadata files remain and no rule above selects one, prompt the user to choose |
| 149 | |
| 150 | Read the selected metadata file and resolve any remaining environment choice in this order: |
| 151 | 1. Environment explicitly named by the user |
| 152 | 2. If the selected metadata file defines exactly one environment, use it |
| 153 | 3. Environment already selected earlier in the session |
| 154 | 4. `defaultEnvironment` from metadata |
| 155 | |
| 156 | If the selected metadata file still contains multiple environments and none of the rules above selects one, prompt the user to choose. Keep the selected agent root, metadata file, environment, and whether context came from azd or metadata visible in every workflow summary. |
| 157 | |
| 158 | If the selected environment exposes older `testSuites[]` metadata but not `evaluationSuites[]`, treat `testSuites[]` as the source for this session and normalize each entry in memory to the `evaluationSuites[]` shape before continuing. If the metadata is older still and only exposes legacy `testCases[]`, normalize that list the same way. Preserve dataset and evaluator fields, keep any existing `tags`, and map legacy `priority` to `tags.tier` only when `tags.tier` is missing: `P0` -> `smoke`, `P1` -> `regression`, `P2` -> `coverage`. |
| 159 | |
| 160 | ### Step 4: Resolve eval.yaml Local Evaluation Intent |
| 161 | |
| 162 | If `eval.yaml` exists in the selected agent root, parse it before generating new suites: |
| 163 | |
| 164 | - `agent.name` -> target agent candidate; verify it matches the selected azd/metadata agent before using it. |
| 165 | - `dataset.local_uri` -> local seed dataset candidate; legacy `dataset_file` may be normalized in memory. |
| 166 | - `dataset.name` / `dataset.version` -> registered dataset candidate. |
| 167 | - `validation_dataset` -> optional validation dataset candidate. |
| 168 | - `evaluators[]` -> candidate Foundry evaluator names; verify with `evaluator_catalog_get` before treating them as remote evaluators. |
| 169 | - `name` -> local eval/suite candidate; verify remotely before persisting as `suiteName`. |
| 170 | - `options.eval_model`, `options.optimization_model`, `options.max_candidates`, `options.optimization_config.model_search_space`, `options.pass_threshold`, `max_samples`, `trace_days`, and `generation_instruction` -> setup defaults. |
| 171 | |
| 172 | Treat `eval.yaml` as local evaluation intent, not proof that a Foundry suite exists. Persist synced suite/dataset/evaluator references to `.foundry` only after remote lookup or registration succeeds. |
| 173 | |
| 174 | ### Step 5: Resolve Common Configuration |
| 175 | |
| 176 | Layer sources in this order: |
| 177 | |
| 178 | 1. Explicit user input and values already selected in the session |
| 179 | 2. azd environment values for deployment context |
| 180 | 3. `.foundry/agent-metadata*.yaml` overlay values and remote suite/cache references |
| 181 | 4. `azure.yaml` and `eval.yaml` local source configuration |
| 182 | 5. User prompts for anything still missing |
| 183 | |
| 184 | If azd and metadata both provide the same value and they differ, stop and ask which source is authoritative. If they match, use the azd value and avoid rewriting the duplicate on future metadata writes. |
| 185 | |
| 186 | | Effective Value | Preferred Source | Used By | |
| 187 | |-----------------|------------------|---------| |
| 188 | | Project endpoint | azd env | deploy, invoke, observe, trace, troubleshoot | |
| 189 | | Agent name/version | azd agent variables, then `azure.yaml` | invoke, observe, trace, troubleshoot | |
| 190 | | ACR | azd env | deploy | |
| 191 | | Evaluation suites and cache paths | `.foundry/agent-metadata*.yaml` | observe, eval-datasets | |
| 192 | | Local seed dataset/evaluator intent | `eval.yaml` | observe, eval-datasets | |
| 193 | |
| 194 | ### Step 6: Write Metadata Overlay (Create/Deploy/Observe Only) |
| 195 | |
| 196 | On any metadata write (deploy, auto-setup, dataset refresh, or trace-to-dataset update), persist only n |
Related
6 picksType
- setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout.Jul 2026
- azure-deployExecute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files.Jul 2026
- azure-preparePrepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd)…Jul 2026
- azure-validatePre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity…Jul 2026
- azure-aigatewayConfigure Azure API Management as an AI Gateway for AI models, MCP tools, and agents.Jul 2026
- azure-cloud-migrateAssess and migrate cross-cloud workloads to Azure with reports and code conversion.Jul 2026
