test-auth
starsforks
How to install test-auth?
microsoft/skills-for-copilot-studio/test-auth
$npx -y skills add microsoft/skills-for-copilot-studio --skill test-authAuthenticate for Copilot Studio evaluation API and SDK chat. Caches a token that is shared across run-eval and chat-sdk skills. Run this before any eval or SDK chat workflow. Requires an App Registration with MakerOperations and Copilots.Invoke permissions.
SKILL.md
| 1 | # Test Agent Authentication |
| 2 | |
| 3 | Authenticate with the Power Platform API for evaluation and SDK chat workflows. |
| 4 | This caches a token in the `"test-agent"` slot that is shared by `run-eval` and `chat-sdk`. |
| 5 | |
| 6 | ## Step 1: Find the workspace |
| 7 | |
| 8 | Locate the agent workspace (directory containing `.mcs/conn.json`): |
| 9 | ```bash |
| 10 | Glob: **/agent.mcs.yml |
| 11 | ``` |
| 12 | |
| 13 | The `conn.json` provides the tenant ID automatically — no need to ask the user for it. |
| 14 | |
| 15 | ## Step 2: Get the client ID |
| 16 | |
| 17 | **CRITICAL: You MUST present the FULL configuration checklist below when asking for the client ID. Do NOT omit any items — users frequently miss the redirect URI and public client settings, which causes auth failures.** |
| 18 | |
| 19 | Ask the user for their **App Registration Client ID** and show them **all** of the following requirements in your message: |
| 20 | |
| 21 | > What is your App Registration Client ID for the Evaluation API? |
| 22 | > |
| 23 | > Before sharing it, please verify your App Registration has **all** of these settings: |
| 24 | > |
| 25 | > 1. **Redirect URI**: Under **Authentication** > **Mobile and desktop applications** > redirect URI set to `http://localhost` |
| 26 | > 2. **Public client**: Under **Authentication** > **Advanced settings** > **Allow public client flows** set to **Yes** |
| 27 | > 3. **API permissions**: Under **API permissions**, these **delegated** permissions on the **Power Platform API**: |
| 28 | > - `CopilotStudio.MakerOperations.Read` (list test sets, get results) |
| 29 | > - `CopilotStudio.MakerOperations.ReadWrite` (start evaluation runs) |
| 30 | > - `CopilotStudio.Copilots.Invoke` (SDK chat — send utterances to the agent) |
| 31 | > 4. **Admin consent**: Granted for the tenant (green checkmarks in the API permissions list) |
| 32 | > |
| 33 | > Missing any of these will cause authentication to fail. |
| 34 | |
| 35 | If the user doesn't have one, guide them: |
| 36 | |
| 37 | > **How to create an App Registration:** |
| 38 | > 1. Go to [Azure Portal > App Registrations](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade) |
| 39 | > 2. Click **New registration**, name it (e.g., "Copilot Studio Eval") |
| 40 | > 3. Under **API permissions** > **Add a permission** > **APIs my organization uses** > search **Power Platform API** |
| 41 | > 4. Select **Delegated permissions** and add: |
| 42 | > - `CopilotStudio.MakerOperations.Read` |
| 43 | > - `CopilotStudio.MakerOperations.ReadWrite` |
| 44 | > - `CopilotStudio.Copilots.Invoke` |
| 45 | > 5. Click **Grant admin consent** for the tenant |
| 46 | > 6. Under **Authentication** > **Add a platform** > **Mobile and desktop applications** > set redirect URI to `http://localhost` |
| 47 | > 7. Still in **Authentication**, scroll to **Advanced settings** and set **Allow public client flows** to **Yes** (required — without this the interactive browser login will fail with an AADSTS error) |
| 48 | > 8. Copy the **Application (client) ID** from the Overview page |
| 49 | |
| 50 | ## Step 3: Authenticate |
| 51 | |
| 52 | Run the auth command (opens a browser — no device code needed): |
| 53 | |
| 54 | ```bash |
| 55 | node ${CLAUDE_SKILL_DIR}/../../scripts/eval-api.bundle.js auth --workspace <path> --client-id <id> |
| 56 | ``` |
| 57 | |
| 58 | If successful, you'll see: |
| 59 | ```json |
| 60 | {"status": "ok", "message": "Authentication successful. Token cached for subsequent commands."} |
| 61 | ``` |
| 62 | |
| 63 | **Return the client ID and workspace path to the caller** — they need these for subsequent `run-eval` and `chat-sdk` calls. |
| 64 | |
| 65 | ## Troubleshooting |
| 66 | |
| 67 | - **HTTP 403 / InsufficientDelegatedPermissions**: The app registration is missing required permissions. Guide the user through adding them (Step 2). |
| 68 | - **AADSTS700016 / app not found in directory**: The client ID doesn't exist in the agent's tenant. The app must be registered in the same tenant as the Copilot Studio environment (check `conn.json` > `AccountInfo.TenantId`). |
| 69 | - **AADSTS7000218 / request body must contain client_secret or client_assertion**: The app registration is not configured as a public client. Go to **Authentication** > **Advanced settings** > set **Allow public client flows** to **Yes**. |
| 70 | - **SDK chat hangs after auth**: The cached token may have stale permissions. Clear the cache and re-authenticate: |
| 71 | - Delete `~/.copilot-studio-cli/test-agent.cache.json` |
| 72 | - Run `security delete-generic-password -s "copilot-studio-cli" -a "test-agent"` to clear the macOS Keychain entry |
| 73 | - Re-run the auth command |
People who liked this also viewed
6 picksType
- tddSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026
- test-driven-developmentUse when implementing any feature or bugfix, before writing implementation codeSkillsJul 2026
- qaSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026
- verification-before-completionUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidenceSkillsJul 2026
- webapp-testingToolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser lSkillsJul 2026
- playwright-best-practicesUse when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, teSkillsJul 2026
