$npx -y skills add butterbase-ai/butterbase-skills --skill aiUse when calling the app's AI gateway from agent tools — chat completions, embeddings, listing models, configuring defaults or BYOK, reading token/cost usage
| 1 | # Butterbase AI Gateway |
| 2 | |
| 3 | Every app has an LLM gateway with chat, embeddings, model listing, configuration, and usage reporting. One umbrella tool: **`manage_ai`**. |
| 4 | |
| 5 | | Action | What it does | Returns | |
| 6 | |---|---|---| |
| 7 | | `chat` | Synchronous chat completion (no streaming) | OpenAI-shaped `{ choices: [...] }` | |
| 8 | | `embed` | Vector embeddings for string or string[] | OpenAI-shaped `{ data: [{ embedding: [...] }] }` | |
| 9 | | `list_models` | Available models with capabilities | `{ models: AiModel[] }` | |
| 10 | | `get_config` | Current AI config (default model, BYOK key flag, etc.) | `AiConfig` | |
| 11 | | `update_config` | Set defaults, allowed models, max tokens, BYOK | `AiConfig` | |
| 12 | | `get_usage` | Token + cost aggregate over a window | usage record | |
| 13 | |
| 14 | --- |
| 15 | |
| 16 | ## 1. Chat |
| 17 | |
| 18 | ``` |
| 19 | manage_ai({ |
| 20 | action: "chat", |
| 21 | app_id, |
| 22 | messages: [ |
| 23 | { role: "system", content: "You are a helpful assistant." }, |
| 24 | { role: "user", content: "What's RAG?" } |
| 25 | ], |
| 26 | model: "openai/gpt-4o-mini", // optional — falls back to app's default |
| 27 | temperature: 0.2, // optional |
| 28 | max_tokens: 500 // optional |
| 29 | }) |
| 30 | ``` |
| 31 | |
| 32 | This action sets `stream: false` deliberately — agent tools don't stream. If you need partial-token deltas, drive the SDK's `ai.chatStream(…)` from inside a function or DO instead. |
| 33 | |
| 34 | `messages[].content` can be a string or an array of content parts (`{ type: "text", text }`, `{ type: "image_url", image_url: {...} }`, `{ type: "video_url", video_url: {...} }`). |
| 35 | |
| 36 | --- |
| 37 | |
| 38 | ## 2. Embed |
| 39 | |
| 40 | ``` |
| 41 | manage_ai({ |
| 42 | action: "embed", |
| 43 | app_id, |
| 44 | input: "hello world", // or ["a", "b", "c"] |
| 45 | model: "openai/text-embedding-3-small", // optional |
| 46 | encoding_format: "float" // or "base64" |
| 47 | }) |
| 48 | ``` |
| 49 | |
| 50 | --- |
| 51 | |
| 52 | ## 3. List models |
| 53 | |
| 54 | ``` |
| 55 | manage_ai({ action: "list_models", app_id }) |
| 56 | // → { models: [{ id, provider, capabilities: ["chat", "embed", ...], context_window, pricing }, ...] } |
| 57 | ``` |
| 58 | |
| 59 | Use this to discover what the app can call — capabilities + context window matter when picking a model. |
| 60 | |
| 61 | --- |
| 62 | |
| 63 | ## 4. Configure |
| 64 | |
| 65 | ``` |
| 66 | manage_ai({ |
| 67 | action: "update_config", |
| 68 | app_id, |
| 69 | config: { |
| 70 | defaultModel: "openai/gpt-4o-mini", |
| 71 | allowedModels: ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"], |
| 72 | maxTokensPerRequest: 4000, |
| 73 | byokKey: "..." // optional — rotates the customer-supplied OpenRouter / Anthropic key |
| 74 | } |
| 75 | }) |
| 76 | ``` |
| 77 | |
| 78 | - `maxTokensPerRequest` is server-clamped to 1–100000. |
| 79 | - `allowedModels` is a whitelist — empty means all models the provider exposes. |
| 80 | - Setting `byokKey` switches the app to route through that customer key. Clear it by passing `byokKey: ""` (returns to platform pool). |
| 81 | |
| 82 | --- |
| 83 | |
| 84 | ## 5. Usage |
| 85 | |
| 86 | ``` |
| 87 | manage_ai({ |
| 88 | action: "get_usage", |
| 89 | app_id, |
| 90 | startDate: "2026-05-01", |
| 91 | endDate: "2026-05-31" |
| 92 | }) |
| 93 | ``` |
| 94 | |
| 95 | Returns aggregate token counts + cost. Useful for billing reconciliation, spending-cap diagnostics, and showing dashboards. |
| 96 | |
| 97 | --- |
| 98 | |
| 99 | ## 6. Common pitfalls |
| 100 | |
| 101 | - **Trying to stream from a tool** — `manage_ai` is synchronous. Use the SDK inside a function for streamed deltas. |
| 102 | - **Sending `stream: true` in the body** — the tool ignores it; always wired to `false`. |
| 103 | - **Hardcoding `model`** — better to omit, let the app's `defaultModel` win, and surface that knob via `update_config`. |
| 104 | - **Skipping `list_models` before suggesting one** — model availability shifts; verify before recommending. |
| 105 | |
| 106 | --- |
| 107 | |
| 108 | ## 7. What this skill does NOT cover |
| 109 | |
| 110 | - Streaming chat — use the SDK (`ai.chatStream`) inside a function or DO. |
| 111 | - Vector storage / retrieval — see `butterbase-skills:rag-dev` (RAG collections wrap embeddings + search together). |
| 112 | - AI in deployed functions — they import `@butterbase/sdk` and call `client.ai.*` directly; no MCP needed at runtime. |
| 113 | |
| 114 | --- |
| 115 | |
| 116 | If a `docs/butterbase/00-state.md` exists in the working directory, prefer invoking via `/butterbase-skills:journey-ai` so the journey orchestrator stays in sync. |