$npx -y skills add mjunaidca/mjs-agent-skills --skill building-chatgpt-appsGuides creation of ChatGPT Apps with interactive widgets using OpenAI Apps SDK and MCP servers. Use when building ChatGPT custom apps with visual UI components, embedded widgets, or rich interactive experiences. Covers widget architecture, MCP server setup with FastMCP, response
| 1 | # ChatGPT Apps SDK Development Guide |
| 2 | |
| 3 | ## Overview |
| 4 | |
| 5 | Create ChatGPT Apps with interactive widgets that render rich UI inside ChatGPT conversations. Apps combine MCP servers (providing tools) with embedded HTML widgets that communicate via the `window.openai` API. |
| 6 | |
| 7 | --- |
| 8 | |
| 9 | ## window.openai API Reference |
| 10 | |
| 11 | Widgets communicate with ChatGPT through these APIs: |
| 12 | |
| 13 | ### sendFollowUpMessage (Recommended for Actions) |
| 14 | |
| 15 | Send a follow-up prompt to ChatGPT on behalf of the user: |
| 16 | |
| 17 | ```javascript |
| 18 | // Trigger a follow-up conversation |
| 19 | if (window.openai?.sendFollowUpMessage) { |
| 20 | await window.openai.sendFollowUpMessage({ |
| 21 | prompt: 'Summarize this chapter for me' |
| 22 | }); |
| 23 | } |
| 24 | ``` |
| 25 | |
| 26 | **Use for**: Action buttons that suggest next steps (summarize, explain, etc.) |
| 27 | |
| 28 | ### toolOutput |
| 29 | |
| 30 | Send structured data back from widget interactions: |
| 31 | |
| 32 | ```javascript |
| 33 | // Send data back to ChatGPT |
| 34 | if (window.openai?.toolOutput) { |
| 35 | window.openai.toolOutput({ |
| 36 | action: 'chapter_selected', |
| 37 | chapter: 1, |
| 38 | title: 'Introduction' |
| 39 | }); |
| 40 | } |
| 41 | ``` |
| 42 | |
| 43 | **Use for**: Selections, form submissions, user choices that feed into tool responses. |
| 44 | |
| 45 | ### callTool |
| 46 | |
| 47 | Call another MCP tool from within a widget: |
| 48 | |
| 49 | ```javascript |
| 50 | // Call a tool directly |
| 51 | if (window.openai?.callTool) { |
| 52 | await window.openai.callTool({ |
| 53 | name: 'read-chapter', |
| 54 | arguments: { chapter: 2 } |
| 55 | }); |
| 56 | } |
| 57 | ``` |
| 58 | |
| 59 | **Use for**: Navigation between content, chaining tool calls. |
| 60 | |
| 61 | --- |
| 62 | |
| 63 | ## Critical: Button Interactivity Limitations |
| 64 | |
| 65 | **Important Discovery**: Widget buttons may render as **static UI elements** rather than interactive JavaScript buttons. ChatGPT renders widgets in a sandboxed iframe where some click handlers don't fire reliably. |
| 66 | |
| 67 | ### What Works |
| 68 | - `sendFollowUpMessage` - Reliably triggers follow-up prompts |
| 69 | - Simple onclick handlers for `toolOutput` calls |
| 70 | - CSS hover effects and visual feedback |
| 71 | |
| 72 | ### What May Not Work |
| 73 | - Complex interactive JavaScript (selection APIs, etc.) |
| 74 | - Multiple chained tool calls from buttons |
| 75 | - `window.getSelection()` for text selection features |
| 76 | |
| 77 | ### Recommended Pattern: Suggestion Buttons |
| 78 | |
| 79 | Instead of complex interactions, use simple buttons that suggest prompts: |
| 80 | |
| 81 | ```html |
| 82 | <div class="action-buttons"> |
| 83 | <button class="btn btn-primary" id="summarizeBtn"> |
| 84 | 📝 Summarize Chapter |
| 85 | </button> |
| 86 | <button class="btn btn-primary" id="explainBtn"> |
| 87 | 💡 Explain Key Concepts |
| 88 | </button> |
| 89 | </div> |
| 90 | |
| 91 | <script> |
| 92 | document.getElementById('summarizeBtn')?.addEventListener('click', async () => { |
| 93 | if (window.openai?.sendFollowUpMessage) { |
| 94 | await window.openai.sendFollowUpMessage({ |
| 95 | prompt: 'Summarize this chapter for me' |
| 96 | }); |
| 97 | } |
| 98 | }); |
| 99 | |
| 100 | document.getElementById('explainBtn')?.addEventListener('click', async () => { |
| 101 | if (window.openai?.sendFollowUpMessage) { |
| 102 | await window.openai.sendFollowUpMessage({ |
| 103 | prompt: 'Explain the key concepts from this chapter' |
| 104 | }); |
| 105 | } |
| 106 | }); |
| 107 | </script> |
| 108 | ``` |
| 109 | |
| 110 | --- |
| 111 | |
| 112 | ## Architecture Summary |
| 113 | |
| 114 | ``` |
| 115 | ┌─────────────────────────────────────────────────────────────────┐ |
| 116 | │ ChatGPT UI │ |
| 117 | │ ┌─────────────────────────────────────────────────────────────┐│ |
| 118 | │ │ Widget (iframe) ││ |
| 119 | │ │ HTML + CSS + JS ││ |
| 120 | │ │ Calls: window.openai.toolOutput({action: "...", ...}) ││ |
| 121 | │ └─────────────────────────────────────────────────────────────┘│ |
| 122 | │ │ │ |
| 123 | │ ▼ │ |
| 124 | │ ChatGPT Backend │ |
| 125 | │ │ │ |
| 126 | │ ▼ │ |
| 127 | │ MCP Server (FastMCP + HTTP) │ |
| 128 | │ - Tools: open-book, read-chapter, etc. │ |
| 129 | │ - Resources: widget HTML (text/html+skybridge) │ |
| 130 | │ - Response includes: _meta["openai.com/widget"] │ |
| 131 | └─────────────────────────────────────────────────────────────────┘ |
| 132 | ``` |
| 133 | |
| 134 | --- |
| 135 | |
| 136 | ## Quick Start |
| 137 | |
| 138 | 1. **Create MCP server** with FastMCP and widget resources |
| 139 | 2. **Define widget HTML** that uses `window.openai.toolOutput` |
| 140 | 3. **Add response metadata** with `_meta["openai.com/widget"]` |
| 141 | 4. **Expose via ngrok** for ChatGPT access |
| 142 | 5. **Register in ChatGPT** Developer Mode settings |
| 143 | |
| 144 | --- |
| 145 | |
| 146 | ## Widget HTML Requirements |
| 147 | |
| 148 | ### Basic Widget Template |
| 149 | |
| 150 | ```html |
| 151 | <!DOCTYPE html> |
| 152 | <html lang="en"> |
| 153 | <head> |
| 154 | <meta charset="UTF-8"> |
| 155 | <meta name="viewport" content="width=dev |