$npx -y skills add aliyun/alibabacloud-bigdata-skills --skill open-apiOperate Alibaba Cloud DataWorks through dynamic API discovery and official SDKs (Node.js, Python, Java). Covers data development, workflow operations, data integration, data quality, metadata lineage, workspace management, and more. APIs are discovered at runtime from official do
| 1 | # DataWorks (dataworks-public) |
| 2 | |
| 3 | Use Alibaba Cloud OpenAPI (RPC) with official SDKs or OpenAPI Explorer to manage all DataWorks resources. MCP Server is available as an alternative integration. |
| 4 | |
| 5 | ## Workflow |
| 6 | |
| 7 | 1. Confirm region, project/resource identifiers, and desired action. |
| 8 | 2. Discover available APIs and required parameters (see API discovery below). |
| 9 | 3. Call API via SDK. |
| 10 | 4. Verify results with describe/list/get APIs. |
| 11 | |
| 12 | ### Troubleshooting fallback chain (follow this order) |
| 13 | |
| 14 | If execution fails at any step, escalate to the next level: |
| 15 | |
| 16 | 1. **Cookbook** — check `references/cookbook.md` first. It contains verified API patterns, pitfalls, error recovery, the full end-to-end lifecycle (Create → Submit → Deploy → Run → Monitor), and reusable code snippets. Most common issues are already documented there. |
| 17 | 2. **Official help docs** — read the API overview and per-API doc pages. |
| 18 | - If an API call fails or the request/error is not understood, fetch the API directory page to locate the relevant API documentation: |
| 19 | `https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-dir` |
| 20 | Browse the category list, find the matching API name, and navigate to its detail page (URL pattern: `https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-{apinameincamelcase}`) for parameter details, error codes, and usage examples. |
| 21 | For example, if `CreateNode` fails, find it under "数据开发(新版)" in the directory, then read its detail page: `https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-createnode` |
| 22 | 3. **OpenAPI metadata** — fetch the raw JSON spec for exact parameter names, types, and required fields. |
| 23 | 4. **SDK API docs** — check the typed SDK reference for correct method signatures and request classes. |
| 24 | |
| 25 | - Java: `https://aliyunsdk-pages.alicdn.com/apidocs/{PRODUCT_CODE}/{API_VERSION}/java-async-tea/8.0.3/index.html` |
| 26 | - Node.js / Python: see `references/sources.md` |
| 27 | |
| 28 | 5. **GitHub source code** — read the Java SDK source for model/request class definitions: `https://github.com/aliyun/alibabacloud-java-sdk/tree/master/dataworks-public-20240518` (browse `src/main/java/com/aliyun/dataworks_public20240518/` for request/response models). |
| 29 | 6. **MCP Server** — use the MCP Server to call APIs directly (see `references/mcp_server.md`). |
| 30 | 7. **Browser** — use browser-use to visit the DataWorks Homepage (`https://dataworks.data.aliyun.com/${ALIBABA_CLOUD_REGION_ID}/#/index`) and inspect the UI behavior, network requests, or page content for clues. |
| 31 | |
| 32 | ## Credentials priority (must follow) |
| 33 | |
| 34 | 1. Environment variables: `ALIBABA_CLOUD_ACCESS_KEY_ID` / `ALIBABA_CLOUD_ACCESS_KEY_SECRET` / `ALIBABA_CLOUD_REGION_ID` |
| 35 | 2. Credentials URI: `ALIBABA_CLOUD_CREDENTIALS_URI` (e.g. `http://localhost:7002/api/v1/credentials/0`). The SDK credentials provider automatically fetches and refreshes AK/SK/STS tokens from this HTTP endpoint. |
| 36 | 3. Shared config file: `~/.alibabacloud/credentials` |
| 37 | |
| 38 | Region policy: `ALIBABA_CLOUD_REGION_ID` is an optional default. If unset, decide the most reasonable region for the task; if unclear, ask the user. |
| 39 | |
| 40 | ## API discovery |
| 41 | |
| 42 | Constants (use these values throughout): |
| 43 | |
| 44 | - **PRODUCT_CODE**: `dataworks-public` |
| 45 | - **API_VERSION**: `2024-05-18` |
| 46 | - Use OpenAPI metadata endpoints to list APIs and get schemas (see references). |
| 47 | |
| 48 | Do NOT rely on a hardcoded API list. Always discover APIs dynamically. Methods are listed in priority order — try them top-down: |
| 49 | |
| 50 | ### Method 1: Fetch from official help docs |
| 51 | |
| 52 | Run the discovery script to curl the official documentation page and extract the complete API overview: |
| 53 | |
| 54 | ```bash |
| 55 | python scripts/fetch_api_overview.py |
| 56 | ``` |
| 57 | |
| 58 | This fetches `https://help.aliyun.com/zh/dataworks/developer-reference/api-{PRODUCT_CODE}-{API_VERSION}-overview`, extracts the API list from `window.__ICE_ |