rudder-data-graphs
starsforks
How to install rudder-data-graphs?
rudderlabs/rudder-agent-skills/rudder-data-graphs
$npx -y skills add rudderlabs/rudder-agent-skills --skill rudder-data-graphsProduces Data Graph YAML from RETL sources for Audiences. Use when designing Data Graphs, mapping RETL to entities/events, or assessing customer fit for Audiences.
SKILL.md
| 1 | # RETL Connection Analysis & Data Graph Design |
| 2 | |
| 3 | ## What this skill produces |
| 4 | |
| 5 | Given a customer name (or workspace / org identifier), this skill produces five deliverables, in order: |
| 6 | |
| 7 | 1. **Source Inventory** — every relevant RETL source categorized as entity / event / audience / supporting-only. |
| 8 | 2. **Untapped Segment List** — realistic, filterable business dimensions the customer has not yet expressed as audiences. |
| 9 | 3. **Data Graph YAML** — one per workspace / domain, ready for validation in `rudder-cli` or the visual builder. |
| 10 | 4. **Demo Warehouse Spec** — tables, key columns, joins, and known / inferred schema details needed to mock the graph. |
| 11 | 5. **Action Items** — priority-ordered next steps, including the upgrade narrative and any assumptions to confirm. |
| 12 | |
| 13 | The end-to-end flow is: find workspaces → list RETL sources → shortlist relevant sources → fetch full configs for the shortlist → categorize → design graph → validate → hand off. Each step below says what to do and *why*, so you can adapt when the customer doesn't match the common shapes. |
| 14 | |
| 15 | --- |
| 16 | |
| 17 | ## Step 1 — Find the customer's workspaces |
| 18 | |
| 19 | 1. `admin_search_organizations(company_name=...)` to get the `org_id`. |
| 20 | 2. `admin_search_workspaces(search_by=organizationId)` using that id. |
| 21 | 3. Keep only `status=ACTIVE` workspaces. *Why:* inactive workspaces often have stale or disconnected configs and will produce misleading source lists. |
| 22 | 4. Note the likely warehouse-backed workspaces, but **do not assume you already have the correct `accountId` at this stage**. *Why:* the Data Graph YAML embeds `account_id`, and that value is the warehouse account the graph runs against. The most authoritative source for it is `rudder-cli workspace accounts list --json` (see Step 4a); a RETL source config is a convenient secondary hint when one already exists, but it is not a prerequisite. |
| 23 | |
| 24 | --- |
| 25 | |
| 26 | ## Step 2 — Discover sources, anchored on audiences |
| 27 | |
| 28 | Run `list_sources(workspace_id=..., retl_type=all, includeConfig=false)` per workspace. *Why `includeConfig=false`:* full configs blow the context budget on large workspaces; start broad, then fetch detail only for sources you actually need. |
| 29 | |
| 30 | **Important:** `includeConfig=false` is a discovery pass only. You cannot extract `filterSpec`, SQL, join hints, or reliable `accountId` values from that response alone. After shortlisting sources, fetch full config for the shortlisted source ids before making graph decisions. |
| 31 | |
| 32 | **Audience sources are the anchor signal.** *Why:* this skill exists to build Audiences. Audience RETL sources are the customer's own pre-validated statements about what they segment on. A workspace can have hundreds of table sources but only a handful of audiences, and the audiences point directly at the entities that matter for activation. |
| 33 | |
| 34 | **Discovery order:** |
| 35 | |
| 36 | 1. **List every audience source first** from the discovery pass. |
| 37 | 2. **Fetch full config for those audience sources** and extract: |
| 38 | - `config.filterSpec.filterGroups[].filters[].fieldName` — the fields the customer actually filters on. |
| 39 | - The underlying table/model reference — this is the anchor entity for that audience. |
| 40 | - `primaryKey` and any source-level `accountId` if present. |
| 41 | 3. **Pull in the tables/models that back those audiences.** Fetch full config for those backing sources too. These are non-negotiable: you need their schema shape to size the entity properly and to compute untapped segments in Step 6. |
| 42 | 4. **Scan remaining tables/models for supporting roles:** |
| 43 | - **Event candidates** — tables/models with a real business-time timestamp (`ordered_at`, `sent_at`, `opened_at`, etc.) and a clear relationship back to an entity. Pull these in; events are a major Data Graph unlock that audiences alone can't surface. |
| 44 | - **FK targets** — tables/models referenced by foreign keys from anchor entities (e.g., if the anchor `contact` has a `branch_id`, pull in `branch`). Pull these in. |
| 45 | - **Everything else** — keep in the Source Inventory for completeness, but do not force into the Data Graph unless it materially improves segmentation or the customer asks. Noise suppression matters on large workspaces. |
| 46 | 5. **Resolve `accountId` only after the shortlist exists.** Cross-check the warehouse account id from the specific RETL source configs you are actually using in the graph against `rudder-cli workspace accounts list` (see Step 4a). If shortlisted sources disagree on `accountId`, stop and flag it; one graph should not span multiple warehouse accounts. |
| 47 | |
| 48 | **What to extract, by sourceType:** |
| 49 | |
| 50 | | `sourceType` | Fields to read | What you can trust from it | |
| 51 | |---|---|---| |
| 52 | | `audience` | `config.filterSpec.filterGroups[].filters[].fieldName`, underlying table/model ref, `primaryKey`, `accountId` if present | fields alrea |
People who liked this also viewed
6 picksType
- azure-kustoQuery and analyze data in Azure Data Explorer (Kusto/ADX) using KQL for log analytics, telemetry, and time series analysis. WHEN: KQL queries, Kusto database queries, Azure Data Explorer, ADX clustersSkillsJul 2026
- lark-base飞书多维表格（Base）操作：建表、字段、记录、视图、统计、公式/lookup、表单、仪表盘、workflow、角色权限；遇到 Base/多维表格/bitable 或 /base/ 链接时使用。文件导入转 lark-drive，认证/授权转 lark-shared。SkillsJul 2026
- lark-calendar飞书日历：管理日历日程和会议室。查看/搜索日程、创建/更新日程、管理参会人、查询忙闲和推荐时段、预定会议室。当用户需要查看日程安排、创建/修改会议、查询/预定会议室时使用。不负责：查询过去的视频会议记录（走 lark-vc）、待办任务（走 lark-task）。SkillsJul 2026
- shadcnManages shadcn components and projects — adding, searching, fixing, debugging, styling, and composing UI, including chat interfaces. Provides project context, component docs, and usage examples. AppliSkillsJul 2026
- xlsxUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.SkillsJul 2026
- firebase-data-connectAgent Skills for FirebaseSkillsJul 2026
