analyzing-data

Name: analyzing-data
Author: astronomer

1.2kinstalls55forks

How to install analyzing-data?

astronomer/agents/analyzing-data

$npx -y skills add astronomer/agents --skill analyzing-data

Installs into the current project.

Summary

Queries the data warehouse with SQL and answers business questions about data. Use when answering anything that needs warehouse data - counts, metrics, trends, aggregations, joins across tables, data lookups, or ad-hoc SQL analysis (for example "who uses X", "how many Y", "show m

Files · 1

View on GitHub

SKILL.md

1	# Data Analysis
2
3	Answer business questions by querying the data warehouse. The kernel auto-starts on first `exec` call.
4
5	All CLI commands below are relative to this skill's directory. Before running any `scripts/cli.py` command, `cd` to the directory containing this file.
6
7	## Workflow
8
9	1. Pattern lookup — Check for a cached query strategy:
10	```bash
11	uv run scripts/cli.py pattern lookup "<user's question>"
12	```
13	If a pattern exists, follow its strategy. Record the outcome after executing:
14	```bash
15	uv run scripts/cli.py pattern record <name> --success # or --failure
16	```
17
18	2. Concept lookup — Find known table mappings:
19	```bash
20	uv run scripts/cli.py concept lookup <concept>
21	```
22
23	3. Table discovery — If cache misses, search the codebase (`Grep pattern="<concept>" glob="*/.sql"`) or query `INFORMATION_SCHEMA`. See [reference/discovery-warehouse.md](reference/discovery-warehouse.md).
24
25	4. Execute query:
26	```bash
27	uv run scripts/cli.py exec "df = run_sql('SELECT ...')"
28	uv run scripts/cli.py exec "print(df)"
29	```
30
31	5. Cache learnings — Always cache before presenting results:
32	```bash
33	# Cache concept → table mapping
34	uv run scripts/cli.py concept learn <concept> <TABLE> -k <KEY_COL>
35	# Cache query strategy (if discovery was needed)
36	uv run scripts/cli.py pattern learn <name> -q "question" -s "step" -t "TABLE" -g "gotcha"
37	```
38
39	6. Present findings to user.
40
41	## Kernel Functions
42
43	\| Function \| Returns \|
44	\|----------\|---------\|
45	\| `run_sql(query, limit=100)` \| Polars DataFrame \|
46	\| `run_sql_pandas(query, limit=100)` \| Pandas DataFrame \|
47	\| `run_sql_many(queries, limit=100)` \| List of Polars DataFrames (one per query) \|
48
49	`pl` (Polars) and `pd` (Pandas) are pre-imported.
50
51	Run independent queries together with `run_sql_many` — they execute concurrently (Snowflake async / connection-pool fan-out) instead of one at a time:
52
53	```bash
54	uv run scripts/cli.py exec "dfs = run_sql_many(['SELECT ...', 'SELECT ...']); print(dfs[0])"
55	```
56
57	`run_sql_many` is fail-fast: if any query errors, the call raises and the results of the queries that succeeded are discarded. Use separate `run_sql` calls if you need partial results.
58
59	Timeouts: `exec` waits up to 120s by default, then interrupts the query and returns a "client stopped waiting" message (the query may still finish server-side). Raise it for known long-running queries: `uv run scripts/cli.py exec "..." -t 600`.
60
61	Idle kernel: the kernel self-terminates after 2h idle (preserving state until then). Override with `ASTRO_KERNEL_IDLE_TIMEOUT` (seconds; `0` disables).
62
63	## CLI Reference
64
65	### Kernel
66
67	```bash
68	uv run scripts/cli.py warehouse list # List warehouses
69	uv run scripts/cli.py start [-w name] # Start kernel (with optional warehouse)
70	uv run scripts/cli.py exec "..." # Execute Python code
71	uv run scripts/cli.py status # Kernel status
72	uv run scripts/cli.py restart # Restart kernel
73	uv run scripts/cli.py stop # Stop kernel
74	uv run scripts/cli.py install <pkg> # Install package
75	```
76
77	### Concept Cache
78
79	```bash
80	uv run scripts/cli.py concept lookup <name> # Look up
81	uv run scripts/cli.py concept learn <name> <TABLE> -k <KEY_COL> # Learn
82	uv run scripts/cli.py concept list # List all
83	uv run scripts/cli.py concept import -p /path/to/warehouse.md # Bulk import
84	```
85
86	### Pattern Cache
87
88	```bash
89	uv run scripts/cli.py pattern lookup "question" # Look up
90	uv run scripts/cli.py pattern learn <name> -q "..." -s "..." -t "TABLE" -g "gotcha" # Learn
91	uv run scripts/cli.py pattern record <name> --success # Record outcome
92	uv run scripts/cli.py pattern list # List all
93	uv run scripts/cli.py pattern delete <name> # Delete
94	```
95
96	### Table Schema Cache
97
98	```bash
99	uv run scripts/cli.py table lookup <TABLE> # Look up schema
100	uv run scripts/cli.py table cache <TABLE> -c '[...]' # Cache schema
101	uv run scripts/cli.py table list # List cached
102	uv run scripts/cli.py table delete <TABLE> # Delete
103	```
104
105	### Cache Management
106
107	```bash
108	uv run scripts/cli.py cache status # Stats
109	uv run scripts/cli.py cache clear [--stale-only] # Clear
110	```
111
112	## References
113
114	- [reference/discovery-warehouse.md](reference/discovery-warehouse.md) — Large table handling, warehouse exploration, INFORMATION_SCHEMA queries
115	- [reference/common-patterns.md](reference/common-patterns.md) — SQL

People who liked this also viewed

6 picks

Type

analyzing-data

1.2kinstalls55forks

View on GitHub

How to install analyzing-data?

astronomer/agents/analyzing-data

$npx -y skills add astronomer/agents --skill analyzing-data

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Data Analysis
2
3	Answer business questions by querying the data warehouse. The kernel auto-starts on first `exec` call.
4
5	All CLI commands below are relative to this skill's directory. Before running any `scripts/cli.py` command, `cd` to the directory containing this file.
6
7	## Workflow
8
9	1. Pattern lookup — Check for a cached query strategy:
10	```bash
11	uv run scripts/cli.py pattern lookup "<user's question>"
12	```
13	If a pattern exists, follow its strategy. Record the outcome after executing:
14	```bash
15	uv run scripts/cli.py pattern record <name> --success # or --failure
16	```
17
18	2. Concept lookup — Find known table mappings:
19	```bash
20	uv run scripts/cli.py concept lookup <concept>
21	```
22
23	3. Table discovery — If cache misses, search the codebase (`Grep pattern="<concept>" glob="*/.sql"`) or query `INFORMATION_SCHEMA`. See [reference/discovery-warehouse.md](reference/discovery-warehouse.md).
24
25	4. Execute query:
26	```bash
27	uv run scripts/cli.py exec "df = run_sql('SELECT ...')"
28	uv run scripts/cli.py exec "print(df)"
29	```
30
31	5. Cache learnings — Always cache before presenting results:
32	```bash
33	# Cache concept → table mapping
34	uv run scripts/cli.py concept learn <concept> <TABLE> -k <KEY_COL>
35	# Cache query strategy (if discovery was needed)
36	uv run scripts/cli.py pattern learn <name> -q "question" -s "step" -t "TABLE" -g "gotcha"
37	```
38
39	6. Present findings to user.
40
41	## Kernel Functions
42
43	\| Function \| Returns \|
44	\|----------\|---------\|
45	\| `run_sql(query, limit=100)` \| Polars DataFrame \|
46	\| `run_sql_pandas(query, limit=100)` \| Pandas DataFrame \|
47	\| `run_sql_many(queries, limit=100)` \| List of Polars DataFrames (one per query) \|
48
49	`pl` (Polars) and `pd` (Pandas) are pre-imported.
50
51	Run independent queries together with `run_sql_many` — they execute concurrently (Snowflake async / connection-pool fan-out) instead of one at a time:
52
53	```bash
54	uv run scripts/cli.py exec "dfs = run_sql_many(['SELECT ...', 'SELECT ...']); print(dfs[0])"
55	```
56
57	`run_sql_many` is fail-fast: if any query errors, the call raises and the results of the queries that succeeded are discarded. Use separate `run_sql` calls if you need partial results.
58
59	Timeouts: `exec` waits up to 120s by default, then interrupts the query and returns a "client stopped waiting" message (the query may still finish server-side). Raise it for known long-running queries: `uv run scripts/cli.py exec "..." -t 600`.
60
61	Idle kernel: the kernel self-terminates after 2h idle (preserving state until then). Override with `ASTRO_KERNEL_IDLE_TIMEOUT` (seconds; `0` disables).
62
63	## CLI Reference
64
65	### Kernel
66
67	```bash
68	uv run scripts/cli.py warehouse list # List warehouses
69	uv run scripts/cli.py start [-w name] # Start kernel (with optional warehouse)
70	uv run scripts/cli.py exec "..." # Execute Python code
71	uv run scripts/cli.py status # Kernel status
72	uv run scripts/cli.py restart # Restart kernel
73	uv run scripts/cli.py stop # Stop kernel
74	uv run scripts/cli.py install <pkg> # Install package
75	```
76
77	### Concept Cache
78
79	```bash
80	uv run scripts/cli.py concept lookup <name> # Look up
81	uv run scripts/cli.py concept learn <name> <TABLE> -k <KEY_COL> # Learn
82	uv run scripts/cli.py concept list # List all
83	uv run scripts/cli.py concept import -p /path/to/warehouse.md # Bulk import
84	```
85
86	### Pattern Cache
87
88	```bash
89	uv run scripts/cli.py pattern lookup "question" # Look up
90	uv run scripts/cli.py pattern learn <name> -q "..." -s "..." -t "TABLE" -g "gotcha" # Learn
91	uv run scripts/cli.py pattern record <name> --success # Record outcome
92	uv run scripts/cli.py pattern list # List all
93	uv run scripts/cli.py pattern delete <name> # Delete
94	```
95
96	### Table Schema Cache
97
98	```bash
99	uv run scripts/cli.py table lookup <TABLE> # Look up schema
100	uv run scripts/cli.py table cache <TABLE> -c '[...]' # Cache schema
101	uv run scripts/cli.py table list # List cached
102	uv run scripts/cli.py table delete <TABLE> # Delete
103	```
104
105	### Cache Management
106
107	```bash
108	uv run scripts/cli.py cache status # Stats
109	uv run scripts/cli.py cache clear [--stale-only] # Clear
110	```
111
112	## References
113
114	- [reference/discovery-warehouse.md](reference/discovery-warehouse.md) — Large table handling, warehouse exploration, INFORMATION_SCHEMA queries
115	- [reference/common-patterns.md](reference/common-patterns.md) — SQL

People who liked this also viewed

6 picks

Type