chdb-datastore

Name: chdb-datastore
Author: ClickHouse

494stars32forks

How to install chdb-datastore?

clickhouse/agent-skills/chdb-datastore

$npx -y skills add ClickHouse/agent-skills --skill chdb-datastore

Installs into the current project.

Summary

Use when the user has tabular data (pandas DataFrame, parquet, csv, Arrow, json) and wants to filter, group, aggregate, join, or speed up slow pandas. Provides chDB DataStore — same pandas API, ClickHouse engine underneath. Also handles reading from S3, MySQL, PostgreSQL, MongoDB

Files · 1

View on GitHub

SKILL.md

1	# chdb DataStore — It's Just Faster Pandas
2
3	## The Key Insight
4
5	```python
6	# Change this:
7	import pandas as pd
8	# To this:
9	import chdb.datastore as pd
10	# Everything else stays the same.
11	```
12
13	DataStore is a lazy, ClickHouse-backed pandas replacement. Your existing pandas code works unchanged — but operations compile to optimized SQL and execute only when results are needed (e.g., `print()`, `len()`, iteration).
14
15	```bash
16	pip install chdb
17	```
18
19	## Decision Tree: Pick the Right Approach
20
21	```
22	1. "I have a file/database and want to analyze it with pandas"
23	→ DataStore.from_file() / from_mysql() / from_s3() etc.
24	→ See references/connectors.md
25
26	2. "I need to join data from different sources"
27	→ Create DataStores from each source, use .join()
28	→ See examples/examples.md #3-5
29
30	3. "My pandas code is too slow"
31	→ import chdb.datastore as pd — change one line, keep the rest
32
33	4. "I need raw SQL queries"
34	→ Use the chdb-sql skill instead
35	```
36
37	## Connect to Any Data Source — One Pattern
38
39	```python
40	from datastore import DataStore
41
42	# Local file (auto-detects .parquet, .csv, .json, .arrow, .orc, .avro, .tsv, .xml)
43	ds = DataStore.from_file("sales.parquet")
44
45	# Database
46	ds = DataStore.from_mysql(host="db:3306", database="shop", table="orders", user="root", password="pass")
47
48	# Cloud storage
49	ds = DataStore.from_s3("s3://bucket/data.parquet", nosign=True)
50
51	# URI shorthand — auto-detects source type
52	ds = DataStore.uri("mysql://root:pass@db:3306/shop/orders")
53	```
54
55	All 16+ sources and URI schemes → [connectors.md](references/connectors.md)
56
57	## After Connecting — Full Pandas API
58
59	```python
60	result = ds[ds["age"] > 25] # filter
61	result = ds[["name", "city"]] # select columns
62	result = ds.sort_values("revenue", ascending=False) # sort
63	result = ds.groupby("dept")["salary"].mean() # groupby
64	result = ds.assign(margin=lambda x: x["profit"] / x["revenue"]) # computed column
65	ds["name"].str.upper() # string accessor
66	ds["date"].dt.year # datetime accessor
67	result = ds1.join(ds2, on="id") # join
68	result = ds.head(10) # preview
69	print(ds.to_sql()) # see generated SQL
70	```
71
72	209 DataFrame methods supported. Full API → [api-reference.md](references/api-reference.md)
73
74	## Cross-Source Join — The Killer Feature
75
76	```python
77	from datastore import DataStore
78
79	customers = DataStore.from_mysql(host="db:3306", database="crm", table="customers", user="root", password="pass")
80	orders = DataStore.from_file("orders.parquet")
81
82	result = (orders
83	.join(customers, left_on="customer_id", right_on="id")
84	.groupby("country")
85	.agg({"amount": "sum", "rating": "mean"})
86	.sort_values("sum", ascending=False))
87	print(result)
88	```
89
90	More join examples → [examples.md](examples/examples.md)
91
92	## Writing Data
93
94	```python
95	source = DataStore.from_mysql(host="db:3306", database="shop", table="orders", user="root", password="pass")
96	target = DataStore("file", path="summary.parquet", format="Parquet")
97
98	target.insert_into("category", "total", "count").select_from(
99	source.groupby("category").select("category", "sum(amount) AS total", "count() AS count")
100	).execute()
101	```
102
103	## Troubleshooting
104
105	\| Problem \| Fix \|
106	\|---------\|-----\|
107	\| `ImportError: No module named 'chdb'` \| `pip install chdb` \|
108	\| `ImportError: cannot import 'DataStore'` \| Use `from datastore import DataStore` or `from chdb.datastore import DataStore` \|
109	\| Database connection timeout \| Include port in host: `host="db:3306"` not `host="db"` \|
110	\| Join returns empty result \| Check key types match (both int or both string); use `.to_sql()` to inspect \|
111	\| Unexpected results \| Call `ds.to_sql()` to see the generated SQL and debug \|
112	\| Environment check \| Run `python scripts/verify_install.py` (from skill directory) \|
113
114	## References
115
116	- [API Reference](references/api-re

People who liked this also viewed

6 picks

Type

chdb-datastore

494stars32forks

View on GitHub

How to install chdb-datastore?

clickhouse/agent-skills/chdb-datastore

$npx -y skills add ClickHouse/agent-skills --skill chdb-datastore

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# chdb DataStore — It's Just Faster Pandas
2
3	## The Key Insight
4
5	```python
6	# Change this:
7	import pandas as pd
8	# To this:
9	import chdb.datastore as pd
10	# Everything else stays the same.
11	```
12
13	DataStore is a lazy, ClickHouse-backed pandas replacement. Your existing pandas code works unchanged — but operations compile to optimized SQL and execute only when results are needed (e.g., `print()`, `len()`, iteration).
14
15	```bash
16	pip install chdb
17	```
18
19	## Decision Tree: Pick the Right Approach
20
21	```
22	1. "I have a file/database and want to analyze it with pandas"
23	→ DataStore.from_file() / from_mysql() / from_s3() etc.
24	→ See references/connectors.md
25
26	2. "I need to join data from different sources"
27	→ Create DataStores from each source, use .join()
28	→ See examples/examples.md #3-5
29
30	3. "My pandas code is too slow"
31	→ import chdb.datastore as pd — change one line, keep the rest
32
33	4. "I need raw SQL queries"
34	→ Use the chdb-sql skill instead
35	```
36
37	## Connect to Any Data Source — One Pattern
38
39	```python
40	from datastore import DataStore
41
42	# Local file (auto-detects .parquet, .csv, .json, .arrow, .orc, .avro, .tsv, .xml)
43	ds = DataStore.from_file("sales.parquet")
44
45	# Database
46	ds = DataStore.from_mysql(host="db:3306", database="shop", table="orders", user="root", password="pass")
47
48	# Cloud storage
49	ds = DataStore.from_s3("s3://bucket/data.parquet", nosign=True)
50
51	# URI shorthand — auto-detects source type
52	ds = DataStore.uri("mysql://root:pass@db:3306/shop/orders")
53	```
54
55	All 16+ sources and URI schemes → [connectors.md](references/connectors.md)
56
57	## After Connecting — Full Pandas API
58
59	```python
60	result = ds[ds["age"] > 25] # filter
61	result = ds[["name", "city"]] # select columns
62	result = ds.sort_values("revenue", ascending=False) # sort
63	result = ds.groupby("dept")["salary"].mean() # groupby
64	result = ds.assign(margin=lambda x: x["profit"] / x["revenue"]) # computed column
65	ds["name"].str.upper() # string accessor
66	ds["date"].dt.year # datetime accessor
67	result = ds1.join(ds2, on="id") # join
68	result = ds.head(10) # preview
69	print(ds.to_sql()) # see generated SQL
70	```
71
72	209 DataFrame methods supported. Full API → [api-reference.md](references/api-reference.md)
73
74	## Cross-Source Join — The Killer Feature
75
76	```python
77	from datastore import DataStore
78
79	customers = DataStore.from_mysql(host="db:3306", database="crm", table="customers", user="root", password="pass")
80	orders = DataStore.from_file("orders.parquet")
81
82	result = (orders
83	.join(customers, left_on="customer_id", right_on="id")
84	.groupby("country")
85	.agg({"amount": "sum", "rating": "mean"})
86	.sort_values("sum", ascending=False))
87	print(result)
88	```
89
90	More join examples → [examples.md](examples/examples.md)
91
92	## Writing Data
93
94	```python
95	source = DataStore.from_mysql(host="db:3306", database="shop", table="orders", user="root", password="pass")
96	target = DataStore("file", path="summary.parquet", format="Parquet")
97
98	target.insert_into("category", "total", "count").select_from(
99	source.groupby("category").select("category", "sum(amount) AS total", "count() AS count")
100	).execute()
101	```
102
103	## Troubleshooting
104
105	\| Problem \| Fix \|
106	\|---------\|-----\|
107	\| `ImportError: No module named 'chdb'` \| `pip install chdb` \|
108	\| `ImportError: cannot import 'DataStore'` \| Use `from datastore import DataStore` or `from chdb.datastore import DataStore` \|
109	\| Database connection timeout \| Include port in host: `host="db:3306"` not `host="db"` \|
110	\| Join returns empty result \| Check key types match (both int or both string); use `.to_sql()` to inspect \|
111	\| Unexpected results \| Call `ds.to_sql()` to see the generated SQL and debug \|
112	\| Environment check \| Run `python scripts/verify_install.py` (from skill directory) \|
113
114	## References
115
116	- [API Reference](references/api-re

People who liked this also viewed

6 picks

Type