weaviate-data-ingestion

Name: weaviate-data-ingestion
Author: saskinosie

39stars5forks

How to install weaviate-data-ingestion?

saskinosie/weaviate-claude-skills/weaviate-data-ingestion

$npx -y skills add saskinosie/weaviate-claude-skills --skill weaviate-data-ingestion

Installs into the current project.

Summary

Upload and process data into local Weaviate collections with support for single objects, batch uploads, and multi-modal content

Files · 1

View on GitHub

SKILL.md

1	# Weaviate Data Ingestion Skill
2
3	This skill helps you upload data to your local Weaviate collections efficiently, handling everything from single objects to large batch imports.
4
5	## Important Note
6
7	This skill is designed for LOCAL Weaviate instances only. Ensure you have Weaviate running locally in Docker before using this skill.
8
9	## Purpose
10
11	Add data to your local Weaviate collections with automatic vectorization, proper error handling, and progress tracking.
12
13	## When to Use This Skill
14
15	- User wants to add data to a collection
16	- User needs to upload documents, articles, or records
17	- User has images or multi-modal content to ingest
18	- User wants to import data from files (JSON, CSV, text)
19	- User asks about batch uploading or bulk data import
20
21	## Prerequisites Check
22
23	Claude should verify these prerequisites before proceeding:
24
25	1. ✅ weaviate-local-setup completed - Python environment and dependencies installed
26	2. ✅ weaviate-connection completed - Successfully connected to Weaviate
27	3. ✅ weaviate-collection-manager used - Target collection exists
28	4. ✅ Docker container running - Weaviate is accessible at localhost:8080
29
30	If any prerequisites are missing, Claude should:
31	- Load the required prerequisite skill first
32	- Guide the user through the setup
33	- Then return to this skill
34
35	## Prerequisites
36
37	- Local Weaviate running in Docker (see weaviate-local-setup skill)
38	- Active Weaviate connection (use weaviate-connection skill first)
39	- Existing collection (use weaviate-collection-manager skill to create)
40	- Python weaviate-client library installed
41
42	## Operations
43
44	### 1. Add a Single Object
45
46	```python
47	import weaviate
48	from weaviate.classes.data import DataObject
49
50	# Assuming client is already connected
51	collection = client.collections.get("Articles")
52
53	# Add one object
54	uuid = collection.data.insert(
55	properties={
56	"title": "Introduction to Vector Databases",
57	"content": "Vector databases enable semantic search by storing embeddings...",
58	"author": "John Doe",
59	"publishDate": "2025-01-20T10:00:00Z"
60	}
61	)
62
63	print(f"✅ Object created with UUID: {uuid}")
64	```
65
66	### 2. Add Object with Custom Vector
67
68	```python
69	# If you bring your own embeddings
70	collection = client.collections.get("CustomEmbeddings")
71
72	uuid = collection.data.insert(
73	properties={
74	"text": "Your content here",
75	"metadata": "Additional info"
76	},
77	vector=[0.1, 0.2, 0.3, ...] # Your pre-computed embedding
78	)
79	```
80
81	### 3. Batch Upload Multiple Objects
82
83	#### Simple Batch Insert
84
85	```python
86	from weaviate.util import generate_uuid5
87
88	collection = client.collections.get("Articles")
89
90	# Prepare your data
91	articles = [
92	{
93	"title": "AI in 2025",
94	"content": "Artificial intelligence continues to evolve...",
95	"author": "Jane Smith",
96	"publishDate": "2025-01-15T00:00:00Z"
97	},
98	{
99	"title": "Vector Search Explained",
100	"content": "Vector search allows you to find similar items...",
101	"author": "Bob Johnson",
102	"publishDate": "2025-01-18T00:00:00Z"
103	},
104	# ... more articles
105	]
106
107	# Batch insert with context manager (recommended)
108	with collection.batch.dynamic() as batch:
109	for article in articles:
110	batch.add_object(
111	properties=article,
112	# Optional: provide deterministic UUID based on content
113	uuid=generate_uuid5(article['title'])
114	)
115
116	print(f"✅ Inserted {len(articles)} articles")
117	```
118
119	#### Batch with Error Handling
120
121	```python
122	from weaviate.util import generate_uuid5
123
124	collection = client.collections.get("Articles")
125	failed_objects = []
126
127	with collection.batch.dynamic() as batch:
128	for i, article in enumerate(articles):
129	try:
130	batch.add_object(
131	properties=article,
132	uuid=generate_uuid5(article.get('title', str(i)))
133	)
134	except Exception as e:
135	failed_objects.append({"index": i, "error": str(e), "data": article})
136	print(f"⚠️ Failed to add article {i}: {str(e)}")
137
138	# Check batch results
139	if batch.failed_objects:
140	print(f"❌ {len(batch.failed_objects)} objects failed")
141	for failed in batch.failed_objects:
142	print(f" Error: {failed.message}")
143	else:
144	print(f"✅ All {len(articles)} objects inserted successfully")
145	```
146
147	### 4. Upload Data from JSON File
148
149	```python
150	import json
151
152	# Read JSON file
153	with open("articles.json", "r") as f:
154	data = json.load(f)
155
156	collection = client.collections.get("Articles")
157
158	# Batch insert
159	with collection.batch.dynamic() as batch:
160	for item in data:
161	batch.add_object(properties=item)
162
163	print(f"✅ Imported {len(data)} objects from JSON")
164	```
165
166	### 5.

People who liked this also viewed

6 picks

Type

azure-aiUse for Azure AI: Search, Speech, OpenAI, Document Intelligence. Helps with search, vector/hybrid search, speech-to-text, text-to-speech, transcription, OCR. WHEN: AI Search, query search, vector searSkillsJul 2026472k1.3k
just-scrapeSearch, scrape, crawl, extract structured data, and monitor web pages via the ScrapeGraph AI CLI. Use when the user asks to search the web, scrape a webpage, grab content from a URL, extract JSON fromSkillsJul 2026245k37
firecrawlCLI and Agent Skill for Firecrawl - Add scrape, search, and browsing capabilities to your AI agentsSkillsJul 202694k531
firecrawl-searchWeb search with full page content extraction. Use this skill whenever the user asks to search the web, find articles, research a topic, look something up, find recent news, discover sources, or says "SkillsJul 202676k531
firecrawl-agentAI-powered autonomous data extraction that navigates complex sites and returns structured JSON. Use this skill when the user wants structured data from websites, needs to extract pricing tiers, producSkillsJul 202674k531
firecrawl-mapDiscover and list all URLs on a website, with optional search filtering. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where sSkillsJul 202674k531

weaviate-data-ingestion

39stars5forks

View on GitHub

How to install weaviate-data-ingestion?

saskinosie/weaviate-claude-skills/weaviate-data-ingestion

$npx -y skills add saskinosie/weaviate-claude-skills --skill weaviate-data-ingestion

Installs into the current project.

Summary

Upload and process data into local Weaviate collections with support for single objects, batch uploads, and multi-modal content

Files · 1

View on GitHub

SKILL.md

1	# Weaviate Data Ingestion Skill
2
3	This skill helps you upload data to your local Weaviate collections efficiently, handling everything from single objects to large batch imports.
4
5	## Important Note
6
7	This skill is designed for LOCAL Weaviate instances only. Ensure you have Weaviate running locally in Docker before using this skill.
8
9	## Purpose
10
11	Add data to your local Weaviate collections with automatic vectorization, proper error handling, and progress tracking.
12
13	## When to Use This Skill
14
15	- User wants to add data to a collection
16	- User needs to upload documents, articles, or records
17	- User has images or multi-modal content to ingest
18	- User wants to import data from files (JSON, CSV, text)
19	- User asks about batch uploading or bulk data import
20
21	## Prerequisites Check
22
23	Claude should verify these prerequisites before proceeding:
24
25	1. ✅ weaviate-local-setup completed - Python environment and dependencies installed
26	2. ✅ weaviate-connection completed - Successfully connected to Weaviate
27	3. ✅ weaviate-collection-manager used - Target collection exists
28	4. ✅ Docker container running - Weaviate is accessible at localhost:8080
29
30	If any prerequisites are missing, Claude should:
31	- Load the required prerequisite skill first
32	- Guide the user through the setup
33	- Then return to this skill
34
35	## Prerequisites
36
37	- Local Weaviate running in Docker (see weaviate-local-setup skill)
38	- Active Weaviate connection (use weaviate-connection skill first)
39	- Existing collection (use weaviate-collection-manager skill to create)
40	- Python weaviate-client library installed
41
42	## Operations
43
44	### 1. Add a Single Object
45
46	```python
47	import weaviate
48	from weaviate.classes.data import DataObject
49
50	# Assuming client is already connected
51	collection = client.collections.get("Articles")
52
53	# Add one object
54	uuid = collection.data.insert(
55	properties={
56	"title": "Introduction to Vector Databases",
57	"content": "Vector databases enable semantic search by storing embeddings...",
58	"author": "John Doe",
59	"publishDate": "2025-01-20T10:00:00Z"
60	}
61	)
62
63	print(f"✅ Object created with UUID: {uuid}")
64	```
65
66	### 2. Add Object with Custom Vector
67
68	```python
69	# If you bring your own embeddings
70	collection = client.collections.get("CustomEmbeddings")
71
72	uuid = collection.data.insert(
73	properties={
74	"text": "Your content here",
75	"metadata": "Additional info"
76	},
77	vector=[0.1, 0.2, 0.3, ...] # Your pre-computed embedding
78	)
79	```
80
81	### 3. Batch Upload Multiple Objects
82
83	#### Simple Batch Insert
84
85	```python
86	from weaviate.util import generate_uuid5
87
88	collection = client.collections.get("Articles")
89
90	# Prepare your data
91	articles = [
92	{
93	"title": "AI in 2025",
94	"content": "Artificial intelligence continues to evolve...",
95	"author": "Jane Smith",
96	"publishDate": "2025-01-15T00:00:00Z"
97	},
98	{
99	"title": "Vector Search Explained",
100	"content": "Vector search allows you to find similar items...",
101	"author": "Bob Johnson",
102	"publishDate": "2025-01-18T00:00:00Z"
103	},
104	# ... more articles
105	]
106
107	# Batch insert with context manager (recommended)
108	with collection.batch.dynamic() as batch:
109	for article in articles:
110	batch.add_object(
111	properties=article,
112	# Optional: provide deterministic UUID based on content
113	uuid=generate_uuid5(article['title'])
114	)
115
116	print(f"✅ Inserted {len(articles)} articles")
117	```
118
119	#### Batch with Error Handling
120
121	```python
122	from weaviate.util import generate_uuid5
123
124	collection = client.collections.get("Articles")
125	failed_objects = []
126
127	with collection.batch.dynamic() as batch:
128	for i, article in enumerate(articles):
129	try:
130	batch.add_object(
131	properties=article,
132	uuid=generate_uuid5(article.get('title', str(i)))
133	)
134	except Exception as e:
135	failed_objects.append({"index": i, "error": str(e), "data": article})
136	print(f"⚠️ Failed to add article {i}: {str(e)}")
137
138	# Check batch results
139	if batch.failed_objects:
140	print(f"❌ {len(batch.failed_objects)} objects failed")
141	for failed in batch.failed_objects:
142	print(f" Error: {failed.message}")
143	else:
144	print(f"✅ All {len(articles)} objects inserted successfully")
145	```
146
147	### 4. Upload Data from JSON File
148
149	```python
150	import json
151
152	# Read JSON file
153	with open("articles.json", "r") as f:
154	data = json.load(f)
155
156	collection = client.collections.get("Articles")
157
158	# Batch insert
159	with collection.batch.dynamic() as batch:
160	for item in data:
161	batch.add_object(properties=item)
162
163	print(f"✅ Imported {len(data)} objects from JSON")
164	```
165
166	### 5.

People who liked this also viewed

6 picks

Type