langsmith-fetch

Name: langsmith-fetch
Author: ECNU-ICALK

524stars49forks

How to install langsmith-fetch?

ecnu-icalk/autoskill/langsmith-fetch

$npx -y skills add ECNU-ICALK/AutoSkill --skill langsmith-fetch

Installs into the current project.

Summary

Debug LangChain and LangGraph agents by fetching execution traces from LangSmith Studio. Use when debugging agent behavior, investigating errors, analyzing tool calls, checking memory operations, or examining agent performance. Automatically fetches recent traces and analyzes exe

Files · 1

View on GitHub

SKILL.md

1	# LangSmith Fetch - Agent Debugging Skill
2
3	Debug LangChain and LangGraph agents by fetching execution traces directly from LangSmith Studio in your terminal.
4
5	## When to Use This Skill
6
7	Automatically activate when user mentions:
8	- 🐛 "Debug my agent" or "What went wrong?"
9	- 🔍 "Show me recent traces" or "What happened?"
10	- ❌ "Check for errors" or "Why did it fail?"
11	- 💾 "Analyze memory operations" or "Check LTM"
12	- 📊 "Review agent performance" or "Check token usage"
13	- 🔧 "What tools were called?" or "Show execution flow"
14
15	## Prerequisites
16
17	### 1. Install langsmith-fetch
18	```bash
19	pip install langsmith-fetch
20	```
21
22	### 2. Set Environment Variables
23	```bash
24	export LANGSMITH_API_KEY="your_langsmith_api_key"
25	export LANGSMITH_PROJECT="your_project_name"
26	```
27
28	Verify setup:
29	```bash
30	echo $LANGSMITH_API_KEY
31	echo $LANGSMITH_PROJECT
32	```
33
34	## Core Workflows
35
36	### Workflow 1: Quick Debug Recent Activity
37
38	When user asks: "What just happened?" or "Debug my agent"
39
40	Execute:
41	```bash
42	langsmith-fetch traces --last-n-minutes 5 --limit 5 --format pretty
43	```
44
45	Analyze and report:
46	1. ✅ Number of traces found
47	2. ⚠️ Any errors or failures
48	3. 🛠️ Tools that were called
49	4. ⏱️ Execution times
50	5. 💰 Token usage
51
52	Example response format:
53	```
54	Found 3 traces in the last 5 minutes:
55
56	Trace 1: ✅ Success
57	- Agent: memento
58	- Tools: recall_memories, create_entities
59	- Duration: 2.3s
60	- Tokens: 1,245
61
62	Trace 2: ❌ Error
63	- Agent: cypher
64	- Error: "Neo4j connection timeout"
65	- Duration: 15.1s
66	- Failed at: search_nodes tool
67
68	Trace 3: ✅ Success
69	- Agent: memento
70	- Tools: store_memory
71	- Duration: 1.8s
72	- Tokens: 892
73
74	💡 Issue found: Trace 2 failed due to Neo4j timeout. Recommend checking database connection.
75	```
76
77	---
78
79	### Workflow 2: Deep Dive Specific Trace
80
81	When user provides: Trace ID or says "investigate that error"
82
83	Execute:
84	```bash
85	langsmith-fetch trace <trace-id> --format json
86	```
87
88	Analyze JSON and report:
89	1. 🎯 What the agent was trying to do
90	2. 🛠️ Which tools were called (in order)
91	3. ✅ Tool results (success/failure)
92	4. ❌ Error messages (if any)
93	5. 💡 Root cause analysis
94	6. 🔧 Suggested fix
95
96	Example response format:
97	```
98	Deep Dive Analysis - Trace abc123
99
100	Goal: User asked "Find all projects in Neo4j"
101
102	Execution Flow:
103	1. ✅ search_nodes(query: "projects")
104	→ Found 24 nodes
105
106	2. ❌ get_node_details(node_id: "proj_123")
107	→ Error: "Node not found"
108	→ This is the failure point
109
110	3. ⏹️ Execution stopped
111
112	Root Cause:
113	The search_nodes tool returned node IDs that no longer exist in the database,
114	possibly due to recent deletions.
115
116	Suggested Fix:
117	1. Add error handling in get_node_details tool
118	2. Filter deleted nodes in search results
119	3. Update cache invalidation strategy
120
121	Token Usage: 1,842 tokens ($0.0276)
122	Execution Time: 8.7 seconds
123	```
124
125	---
126
127	### Workflow 3: Export Debug Session
128
129	When user says: "Save this session" or "Export traces"
130
131	Execute:
132	```bash
133	# Create session folder with timestamp
134	SESSION_DIR="langsmith-debug/session-$(date +%Y%m%d-%H%M%S)"
135	mkdir -p "$SESSION_DIR"
136
137	# Export traces
138	langsmith-fetch traces "$SESSION_DIR/traces" --last-n-minutes 30 --limit 50 --include-metadata
139
140	# Export threads (conversations)
141	langsmith-fetch threads "$SESSION_DIR/threads" --limit 20
142	```
143
144	Report:
145	```
146	✅ Session exported successfully!
147
148	Location: langsmith-debug/session-20251224-143022/
149	- Traces: 42 files
150	- Threads: 8 files
151
152	You can now:
153	1. Review individual trace files
154	2. Share folder with team
155	3. Analyze with external tools
156	4. Archive for future reference
157
158	Session size: 2.3 MB
159	```
160
161	---
162
163	### Workflow 4: Error Detection
164
165	When user asks: "Show me errors" or "What's failing?"
166
167	Execute:
168	```bash
169	# Fetch recent traces
170	langsmith-fetch traces --last-n-minutes 30 --limit 50 --format json > recent-traces.json
171
172	# Search for errors
173	grep -i "error\\|failed\\|exception" recent-traces.json
174	```
175
176	Analyze and report:
177	1. 📊 Total errors found
178	2. ❌ Error types and frequency
179	3. 🕐 When errors occurred
180	4. 🎯 Which agents/tools failed
181	5. 💡 Common patterns
182
183	Example response format:
184	```
185	Error Analysis - Last 30 Minutes
186
187	Total Traces: 50
188	Failed Traces: 7 (14% failure rate)
189
190	Error Breakdown:
191	1. Neo4j Connection Timeout (4 occurrences)
192	- Agent: cypher
193	- Tool: search_nodes
194	- First occurred: 14:32
195	- Last occurred: 14:45
196	- Pattern: Happens during peak load
197
198	2. Memory Store Failed (2 occurrences)
199	- Agent: memento
200	- Tool: store_memory
201	- Error: "Pinecone rate limit exceeded"
202	- Occurred: 14:38, 14:41
203
204	3. Tool Not Found (1 occurrence)
205	- Agent: sqlcrm
206	- Attempted tool: "export_report" (doesn't exist)
207	-

People who liked this also viewed

6 picks

Type

langsmith-fetch

524stars49forks

View on GitHub

How to install langsmith-fetch?

ecnu-icalk/autoskill/langsmith-fetch

$npx -y skills add ECNU-ICALK/AutoSkill --skill langsmith-fetch

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# LangSmith Fetch - Agent Debugging Skill
2
3	Debug LangChain and LangGraph agents by fetching execution traces directly from LangSmith Studio in your terminal.
4
5	## When to Use This Skill
6
7	Automatically activate when user mentions:
8	- 🐛 "Debug my agent" or "What went wrong?"
9	- 🔍 "Show me recent traces" or "What happened?"
10	- ❌ "Check for errors" or "Why did it fail?"
11	- 💾 "Analyze memory operations" or "Check LTM"
12	- 📊 "Review agent performance" or "Check token usage"
13	- 🔧 "What tools were called?" or "Show execution flow"
14
15	## Prerequisites
16
17	### 1. Install langsmith-fetch
18	```bash
19	pip install langsmith-fetch
20	```
21
22	### 2. Set Environment Variables
23	```bash
24	export LANGSMITH_API_KEY="your_langsmith_api_key"
25	export LANGSMITH_PROJECT="your_project_name"
26	```
27
28	Verify setup:
29	```bash
30	echo $LANGSMITH_API_KEY
31	echo $LANGSMITH_PROJECT
32	```
33
34	## Core Workflows
35
36	### Workflow 1: Quick Debug Recent Activity
37
38	When user asks: "What just happened?" or "Debug my agent"
39
40	Execute:
41	```bash
42	langsmith-fetch traces --last-n-minutes 5 --limit 5 --format pretty
43	```
44
45	Analyze and report:
46	1. ✅ Number of traces found
47	2. ⚠️ Any errors or failures
48	3. 🛠️ Tools that were called
49	4. ⏱️ Execution times
50	5. 💰 Token usage
51
52	Example response format:
53	```
54	Found 3 traces in the last 5 minutes:
55
56	Trace 1: ✅ Success
57	- Agent: memento
58	- Tools: recall_memories, create_entities
59	- Duration: 2.3s
60	- Tokens: 1,245
61
62	Trace 2: ❌ Error
63	- Agent: cypher
64	- Error: "Neo4j connection timeout"
65	- Duration: 15.1s
66	- Failed at: search_nodes tool
67
68	Trace 3: ✅ Success
69	- Agent: memento
70	- Tools: store_memory
71	- Duration: 1.8s
72	- Tokens: 892
73
74	💡 Issue found: Trace 2 failed due to Neo4j timeout. Recommend checking database connection.
75	```
76
77	---
78
79	### Workflow 2: Deep Dive Specific Trace
80
81	When user provides: Trace ID or says "investigate that error"
82
83	Execute:
84	```bash
85	langsmith-fetch trace <trace-id> --format json
86	```
87
88	Analyze JSON and report:
89	1. 🎯 What the agent was trying to do
90	2. 🛠️ Which tools were called (in order)
91	3. ✅ Tool results (success/failure)
92	4. ❌ Error messages (if any)
93	5. 💡 Root cause analysis
94	6. 🔧 Suggested fix
95
96	Example response format:
97	```
98	Deep Dive Analysis - Trace abc123
99
100	Goal: User asked "Find all projects in Neo4j"
101
102	Execution Flow:
103	1. ✅ search_nodes(query: "projects")
104	→ Found 24 nodes
105
106	2. ❌ get_node_details(node_id: "proj_123")
107	→ Error: "Node not found"
108	→ This is the failure point
109
110	3. ⏹️ Execution stopped
111
112	Root Cause:
113	The search_nodes tool returned node IDs that no longer exist in the database,
114	possibly due to recent deletions.
115
116	Suggested Fix:
117	1. Add error handling in get_node_details tool
118	2. Filter deleted nodes in search results
119	3. Update cache invalidation strategy
120
121	Token Usage: 1,842 tokens ($0.0276)
122	Execution Time: 8.7 seconds
123	```
124
125	---
126
127	### Workflow 3: Export Debug Session
128
129	When user says: "Save this session" or "Export traces"
130
131	Execute:
132	```bash
133	# Create session folder with timestamp
134	SESSION_DIR="langsmith-debug/session-$(date +%Y%m%d-%H%M%S)"
135	mkdir -p "$SESSION_DIR"
136
137	# Export traces
138	langsmith-fetch traces "$SESSION_DIR/traces" --last-n-minutes 30 --limit 50 --include-metadata
139
140	# Export threads (conversations)
141	langsmith-fetch threads "$SESSION_DIR/threads" --limit 20
142	```
143
144	Report:
145	```
146	✅ Session exported successfully!
147
148	Location: langsmith-debug/session-20251224-143022/
149	- Traces: 42 files
150	- Threads: 8 files
151
152	You can now:
153	1. Review individual trace files
154	2. Share folder with team
155	3. Analyze with external tools
156	4. Archive for future reference
157
158	Session size: 2.3 MB
159	```
160
161	---
162
163	### Workflow 4: Error Detection
164
165	When user asks: "Show me errors" or "What's failing?"
166
167	Execute:
168	```bash
169	# Fetch recent traces
170	langsmith-fetch traces --last-n-minutes 30 --limit 50 --format json > recent-traces.json
171
172	# Search for errors
173	grep -i "error\\|failed\\|exception" recent-traces.json
174	```
175
176	Analyze and report:
177	1. 📊 Total errors found
178	2. ❌ Error types and frequency
179	3. 🕐 When errors occurred
180	4. 🎯 Which agents/tools failed
181	5. 💡 Common patterns
182
183	Example response format:
184	```
185	Error Analysis - Last 30 Minutes
186
187	Total Traces: 50
188	Failed Traces: 7 (14% failure rate)
189
190	Error Breakdown:
191	1. Neo4j Connection Timeout (4 occurrences)
192	- Agent: cypher
193	- Tool: search_nodes
194	- First occurred: 14:32
195	- Last occurred: 14:45
196	- Pattern: Happens during peak load
197
198	2. Memory Store Failed (2 occurrences)
199	- Agent: memento
200	- Tool: store_memory
201	- Error: "Pinecone rate limit exceeded"
202	- Occurred: 14:38, 14:41
203
204	3. Tool Not Found (1 occurrence)
205	- Agent: sqlcrm
206	- Attempted tool: "export_report" (doesn't exist)
207	-

People who liked this also viewed

6 picks

Type