braintrust-tracing

Name: braintrust-tracing
Author: vibeeval

519stars44forks

How to install braintrust-tracing?

vibeeval/vibecosystem/braintrust-tracing

$npx -y skills add vibeeval/vibecosystem --skill braintrust-tracing

Installs into the current project.

Summary

--- name: braintrust-tracing description: Braintrust tracing for Claude Code - hook architecture, sub-agent correlation, debugging user-invocable: false ---

Files · 1

View on GitHub

SKILL.md

1	# Braintrust Tracing for Claude Code
2
3	Comprehensive guide to tracing Claude Code sessions in Braintrust, including sub-agent correlation.
4
5	## Architecture Overview
6
7	```
8	PARENT SESSION
9	+---------------------+
10	\| SessionStart \|
11	\| (creates root) \|
12	+----------+----------+
13	\|
14	+----------v----------+
15	\| UserPromptSubmit \|
16	\| (creates Turn) \|
17	+----------+----------+
18	\|
19	+--------------------+--------------------+
20	\| \| \|
21	+---------v--------+ +--------v--------+ +--------v--------+
22	\| PostToolUse \| \| PostToolUse \| \| PreToolUse \|
23	\| (Read span) \| \| (Edit span) \| \| (Task - inject) \|
24	+------------------+ +-----------------+ +--------+--------+
25	\|
26	+----------v----------+
27	\| SUB-AGENT \|
28	\| SessionStart \|
29	\| (NEW root_span_id)\|
30	+----------+----------+
31	\|
32	+----------v----------+
33	\| SubagentStop \|
34	\| (has session_id) \|
35	+---------------------+
36	```
37
38	## Hook Event Flow
39
40	\| Hook \| Trigger \| Creates \| Key Fields \|
41	\|------\|---------\|---------\|------------\|
42	\| SessionStart \| Session begins \| Root span \| `session_id`, `root_span_id` \|
43	\| UserPromptSubmit \| User sends prompt \| Turn span \| `prompt`, `turn_number` \|
44	\| PreToolUse \| Before tool runs \| (modifies Task prompts) \| `tool_input.prompt` \|
45	\| PostToolUse \| After tool runs \| Tool span \| `tool_name`, `input`, `output` \|
46	\| Stop \| Turn completes \| LLM spans \| `model`, `tokens`, `tool_calls` \|
47	\| SubagentStop \| Sub-agent finishes \| (no span) \| `session_id` of sub-agent \|
48	\| SessionEnd \| Session ends \| (finalizes root) \| `turn_count`, `tool_count` \|
49
50	## Trace Hierarchy
51
52	```
53	Session (task span) - root_span_id = session_id
54	\|
55	+-- Turn 1 (task span)
56	\| \|
57	\| +-- claude-sonnet (llm span) - model call with tool_use
58	\| +-- Read (tool span)
59	\| +-- Edit (tool span)
60	\| +-- claude-sonnet (llm span) - response after tools
61	\|
62	+-- Turn 2 (task span)
63	\| \|
64	\| +-- claude-sonnet (llm span)
65	\| +-- Task (tool span) -----> [Sub-agent session - SEPARATE trace]
66	\| +-- claude-sonnet (llm span)
67	\|
68	+-- Turn 3 ...
69	```
70
71	## Sub-Agent Tracing: What Works and What Doesn't
72
73	### What Doesn't Work
74
75	SessionStart doesn't receive the Task prompt.
76
77	We tried injecting trace context into Task prompts via PreToolUse:
78
79	```bash
80	# PreToolUse hook injects:
81	[BRAINTRUST_TRACE_CONTEXT]
82	{"root_span_id": "abc", "parent_span_id": "xyz", "project_id": "123"}
83	[/BRAINTRUST_TRACE_CONTEXT]
84	```
85
86	But SessionStart only receives session metadata, not the modified prompt. The injected context is lost.
87
88	### What DOES Work
89
90	Task spans in parent session contain everything:
91	- `agentId` - identifier for the sub-agent run
92	- `totalTokens`, `totalToolUseCount` - metrics
93	- `content` - full agent response/summary
94	- `tool_input.prompt` - original task prompt
95	- `tool_input.subagent_type` - agent type (e.g., "oracle")
96
97	SubagentStop hook receives the sub-agent's `session_id`:
98	- This equals the sub-agent's orphaned trace `root_span_id`
99	- Allows correlation between parent Task span and child trace
100
101	### The Correlation Pattern
102
103	Current state: Sub-agents create orphaned traces (new `root_span_id`).
104
105	Correlation method:
106	1. Query parent session's Task spans for agent metadata
107	2. Match `agentId` or timing with orphaned traces
108	3. Sub-agent's `session_id` = its trace's `root_span_id`
109
110	Future solution (not yet implemented):
111	```
112	SubagentStop fires -> writes session_id to temp file
113	PostToolUse (Task) -> reads temp file -> adds child_session_id to Task span metadata
114	```
115
116	This would link: `Task.agentId` + `Task.child_session_id` -> orphaned trace `root_span_id`
117
118	## State Management
119
120	### Per-Session State Files
121
122	```
123	~/.claude/state/braintrust_sessions/
124	{session_id}.json # Per-session state
125	```
126
127	Each session file contains:
128	```json
129	{
130	"root_span_id": "abc-123",
131	"project_id": "proj-456",
132	"turn_count": 5,
133	"tool_count": 23,
134	"current_turn_span_id": "turn-789",
135	"current_turn_start": 17034567

People who liked this also viewed

6 picks

Type

tddSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026501k181k
test-driven-developmentUse when implementing any feature or bugfix, before writing implementation codeSkillsJul 2026175k259k
qaSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026160k181k
verification-before-completionUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidenceSkillsJul 2026153k259k
webapp-testingToolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser lSkillsJul 2026120k163k
playwright-best-practicesUse when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, teSkillsJul 202664k338

braintrust-tracing

519stars44forks

View on GitHub

How to install braintrust-tracing?

vibeeval/vibecosystem/braintrust-tracing

$npx -y skills add vibeeval/vibecosystem --skill braintrust-tracing

Installs into the current project.

Summary

--- name: braintrust-tracing description: Braintrust tracing for Claude Code - hook architecture, sub-agent correlation, debugging user-invocable: false ---

Files · 1

View on GitHub

SKILL.md

1	# Braintrust Tracing for Claude Code
2
3	Comprehensive guide to tracing Claude Code sessions in Braintrust, including sub-agent correlation.
4
5	## Architecture Overview
6
7	```
8	PARENT SESSION
9	+---------------------+
10	\| SessionStart \|
11	\| (creates root) \|
12	+----------+----------+
13	\|
14	+----------v----------+
15	\| UserPromptSubmit \|
16	\| (creates Turn) \|
17	+----------+----------+
18	\|
19	+--------------------+--------------------+
20	\| \| \|
21	+---------v--------+ +--------v--------+ +--------v--------+
22	\| PostToolUse \| \| PostToolUse \| \| PreToolUse \|
23	\| (Read span) \| \| (Edit span) \| \| (Task - inject) \|
24	+------------------+ +-----------------+ +--------+--------+
25	\|
26	+----------v----------+
27	\| SUB-AGENT \|
28	\| SessionStart \|
29	\| (NEW root_span_id)\|
30	+----------+----------+
31	\|
32	+----------v----------+
33	\| SubagentStop \|
34	\| (has session_id) \|
35	+---------------------+
36	```
37
38	## Hook Event Flow
39
40	\| Hook \| Trigger \| Creates \| Key Fields \|
41	\|------\|---------\|---------\|------------\|
42	\| SessionStart \| Session begins \| Root span \| `session_id`, `root_span_id` \|
43	\| UserPromptSubmit \| User sends prompt \| Turn span \| `prompt`, `turn_number` \|
44	\| PreToolUse \| Before tool runs \| (modifies Task prompts) \| `tool_input.prompt` \|
45	\| PostToolUse \| After tool runs \| Tool span \| `tool_name`, `input`, `output` \|
46	\| Stop \| Turn completes \| LLM spans \| `model`, `tokens`, `tool_calls` \|
47	\| SubagentStop \| Sub-agent finishes \| (no span) \| `session_id` of sub-agent \|
48	\| SessionEnd \| Session ends \| (finalizes root) \| `turn_count`, `tool_count` \|
49
50	## Trace Hierarchy
51
52	```
53	Session (task span) - root_span_id = session_id
54	\|
55	+-- Turn 1 (task span)
56	\| \|
57	\| +-- claude-sonnet (llm span) - model call with tool_use
58	\| +-- Read (tool span)
59	\| +-- Edit (tool span)
60	\| +-- claude-sonnet (llm span) - response after tools
61	\|
62	+-- Turn 2 (task span)
63	\| \|
64	\| +-- claude-sonnet (llm span)
65	\| +-- Task (tool span) -----> [Sub-agent session - SEPARATE trace]
66	\| +-- claude-sonnet (llm span)
67	\|
68	+-- Turn 3 ...
69	```
70
71	## Sub-Agent Tracing: What Works and What Doesn't
72
73	### What Doesn't Work
74
75	SessionStart doesn't receive the Task prompt.
76
77	We tried injecting trace context into Task prompts via PreToolUse:
78
79	```bash
80	# PreToolUse hook injects:
81	[BRAINTRUST_TRACE_CONTEXT]
82	{"root_span_id": "abc", "parent_span_id": "xyz", "project_id": "123"}
83	[/BRAINTRUST_TRACE_CONTEXT]
84	```
85
86	But SessionStart only receives session metadata, not the modified prompt. The injected context is lost.
87
88	### What DOES Work
89
90	Task spans in parent session contain everything:
91	- `agentId` - identifier for the sub-agent run
92	- `totalTokens`, `totalToolUseCount` - metrics
93	- `content` - full agent response/summary
94	- `tool_input.prompt` - original task prompt
95	- `tool_input.subagent_type` - agent type (e.g., "oracle")
96
97	SubagentStop hook receives the sub-agent's `session_id`:
98	- This equals the sub-agent's orphaned trace `root_span_id`
99	- Allows correlation between parent Task span and child trace
100
101	### The Correlation Pattern
102
103	Current state: Sub-agents create orphaned traces (new `root_span_id`).
104
105	Correlation method:
106	1. Query parent session's Task spans for agent metadata
107	2. Match `agentId` or timing with orphaned traces
108	3. Sub-agent's `session_id` = its trace's `root_span_id`
109
110	Future solution (not yet implemented):
111	```
112	SubagentStop fires -> writes session_id to temp file
113	PostToolUse (Task) -> reads temp file -> adds child_session_id to Task span metadata
114	```
115
116	This would link: `Task.agentId` + `Task.child_session_id` -> orphaned trace `root_span_id`
117
118	## State Management
119
120	### Per-Session State Files
121
122	```
123	~/.claude/state/braintrust_sessions/
124	{session_id}.json # Per-session state
125	```
126
127	Each session file contains:
128	```json
129	{
130	"root_span_id": "abc-123",
131	"project_id": "proj-456",
132	"turn_count": 5,
133	"tool_count": 23,
134	"current_turn_span_id": "turn-789",
135	"current_turn_start": 17034567

People who liked this also viewed

6 picks

Type