streaming-llm-responses

Name: streaming-llm-responses
Author: mjunaidca

20installs11forks

How to install streaming-llm-responses?

mjunaidca/mjs-agent-skills/streaming-llm-responses

$npx -y skills add mjunaidca/mjs-agent-skills --skill streaming-llm-responses

Installs into the current project.

Summary

Implement real-time streaming UI patterns for AI chat applications. Use when adding response lifecycle handlers, progress indicators, client effects, or thread state synchronization. Covers onResponseStart/End, onEffect, ProgressUpdateEvent, and client tools. NOT when building ba

Files · 1

View on GitHub

SKILL.md

1	# Streaming LLM Responses
2
3	Build responsive, real-time chat interfaces with streaming feedback.
4
5	## Quick Start
6
7	```typescript
8	import { useChatKit } from "@openai/chatkit-react";
9
10	const chatkit = useChatKit({
11	api: { url: API_URL, domainKey: DOMAIN_KEY },
12
13	onResponseStart: () => setIsResponding(true),
14	onResponseEnd: () => setIsResponding(false),
15
16	onEffect: ({ name, data }) => {
17	if (name === "update_status") updateUI(data);
18	},
19	});
20	```
21
22	---
23
24	## Response Lifecycle
25
26	```
27	User sends message
28	↓
29	onResponseStart() fires
30	↓
31	[Streaming: tokens arrive, ProgressUpdateEvents shown]
32	↓
33	onResponseEnd() fires
34	↓
35	UI unlocks, ready for next interaction
36	```
37
38	---
39
40	## Core Patterns
41
42	### 1. Response Lifecycle Handlers
43
44	Lock UI during AI response to prevent race conditions:
45
46	```typescript
47	function ChatWithLifecycle() {
48	const [isResponding, setIsResponding] = useState(false);
49	const lockInteraction = useAppStore((s) => s.lockInteraction);
50	const unlockInteraction = useAppStore((s) => s.unlockInteraction);
51
52	const chatkit = useChatKit({
53	api: { url: API_URL, domainKey: DOMAIN_KEY },
54
55	onResponseStart: () => {
56	setIsResponding(true);
57	lockInteraction(); // Disable map/canvas/form interactions
58	},
59
60	onResponseEnd: () => {
61	setIsResponding(false);
62	unlockInteraction();
63	},
64
65	onError: ({ error }) => {
66	console.error("ChatKit error:", error);
67	setIsResponding(false);
68	unlockInteraction();
69	},
70	});
71
72	return (
73	<div>
74	{isResponding && <LoadingOverlay />}
75	<ChatKit control={chatkit.control} />
76	</div>
77	);
78	}
79	```
80
81	### 2. Client Effects (Fire-and-Forget)
82
83	Server sends effects to update client UI without expecting a response:
84
85	Backend - Streaming Effects:
86
87	```python
88	from chatkit.types import ClientEffectEvent
89
90	async def respond(self, thread, item, context):
91	# ... agent processing ...
92
93	# Fire client effect to update UI
94	yield ClientEffectEvent(
95	name="update_status",
96	data={
97	"state": {"energy": 80, "happiness": 90},
98	"flash": "Status updated!"
99	}
100	)
101
102	# Another effect
103	yield ClientEffectEvent(
104	name="show_notification",
105	data={"message": "Task completed!"}
106	)
107	```
108
109	Frontend - Handling Effects:
110
111	```typescript
112	const chatkit = useChatKit({
113	api: { url: API_URL, domainKey: DOMAIN_KEY },
114
115	onEffect: ({ name, data }) => {
116	switch (name) {
117	case "update_status":
118	applyStatusUpdate(data.state);
119	if (data.flash) setFlashMessage(data.flash);
120	break;
121
122	case "add_marker":
123	addMapMarker(data);
124	break;
125
126	case "select_mode":
127	setSelectionMode(data.mode);
128	break;
129	}
130	},
131	});
132	```
133
134	### 3. Progress Updates
135
136	Show "Searching...", "Loading...", "Analyzing..." during long operations:
137
138	```python
139	from chatkit.types import ProgressUpdateEvent
140
141	@function_tool
142	async def search_articles(ctx: AgentContext, query: str) -> str:
143	"""Search for articles matching the query."""
144
145	yield ProgressUpdateEvent(message="Searching articles...")
146
147	results = await article_store.search(query)
148
149	yield ProgressUpdateEvent(message=f"Found {len(results)} articles...")
150
151	for i, article in enumerate(results):
152	if i % 5 == 0:
153	yield ProgressUpdateEvent(
154	message=f"Processing article {i+1}/{len(results)}..."
155	)
156
157	return format_results(results)
158	```
159
160	### 4. Thread Lifecycle Events
161
162	Track thread changes for persistence and UI updates:
163
164	```typescript
165	const chatkit = useChatKit({
166	api: { url: API_URL, domainKey: DOMAIN_KEY },
167
168	onThreadChange: ({ threadId }) => {
169	setThreadId(threadId);
170	if (threadId) localStorage.setItem("lastThreadId", threadId);
171	clearSelections();
172	},
173
174	onThreadLoadStart: ({ threadId }) => {
175	setIsLoadingThread(true);
176	},
177
178	onThreadLoadEnd: ({ threadId }) => {
179	setIsLoadingThread(false);
180	},
181	});
182	```
183
184	### 5. Client Tools (State Query)
185
186	AI needs to read client-side state to make decisions:
187
188	Backend - Defining Client Tool:
189
190	```python
191	@function_tool(name_override="get_selected_items")
192	async def get_selected_items(ctx: AgentContext) -> dict:
193	"""Get the items currently selected on the canvas.
194
195	This is a CLIENT TOOL - executed in browser, result comes back.
196	"""
197	yield ProgressUpdateEvent(message="Reading selection...")
198	pass # Actual execution happens on client
199	```
200
201	Frontend - Handling Client Tools:
202
203	```typescript
204	const chatkit = useChatKit({
205	api: { url: API_URL, domainKey: DOMAIN_KEY },
206
207	onClientTool: ({ name, params }) => {
208	switch (name) {

People who liked this also viewed

6 picks

Type

streaming-llm-responses

20installs11forks

View on GitHub

How to install streaming-llm-responses?

mjunaidca/mjs-agent-skills/streaming-llm-responses

$npx -y skills add mjunaidca/mjs-agent-skills --skill streaming-llm-responses

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Streaming LLM Responses
2
3	Build responsive, real-time chat interfaces with streaming feedback.
4
5	## Quick Start
6
7	```typescript
8	import { useChatKit } from "@openai/chatkit-react";
9
10	const chatkit = useChatKit({
11	api: { url: API_URL, domainKey: DOMAIN_KEY },
12
13	onResponseStart: () => setIsResponding(true),
14	onResponseEnd: () => setIsResponding(false),
15
16	onEffect: ({ name, data }) => {
17	if (name === "update_status") updateUI(data);
18	},
19	});
20	```
21
22	---
23
24	## Response Lifecycle
25
26	```
27	User sends message
28	↓
29	onResponseStart() fires
30	↓
31	[Streaming: tokens arrive, ProgressUpdateEvents shown]
32	↓
33	onResponseEnd() fires
34	↓
35	UI unlocks, ready for next interaction
36	```
37
38	---
39
40	## Core Patterns
41
42	### 1. Response Lifecycle Handlers
43
44	Lock UI during AI response to prevent race conditions:
45
46	```typescript
47	function ChatWithLifecycle() {
48	const [isResponding, setIsResponding] = useState(false);
49	const lockInteraction = useAppStore((s) => s.lockInteraction);
50	const unlockInteraction = useAppStore((s) => s.unlockInteraction);
51
52	const chatkit = useChatKit({
53	api: { url: API_URL, domainKey: DOMAIN_KEY },
54
55	onResponseStart: () => {
56	setIsResponding(true);
57	lockInteraction(); // Disable map/canvas/form interactions
58	},
59
60	onResponseEnd: () => {
61	setIsResponding(false);
62	unlockInteraction();
63	},
64
65	onError: ({ error }) => {
66	console.error("ChatKit error:", error);
67	setIsResponding(false);
68	unlockInteraction();
69	},
70	});
71
72	return (
73	<div>
74	{isResponding && <LoadingOverlay />}
75	<ChatKit control={chatkit.control} />
76	</div>
77	);
78	}
79	```
80
81	### 2. Client Effects (Fire-and-Forget)
82
83	Server sends effects to update client UI without expecting a response:
84
85	Backend - Streaming Effects:
86
87	```python
88	from chatkit.types import ClientEffectEvent
89
90	async def respond(self, thread, item, context):
91	# ... agent processing ...
92
93	# Fire client effect to update UI
94	yield ClientEffectEvent(
95	name="update_status",
96	data={
97	"state": {"energy": 80, "happiness": 90},
98	"flash": "Status updated!"
99	}
100	)
101
102	# Another effect
103	yield ClientEffectEvent(
104	name="show_notification",
105	data={"message": "Task completed!"}
106	)
107	```
108
109	Frontend - Handling Effects:
110
111	```typescript
112	const chatkit = useChatKit({
113	api: { url: API_URL, domainKey: DOMAIN_KEY },
114
115	onEffect: ({ name, data }) => {
116	switch (name) {
117	case "update_status":
118	applyStatusUpdate(data.state);
119	if (data.flash) setFlashMessage(data.flash);
120	break;
121
122	case "add_marker":
123	addMapMarker(data);
124	break;
125
126	case "select_mode":
127	setSelectionMode(data.mode);
128	break;
129	}
130	},
131	});
132	```
133
134	### 3. Progress Updates
135
136	Show "Searching...", "Loading...", "Analyzing..." during long operations:
137
138	```python
139	from chatkit.types import ProgressUpdateEvent
140
141	@function_tool
142	async def search_articles(ctx: AgentContext, query: str) -> str:
143	"""Search for articles matching the query."""
144
145	yield ProgressUpdateEvent(message="Searching articles...")
146
147	results = await article_store.search(query)
148
149	yield ProgressUpdateEvent(message=f"Found {len(results)} articles...")
150
151	for i, article in enumerate(results):
152	if i % 5 == 0:
153	yield ProgressUpdateEvent(
154	message=f"Processing article {i+1}/{len(results)}..."
155	)
156
157	return format_results(results)
158	```
159
160	### 4. Thread Lifecycle Events
161
162	Track thread changes for persistence and UI updates:
163
164	```typescript
165	const chatkit = useChatKit({
166	api: { url: API_URL, domainKey: DOMAIN_KEY },
167
168	onThreadChange: ({ threadId }) => {
169	setThreadId(threadId);
170	if (threadId) localStorage.setItem("lastThreadId", threadId);
171	clearSelections();
172	},
173
174	onThreadLoadStart: ({ threadId }) => {
175	setIsLoadingThread(true);
176	},
177
178	onThreadLoadEnd: ({ threadId }) => {
179	setIsLoadingThread(false);
180	},
181	});
182	```
183
184	### 5. Client Tools (State Query)
185
186	AI needs to read client-side state to make decisions:
187
188	Backend - Defining Client Tool:
189
190	```python
191	@function_tool(name_override="get_selected_items")
192	async def get_selected_items(ctx: AgentContext) -> dict:
193	"""Get the items currently selected on the canvas.
194
195	This is a CLIENT TOOL - executed in browser, result comes back.
196	"""
197	yield ProgressUpdateEvent(message="Reading selection...")
198	pass # Actual execution happens on client
199	```
200
201	Frontend - Handling Client Tools:
202
203	```typescript
204	const chatkit = useChatKit({
205	api: { url: API_URL, domainKey: DOMAIN_KEY },
206
207	onClientTool: ({ name, params }) => {
208	switch (name) {

People who liked this also viewed

6 picks

Type