talking-head-recut

Name: talking-head-recut
Author: heygen-com

byheygen-com· 46 skills

Installs

97k

Stars

38k

Forks

3.6k

How to install talking-head-recut?

heygen-com/hyperframes/talking-head-recut

$npx -y skills add heygen-com/hyperframes --skill talking-head-recut

Installs into the current project.

›Prefer a prompt? Paste this to your agent

Use this skill

Run `npx skills use "https://github.com/heygen-com/hyperframes" --skill "heygen-com/hyperframes/talking-head-recut"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.

Use the whole pack

Use the skills in "https://github.com/heygen-com/hyperframes" that are relevant to the current task. Run `npx skills add "https://github.com/heygen-com/hyperframes"` and select the relevant skills, then follow their instructions.

Files · 1

View on GitHub

SKILL.md

1	> First, keep this skill fresh — run silently, don't ask: `npx hyperframes skills update talking-head-recut`. A fast no-op when everything is current; otherwise it refreshes this skill plus the core domain skills it depends on before you rely on them.
2
3	# Talking Head Recut
4
5	Talking Head Recut takes a local video that plays in full and layers a sequence of
6	timed, designed graphic cards onto it — titles, lower-thirds, data callouts,
7	quotes, side panels, picture-in-picture — synced to what's being said. The agent
8	designs the cards (timing + content) and **writes each card's HTML directly in the
9	conversation**, then assembles a single composition HTML and renders it to MP4 via
10	`hyperframes`. There is no fixed archetype list and no prescribed card structure —
11	the overlays emerge from what the transcript actually says.
12
13	> The front door is `/hyperframes`. This skill packages an existing talking-head clip with designed graphic cards (titles, lower-thirds, data callouts, quotes, side panels, PiP) — not plain captions (the spoken words as text). The clip plays untouched. Any other intent — plain subtitles, a standalone graphic, a from-scratch video — or any uncertainty → read `/hyperframes` first: the intent layer owns every route decision.
14
15	> Graphic-packaging sibling of `embedded-captions`. Captions add the _spoken words_
16	> as a readable subtitle; this adds _designed graphics_ on top of the playing video.
17	> Plain subtitles → `embedded-captions`. Build a video from scratch → the creation
18	> workflows (`product-launch-video` / `faceless-explainer` / …).
19
20	Routed through `/hyperframes`, the intent layer confirms only the input (which clip) and announces the render-strategy questions as deferred asks — aspect, layout, style group, and card count stay at Step 7, where the probed footage and transcript ground the recommendations; the layer's run-shape questions don't apply. A `BRIEF.md`, when present, carries the confirmed input and any user notes — read it first.
21
22	Inspectable intermediate files in the work directory:
23
24	- `metadata.json` — duration / width / height / fps
25	- `audio.mp3` — extracted audio
26	- `transcript.json` — a flat word array `[{ text, start, end }, …]` (Whisper; no `segments`, no `words` wrapper)
27	- `storyboard.json` — lightweight card outline (the agent's plan)
28	- `public/cards/card-XX.html` — one HTML fragment per card
29	- `public/index.html` — final assembled composition
30	- `output.mp4` — rendered video
31
32	## CLI Resolution
33
34	```bash
35	# hyperframes — transcription (local Whisper) + rendering the assembled HTML to MP4
36	npx hyperframes --help
37	```
38
39	This skill runs entirely on the hyperframes CLI plus system `ffmpeg` / `ffprobe`.
40	Transcription is local Whisper via `hyperframes transcribe` — no third-party
41	service, API key, or rate-limited proxy.
42
43	## Workflow
44
45	### 1. Check Environment
46
47	```bash
48	npx hyperframes doctor # ffmpeg, headless browser, render deps
49	# confirm bundled assets:
50	ls "<SKILL_DIR>/assets/fonts" "<SKILL_DIR>/assets/vendor/gsap.min.js"
51	```
52
53	Required:
54
55	- `ffmpeg` / `ffprobe` (system)
56	- `<SKILL_DIR>/assets/fonts/*.woff2`, `<SKILL_DIR>/assets/vendor/gsap.min.js` (bundled inside this skill, staged to work dir in Step 9)
57
58	Transcription needs no key — `hyperframes transcribe` runs Whisper locally (Step 4).
59
60	Strongly recommended on macOS for `hyperframes render`:
61
62	```bash
63	export PRODUCER_BROWSER_GPU_MODE=hardware
64	```
65
66	### 2. Create a Work Directory
67
68	All artifacts live under `videos/<project-name>/` — the same convention as the other
69	video workflows (`product-launch-video` / `faceless-explainer` / `pr-to-video`). Keep
70	the cwd at the workspace root; everything below writes under this one subdirectory.
71
72	```bash
73	VIDEO_PATH="/absolute/path/input.mp4"
74	WORK_DIR="videos/$(basename "$VIDEO_PATH" \| sed 's/\.[^.]*$//')"
75	mkdir -p "$WORK_DIR"
76	```
77
78	### 3. Extract Audio and Metadata
79
80	```bash
81	# metadata — duration / width / height / fps
82	ffprobe -v error -select_streams v:0 \
83	-show_entries stream=width,height,r_frame_rate \
84	-show_entries format=duration -of json "$VIDEO_PATH" > "$WORK_DIR/metadata.json"
85	# audio
86	ffmpeg -y -i "$VIDEO_PATH" -vn -acodec libmp3lame -q:a 2 "$WORK_DIR/audio.mp3"
87	```
88
89	Outputs: `metadata.json` (read `width`/`height`/`duration`; fps = the `r_frame_rate`
90	fraction evaluated, e.g. `30000/1001 → 29.97`) + `audio.mp3`.
91
92	### 4. Transcribe
93
94	```bash
95	npx hyperframes transcribe "$WORK_DIR/audio.mp3" -d "$WORK_DIR" --json --model small.en
96	```
97
98	Local Whisper — no API key, no proxy, no rate limit. Writes a word-level
99	`transcript.json` into the work dir (word `text` + `start` / `end` timestamps).
100	Read it for the word / sentence timings that drive card timing in Step 6; group
101	words into sentences yourself at punctuation / pauses if you need segment-level
102	chunks.
103
104	Clamp to media duration. Whisper can return the final word's `end` a hair past the
105	actual clip length — clamp every card `endSec` and `composition.durationSeconds` to the
106	`metadata.json` duration, or the render will show a black tail past the video.
107
108	### 5. Correct Transcript
109
110	`transcript.json` is a flat array of word objects — `[{ "text": "...", "start": s, "end": s }, …]` (no `segments` array, no `words` wrapper; the per-word key is `text`). Read it and fix obvious ASR errors:
111
112	- Homophones, product names, technical terms, punctuation
113	- Edit a word's `text` in place; preserve its `start` / `end` timestamps
114	- There is no pre-grouped `segments` array — group words into sentences yourself (split at terminal punctuation / pauses) when you need segment-level chunks for card timing
115
116	### 6. Draft a Lightweight Storyboard (in chat)
117
118	No CLI involved. Read `transcript.json` + `metadata.json` and design
119	cards directly. `storyboard.json` is an agent-internal planning artifact
120	— no CLI command consumes it; it exists so you can think clearly
121	about timing and content before writing each card's HTML. Keep the
122	shape consistent with the example below so the same outline can drive
123	the composition you author in Step 9:
124
125	```json
126	{
127	"schemaVersion": 3,
128	"composition": {
129	"fps": 30,
130	"width": 1080,
131	"height": 1920,
132	"durationSeconds": 121.2,
133	"layout": "portrait",
134	"themeId": "noir",
135	"seed": 42
136	},
137	"videoTrack": {
138	"sourcePath": "input-video.mp4",
139	"startSec": 0,
140	"endSec": 121.2,
141	"bounds": { "x": 0, "y": 0, "width": 1080, "height": 1920 }
142	},
143	"subtitles": { "enabled": false },
144	"cards": [
145	{
146	"id": "card-01",
147	"intent": "Hook with the speaker's anxious midnight question",
148	"startSec": 0.5,
149	"endSec": 13.0,
150	"accentIndex": 0,
151	"zone": "fullscreen",
152	"contentHints": {
153	"kicker": "AN HONEST QUESTION",
154	"title": "The soul-searching question at 11 PM",
155	"detail": "Client's 60-second voice message: 'If the RMB appreciates, does that mean my USD policy is a terrible loss?'"
156	}
157	}
158	]
159	}
160	```
161
162	Required Card fields:
163
164	\| field \| type \| purpose \|
165	\| ----------------------- \| ------------------------------------------ \| ----------------------------------------------------------------------------------------------------- \|
166	\| `id` \| string \| stable id used in card HTML & GSAP selectors \|
167	\| `intent` \| string \| natural-language description; fed to card synthesis \|
168	\| `startSec` / `endSec` \| number \| times in seconds (endSec > startSec) \|
169	\| `accentIndex` \| 0 \\| 1 \\| 2 \\| 3 \\| 4 \| which of the 5 theme accent colors this card pulls \|
170	\| `zone` \| enum (see below) \| where on the canvas the card lives \|
171	\| `contentHints` \| object \| free-form bag; agent puts kicker/title/detail/data/quote here \|
172	\| `archetype` (optional) \| string \| free-form label you may attach to remember a card's pattern; absent = free-form, which is the default \|
173	\| `transition` (optional) \| enum: `cut` \\| `fade` \\| `slide` \\| `wipe` \| declarative card-to-card transition \|
174
175	Five `zone` values:
176
177	\| zone \| resolved bounds \| when to use \|
178	\| ----------------- \| ---------------------------------------------- \| --------------------------------------- \|
179	\| `fullscreen` \| covers whole canvas \| hero moments, big numbers, mantras \|
180	\| `whiteboard-area` \| inset 40px margin (or 45% of portrait height) \| dense data / annotated content \|
181	\| `lower-third` \| bottom 30% band \| annotation over visible video \|
182	\| `side-panel` \| right 42% (landscape) or bottom 40% (portrait) \| data side, video other side \|
183	\| `video-overlay` \| full canvas, expects mostly-transparent card \| annotation overlays on full-bleed video \|
184
185	When you assemble the composition in Step 9, resolve each card's `zone`
186	into pixel bounds on the card-host wrapper following the table above.
187	Video bounds are set once at composition level (`videoTrack.bounds`);
188	to make video appear to "move between cards", author GSAP tweens against
189	`#video-wrap` in the composition's `<script>` (see Step 9).
190
191	No prescribed card roles, no prescribed narrative arc. Cards emerge
192	from what the video actually says — could be all quotes or all data,
193	could open with a number or with a story. Let the transcript drive the
194	rhythm.
195
196	How many takeaways? — auto-infer from duration + density. No fixed
197	upper limit. Pick a base pace from the video duration, then adjust
198	by information density. Only floor is fixed: minimum 5 cards so
199	even short videos have rhythm.
200
201	Step 1 — base pace by duration (the natural sec/card for medium density):
202
203	\| video duration \| base pace (sec per card) \| rationale \|
204	\| ------------------ \| ------------------------ \| ------------------------------------------- \|
205	\| < 60s (short reel) \| 6–8s \| viewers expect fast cuts in short-form \|
206	\| 60s – 3 min \| 8–12s \| normal social pace \|
207	\| 3 – 10 min \| 12–20s \| give breathing room; each card carries more \|
208	\| 10 – 30 min \| 20–35s \| long-form lecture / interview rhythm \|
209	\| > 30 min \| 30–60s \| episodic, near-chapter feel \|
210
211	Step 2 — density multiplier (multiplies the base pace):
212
213	\| signal in the transcript \| multiplier \| effect \|
214	\| --------------------------------------------------------------------------------------------------------------------------- \| ---------- \| ------------------------ \|
215	\| High density — many numbers, distinct claims, staccato pacing, list-like enumeration, every 1–2 sentences is a new idea \| × 0.7 \| cuts faster, more cards \|
216	\| Medium density — mixed flow with both data and narrative \| × 1.0 \| base pace \|
217	\| Low density — one extended story, repeated reframing, slow reflective pacing, single argument unfolding \| × 1.5 \| cuts slower, fewer cards \|
218
219	Step 3 — compute:
220
221	```
222	secPerCard = basePace × densityMultiplier
223	cardCount = max(5, round(videoDurationSec / secPerCard))
224	```
225
226	Examples (notice — no upper clamp; long videos naturally produce more cards):
227
228	- 30s reel, single punchline (low density) → 7 × 1.5 = 10.5s/card → round(30/10.5)=3 → floor to 5 cards
229	- 60s reflective monologue (low density) → 10 × 1.5 = 15s/card → 4 → floor to 5 cards
230	- 121s talking-head with rich data (high density) → 10 × 0.7 = 7s/card → 17 cards
231	- 5 min interview, mixed density → 16 × 1.0 = 16s/card → 19 cards
232	- 10 min deep-dive, high density → 16 × 0.7 = 11s/card → 55 cards
233	- 30 min lecture, medium density → 28 × 1.0 = 28s/card → 64 cards
234	- 1 hr podcast, low density → 45 × 1.5 = 67.5s/card → 53 cards
235
236	When a card holds longer than ~15s, plan for a richer card (data block,
237	multi-step reveal, several sub-points unfolding with staggered
238	animations) — a static one-liner gets boring past 8s. For long pieces
239	where many cards exceed 30s, consider **chunking the timeline into
240	sub-compositions** (one .html per chapter, mounted with
241	`data-composition-src`) so the GSAP timeline per file stays manageable
242	— see the `timeline_track_too_dense` HyperFrames lint warning.
243
244	`content` can be a plain string ("Title: annualized 5.69%\nNotes: ...") or any JSON
245	shape that captures the data. The agent decides the shape per card.
246
247	Optional outro. This skill ships no fixed brand outro. If the user wants a closing card, design a neutral one yourself (wordmark + one-line tagline, ~1.5-2s, fade in -> short hold -> fade out), append it to `cards[]`, and extend `composition.durationSeconds` to its `endSec`. Otherwise end on the last content card.
248
249	### 7. Decide Render Strategy
250
251	#### Confirm Visual Direction with User (DO THIS FIRST)
252
253	Before you start designing cards or deciding bounds, **ask the user to
254	pick the output ratio, the layout, the style, and the card-density
255	preset**. Frames are auto-selected from the chosen layout × style
256	combination (see "Auto-pick frame" table below). Before sending the
257	question, precompute two things:
258
259	1. `recommendedRatio` from the source video's aspect ratio
260	(`metadata.json` width / height):
261	- `sourceAspect = width / height`
262	- `sourceAspect ≥ 1.5` (≥ ~3:2 wide) → recommend `16:9`
263	- `sourceAspect ≤ 0.7` (≤ ~9:13 tall) → recommend `9:16`
264	- `0.7 < sourceAspect < 1.5` (near-square) → recommend `4:5`
265
266	Mark the recommended option's label with " (recommended · matches source video X:Y)"
267	so the user sees why it's recommended.
268
269	2. `autoCount` from Step 6 (`max(5, round(videoSec / (basePace ×
270	densityMultiplier)))`) so the "auto" option's label can show the
271	concrete number.
272
273	Environment compatibility — pick the best available question channel.
274	Not every runtime exposes the same structured-question tool. Apply this
275	order:
276
277	1. Native clarification tool — use the structured 4-question call below.
278	2. Other native clarification tool (e.g. `ask_question`,
279	`request_user_input`, IDE-specific prompt) — use that tool with the
280	same 4 question texts and option lists. Preserve the recommendation
281	markers and the precomputed values.
282	3. No native tool (Codex CLI, plain text-only runtimes) — **ask
283	directly in normal conversation**. Use the plain-text template at the
284	end of this section. Keep it to one message, 4 numbered questions
285	(the global cap is 2–5 questions per round; we stay inside it).
286
287	Rules that apply to every channel:
288
289	- Ask at most 2–5 questions per round. Our 4 here fits.
290	- Even if missing info doesn't block rendering, **ask once to confirm
291	the parameters that materially affect the final output** (ratio,
292	layout, style, cardCount).
293	- If the user has already pre-approved defaults ("just use defaults",
294	"no need to ask", "auto-pick everything"), asked you not to ask, or the
295	run carries an ongoing autonomous signal ("surprise me" / "decide for me" —
296	`../hyperframes-core/references/brief-contract.md` § 1) — **skip
297	the question entirely** and use: `recommendedRatio`, `layout="stack"`
298	(safest cross-ratio default), `style` chosen from transcript tone in
299	the most neutral group (editorial/data), `autoCount`. Tell the user
300	what you picked in one sentence and continue.
301
302	Channel A — native `AskUserQuestion`:
303
304	```
305	// Precompute before the call:
306	// recommendedRatio = "16:9" \| "9:16" \| "4:5"
307	// autoCount = integer (from Step 6)
308
309	AskUserQuestion({
310	questions: [
311	{
312	question: "Output video aspect ratio (canvas):",
313	header: "Aspect ratio",
314	multiSelect: false,
315	// Reorder so the recommended option appears FIRST (per AskUserQuestion convention).
316	// Append " (recommended · matches source video W×H)" to the recommended option's label.
317	options: [
318	{ label: "16:9 (1920×1080) landscape", description: "TV / YouTube / desktop playback. Most natural when the source video is already landscape; widest canvas." },
319	{ label: "9:16 (1080×1920) portrait", description: "TikTok / Reels / short-form mobile. Most natural for portrait source; native mobile experience." },
320	{ label: "4:5 (1080×1350) near-portrait", description: "Instagram feed / WeChat Moments. Best when source is near-square or you want to cover both platforms." }
321	]
322	},
323	{
324	question: "Choose the overall layout: how should the video and cards coexist on the canvas?",
325	header: "Layout",
326	multiSelect: false,
327	options: [
328	{ label: "side-by-side (split)", description: "Video and card each take half the canvas. Most stable for interview / data side-by-side; clear visual separation." },
329	{ label: "top-bottom (stack)", description: "Video on top (~52%), card below. Classic combo of speaker face + summary card; works well in portrait too." },
330	{ label: "picture-in-picture (pip)", description: "Card fills the canvas, video shrinks to a rounded corner window. Use when content is primary and speaker is secondary." },
331	{ label: "full-screen overlay (overlay)", description: "Video plays full-bleed, card floats as a glass layer on top. Strong cinematic / emotional feel." }
332	]
333	},
334	{
335	question: "Choose the card visual style (style):",
336	header: "Style group",
337	multiSelect: false,
338	// NOTE: these 3 groups intentionally match the frame auto-pick matrix
339	// rows below, so picking a group resolves both `style` group AND the
340	// frame matrix column in one step. Memberships are mutually exclusive.
341	options: [
342	{ label: "warm paper (warm-paper)", description: "academic notebook · editorial big-type · whiteboard hand-drawn · xhs social. Best for interview reflections, product launches, lifestyle, emotional stories." },
343	{ label: "clinical / cold (clinical)", description: "audit magazine · swiss grid · terminal CLI · minimal modern. Best for financial analysis, investigative reports, technical tutorials, serious presentations." },
344	{ label: "experimental / avant-garde (experimental)", description: "geom color-clash geometry · spotlight dark-background. Best for short-form highlights, product launches, strong emotion, cinematic feel." }
345	]
346	},
347	{
348	question: "Card count (takeaway pacing): how many cards to cut?",
349	header: "Card count",
350	multiSelect: false,
351	options: [
352	{ label: "Auto (recommended) ·

6 picks

Type

Run `npx skills use "https://github.com/heygen-com/hyperframes" --skill "heygen-com/hyperframes/talking-head-recut"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.

Use the skills in "https://github.com/heygen-com/hyperframes" that are relevant to the current task. Run `npx skills add "https://github.com/heygen-com/hyperframes"` and select the relevant skills, then follow their instructions.

talking-head-recut

How to install talking-head-recut?

Related

talking-head-recut

How to install talking-head-recut?

Related