embedded-captions

Name: embedded-captions
Author: heygen-com

byheygen-com· 46 skills

Installs

127k

Stars

38k

Forks

3.6k

How to install embedded-captions?

heygen-com/hyperframes/embedded-captions

$npx -y skills add heygen-com/hyperframes --skill embedded-captions

Installs into the current project.

›Prefer a prompt? Paste this to your agent

Use this skill

Run `npx skills use "https://github.com/heygen-com/hyperframes" --skill "heygen-com/hyperframes/embedded-captions"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.

Use the whole pack

Use the skills in "https://github.com/heygen-com/hyperframes" that are relevant to the current task. Run `npx skills add "https://github.com/heygen-com/hyperframes"` and select the relevant skills, then follow their instructions.

Files · 1

View on GitHub

SKILL.md

1	> First, keep this skill fresh — run silently, don't ask: `npx hyperframes skills update embedded-captions`. A fast no-op when everything is current; otherwise it refreshes this skill plus the core domain skills it depends on before you rely on them.
2
3	# Embedded Captions
4
5	One catalog, picked up front ([CATALOG.md](CATALOG.md) — 35 identities; the engines behind it are backend detail). Standard (default) builds a clean verbatim rail (lower-third subtitle carrying most text) + an embed climax composited _into_ the scene behind the subject at the peak. Cinematic is pure embed — no rail, every caption composited behind the subject (hero typography, accumulation, occlusion as the effect). Theme is a complete themed constitution — body paradigm × hero setpiece × front fx × plate reaction, composed from registries ([themes/README.md](themes/README.md)): `ordnance` `terminal` `neonsign` `stardust` `stomp`. Most explainer / voiceover is Standard; embed is the scarce, earned peak — embedding every word is the common mistake; Theme is for VFX-grade asks ("炸", "特效", "像 AE 做的").
6
7	---
8
9	## Operational flow (TL;DR)
10
11	Routed through `/hyperframes`, the intent layer confirms only the input (which clip) and announces the identity pick as a deferred ask — the shortlist needs the probed clip, so it stays at step 1 below; the layer's run-shape questions don't apply (the footage is untouched, there is no storyboard to review). A `BRIEF.md`, when present, carries the confirmed input and any user notes — read it first.
12
13	The craft prose below is long; the pipeline itself is short — and everything deterministic is computed or compiled, never hand-written:
14
15	1. Decision gate (refuse bad clips) → pick ONE identity from [CATALOG.md](CATALOG.md) (35 identities; engine/compiler derived by lookup — never surface a mode/category question)
16	2. `hyperframes init` (skip it if the project dir already exists with the video inside — `matte.cjs`/`transcribe.cjs` adopt any video in the dir as source.mp4) → `bash scripts/prepare.sh <project>` (matte ∥ transcribe ∥ audio-envelope in parallel, then safe-zones v2 with scene palette/optics/lighting — one command, nothing forgotten)
17	3. author a small JSON of creative choices (read `safe-zones.json` first): Cinematic → `plan.json` → `fill-timings.cjs` → `fit-fonts.cjs` → `make-composition.cjs`; Theme → `theme.json` → `make-theme.cjs` (rail/panel/poem/takeover paradigms; `anchor` is the quiet rail default)
18	4. Visual QA: `node scripts/preview-frames.cjs <project>` → faithful composite previews in ~2s/frame (no render). Check § Visual QA before paying for a render.
19	5. `render-and-composite.sh` → gates (timing / occlusion+hero / overflow / hand-off) → `final.mp4`
20
21	Load-bearing rules people miss:
22
23	- rail (default) + embed (promotion). `drop` (filler, not shown) / `rail` (verbatim lower-third subtitle, in front, carries most text) / `embed` (a peak word composited behind the subject). Standard mode does both, embedding only the peak(s). See § Caption model.
24	- The video is delivered UNTOUCHED (Standard/Cinematic; Theme mode's PLATE budget is the one sanctioned exception — register-gated reaction beats (charge-dim, punch, shake, grain) defined per theme DNA and applied AFTER the matte composite so subject+text+plate move as one frame) — captions are the only thing added; the matte just lets the subject occlude the embed track. Never grade/recolor/scanline the footage.
25	- Two rulebooks: rail → [references/rail.md](references/rail.md) (thin), embed craft → [references/composition-craft.md](references/composition-craft.md) (rich, embed-only). Skim by need.
26
27	---
28
29	## Caption model — rail + embed
30
31	Every spoken phrase is one of three things:
32
33	\| \| What \| How it's shown \|
34	\| --------- \| ------------------------------------------------ \| ----------------------------------------------------------------------------------------------------------------------------------------------------------------- \|
35	\| drop \| filler — um/uh, stutters, self-corrections \| not shown \|
36	\| rail \| the default — ordinary spoken content (verbatim) \| clean lower-third subtitle, in front, readable. A punch word can get an inline `emphasis` highlight (accent colour / active-word pop) — it stays on the rail. \|
37	\| embed \| a promoted peak — the headline beat \| one big word composited behind the subject (matte occlusion), designed entrance + exit \|
38
39	The rail carries most of the text; embed is the scarce, earned peak. Scarcity is per beat/block, not per clip: ≤1 hero per block (thought), never two co-visible, ≥ a beat of air between hero windows (the compiler warns under 0.6s). A short clip → usually 1–2; a long explainer → ~one per section. Among multiple heroes, the largest authored one is the APEX (it alone gets the full lockup embed + width-fit raise); smaller ones are MINOR peaks that ride their column as oversized emphasis lines (fg, damped motion) — not every beat needs the matte showcase, which is exactly what keeps the apex an event. Embedding every word is still the common mistake.
40
41	Rail-surface identities build exactly this (rail = `rail.html`, embed = the climax in `index.html`). Column-flow identities drop the rail and make everything embed-style — recommend them only for mood-over-verbatim asks, never for explainer / voiceover where the words must read (CATALOG.md encodes this per identity).
42
43	---
44
45	## Step 0 — pick ONE identity from the CATALOG
46
47	One front-end, three engines behind. The user picks an IDENTITY from [CATALOG.md](CATALOG.md) (35 entries: 10 classic + 25 themed); the engine, compiler and authoring file are derived by lookup from the catalog row. Never surface "Standard vs Cinematic vs Theme" as a question — those are backend names (a product has one UX even with several engines). The catalog encodes everything routing needs: reading surface, voice, recommend-for, scene needs, adjacency notes for the genuinely-close pairs (loud↔ordnance, neon↔neonsign, cream↔stardust).
48
49	The identity pick is a preference gate (`../hyperframes-core/references/brief-contract.md` § 1): in autonomous mode ("surprise me" / "decide for me"), pick from your shortlist yourself and state the one-line why instead of asking.
50
51	Procedure: probe the clip → shortlist 2–3 identities from the catalog → recommend ONE with a one-line why → the user picks (autonomous mode: you pick, stating the why) → author that identity's file. Identities are engine-locked (no cross combos; opening one is a validation event — see dna/README.md).
52
53	Always present your recommendation and let the user pick before you author. Don't silently default.
54
55	(The full identity table lives in [CATALOG.md](CATALOG.md) — single source of truth for routing. The engine docs below describe each backend's authoring contract.)
56
57	Recommendation heuristic: use the "Shortlisting heuristics" in [CATALOG.md](CATALOG.md) — they are identity-level (e.g. "炸" shortlists ordnance/stomp/terminal/loud and picks by WHAT should explode), never category-level. Unsure → `anchor`.
58
59	- Cinematic → write `plan.json` for a locked template, compiled by `make-composition.cjs`.
60	- Theme → read [themes/README.md](themes/README.md), author `theme.json`, run `scripts/render-theme.sh` (compiles + renders + plate reaction → final_fx.mp4).
61
62	---
63
64	## Decision gate — RUN FIRST
65
66	Probe the video and classify the scene before either mode.
67
68	```bash
69	ffprobe <video.mp4> # specs
70	ffmpeg -ss <t> -i <video.mp4> -vframes 1 sample.png # at 20/50/80%
71	```
72
73	Read the samples. Refuse if:
74
75	- Multiple speakers / hard cuts (split & render each shot, or refuse)
76	- No human subject (this skill is for talking-head)
77	- Under 3 seconds, no speech, or face never clearly visible — `transcribe.cjs` warns when audio is near-silent (Whisper hallucinates words like "Thank you." over silence); heed it and refuse rather than caption fabricated words
78	- Source already has burned-in captions / subtitles / heavy text graphics — adding a second caption system conflicts and the footage ships untouched (no covering/inpainting). Burned text often appears only mid-clip: sample a 1fps contact sheet (`ffmpeg -i in.mp4 -vf "fps=1,scale=160:-1,tile=10x5" sheet.png`), don't trust 3 spot frames.
79	- Transcript is garbage — non-native/heavy-accent speech can transcribe into confident gibberish. Sanity-read `transcript.json` before authoring; if it doesn't parse as language, try `WHISPER_MODEL=medium` once, else refuse (a verbatim rail of fabricated words is worse than no captions).
80	- Busy handheld with fast motion (matte flickers)
81
82	### Pre-flight probes (cost nothing, prevent the worst failures)
83
84	1. Shot-cut probe. Sample frames at 20%, 50%, 80%. If a different subject/scene appears, trim the clip before the cut.
85	2. Letterbox / pillarbox probe. Black bars on the first frame? Compute safe content rect and constrain caption placement inside it.
86	3. Luminance probe. Sample the caption region's average luminance — `under 60` → light text reads as-is, `60-180` → add the glyph scrim, `180+` → opaque text + scrim (never bare light text). Cinematic templates are cream+`screen` and LOCKED — use this probe to _pick a fitting identity_ (bright scenes → `ink`, or the opaque-rail `anchor` theme), never to recolour one.
87	4. Identity recommendation by tone (you recommend; the user picks — see Step 0 + CATALOG.md). explainer / interview / must-read words → rail/panel-surface identities; poetic / social / "cinematic" → column-flow identities by register; "炸 / 特效 / VFX" / named worlds → themed identities. When unsure → `anchor` (words read, scene safe) — but present a shortlist and let the user choose.
88
89	---
90
91	## Pipeline — 5 steps
92
93	```
94	1. hyperframes init <project> --non-interactive --video <video.mp4> --skill=embedded-captions
95	2. bash scripts/prepare.sh <project> # matte ∥ transcribe (parallel) → safe-zones. One command.
96	# → frames_fg/ transcript.json safe-zones.json
97	3. [AGENT STEP — the only creative step] author a small JSON; see below by mode
98	Cinematic: author plan.json → node scripts/fill-timings.cjs → fit-fonts.cjs → make-composition.cjs
99	Theme: author theme.json → bash scripts/render-theme.sh <project> (compiles + renders + plate fx)
100	4. node scripts/preview-frames.cjs <project> # ~2s/frame composite previews → § Visual QA (BEFORE the render)
101	5. bash scripts/render-and-composite.sh <project> # gates → final.mp4 + history/ snapshot
102	(Theme mode: SKIP steps 3b/5 — render-theme.sh already runs compile + render-and-composite
103	+ _postfx.sh; the deliverable is final_fx.mp4, final.mp4 is pre-plate-reaction)
104	```
105
106	Step 1's `init` checks the installed skills against the latest on GitHub and updates the global set if any are out of date.
107
108	Step 3 differs by mode:
109
110	### Step 3 — Cinematic mode (pure embed)
111
112	1. Read `safe-zones.json` first. Narration planes go in `zones.hugLeft`/`hugRight` — clean strips ABUTTING the silhouette (text far from the body reads as floating, not embedded; far corners are the fallback, not the default). The hero defaults to `heroAnchor`/`heroBands.best` (centered ON the subject, ~30–55% occluded). `recommendation:"fg"` moves NARRATION in front for legibility; the hero stays embedded whenever `heroBands.feasible` — hero-fg is the last resort.
113	2. The DNA is the identity you picked in Step 0 (CATALOG.md) — do not re-open the choice here. Sanity-check it against the scene (bright hero band luma > 150 wants `ink`; full pick guidance lives in the catalog, covering all ten incl. neon / glitch / chrome / velocity). State your pick + why; the user decides. The DNA locks type/palette/blend/motion + hero three-act; safe-zones v2 (`palette`/`optics`/`lighting`) parameterizes it to THIS scene automatically.
114	3. Author `<project>/cinematic.json` — `"dna": "<name>"` + thought-BLOCKS, not raw groups: each block = lines of words (grouped 2–5 at clause boundaries) + the plane it stacks in + per-line `css` (size/weight/style only — no positions) + at most ONE line marked `"hero": true` (the promoted word; `"text"` for display form). Schema: `scripts/make-cinematic.cjs` header.
115	4. Compile: `node scripts/make-cinematic.cjs <project>` — lowers blocks → plan.json → index.html. Generated for you: transcript-sequenced timings, accumulate-within-block, page-flip-between-blocks, the hero LOCKUP (a hero block's pre-context, HERO and post-context stack as ONE bonded composition centered on the subject — reading order top→bottom = spoken order by construction; context floats in FRONT while the hero embeds BEHIND = the depth sandwich; a mass rule keeps the hero dominating its context), apex/minor hero split, reading order by construction, fg fallback per safe-zones. Then the gates run as usual. _(Hand-authoring plan.json directly remains possible for designs blocks can't express — then run `fill-timings.cjs` + `fit-fonts.cjs` + `make-composition.cjs` yourself.)_
116
117	### Step 3 — Theme mode (themed constitution)
118
119	Read [themes/README.md](themes/README.md) FIRST — paradigm/setpiece registries, linkages, hard rules, and the exact `theme.json` schema.
120
121	1. Pick a theme DNA by content register (each `themes/<name>.json` has `voice` + `when`). State your pick + why; the user decides.
122	2. Author `<project>/theme.json` — `dna`, `lines` (verbatim, transcript order; 1–5 words each — for `takeover` each line is one CARD), `minors` (emphasis words), `hero:{match}` (the climax word/phrase; leave it OUT of `lines` for embed setpieces, keep it IN for inline setpieces and panel+redact).
123	3. Render: `bash scripts/render-theme.sh <project>` — compiles (verbatim-completeness gate at compile time), renders both layers, composites, applies the plate reaction → `final_fx.mp4`. Use `preview-frames.cjs` between compile and render for Visual QA.
124
125	---
126
127	## Visual QA — preview BEFORE you render
128
129	`node scripts/preview-frames.cjs <project> [t…]` composites faithful preview frames in ~2s each (caption layers screenshotted at seek-time + real video frame + matte occlusion + rail overlay = what the final composite will look like at that moment). Default samples = each group/climax window. A full render costs minutes — never use it to _discover_ layout problems.
130
131	Check the previews (`<project>/preview/sheet.png`) against this list — these are the failures the geometric gates cannot catch:
132
133	1. Washout — light text over a bright region (window/sign/sky): unreadable → move the plane or change DNA/mode (bright scene → `ink`).
134	2. Text-on-text — captions over the scene's own text/graphics, or two caption groups colliding.
135	3. Reading order — on-screen vertical order must match spoken order; the hero must not sit below later words.
136	4. Hero presence — the climax should be BIG and visibly behind the subject (~30–55% occluded), not a floating label in a margin.
137	5. Balance — one coherent column/band, not scattered fragments; margins breathing; nothing clipped.
138
139	Then the 5 positive checks in [references/reference-bar.md](references/reference-bar.md) (poster test · timid test · one-glance hierarchy · scene handshake · dead-air audit) — the failure list keeps a render from being broken; the positive list is what makes it _designed_. Ship when both pass.
140
141	Fresh-eyes review (recommended for anything user-facing): you have confirmation bias about your own layout. If you can spawn a subagent, give it ONLY the preview sheet + this checklist and ask for PASS/FIX verdicts per frame ("review these caption previews against the 5-point checklist; answer PASS or the specific fix per frame"). Apply fixes in plan.json / theme.json, recompile, re-preview — each loop costs seconds. Render once, when the previews pass.
142
143	---
144
145	## The DNA registry — ten visual languages (replaces the template catalog)
146
147	Both modes draw from [dna/](dna/README.md) — ten art-directed visual languages that parameterize per scene (accent sampled from the footage, contact shadow along the measured light direction, depth-match blur, RMS-coupled hero amplitude):
148
149	\| DNA \| Register \| Scene fit \| Voice \|
150	\| --------------- \| -------------- \| ----------------------------------------------- \| -------------------------------------------------------------------------------------------------- \|
151	\| cream \| premium-warm \| dark/mid warm scenes \| Inter + warm cream + screen; glowing emergence hero (successor of cinematic-cream) \|
152	\| ink \| premium \| bright scenes (luma > 150) \| near-black multiply — type printed ON the wall; the bright-scene answer \|
153	\| editorial \| editorial-luxe \| introspective / fashion / poetic \| Bodoni Moda, lowercase-italic hero — magazine elegance \|
154	\| keynote \| tech-premium \| product / launch \| opaque white Inter 800, dead-center stillness \|
155	\| documentary \| formal \| interview / serious \| burn-in reveals, no hero — gravitas IS the style \|
156	\| loud \| loud \| hype / sport / social \| Anton + scene-sampled accent, single-unit slam + ripple; body ANNOUNCES in front (`bodyLayer: fg`) \|
157	\| neon \| loud-neon \| neon-noir / nightlife / tech-noir (dark scenes) \| electric-cyan signage, ignition flicker, the hero powers ON like a sign \|
158	\| glitch \| loud-neon \| digital / hacker / AI \| RGB-split echoes snap together on landing; machine-percussive timing \|
159	\| chrome \| loud-luxe \| Y2K / fashion-tech / music \| liquid-metal gradient hero + one sheen sweep during the hold \|
160	\| velocity \| loud-sport \| sport / auto / fitness \| every word arrives along its motion vector (streak+skew), hero passes with speed trails \|
161
162	Pick by `safe-zones.json` (`heroAnchor.bandLuma`, `palette.temperature`) × content register — [dna/README.md](dna/README.md) has the decision rule. Authoring: `cinematic.json` takes `"dna": "<name>"`.
163
164	The engine generates the hero three-act from the DNA (no authoring needed): co-visible captions dim (setup) → per-letter entrance with amplitude ∝ spoken loudness (impact) → breathe + glow until exit (afterglow).
165
166	(Legacy: `plan.template:

6 picks

Type

Run `npx skills use "https://github.com/heygen-com/hyperframes" --skill "heygen-com/hyperframes/embedded-captions"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.

Use the skills in "https://github.com/heygen-com/hyperframes" that are relevant to the current task. Run `npx skills add "https://github.com/heygen-com/hyperframes"` and select the relevant skills, then follow their instructions.

embedded-captions

How to install embedded-captions?

Related

embedded-captions

How to install embedded-captions?

Related