video-interaction-mapper

Name: video-interaction-mapper
Author: figma

1.8kstars163forks

How to install video-interaction-mapper?

figma/mcp-server-guide/video-interaction-mapper

$npx -y skills add figma/mcp-server-guide --skill video-interaction-mapper

Installs into the current project.

Summary

This skill should be used when the user asks to analyze a UI screen recording and map interaction states into Figma. Trigger for requests such as "put video frames in Figma", "extract states from my recording", "map interactions from video to Figma", "analyze this screen recordin

Files · 1

View on GitHub

SKILL.md

1	# Video Interaction Mapper
2
3	Turn a UI recording into a static, annotated Figma storyboard. Extract the important
4	before/after states, infer what triggered each change, then place clean screenshots,
5	blue interaction markers, and concise annotations into a Figma Design file.
6
7	## Required Inputs and Tools
8
9	Start with:
10
11	- A local video file path (`.mp4`, `.mov`, `.webm`, or `.avi`).
12	- `ffmpeg` and `ffprobe` available on the machine.
13	- Python with Pillow installed.
14	- Figma MCP tools available: `create_new_file`, `use_figma`, `upload_assets`, and
15	`get_screenshot` or `get_design_context`.
16
17	Run the bundled files in `scripts/` as executable workflow helpers. They are part
18	of the skill's implementation, not reference material. Read or modify them only
19	when debugging, adapting to an unusual environment, or changing the skill itself.
20
21	Use a Figma Design file (`figma.com/design/...`) as the target. The generated
22	Plugin API code creates pages and image-fill frames, which are Design-file
23	operations. If the user provides a FigJam or Slides URL, ask for a Design file or
24	create a new one.
25
26	Before any `use_figma` call, load the Figma API guidance skill if available
27	(`figma-use`). Before creating a new Figma file, load the file-creation guidance
28	skill if available (`figma-create-new-file`). Pass `skillNames:
29	"figma-use,video-interaction-mapper"` on `use_figma` calls when the client supports
30	that parameter.
31
32	## Workflow
33
34	### 1. Validate and Prepare
35
36	Confirm the video path exists, then check `ffmpeg -version`. If Pillow is missing,
37	install it in the active Python environment:
38
39	```bash
40	pip install Pillow --break-system-packages -q
41	```
42
43	Do local analysis before creating or modifying the Figma file. Create a Figma file
44	only after `key_moments.json`, `upload_manifest.json`, and generated Figma scripts
45	are ready, unless the user explicitly asks to create the file first. This avoids
46	leaving partial pages behind when local frame analysis changes.
47
48	### 2. Scout the Timeline Quickly
49
50	Start with a low-resolution scout pass and contact sheet:
51
52	```bash
53	python <SKILL_DIR>/scripts/extract_key_frames.py \
54	--input "<video_path>" \
55	--output /tmp/vim_frames_<timestamp>/ \
56	--mode scout
57	```
58
59	The scout pass extracts at 1 fps by default, resizes frames to 640 px wide, scores
60	scene changes, writes `<output_dir>/frames_manifest.json`, and creates
61	`<output_dir>/contact_sheet.jpg`. Inspect the contact sheet first to identify likely
62	interaction moments.
63
64	Use the slower full extraction only when the scout pass is insufficient:
65
66	```bash
67	python <SKILL_DIR>/scripts/extract_key_frames.py \
68	--input "<video_path>" \
69	--output /tmp/vim_frames_<timestamp>/full_frames/ \
70	--mode full \
71	--max-width 1600
72	```
73
74	For very long videos, ask whether to focus on a specific time range before
75	processing the whole recording. For too many key frames, re-run extraction with a
76	higher `--scene-threshold`, such as `0.4`.
77
78	### 3. Analyze Key Moments
79
80	Read `frames_manifest.json` and inspect `contact_sheet.jpg` or relevant extracted
81	frames. For each meaningful transition, identify:
82
83	- Visual change: modal, drawer, dropdown, tooltip, snackbar, overlay, navigation,
84	loaded content, selected state, validation state, input focus, or scroll position.
85	- Trigger: click, tap, keyboard input, hover, scroll, swipe, form submit, auto/timer,
86	or unknown trigger when the cause is ambiguous.
87	- Interaction target and result coordinates when visible and useful.
88
89	Use normalized coordinates from `0` to `1` relative to the screenshot. Percent-like
90	values such as `45` or `94` are accepted by the generated Figma script and converted
91	to `0.45` or `0.94`. Coordinates are machine metadata for marker placement only.
92	Do not include coordinate values in visible annotation text, moment labels, or user
93	summaries.
94
95	Only add marker coordinates when the point lands on a clearly visible UI element or
96	state. If a target is inferred, between visible controls, or based mainly on cursor
97	position, mark it with `inferred: true`, `uncertain: true`, `visible: false`, or
98	`confidence: "medium"`/`"low"`. The generated script suppresses marker

People who liked this also viewed

6 picks

Type

video-interaction-mapper

1.8kstars163forks

View on GitHub

How to install video-interaction-mapper?

figma/mcp-server-guide/video-interaction-mapper

$npx -y skills add figma/mcp-server-guide --skill video-interaction-mapper

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Video Interaction Mapper
2
3	Turn a UI recording into a static, annotated Figma storyboard. Extract the important
4	before/after states, infer what triggered each change, then place clean screenshots,
5	blue interaction markers, and concise annotations into a Figma Design file.
6
7	## Required Inputs and Tools
8
9	Start with:
10
11	- A local video file path (`.mp4`, `.mov`, `.webm`, or `.avi`).
12	- `ffmpeg` and `ffprobe` available on the machine.
13	- Python with Pillow installed.
14	- Figma MCP tools available: `create_new_file`, `use_figma`, `upload_assets`, and
15	`get_screenshot` or `get_design_context`.
16
17	Run the bundled files in `scripts/` as executable workflow helpers. They are part
18	of the skill's implementation, not reference material. Read or modify them only
19	when debugging, adapting to an unusual environment, or changing the skill itself.
20
21	Use a Figma Design file (`figma.com/design/...`) as the target. The generated
22	Plugin API code creates pages and image-fill frames, which are Design-file
23	operations. If the user provides a FigJam or Slides URL, ask for a Design file or
24	create a new one.
25
26	Before any `use_figma` call, load the Figma API guidance skill if available
27	(`figma-use`). Before creating a new Figma file, load the file-creation guidance
28	skill if available (`figma-create-new-file`). Pass `skillNames:
29	"figma-use,video-interaction-mapper"` on `use_figma` calls when the client supports
30	that parameter.
31
32	## Workflow
33
34	### 1. Validate and Prepare
35
36	Confirm the video path exists, then check `ffmpeg -version`. If Pillow is missing,
37	install it in the active Python environment:
38
39	```bash
40	pip install Pillow --break-system-packages -q
41	```
42
43	Do local analysis before creating or modifying the Figma file. Create a Figma file
44	only after `key_moments.json`, `upload_manifest.json`, and generated Figma scripts
45	are ready, unless the user explicitly asks to create the file first. This avoids
46	leaving partial pages behind when local frame analysis changes.
47
48	### 2. Scout the Timeline Quickly
49
50	Start with a low-resolution scout pass and contact sheet:
51
52	```bash
53	python <SKILL_DIR>/scripts/extract_key_frames.py \
54	--input "<video_path>" \
55	--output /tmp/vim_frames_<timestamp>/ \
56	--mode scout
57	```
58
59	The scout pass extracts at 1 fps by default, resizes frames to 640 px wide, scores
60	scene changes, writes `<output_dir>/frames_manifest.json`, and creates
61	`<output_dir>/contact_sheet.jpg`. Inspect the contact sheet first to identify likely
62	interaction moments.
63
64	Use the slower full extraction only when the scout pass is insufficient:
65
66	```bash
67	python <SKILL_DIR>/scripts/extract_key_frames.py \
68	--input "<video_path>" \
69	--output /tmp/vim_frames_<timestamp>/full_frames/ \
70	--mode full \
71	--max-width 1600
72	```
73
74	For very long videos, ask whether to focus on a specific time range before
75	processing the whole recording. For too many key frames, re-run extraction with a
76	higher `--scene-threshold`, such as `0.4`.
77
78	### 3. Analyze Key Moments
79
80	Read `frames_manifest.json` and inspect `contact_sheet.jpg` or relevant extracted
81	frames. For each meaningful transition, identify:
82
83	- Visual change: modal, drawer, dropdown, tooltip, snackbar, overlay, navigation,
84	loaded content, selected state, validation state, input focus, or scroll position.
85	- Trigger: click, tap, keyboard input, hover, scroll, swipe, form submit, auto/timer,
86	or unknown trigger when the cause is ambiguous.
87	- Interaction target and result coordinates when visible and useful.
88
89	Use normalized coordinates from `0` to `1` relative to the screenshot. Percent-like
90	values such as `45` or `94` are accepted by the generated Figma script and converted
91	to `0.45` or `0.94`. Coordinates are machine metadata for marker placement only.
92	Do not include coordinate values in visible annotation text, moment labels, or user
93	summaries.
94
95	Only add marker coordinates when the point lands on a clearly visible UI element or
96	state. If a target is inferred, between visible controls, or based mainly on cursor
97	position, mark it with `inferred: true`, `uncertain: true`, `visible: false`, or
98	`confidence: "medium"`/`"low"`. The generated script suppresses marker

People who liked this also viewed

6 picks

Type