automate-this

Name: automate-this
Author: github

37kstars4.6kforks

How to install automate-this?

github/awesome-copilot/automate-this

$npx -y skills add github/awesome-copilot --skill automate-this

Installs into the current project.

Summary

Analyze a screen recording of a manual process and produce targeted, working automation scripts. Extracts frames and audio narration from video files, reconstructs the step-by-step workflow, and proposes automation at multiple complexity levels using tools already installed on th

Files · 1

View on GitHub

SKILL.md

1	# Automate This
2
3	Analyze a screen recording of a manual process and build working automation for it.
4
5	The user records themselves doing something repetitive or tedious, hands you the video file, and you figure out what they're doing, why, and how to script it away.
6
7	## Prerequisites Check
8
9	Before analyzing any recording, verify the required tools are available. Run these checks silently and only surface problems:
10
11	```bash
12	command -v ffmpeg >/dev/null 2>&1 && ffmpeg -version 2>/dev/null \| head -1 \|\| echo "NO_FFMPEG"
13	command -v whisper >/dev/null 2>&1 \|\| command -v whisper-cpp >/dev/null 2>&1 \|\| echo "NO_WHISPER"
14	```
15
16	- ffmpeg is required. If missing, tell the user: `brew install ffmpeg` (macOS) or the equivalent for their OS.
17	- Whisper is optional. Only needed if the recording has narration. If missing AND the recording has an audio track, suggest: `pip install openai-whisper` or `brew install whisper-cpp`. If the user declines, proceed with visual analysis only.
18
19	## Phase 1: Extract Content from the Recording
20
21	Given a video file path (typically on `~/Desktop/`), extract both visual frames and audio:
22
23	### Frame Extraction
24
25	Extract frames at one frame every 2 seconds. This balances coverage with context window limits.
26
27	```bash
28	WORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/automate-this-XXXXXX")
29	chmod 700 "$WORK_DIR"
30	mkdir -p "$WORK_DIR/frames"
31	ffmpeg -y -i "<VIDEO_PATH>" -vf "fps=0.5" -q:v 2 -loglevel warning "$WORK_DIR/frames/frame_%04d.jpg"
32	ls "$WORK_DIR/frames/" \| wc -l
33	```
34
35	Use `$WORK_DIR` for all subsequent temp file paths in the session. The per-run directory with mode 0700 ensures extracted frames are only readable by the current user.
36
37	If the recording is longer than 5 minutes (more than 150 frames), increase the interval to one frame every 4 seconds to stay within context limits. Tell the user you're sampling less frequently for longer recordings.
38
39	### Audio Extraction and Transcription
40
41	Check if the video has an audio track:
42
43	```bash
44	ffprobe -i "<VIDEO_PATH>" -show_streams -select_streams a -loglevel error \| head -5
45	```
46
47	If audio exists:
48
49	```bash
50	ffmpeg -y -i "<VIDEO_PATH>" -ac 1 -ar 16000 -loglevel warning "$WORK_DIR/audio.wav"
51
52	# Use whichever whisper binary is available
53	if command -v whisper >/dev/null 2>&1; then
54	whisper "$WORK_DIR/audio.wav" --model small --language en --output_format txt --output_dir "$WORK_DIR/"
55	cat "$WORK_DIR/audio.txt"
56	elif command -v whisper-cpp >/dev/null 2>&1; then
57	whisper-cpp -m "$(brew --prefix 2>/dev/null)/share/whisper-cpp/models/ggml-small.bin" -l en -f "$WORK_DIR/audio.wav" -otxt -of "$WORK_DIR/audio"
58	cat "$WORK_DIR/audio.txt"
59	else
60	echo "NO_WHISPER"
61	fi
62	```
63
64	If neither whisper binary is available and the recording has audio, inform the user they're missing narration context and ask if they want to install Whisper (`pip install openai-whisper` or `brew install whisper-cpp`) or proceed with visual-only analysis.
65
66	## Phase 2: Reconstruct the Process
67
68	Analyze the extracted frames (and transcript, if available) to build a structured understanding of what the user did. Work through the frames sequentially and identify:
69
70	1. Applications used — Which apps appear in the recording? (browser, terminal, Finder, mail client, spreadsheet, IDE, etc.)
71	2. Sequence of actions — What did the user do, in order? Click-by-click, step-by-step.
72	3. Data flow — What information moved between steps? (copied text, downloaded files, form inputs, etc.)
73	4. Decision points — Were there moments where the user paused, checked something, or made a choice?
74	5. Repetition patterns — Did the user do the same thing multiple times with different inputs?
75	6. Pain points — Where did the process look slow, error-prone, or tedious? The narration often reveals this directly ("I hate this part," "this always takes forever," "I have to do this for every single one").
76
77	Present this reconstruction to the user as a numbered step list and ask them to confirm it's accurate before proposing automation. This is critical — a wrong understanding leads to useless automation.
78
79	Format:
80
81	```
82	Here's what I see you doing in this recording:
83
84	1. Open Chrome and navigate to [specific URL]
85	2. Log in with credentials
86	3. Click through to the reporting dashboard
87	4. Download a CSV export
88	5. Open the CSV in Excel
89	6. Filter rows where column B is "pending"
90	7. Copy those rows into a new spreadsheet
91	8. Email the new spreadsheet to [recipient]
92
93	You repeated steps 3-8 three times for different report types.
94
95	[If narration was present]: You mentioned that the export step is the slowest
96	part and that you do this every Monday morning.
97
98	Does this match what you were doing? Anythin

People who liked this also viewed

6 picks

Type

automate-this

37kstars4.6kforks

View on GitHub

How to install automate-this?

github/awesome-copilot/automate-this

$npx -y skills add github/awesome-copilot --skill automate-this

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Automate This
2
3	Analyze a screen recording of a manual process and build working automation for it.
4
5	The user records themselves doing something repetitive or tedious, hands you the video file, and you figure out what they're doing, why, and how to script it away.
6
7	## Prerequisites Check
8
9	Before analyzing any recording, verify the required tools are available. Run these checks silently and only surface problems:
10
11	```bash
12	command -v ffmpeg >/dev/null 2>&1 && ffmpeg -version 2>/dev/null \| head -1 \|\| echo "NO_FFMPEG"
13	command -v whisper >/dev/null 2>&1 \|\| command -v whisper-cpp >/dev/null 2>&1 \|\| echo "NO_WHISPER"
14	```
15
16	- ffmpeg is required. If missing, tell the user: `brew install ffmpeg` (macOS) or the equivalent for their OS.
17	- Whisper is optional. Only needed if the recording has narration. If missing AND the recording has an audio track, suggest: `pip install openai-whisper` or `brew install whisper-cpp`. If the user declines, proceed with visual analysis only.
18
19	## Phase 1: Extract Content from the Recording
20
21	Given a video file path (typically on `~/Desktop/`), extract both visual frames and audio:
22
23	### Frame Extraction
24
25	Extract frames at one frame every 2 seconds. This balances coverage with context window limits.
26
27	```bash
28	WORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/automate-this-XXXXXX")
29	chmod 700 "$WORK_DIR"
30	mkdir -p "$WORK_DIR/frames"
31	ffmpeg -y -i "<VIDEO_PATH>" -vf "fps=0.5" -q:v 2 -loglevel warning "$WORK_DIR/frames/frame_%04d.jpg"
32	ls "$WORK_DIR/frames/" \| wc -l
33	```
34
35	Use `$WORK_DIR` for all subsequent temp file paths in the session. The per-run directory with mode 0700 ensures extracted frames are only readable by the current user.
36
37	If the recording is longer than 5 minutes (more than 150 frames), increase the interval to one frame every 4 seconds to stay within context limits. Tell the user you're sampling less frequently for longer recordings.
38
39	### Audio Extraction and Transcription
40
41	Check if the video has an audio track:
42
43	```bash
44	ffprobe -i "<VIDEO_PATH>" -show_streams -select_streams a -loglevel error \| head -5
45	```
46
47	If audio exists:
48
49	```bash
50	ffmpeg -y -i "<VIDEO_PATH>" -ac 1 -ar 16000 -loglevel warning "$WORK_DIR/audio.wav"
51
52	# Use whichever whisper binary is available
53	if command -v whisper >/dev/null 2>&1; then
54	whisper "$WORK_DIR/audio.wav" --model small --language en --output_format txt --output_dir "$WORK_DIR/"
55	cat "$WORK_DIR/audio.txt"
56	elif command -v whisper-cpp >/dev/null 2>&1; then
57	whisper-cpp -m "$(brew --prefix 2>/dev/null)/share/whisper-cpp/models/ggml-small.bin" -l en -f "$WORK_DIR/audio.wav" -otxt -of "$WORK_DIR/audio"
58	cat "$WORK_DIR/audio.txt"
59	else
60	echo "NO_WHISPER"
61	fi
62	```
63
64	If neither whisper binary is available and the recording has audio, inform the user they're missing narration context and ask if they want to install Whisper (`pip install openai-whisper` or `brew install whisper-cpp`) or proceed with visual-only analysis.
65
66	## Phase 2: Reconstruct the Process
67
68	Analyze the extracted frames (and transcript, if available) to build a structured understanding of what the user did. Work through the frames sequentially and identify:
69
70	1. Applications used — Which apps appear in the recording? (browser, terminal, Finder, mail client, spreadsheet, IDE, etc.)
71	2. Sequence of actions — What did the user do, in order? Click-by-click, step-by-step.
72	3. Data flow — What information moved between steps? (copied text, downloaded files, form inputs, etc.)
73	4. Decision points — Were there moments where the user paused, checked something, or made a choice?
74	5. Repetition patterns — Did the user do the same thing multiple times with different inputs?
75	6. Pain points — Where did the process look slow, error-prone, or tedious? The narration often reveals this directly ("I hate this part," "this always takes forever," "I have to do this for every single one").
76
77	Present this reconstruction to the user as a numbered step list and ask them to confirm it's accurate before proposing automation. This is critical — a wrong understanding leads to useless automation.
78
79	Format:
80
81	```
82	Here's what I see you doing in this recording:
83
84	1. Open Chrome and navigate to [specific URL]
85	2. Log in with credentials
86	3. Click through to the reporting dashboard
87	4. Download a CSV export
88	5. Open the CSV in Excel
89	6. Filter rows where column B is "pending"
90	7. Copy those rows into a new spreadsheet
91	8. Email the new spreadsheet to [recipient]
92
93	You repeated steps 3-8 three times for different report types.
94
95	[If narration was present]: You mentioned that the export step is the slowest
96	part and that you do this every Monday morning.
97
98	Does this match what you were doing? Anythin

People who liked this also viewed

6 picks

Type