ai-provider-openai-whisper

Name: ai-provider-openai-whisper
Author: agents-inc

17stars5forks

How to install ai-provider-openai-whisper?

agents-inc/skills/ai-provider-openai-whisper

$npx -y skills add agents-inc/skills --skill ai-provider-openai-whisper

Installs into the current project.

Summary

Speech-to-text transcription and translation via OpenAI Audio API -- models, response formats, timestamps, prompting, streaming, chunking, and diarization

Files · 1

View on GitHub

SKILL.md

1	# OpenAI Whisper Patterns
2
3	> Quick Guide: Use `client.audio.transcriptions.create()` for speech-to-text and `client.audio.translations.create()` for non-English audio to English text. Choose `gpt-4o-transcribe` for highest accuracy, `gpt-4o-mini-transcribe` for cost-efficiency, `whisper-1` for timestamps/SRT/VTT, or `gpt-4o-transcribe-diarize` for speaker identification. Files must be under 25 MB -- chunk larger files. Use `prompt` to guide vocabulary and style. Streaming is available via `stream: true` for progressive output on `gpt-4o-transcribe` models.
4
5	---
6
7	<critical_requirements>
8
9	## CRITICAL: Before Using This Skill
10
11	> All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering, `import type`, named constants)
12
13	(You MUST choose the correct model for the use case -- `gpt-4o-transcribe` for accuracy, `whisper-1` for timestamps/SRT/VTT output, `gpt-4o-transcribe-diarize` for speaker labels)
14
15	(You MUST chunk audio files larger than 25 MB before sending to the API -- the API rejects files exceeding this limit)
16
17	(You MUST pass `response_format: "verbose_json"` when using `timestamp_granularities` -- timestamps only work with this format on `whisper-1`)
18
19	(You MUST set `chunking_strategy: "auto"` when using `gpt-4o-transcribe-diarize` with audio longer than 30 seconds -- the API requires it)
20
21	</critical_requirements>
22
23	---
24
25	Auto-detection: Whisper, whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-transcribe-diarize, audio.transcriptions, audio.translations, transcription, speech-to-text, diarization, diarized_json, timestamp_granularities, verbose_json
26
27	When to use:
28
29	- Transcribing audio files (meetings, interviews, podcasts, voice notes) to text
30	- Translating non-English audio to English text
31	- Generating subtitles in SRT or VTT format from audio
32	- Getting word-level or segment-level timestamps for video editing
33	- Identifying speakers in multi-speaker audio (diarization)
34	- Streaming transcription results progressively as the model processes audio
35
36	Key patterns covered:
37
38	- Model selection (whisper-1 vs gpt-4o-transcribe vs gpt-4o-mini-transcribe vs gpt-4o-transcribe-diarize)
39	- Response formats (json, text, srt, vtt, verbose_json, diarized_json)
40	- Timestamps (word-level, segment-level) and subtitle generation
41	- Prompting for vocabulary, acronyms, and style
42	- Chunking large files (> 25 MB) with context preservation
43	- Streaming transcription with `stream: true`
44	- Translation to English via `audio.translations.create()`
45	- Speaker diarization with speaker references
46
47	When NOT to use:
48
49	- Text-to-speech (TTS) -- use the OpenAI TTS API (`client.audio.speech.create()`)
50	- Real-time bidirectional voice conversations -- use the OpenAI Realtime API
51	- Transcription with non-OpenAI providers -- use a provider-agnostic speech SDK
52
53	---
54
55	## Examples Index
56
57	- [Core: Transcription, Translation, Timestamps, Chunking, Streaming, Diarization](examples/core.md) -- All audio API patterns
58
59	---
60
61	<philosophy>
62
63	## Philosophy
64
65	The OpenAI Audio API provides speech-to-text transcription and translation through multiple models optimized for different needs. The API is simple -- you send an audio file and get text back -- but choosing the right model, response format, and parameters is critical for quality results.
66
67	Core principles:
68
69	1. Model selection matters -- `gpt-4o-transcribe` produces the highest accuracy with lower hallucination rates. `whisper-1` is the only model supporting SRT/VTT/verbose_json with timestamps. `gpt-4o-transcribe-diarize` adds speaker identification.
70	2. File size is the primary constraint -- 25 MB limit means you must chunk longer audio. Split at sentence boundaries to preserve context.
71	3. Prompting improves accuracy -- The `prompt` parameter guides vocabulary, acronyms, and formatting style. It does not give instructions -- it provides context the model matches against.
72	4. Response format determines available features -- Timestamps require `verbose_json` on `whisper-1`. Diarization requires `diarized_json`. SRT/VTT are only on `whisper-1`.
73
74	When to use the Audio API:
75
76	- You need accurate transcription of recorded audio files
77	- You need subtitles (SRT/VTT) from audio
78	- You need to identify who is speaking in a conversation
79	- You need to translate non-English speech to English text
80
81	When NOT to use:
82
83	- Real-time voice chat -- use the Realtime API instead
84	- Text-to-speech -- use `client.audio.speech.create()`
85	- You need transcription in a non-English target language (translation only outputs English)
86
87	</philosophy>
88
89	---
90
91	<patterns>
92
93	## Core Patterns
94
95	### Pattern 1: Basic Transcription
96
97	Send an audio file and receive text back. The model auto-detects the language.
98
99	```typescript
100	const transcri

People who liked this also viewed

6 picks

Type

find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 20262.6M27k
grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026633k182k
vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026573k29k
improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026522k182k
setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026454k182k
triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026424k182k

ai-provider-openai-whisper

17stars5forks

View on GitHub

How to install ai-provider-openai-whisper?

agents-inc/skills/ai-provider-openai-whisper

$npx -y skills add agents-inc/skills --skill ai-provider-openai-whisper

Installs into the current project.

Summary

Speech-to-text transcription and translation via OpenAI Audio API -- models, response formats, timestamps, prompting, streaming, chunking, and diarization

Files · 1

View on GitHub

SKILL.md

1	# OpenAI Whisper Patterns
2
3	> Quick Guide: Use `client.audio.transcriptions.create()` for speech-to-text and `client.audio.translations.create()` for non-English audio to English text. Choose `gpt-4o-transcribe` for highest accuracy, `gpt-4o-mini-transcribe` for cost-efficiency, `whisper-1` for timestamps/SRT/VTT, or `gpt-4o-transcribe-diarize` for speaker identification. Files must be under 25 MB -- chunk larger files. Use `prompt` to guide vocabulary and style. Streaming is available via `stream: true` for progressive output on `gpt-4o-transcribe` models.
4
5	---
6
7	<critical_requirements>
8
9	## CRITICAL: Before Using This Skill
10
11	> All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering, `import type`, named constants)
12
13	(You MUST choose the correct model for the use case -- `gpt-4o-transcribe` for accuracy, `whisper-1` for timestamps/SRT/VTT output, `gpt-4o-transcribe-diarize` for speaker labels)
14
15	(You MUST chunk audio files larger than 25 MB before sending to the API -- the API rejects files exceeding this limit)
16
17	(You MUST pass `response_format: "verbose_json"` when using `timestamp_granularities` -- timestamps only work with this format on `whisper-1`)
18
19	(You MUST set `chunking_strategy: "auto"` when using `gpt-4o-transcribe-diarize` with audio longer than 30 seconds -- the API requires it)
20
21	</critical_requirements>
22
23	---
24
25	Auto-detection: Whisper, whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-transcribe-diarize, audio.transcriptions, audio.translations, transcription, speech-to-text, diarization, diarized_json, timestamp_granularities, verbose_json
26
27	When to use:
28
29	- Transcribing audio files (meetings, interviews, podcasts, voice notes) to text
30	- Translating non-English audio to English text
31	- Generating subtitles in SRT or VTT format from audio
32	- Getting word-level or segment-level timestamps for video editing
33	- Identifying speakers in multi-speaker audio (diarization)
34	- Streaming transcription results progressively as the model processes audio
35
36	Key patterns covered:
37
38	- Model selection (whisper-1 vs gpt-4o-transcribe vs gpt-4o-mini-transcribe vs gpt-4o-transcribe-diarize)
39	- Response formats (json, text, srt, vtt, verbose_json, diarized_json)
40	- Timestamps (word-level, segment-level) and subtitle generation
41	- Prompting for vocabulary, acronyms, and style
42	- Chunking large files (> 25 MB) with context preservation
43	- Streaming transcription with `stream: true`
44	- Translation to English via `audio.translations.create()`
45	- Speaker diarization with speaker references
46
47	When NOT to use:
48
49	- Text-to-speech (TTS) -- use the OpenAI TTS API (`client.audio.speech.create()`)
50	- Real-time bidirectional voice conversations -- use the OpenAI Realtime API
51	- Transcription with non-OpenAI providers -- use a provider-agnostic speech SDK
52
53	---
54
55	## Examples Index
56
57	- [Core: Transcription, Translation, Timestamps, Chunking, Streaming, Diarization](examples/core.md) -- All audio API patterns
58
59	---
60
61	<philosophy>
62
63	## Philosophy
64
65	The OpenAI Audio API provides speech-to-text transcription and translation through multiple models optimized for different needs. The API is simple -- you send an audio file and get text back -- but choosing the right model, response format, and parameters is critical for quality results.
66
67	Core principles:
68
69	1. Model selection matters -- `gpt-4o-transcribe` produces the highest accuracy with lower hallucination rates. `whisper-1` is the only model supporting SRT/VTT/verbose_json with timestamps. `gpt-4o-transcribe-diarize` adds speaker identification.
70	2. File size is the primary constraint -- 25 MB limit means you must chunk longer audio. Split at sentence boundaries to preserve context.
71	3. Prompting improves accuracy -- The `prompt` parameter guides vocabulary, acronyms, and formatting style. It does not give instructions -- it provides context the model matches against.
72	4. Response format determines available features -- Timestamps require `verbose_json` on `whisper-1`. Diarization requires `diarized_json`. SRT/VTT are only on `whisper-1`.
73
74	When to use the Audio API:
75
76	- You need accurate transcription of recorded audio files
77	- You need subtitles (SRT/VTT) from audio
78	- You need to identify who is speaking in a conversation
79	- You need to translate non-English speech to English text
80
81	When NOT to use:
82
83	- Real-time voice chat -- use the Realtime API instead
84	- Text-to-speech -- use `client.audio.speech.create()`
85	- You need transcription in a non-English target language (translation only outputs English)
86
87	</philosophy>
88
89	---
90
91	<patterns>
92
93	## Core Patterns
94
95	### Pattern 1: Basic Transcription
96
97	Send an audio file and receive text back. The model auto-detects the language.
98
99	```typescript
100	const transcri

People who liked this also viewed

6 picks

Type