Whisper.cpp: Local Speech-to-Text from the Command Line

AI offers a different set of inputs and outputs for inferences. One of the many inference families is Automatic Speech Recognition (ASR). OpenAI’s Whisper model makes transcribing pre-recorded audio genuinely practical — and whisper.cpp lets you run the model entirely on your own machine: CPU, Apple’s Metal GPU and Core ML on M processors, or a dedicated GPU.

It supports the full range of Whisper models, plus quantized variants that need far less memory and disk via the GGML library. No API keys, no per-minute pricing, no audio leaving your machine.

This post walks from installation to producing an SRT subtitle file from a video — and at the end, how to serve transcription over HTTP. If you are exploring how AI fits into everyday development work, I wrote about how I actually use AI to write Ruby on Rails code as well.

Install and build

Clone the repository (it now lives in the ggml-org organization — the old ggerganov/whisper.cpp URL redirects) and build with CMake:

git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
cmake -B build
cmake --build build -j --config Release

The build produces several binaries under build/bin/ — including whisper-cli (transcription), whisper-server (HTTP), whisper-stream (microphone input), and quantize. Note that main was renamed to whisper-cli; older tutorials referencing ./main mean the new name.

Download a model

Models in ggml format download with a helper script:

sh ./models/download-ggml-model.sh base.en

All available models are listed at huggingface.co/ggerganov/whisper.cpp — tiny, base, small, medium, large-v3, large-v3-turbo, in English-only (.en) and multilingual variants. Pick based on the memory you can spare:

Quantized models cut both disk and memory at a small accuracy cost. You can create one from a downloaded model with the quantize binary — this is where files like ggml-medium.en-q5_0.bin come from:

./build/bin/quantize models/ggml-medium.en.bin models/ggml-medium.en-q5_0.bin q5_0

If you only find a quantized file referenced somewhere, you can also download it directly: the download script accepts quantized names too, e.g. sh ./models/download-ggml-model.sh medium.en-q5_0.

Transcribe your first file

Test with the sample audio shipped in the repository:

./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wav

[00:00:00.000 --> 00:00:11.000]   And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

whisper-cli expects 16-bit WAV input — convert anything else first with ffmpeg:

ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav

Command-line flags worth knowing

• -m <path> — model file to load
• -f <path> — audio file to transcribe
• -l es — force a language instead of auto-detecting
• -osrt, -ovtt, -otxt, -oj — emit SRT, VTT, plain text, or JSON output files
• -of <name> — base name for output files (so -osrt -of video writes video.srt)
• -ml 32 — cap segment length (use -ml 1 on long words for word-level timestamps)
• -t 8 — number of threads
• --print-colors — color-code words by transcription confidence

The direct SRT output is the big quality-of-life win: when I first wrote this post, I piped JSON through a Ruby script to build subtitles. Today -osrt does it for you.

From video to SRT subtitles

Two commands: extract the audio track as 16-bit WAV, then transcribe straight to SRT:

ffmpeg -i video.mp4 -ar 16000 -ac 1 -c:a pcm_s16le video.wav
./build/bin/whisper-cli -m models/ggml-medium.en-q5_0.bin -f video.wav -osrt -of video

You get video.srt ready for any video player. Depending on audio length and model size, this takes seconds to a few minutes.

I still automate the pair with a small Ruby script when batch-processing a folder of videos:

Dir.glob("*.mp4").each do |video|
  base = File.basename(video, ".*")
  system("ffmpeg -i '#{video}' -ar 16000 -ac 1 -c:a pcm_s16le '#{base}.wav'")
  system("./build/bin/whisper-cli -m models/ggml-medium.en-q5_0.bin " \
         "-f '#{base}.wav' -osrt -of '#{base}'")
  File.delete("#{base}.wav")
  puts "Created #{base}.srt"
end

Run it with ruby transcribe.rb inside a folder of .mp4 files and each video ends up with an .srt next to it.

Apple Silicon performance

On M-series Macs, inference runs on the GPU via Metal by default — nothing to configure. For an extra speed-up (around 3x on the encoder), you can move the encoder to the Apple Neural Engine through Core ML. You need Python (3.11 recommended) with a few packages, then generate a Core ML variant of your model and rebuild:

pip install ane_transformers openai-whisper coremltools
./models/generate-coreml-model.sh base.en
cmake -B build -DWHISPER_COREML=1
cmake --build build -j --config Release

The first run takes a moment while the ANE compiles the model for your specific device; subsequent runs are fast.

Serving transcription over HTTP

whisper-server turns the same setup into a local service:

./build/bin/whisper-server -m models/ggml-base.en.bin --host 127.0.0.1 --port 8080

It listens on 127.0.0.1:8080 by default and accepts multipart POSTs at /inference:

curl 127.0.0.1:8080/inference \
  -F "file=@video.wav" \
  -F "response_format=json"

The response shape is deliberately close to OpenAI’s transcription API, so client code written against the hosted API ports easily. If you need the exact path — for a drop-in replacement, say — --inference-path /v1/audio/transcriptions repoints the route.

Wrapping up

The stack — ffmpeg, whisper-cli, a pinch of Ruby — covers everything from a one-off transcript to subtitling a whole folder, fully offline. Start with small.en if you want the quality/speed sweet spot, drop to quantized models when memory is tight, and reach for whisper-server when the transcript needs to live behind an API.