whisper.cpp — C/C++ Port of OpenAI Whisper for CPU, Mobile & Edge

Key takeaways

• ●Same Whisper, no Python. Same model weights as OpenAI Whisper, reimplemented in C/C++ with GGML/GGUF quantized weights. No Python interpreter, no PyTorch, no CUDA runtime.
• ●Runs on CPU, Apple Silicon, iOS, Android, Raspberry Pi. Metal + CoreML acceleration on Macs and iPhones. SIMD instructions on x86/ARM CPUs. No GPU required.
• ●Single-binary deployment. One executable file, plus one model file (base.en is 150 MB, large-v3 quantized is 1.6 GB). Ships as easily as any command-line tool.
• ●Full accuracy at F16. Identical transcription quality to OpenAI Whisper reference. Q5_1 quantization loses less than 1% WER; Q4_0 loses 2-4%.
• ●60-80% of GPU speed on Apple Silicon. M2 Pro with CoreML processes large-v3 at 5-7x real-time. Close to CUDA GPU experience without CUDA setup.
• ●Transcription only. No built-in speaker diarization, no word-level forced alignment. Pair with pyannote.audio or a lightweight alternative if you need speaker labels.

What is whisper.cpp?

whisper.cpp is a plain C/C++ reimplementation of OpenAI Whisper's inference stack, created by Georgi Gerganov in October 2022. It uses the GGML/GGUF tensor format (the same format later adopted by llama.cpp for LLM inference) to store quantized Whisper model weights, and runs the Whisper transformer forward pass without any Python, PyTorch, or CUDA dependency.

The project's purpose is deployment reach. Vanilla OpenAI Whisper requires a full Python environment plus a CUDA-enabled GPU with 10+ GB VRAM for the large-v3 model — impractical for most laptops, all mobile devices, edge deployments, and single-binary CLI tools. whisper.cpp fills that gap: same model quality, ships as a compiled executable, runs on whatever hardware you have.

Common deployment targets: macOS/Linux/Windows CLIs, iOS and Android apps, Raspberry Pi for offline transcription boxes, embedded devices with tight memory budgets, and any environment where installing PyTorch would be disproportionate to the task.

whisper.cpp vs vanilla Whisper

Same weights, different everything else.

Install and run

Clone the repo, build, download a model, run. Total time on Mac/Linux: 2-5 minutes.

# 1. Clone and build
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
make -j

# 2. Download a model (choose one)
bash ./models/download-ggml-model.sh base.en       # 150 MB, practical default
bash ./models/download-ggml-model.sh large-v3      # 3 GB, accuracy tier
bash ./models/download-ggml-model.sh large-v3-q5_1 # 1.6 GB, quantized

# 3. Convert audio to 16 kHz mono WAV if needed
ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le audio.wav

# 4. Transcribe
./main -m models/ggml-base.en.bin -f audio.wav

# 5. With SRT output
./main -m models/ggml-large-v3.bin -f audio.wav -osrt

# 6. Word-level timestamps
./main -m models/ggml-large-v3.bin -f audio.wav -ml 1 -oj

Windows: use CMake — cmake -B build && cmake --build build --config Release. Mac with Apple Silicon: enable Metal — WHISPER_METAL=1 make -j. CUDA on Linux: yes, also supported — WHISPER_CUDA=1 make -j.

Model choice + quantization

Model size drives every other constraint: memory, speed, accuracy. Practical guidance for common deployment scenarios:

Quantization levels: F16 is the reference precision. Q5_1 typically loses less than 1% WER vs F16 while using ~40% less memory — the practical sweet spot for most desktop and laptop deployments. Q4_0 loses 2-4% WER but drops memory by 60%+ — worth it on mobile and Raspberry Pi. Q8_0 sits between F16 and Q5_1 with almost no accuracy loss but modest memory savings.

CoreML on Apple Silicon

On Apple Silicon (M1/M2/M3/M4, plus A-series in iPhones and iPads), whisper.cpp can offload the transformer encoder pass to the Apple Neural Engine (ANE) via CoreML. The decoder still runs on CPU/Metal, but the encoder — which dominates inference time — gets a 2-3x speedup.

# Build with CoreML support
WHISPER_COREML=1 make -j

# Generate CoreML model from GGML weights
./models/generate-coreml-model.sh large-v3

# The models directory now has both:
# - ggml-large-v3.bin (weights for decoder)
# - ggml-large-v3-encoder.mlmodelc (CoreML encoder for ANE)

# Run — whisper.cpp uses both automatically
./main -m models/ggml-large-v3.bin -f audio.wav

This is the fastest path to Whisper Large-v3 transcription on a Mac laptop without a discrete GPU. On an M2 Pro with CoreML, large-v3 processes at roughly 5-7x real-time — close to what a modest CUDA GPU delivers.

iOS and Android deployment

whisper.cpp compiles for both iOS (device and simulator) and Android (via the NDK). The repo includes example apps demonstrating real-time on-device transcription on modern iPhones and Android devices.

Model choice for mobile: tiny.en (75 MB) and base.en (150 MB) run smoothly on-device. medium and large models are typically too memory-intensive for mobile RAM budgets — you can fit them, but the OS will kill your app under memory pressure. For production apps, the common pattern is: ship tiny.en or base.en for real-time transcription, plus a fallback to a cloud endpoint for longer or accuracy-critical files.

Battery impact: continuous Whisper inference on-device is CPU/ANE-intensive. Expect 15-30% battery drain per hour of continuous transcription on a modern iPhone. Design accordingly — most apps trigger transcription on-demand or run it in short bursts rather than continuously.

CPU vs GPU benchmarks

Rough numbers from community benchmarks and internal testing (July 2026). All times assume a 60-minute audio file:

Numbers vary with audio quality, model quantization, and library version. Benchmark on your specific hardware before committing to whisper.cpp for production workloads.

whisper.cpp vs faster-whisper and WhisperX

Three different optimization strategies for the same Whisper models.

Rule of thumb: use whisper.cpp when you need to ship to environments without Python/CUDA (mobile, edge, single-binary CLI). Use faster-whisper when you have GPU infrastructure and want maximum throughput. Use WhisperX when you need built-in diarization + word-level alignment and are already in Python.

The diarization gap

whisper.cpp handles transcription only. No built-in speaker labeling. If your use case needs “who said what” — meeting transcripts, interviews, podcasts — you'll need to layer a diarization library on top.

The common pattern: run whisper.cpp for the transcript with timestamps, run pyannote.audio 3.1 separately for speaker turns, merge the outputs by matching timestamps. This is the same pipeline that WhisperX bundles automatically — but if you're on whisper.cpp specifically to avoid Python, adding pyannote (which is Python-based) defeats the purpose. Alternatives:

• sherpa-onnx / NeMo diarization — ONNX-based or C++-friendly diarization pipelines. Less accurate than pyannote but no Python dependency.
• Custom clustering — for known speaker counts, run whisper.cpp then cluster speaker embeddings from a small C-callable library (e.g., wespeaker exported to ONNX).
• Hosted service — offload diarization to a cloud API and keep whisper.cpp for local transcription only.

For most single-speaker use cases (voice memos, dictation, monologue transcription), the diarization gap doesn't matter. For multi-speaker workflows, plan for a diarization step from day one.

When to use a hosted service instead

whisper.cpp is the right choice for self-hosted, on-device, or edge deployments. It's not the right choice when:

• You need speaker diarization out of the box without assembling a pipeline
• You need 99 languages with translation to 133 without managing weights and post-processing
• You need bulk transcription of long files where CPU inference times add up
• You need SRT/VTT export ready for YouTube Studio without writing your own subtitle formatter
• The MLOps overhead — model updates, quantization tuning, benchmark maintenance — exceeds the value of self-hosting

VexaScribe runs Whisper Large-v3 + pyannote as a hosted service — same model quality as whisper.cpp at F16, with diarization, translation, and multi-format export bundled. 30 minutes free at signup. For most creator, corporate, and research workflows, hosted is faster to production. Self-host with whisper.cpp when the deployment constraints (offline, on-device, air-gapped, edge) require it.