Run Whisper on your Mac.

whisper.cpp: the usual first stop

• Core ML is opt in. The README has you pip install ane_transformers, openai-whisper and coremltools, run ./models/generate-coreml-model.sh base.en, then rebuild with cmake -B build -DWHISPER_COREML=1. It says running the encoder on the Apple Neural Engine this way can be more than three times faster than CPU only, and that the first run is slow while the ANE service compiles the model.
• Homebrew has the project as the whisper.cpp formula, whose old name was whisper-cpp. It installs whisper-cli, and its caveats say model files are not downloaded by default, so you still fetch a .bin yourself.
• The README says whisper-cli currently runs only with 16-bit WAV files. Its own fix is ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav.

whisper.cpp is Whisper rewritten in C and C++ on top of the ggml library. Its README calls Apple silicon a first-class citizen, optimized through ARM NEON, Accelerate, Metal and Core ML, and says inference there runs fully on the GPU via Metal. The README's quick start is six commands.

git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
sh ./models/download-ggml-model.sh base.en
cmake -B build
cmake --build build -j --config Release
./build/bin/whisper-cli -f samples/jfk.wav

MLX Whisper, if you already work in Python

• The default model is mlx-community/whisper-tiny. Pre-converted models sit in the MLX Community organization on Hugging Face and download on first use.
• In Python it is two lines: import mlx_whisper, then mlx_whisper.transcribe(speech_file)["text"]. Pass word_timestamps=True for word level timestamps.

MLX is Apple's array framework for machine learning on Apple silicon. The Whisper example in the ml-explore/mlx-examples repo is published as a pip package, so there is nothing to build.

brew install ffmpeg
pip install mlx-whisper
mlx_whisper audio_file.mp3 --model mlx-community/whisper-large-v3-turbo

WhisperKit, if you are building a Mac or iPhone app

• The README recommends large-v3-v20240930_626MB across iOS and macOS for maximum accuracy, and large-v3-v20240930_turbo on macOS for maximum speed and accuracy.
• The same CLI, which the README runs from source as swift run argmax-cli, can read a microphone instead of a file with --stream, and can run a local server that implements the OpenAI Audio API.
• For an app, you add https://github.com/argmaxinc/argmax-oss-swift as a Swift package and pick the WhisperKit product.

WhisperKit is Argmax's Swift package that runs Whisper through Core ML. It now ships inside the argmax-oss-swift package, next to their text to speech and speaker diarization kits, under the MIT license. The old argmaxinc/WhisperKit address redirects there. The README lists macOS 14.0 or later and Xcode 16.0 or later as prerequisites, and a Homebrew command line app if you only want to try it.

brew install whisperkit-cli
whisperkit-cli transcribe --model large-v3-v20240930_626MB --audio-path audio.wav

The official package, and the apps built on it

• The README's model table runs from tiny at 39M parameters to large at 1550M, with turbo at 809M. OpenAI's model card describes turbo as a pruned and finetuned large-v3 with 4 decoding layers instead of 32.
• Whisper's own tokenizer lists 100 language codes. That breadth is the main reason to pick Whisper. One catch from the README: turbo is not trained for translation, so --task translate needs medium or large, and it only goes into English.
• If you would rather not open a terminal, apps wrap this up. MacWhisper, built by Jordi Bruin and friends at Good Snooze, is one. Its site says you drag a file in and get a transcript, that it transcribes Zoom, Teams, Webex, Skype and Discord calls in the background with no bots joining, and that it runs Whisper, Parakeet and other models locally on your Mac. It also lists dictation, speaker recognition and summaries through AI services you connect, and its help site has a Live Captions section.

openai-whisper is OpenAI's own implementation, in PyTorch, under the MIT license. Pick it when you want Whisper's reference behavior. The README says the codebase is expected to work with Python 3.8 to 3.11, and it needs ffmpeg on the system.

brew install ffmpeg
pip install -U openai-whisper
whisper audio.mp3 --model turbo

Which route suits whom

• One file, today, no Python: whisper.cpp from Homebrew, or an app like MacWhisper.
• A Python project: MLX Whisper. An app you ship: WhisperKit. The reference output: openai-whisper.
• You want captions while the call is still going: that is a different job, covered next.

Whisper is for files. Live captions are another job.

The four command line routes are built around a finished recording. whisper.cpp does ship a stream example that samples the microphone every half second, and its README calls it naive. WhisperKit's CLI can stream a microphone too. Both are documented as microphone tools that print text in a terminal, not as captions for what your Mac is playing, and Whisper's own translate task goes into English only.

macOS already covers half of that. Apple's guide says Live Captions transcribes spoken audio in any app and conversations around you, needs a Mac with Apple silicon, and processes the audio on device. It does not translate. Apple's separate Live Translation works in Messages, FaceTime and the Phone app, not on any audio your Mac plays.

Fraze is the other half, and it does not run Whisper. Speech is NVIDIA's Parakeet TDT 0.6B v3, int8 ONNX files run through sherpa-onnx behind a Silero voice activity detector. Translation is Mozilla's Bergamot, the Firefox Translations models, compiled to WebAssembly. On macOS 26 it can switch to Apple's SpeechAnalyzer, and to Apple's Translation framework in its on-device low latency mode. It listens to system audio, one app, or the microphone, and lines land about half a second to a second after a speaker pauses.

The trade is coverage against speed. In our own runs, word error rate in percent, lower is better: on Earnings-22 conference calls Parakeet scored 11.20, Apple SpeechAnalyzer 12.03 and Whisper large-v3-turbo 12.35. Our Whisper runs went through WhisperKit's Core ML build, and the benchmark README notes that the Open ASR Leaderboard lists the reference PyTorch turbo at about 11.1 on that set, level with Parakeet. On LibriSpeech clean, Whisper turbo at 1.93 beat Parakeet at 2.15, and Apple led at 1.82. On FLEURS read speech, 200 clips per language on a Mac mini, Whisper turbo beat Parakeet in German (5.33 against 6.21), Spanish (5.68 against 6.05), French (6.82 against 7.83) and Russian (7.48 against 9.18). Parakeet v3's model card lists 25 languages and no Japanese, Korean or Chinese. What Parakeet buys back is speed: about 0.02 times real time on that Mac, roughly 50 times faster than the audio, which leaves room to caption a long call as it happens.

Questions