How Humla runs Whisper on-device on Apple Silicon

Humla's pitch is that your meetings can stay on your Mac. That's only worth saying if it's actually true, so this post walks through how the on-device path works: capture, the Whisper runtime, speaker labelling, and where anything touches the network. It's all open source, so you can check every claim here against the code.

Two audio streams, no bot

Most meeting tools send a bot into the call. Humla records your own Mac instead. A small Swift sidecar (audio-capture) captures two things at once: your microphone through AVAudioEngine, and the system's output audio through ScreenCaptureKit. Nothing joins the meeting, so the other side sees nothing in the participant list. There's more on the no-bot approach here.

The important detail is that these are kept as two separate streams end to end — there's no mixdown. The mic writer and the system-audio writer each produce their own WAV files. This keeps "you versus them" clean: on a remote call, everything on the mic stream is you, and everything on the system stream is the people you're talking to. That fact alone does most of the work of labelling the transcript, before any diarization model runs.

Both streams are converted to 16 kHz mono — the format Whisper wants — and sliced into chunks at natural speech pauses (a simple voice-activity gate: a chunk closes after ~500 ms of silence, bounded to 1–15 seconds) rather than on a fixed timer. Each chunk is transcribed as it lands, so the transcript fills in during the meeting.

Whisper on the GPU

Transcription runs through whisper-rs 0.16 with its metal feature. That crate wraps whisper.cpp and compiles it with Metal support, so inference runs on the GPU rather than the CPU. On Apple Silicon the GPU and CPU share the same unified memory, which means the model weights don't have to be copied across a PCIe bus the way they would on a discrete GPU — the model loads once and stays resident, and each chunk reuses it. (If Metal ever fails to initialise, whisper.cpp falls back to a CPU/BLAS path — slower, but it still works.)

This is also the honest answer to "why Apple Silicon only": there is no Intel build. Whisper leans on the Metal GPU, and the speaker-diarization models (below) run as CoreML on the Neural Engine — both tuned for Apple's own silicon, and a separate Intel path wasn't worth it for a solo project. The requirement is macOS 13 or newer on Apple Silicon.

The app bundles three quality presets, which really just bundle two whisper.cpp knobs that move together — the sampling strategy and the no-speech threshold:

Fast — greedy decoding. Snappiest, but borderline clauses can get dropped.
Balanced — beam search, width 3.
Quality — beam search, width 5, with a loose no-speech threshold so almost nothing is silently discarded. This is the default, and it's the right choice for meetings.

In my own testing these land somewhere around 5×, 3× and 2× realtime respectively on an M-series chip — rough numbers, not a benchmark, but even the slowest preset keeps up with a live conversation.

The models, including a Norwegian one

The default model is large-v3-turbo (a q5-quantized GGML build, about half a gigabyte). It's multilingual, handles roughly 100 languages, and is fast enough to run during the meeting. There are heavier options in the picker — full large-v3 and large-v2 for the highest accuracy on dense or accented audio, and a smaller medium for older machines.

There's also a language-specific one: NB Whisper Large, fine-tuned on Norwegian by Nasjonalbiblioteket (the National Library of Norway). It's noticeably better on Norwegian and noticeably worse on everything else, so it isn't a general option — you select it as a per-language override. Humla's transcription config is per-language: you can route Norwegian to NB Whisper, English to a cloud engine, and leave everything else on the default, and it resolves the right provider per chunk based on the note's language.

Models are downloaded on demand from Hugging Face the first time you pick them, into ~/Library/Application Support. After that they're local files.

Diarization runs offline too

"Who said what" is handled by a second Swift sidecar, speaker-diarize, built on the FluidAudio package (Apache-2.0). It runs CoreML models on the Neural Engine — no network, no cloud. Two engines are available: Community-1 (segmentation plus VBx clustering with PLDA scoring) and NVIDIA's Sortformer (an end-to-end transformer with a fixed four-speaker cap, run in batch mode), which handles rapid back-and-forth better.

Diarization runs once, after you stop recording, over the saved full-length WAV — not live. That's a deliberate trade-off: clustering is more accurate when it can see the whole conversation. For a remote call, the mic stream is simply labelled "You" and the diarizer only runs on the system stream. For an in-person meeting there's just the mic stream, so it diarizes that to separate the people in the room. The models are small (~30 MB) and, like the Whisper models, download once on first use.

What actually touches the network

Here is the complete list of things that leave your machine, and when:

One-time model downloads. Whisper models from Hugging Face, diarization models from Hugging Face. After they're on disk, the local path is fully offline — it works on a plane.
Cloud transcription, only if you choose it. You can bring your own OpenAI, Deepgram, or Groq key for speed or accuracy. It's your account and your bill; keys live in the macOS Keychain, and this is entirely opt-in.
Summaries. These can run locally against any OpenAI-compatible server on your own machine (Ollama, LM Studio, llama.cpp) — that traffic is localhost, not the internet — or against a cloud model with your key.
Update checks on launch, against GitHub.

That's it. Notes live in a local SQLite database, there's no telemetry, and nothing about your audio phones home. If you want the reasoning behind that design, I wrote more on the privacy page.

The honest trade-offs

The local path isn't free of downsides. The one-time download is real — half a gigabyte for the default model, ~1.1 GB for NB Whisper. Cloud engines can still beat local Whisper on some hard audio and return results faster, which is exactly why the bring-your-own-key option exists rather than being hidden. Diarization being post-stop means speaker labels appear after you finish, not during. And it's Apple Silicon only.

If those trade-offs are fine for you, you get meeting notes that never leave your Mac. For most of my own calls, they are.

Try it

There's an in-browser demo on the local page that runs Whisper client-side via transformers.js — nothing uploads. Fair warning: it uses whisper-base.en (~75 MB) to keep the download small — far smaller than the model the app ships — so treat it as a taste of the idea, not a measure of real quality.

If you want the actual thing, download Humla — it's free and MIT-licensed, the source is on GitHub, and there's an honest side-by-side with Granola and a rundown of why it's Mac-first if you're weighing it up.