Skip to content

Everything runs on the laptop

· #local-first #apple silicon

Four of the repositories here do unrelated jobs. scriba turns a voice memo into a document that says who said what. molo is a download queue for yt-dlp. vesta puts a garment on a photo of you. argano rebuilds the index of a video file that was never finalised, so the frames become playable again.

What they share is where the work happens. On the machine in front of you, with no account to create and nothing uploaded. That is a decision with an invoice attached, and the invoice is the interesting part.

The bill in seconds

ctranslate2, the engine under faster-whisper, has no Metal backend in any released version. On Apple Silicon it runs entirely on the CPU. Measured on an M4 with 16 GB, warm model cache, 6:45 of Spanish audio: 443.1 seconds of transcription on 8 threads, with the GPU idle for all of it.

There is an open pull request adding a Metal backend (OpenNMT/CTranslate2#2077). Built from that branch, the same recording takes 80.4 seconds and produces the same text: 725 words on the CPU, 724 on Metal. The branch is in no wheel, so that number costs you a C++ build you have to decide to do.

Diarization has the same shape with a better ending. pyannote on the CPU takes 211.0 seconds for that file. On Metal it takes 35.7. The default moved to Metal only after the two outputs were compared line by line: same 165 turns, same labels, boundaries matching to the millisecond.

For scale, Apple’s own on-device recogniser via yap does the same file in 24 seconds. It has no diarization at all, so it does not cover the job, but it is the number sitting on the other side of the choice.

The bill in gigabytes

Local models live on your disk. scriba wants a few gigabytes set aside, and the first run downloads the large-v3 weights before anything happens. Every job then keeps a full 16 kHz copy of the audio it processed, about 2 MB a minute, which is what scriba jobs prune --audio exists to hand back.

vesta draws a harder line. Local generation needs a Mac with Apple Silicon and Python 3.11, because CatVTON and segformer run through Metal. On any other machine the local path is simply not there and cloud mode is what remains.

Sometimes the constraint pays. molo’s desktop window is the system WebView pointed at localhost rather than a bundled Chromium, which is why the build is 44 MB instead of 200.

Things a cloud service would get for free

Getting the software to people. molo’s builds are unsigned. On macOS the first launch is a right-click and Open, on Windows it is More info then Run anyway, on Android it is allowing installs from unknown sources. The Developer ID script is in the repository and has never been run end to end, because this machine has no certificate. The Windows executable and the Android APK compile in CI and nobody has launched either on real hardware. Only the macOS build has been used for a full download.

Dependencies you are not allowed to ship. ffmpeg is not bundled with molo, for licensing reasons written down in the repository. Without it molo still runs and is limited to formats that already carry both streams, which usually means 720p. argano needs libfaad to segment AAC, the one piece that is not a Python package; without it the picture is recovered in full and the tool says the sound was left out. The browser build of argano carries its own AAC parser, written from the format specification, because the C library that does that job is under a licence the project cannot bundle. That parser is tested against 1218 real frame boundaries taken from four recordings.

Reaching your other device. molo listens on 127.0.0.1 until you turn phone access on. After that, every request that does not come from the local machine has to carry a token generated for your installation. A hosted service would have solved this with a login.

Where the line is not clean

Saying nothing calls home is a claim, so it is worth naming the places where something tried.

pyannote 4 ships a telemetry module. Its config.yaml sets metrics_enabled: true and it reports to an endpoint at pyannote.ai on every model load and every file processed. scriba turns it off in scriba/__init__.py, before any import path can reach pyannote, because the flag is read once at import time and setting it afterwards does nothing.

scriba still needs a Hugging Face account, once, to download the pyannote weights. The token goes into the Keychain rather than a file.

molo replaces its own yt-dlp by fetching a wheel from PyPI and checking it against the sha256 PyPI publishes. The README states the limit of that check in the same paragraph: it catches a truncated download, it is not a signature, and it would not stop a compromised release.

vesta is the one that keeps a cloud path on purpose. Generation runs locally, or on a free Hugging Face GPU whose quota runs out and falls back to local, or through gpt-image-1 or gemini-2.5-flash-image with an API key you paste in yourself. Those keys stay in backend/.keys.json, excluded from git, and the API never returns them.

Why the constraint is worth the invoice

“Runs locally” is a claim anybody can print on a page. It is also one of the few claims a user can check without reading a line of source: turn the network off and see what still works. argano’s browser page keeps repairing files with the network down after the first visit, because the file is read from disk by the page itself and never sent anywhere.

That is the reason the constraint is in all four repositories rather than in the marketing copy of one. A promise that breaks visibly is worth more than a promise you have to take on trust.

All posts