Skip to content
Nhat Nguyen (Nate)

2026

Type by Voice: macOS dictation at the cursor

Nhat Nguyen (Nate)

SwiftNest.jsTypeScriptWebSocket

Problem

Developers and knowledge workers need to turn speech into text instantly, to prompt an AI, draft messages, or write faster without typing every word by hand. Like talking to a Jarvis: press a hotkey, speak, and the transcript lands at the cursor. Shipping that on Mac without burning CPU on-device means a thin client and a serious backend.

What I owned

End-to-end product: Swift menu-bar client (global hotkey, Opus capture, floating pill, clipboard paste insertion) and the Nest.js gateway that streams audio and returns a final transcript. Shared HTTP + WebSocket contract so later Windows / mobile clients can reuse the same API.

Type by Voice transcripts window with search, dated notes, copy buttons, and a floating record control

Architecture

The Mac app is LSUIElement only, with no Dock icon. Recording is a state machine (idle → connecting → recording → transcribing → inserting) with timeouts on every non-idle state. The pill stays on Connecting until mic frames and a server ready arrive, then Listening; pre-ready audio is buffered so leading words are not dropped.

Audio is Ogg Opus (~24 kbps) streamed over wss://…/v1/dictate after ready. The server assembles one payload per utterance and runs batch Whisper-class inference (Groq adapter). Text insertion prefers Accessibility paste (save clipboard → paste → restore); without Accessibility, the transcript stays on the clipboard with a notification.

Outcome

A downloadable unsigned macOS DMG on GitHub Releases, wired to a production API. English dictation with word-based weekly quota on the server. A real live app, not a mock.

Open on macOS

Right-click to Open does not work for this build yet. macOS Gatekeeper blocks unsigned apps, so you enable it once in System Settings.

  1. After download, try to open TypeByVoice. You will see "TypeByVoice" Not Opened.

macOS dialog saying TypeByVoice was not opened because Apple could not verify it

  1. Open System Settings → Privacy & Security, find "TypeByVoice" was blocked to protect your Mac, then click Open Anyway.

Privacy and Security with TypeByVoice blocked and an Open Anyway button

  1. Confirm when macOS asks again. After that, TypeByVoice launches normally.