Backend Architecture

The WinSTT backend is Rust (src-tauri/), compiled into the Tauri binary. It captures audio, runs voice-activity detection, loads and runs STT/TTS models on ONNX Runtime (ort), drives the LLM and cloud integrations, and owns settings, history, and the global hotkey. Everything runs in one process — there is no server and no IPC to a model host.

Module Tree

The app's own code lives under src-tauri/src/winstt/. Shared Tauri plumbing (window setup, the recording coordinator, the audio recorder toolkit) sits one level up in src-tauri/src/.

src-tauri/src/
  lib.rs                  App builder: plugins, windows, state, setup() (run())
  main.rs                 Thin binary — parse CliArgs, call run()
  cli.rs                  clap CliArgs (--start-hidden, --debug, …)
  actions.rs              Dictation coordinator — record → decode → paste
  transcription_coordinator.rs  Single thread serializing recording state
  settings.rs             Former AppSettings store, now folded into WinsttSettings.core
  audio_toolkit/          Recorder (cpal), resampler (rubato), Silero VAD
  managers/               audio · transcription · history (core)
  winstt/
    mod.rs                Module tiering + re-exports
    stt/
      backend.rs          SttBackend trait — the ONLY seam the core sees
      mod.rs              Transcriber trait, EngineKind, EngineConfig
      whisper.rs          Whisper / lite-whisper / distil (IoBinding KV-cache)
      moonshine.rs        3-graph encoder/decoder
      families.rs         CTC / RNN-T / TDT / AED engines
      resolver.rs         HuggingFace snapshot resolver + per-quant cache
      mel.rs              Log-mel feature extraction
    tts/
      kokoro.rs           Kokoro-82M @ 24 kHz (CPU-only)
      phonemize.rs        espeak-ng G2P (dlopen, process-separated)
    managers/             llm · cloud_stt · tts · wakeword · diarization ·
                          loopback · realtime · file_transcribe · download · …
    commands/             ~90 #[tauri::command] #[specta::specta] fns
    settings_schema.rs    WinsttSettings (the full typed settings tree)
    catalog.rs            Embedded STT model catalog + quant/EP policy

Manager Layer

lib.rs::initialize_core_logic() constructs every manager once and registers it as Tauri managed state (Arc<T>), so commands receive them via State<'_, Arc<T>>. The core four handle recording, transcription, model metadata, and history; the rest are WinSTT feature managers.

The STT Pipeline

A press-to-talk dictation flows through a single sequential coordinator, so the hotkey, the CLI, and a wake-word can never race the recording state:

hotkey / CLI / wake-word
  → TranscriptionCoordinator (debounced, owns Idle | Recording | Processing)
  → AudioRecordingManager.start_recording()   open mic, resample → 16 kHz mono
  → (release) stop_recording() → Vec<f32>
  → TranscriptionManager.transcribe(samples)
  → SttBackend::decode(...)  → text
  → LLM post-process (optional) → paste (Enigo) / clipboard
  → HistoryManager record + optional WAV

Two traits define the boundary

TranscriptionManager stays intentionally isolated from WinSTT's engine zoo, so the backend exposes exactly one seam — the SttBackend trait (winstt/stt/backend.rs). It routes a model id (cloud:* → cloud, in-catalog → local, else → unsupported) and resolves/loads/decodes in two phases so a model swap stays failure-atomic.

Each engine family implements the Transcriber trait (winstt/stt/mod.rs):

pub trait Transcriber: Send {
    fn kind(&self) -> EngineKind;
    fn is_ready(&self) -> bool;
    fn transcribe(&mut self, audio: &[f32], opts: &TranscribeOptions) -> SttResult<Transcription>;
    fn shutdown(&mut self) {}   // explicit ort session release (Windows DLL-unload safety)
    // …
}

Model families

The catalog spans 73 models across 11 families, each routed to an EngineKind:

Models resolve through the onnx-asr resolver (resolver.rs): a HuggingFace snapshot fetch (hf-hub) that verifies sharded .onnx_data completeness and caches per-quantization (""/fp16/fp16w/int8/int4/q4/q4f16/bnb4/uint8).

Device policy (ort)

WinSTT's Windows build ships both CPU and DirectML execution providers and picks per EngineKind rather than blanket-excluding families. Other platform builds use the providers bundled for that target and keep CPU as the fallback:

Audio, VAD & Realtime

AudioRecordingManager builds an AudioRecorder (cpal) with a rubato resampler to 16 kHz mono and a SileroVad wrapped in a SmoothedVad (onset pre-roll + hangover) for clean speech-edge detection. Every 16 kHz frame is also tapped for the audio-level visualizer and the always-on wake-word detector.

The RealtimeManager is a single daemon that powers the live preview: it try_locks the main engine and runs a non-blocking peek decode of the growing recording window (no second model in memory), feeding a stabilizer that commits text past a watermark.

Text-to-Speech

The shipped engine is Kokoro-82M (winstt/tts/kokoro.rs), 24 kHz, CPU-only (its ConvTranspose crashes on DirectML). Phonemization is espeak-ng via libloading, process-separated for GPL hygiene and installed on demand under the app-data TTS runtime. TtsManager splits text into sentences and streams tts://chunk events for gap-free playback; cloud ElevenLabs synthesis follows the same chunked path. (kitten/piper/supertonic/chatterbox are exploratory benchmark engines, not shipped.)

Threading Model

Key Dependencies

Related