# 07 — STT Module

Source: `src/vocal10n/stt/`.

## Responsibilities

- Capture microphone audio at 16 kHz mono.
- Optionally cancel TTS playback echo before transcription.
- Run FasterWhisper in a sliding-window streaming mode.
- Emit partial (live) and confirmed (stable) text events.
- Filter hallucinations and obvious repetitions.
- Optionally tag speakers (lightweight diarisation).

## Files

| File | Role |
|------|------|
| `audio_capture.py` | `sounddevice` input stream, ring buffer, device enumeration. |
| `playback_aec.py` | `PlaybackTimeline` and `AdaptiveEchoCanceller` (NLMS + DTD). |
| `engine.py` | `STTEngine` — thin FasterWhisper wrapper. |
| `transcript.py` | Segment management; partial vs. confirmed. |
| `filters.py` | Hallucination filter, adjacent-dedup, short-phrase repeat filter, phonetic correction. |
| `diarizer.py` | Optional speaker-id tagging. |
| `worker.py` | Thread that pulls from the ring buffer and drives the engine. |
| `controller.py` | Public API consumed by the STT tab. |

## Streaming Strategy

The capture layer pushes 0.2 s frames into a ring buffer. The worker
runs a loop that:

1. Reads the trailing `window_seconds` (default 6.5) of audio.
2. Calls `STTEngine.transcribe()`. Returns `SegmentResult[]` with text,
   start, end, `avg_logprob`, `no_speech_prob`, and per-word confidences
   when available.
3. Passes each segment through `filters.HallucinationFilter`, then
   adjacent-dedup and short-phrase repeat suppression to cut runaway
   loops Whisper sometimes produces on noise.
4. Splits segments into:
   - **Pending**: segments whose `end` is within `confirm_threshold`
     seconds of the window tail.
   - **Confirmed**: segments older than the threshold *or* older than
     `max_segment_age` (force-flush to avoid stalling on a long
     unsplit utterance).
5. Publishes `STT_PARTIAL` for the pending tail and `STT_CONFIRMED` for
   each freshly-stable segment.

Recognition context — the term files described in
[14 — Knowledge Base and RAG](14-knowledge-base-rag.md) — is concatenated
into Whisper's `initial_prompt` up to `initial_prompt_capacity` terms.

## Hallucination Filter

`filters.py` carries multiple defences:

- **Static filter list.** Phrases loaded from `config/filters.txt`
  (e.g. boilerplate Whisper hallucinations like *"Thank you for
  watching"*). Matched segments are dropped.
- **Adjacent dedup.** A segment whose text is identical or near-identical
  to the previous confirmed segment is suppressed.
- **Short-phrase repeat suppression.** Detects N-times-repeated short
  fragments (a known Whisper failure mode on silence).
- **Phonetic index** (e.g. `stt_terms/context_gaming.txt`) — fuzzy match
  source tokens to expected domain terms to fix "似听非听" cases.

The full filter list is editable from the Knowledge Base tab via
`vocal10n.ui.widgets.filter_list_editor`.

## Playback-Aware AEC

When TTS is enabled, the played-back synthetic voice can re-enter the
microphone and be re-transcribed, producing an echo loop. The AEC layer
in `playback_aec.py` prevents this:

```mermaid
flowchart LR
    Mic([mic]) --> AEC["AdaptiveEchoCanceller<br/>(NLMS, length filter_taps)"]
    TTSPlay([TTS playback]) --> Timeline[PlaybackTimeline]
    Timeline -->|reference signal| AEC
    AEC --> DTD{"Double-talk?<br/>mic > dt_threshold &times; echo"}
    DTD -- yes --> Freeze["Freeze NLMS weights<br/>(still apply filter)"]
    DTD -- no --> Adapt[Adapt NLMS weights]
    Freeze --> Out([clean mic to Whisper])
    Adapt --> Out
```


- `PlaybackTimeline` is a thread-safe ring buffer of recently played
  audio, time-tagged at the moment `sd.play()` is called by
  `AudioPlayer`.
- `AdaptiveEchoCanceller` runs per mic chunk:
  1. Looks up the matching reference signal from `PlaybackTimeline`,
     allowing for a bulk acoustic delay estimated periodically by
     cross-correlation (capped at `aec.max_delay_ms`).
  2. Applies a block-NLMS adaptive filter of length `aec.filter_taps`.
     Adaptation step is `aec.step_size`.
  3. Runs a double-talk detector: if mic energy exceeds
     `aec.dt_threshold` × estimated-echo energy, weight adaptation is
     **frozen** while the existing filter is still applied. This keeps
     user speech intact while preventing the adapter from drifting
     during overlap.

This was added in commit `f334773` after several iterations on simpler
correlation-based subtraction proved insufficient for typical desktop
rooms.

## Speaker Diarisation

`diarizer.py` provides a lightweight per-segment speaker tagger that
prefixes confirmed segments with `[S1]`, `[S2]`, … when
`SystemState.speaker_tagging` is on. It is intentionally simple — no
embedding model — and is meant for *streaming* contexts rather than
post-hoc analysis.
