# 08 — LLM Translation Module

Source: `src/vocal10n/llm/`.

## Responsibilities

The LLM stage performs two distinct jobs on the transcript stream:

1. **Source correction.** Takes raw STT text, applies glossary / RAG
   substitutions, and (optionally) restores punctuation.
2. **Translation.** Takes the corrected source text and produces the
   target-language version, optionally conditioned on the previous
   `context_window_size` translation pairs.

A single Qwen3-4B instance handles both calls. The model is small enough
that the per-call overhead at `n_ctx=512` is well under the
`translation.target_latency_ms` budget.

## Backends

`engine.py` wraps `llama-cpp-python` for the local GGUF backend.
`api_backend.py` wraps any OpenAI-compatible HTTP server (LM Studio,
Ollama with the OpenAI shim, vLLM, OpenAI itself) using `translation.api_*`
config keys. The active backend is selected by `translation.backend`.

The LLM tab can flip backends at runtime; the controller hot-swaps the
implementation while preserving prompt and KB state.

## Prompt Format

Prompts are ChatML-style messages ending in an `assistant` cue, which
empirically gave the most stable JSON-free outputs from Qwen3-4B-Instruct
in this size class. The prompt builder lives in `translator.py` and
includes:

- A system message with the source/target language pair and behavioural
  rules ("Translate, do not explain. Preserve proper nouns. Use the
  glossary when applicable.").
- An optional **glossary** block — either inline (when small) or
  retrieved by `rag.py` (when the glossary exceeds `rag_threshold`
  terms).
- A short **context window** of recent (source, target) pairs to keep
  pronoun and tense choices consistent across utterances.
- The current source text.

Generation defaults (`temperature=0.0`, `top_k=1`, `top_p=1.0`,
`max_tokens=64`) intentionally produce short, deterministic outputs.

## Corrector

`corrector.py` runs *before* translation when enabled:

- Replaces glossary keys in the source text with their canonical form.
- Optionally invokes the LLM with a "punctuate and clean up" prompt for
  the rare confirmed segment that arrived without punctuation.

It is a separate path so that translation calls remain short and
predictable.

## RAG

`rag.py` is the vector-retrieval implementation that activates when the
mounted glossary has more than `translation.rag_threshold` terms. It:

- Embeds glossary terms with a small local embedding model.
- On each translation call, embeds the source text and retrieves the
  top-K relevant entries to inject into the prompt instead of the full
  glossary.

This keeps context windows tractable when a domain glossary is large
(thousands of terms).

## Controller

`controller.py` ties the engine together with the dispatcher:

- Subscribes to `STT_CONFIRMED` and (optionally) the partial stream.
- Debounces partial calls by `pipeline.translation_debounce_ms`.
- Batches confirmed text using `pipeline.confirmed_batch_delay_ms` so
  that punctuation-broken Whisper segments are joined into a single
  translation request.
- Maintains the rolling translation context (source/target pairs).
- Publishes `TRANSLATION_PARTIAL` and `TRANSLATION_CONFIRMED`.
- Updates `SystemState.current_translation` and
  `SystemState.accumulated_translation` for the UI.

Manual-input mode (no STT) is supported by routing UI text input
directly to the controller through the same translation entry point;
when STT is disabled the controller does not gate output behind the
"confirmed" event.
