# Audio Input Limits

How transcribe.cpp bounds audio input length, what a caller observes when a
limit is reached, and how to discover the limit before calling. This is both
the design reference for the library and the contract a consumer can rely on.

## The one-paragraph version

There is no single global length limit. Each model family falls into one of
three buckets. Some families chunk long audio internally and have no practical
limit; some have a hard context-window limit and reject over-length input
*before* running; some accept any length but lose accuracy past the window they
were trained on and warn when you cross it. Whatever the bucket, the library
never truncates silently: an over-length input is rejected up front with
`TRANSCRIBE_ERR_INPUT_TOO_LONG`; a transcript that runs into the context or
generation budget mid-decode returns the hard status
`TRANSCRIBE_ERR_OUTPUT_TRUNCATED` (with the partial transcript still readable
and `transcribe_was_truncated()` set); and a soft-window family logs a `WARN`
and proceeds. Every model reports its usable limit through
`transcribe_capabilities::max_audio_ms` (or, per-session,
`transcribe_session_get_limits()`) so you can check before you call. (Streaming
is the one exception to the non-OK truncation rule — see below.)

## Discovering the limit (consumer-facing)

Read `transcribe_capabilities::max_audio_ms` after loading a model:

```c
struct transcribe_capabilities caps;
transcribe_capabilities_init(&caps);
transcribe_model_get_capabilities(model, &caps);

if (caps.max_audio_ms == 0) {
    /* No practical limit. See the family notes for any absolute safety cap. */
} else {
    /* Usable ceiling in milliseconds of 16 kHz audio. Longer input is
       either rejected (hard-cap families) or warned-and-degraded
       (soft-window families) — see the family's bucket below. */
}
```

`max_audio_ms` is a single honest number derived from the model's real
metadata, not a hardcoded guess. For hard-cap families it is computed from the
decoder context window minus a **representative** prompt and a generation
reserve — the same shape as the upfront gate, but the gate uses the *exact*
prompt for each call (which shifts a little with the language hint and other
options), so `max_audio_ms` is advisory and slightly conservative, not an
exact per-call bound. For soft-window families it is the advisory window the
model was trained on. `0` means "no practical limit." It is a model-level
value at the default context; for the effective limit under a lowered
`n_ctx`, use `transcribe_session_get_limits()`.

## The three buckets

### 1. Chunked / unbounded — "just works"

| Families | Limit | Behavior |
| --- | --- | --- |
| whisper, parakeet (all variants), voxtral_realtime | none (`max_audio_ms = 0`) | Long audio is windowed internally and stitched (whisper), processed by an unbounded/streaming encoder (parakeet, voxtral_realtime), or padded if short. No practical limit; all three **ignore `n_ctx`** (it cannot lower a limit they don't have). whisper and parakeet never error on length. voxtral_realtime has one exception — its absolute `dec_max_position` cap (~2.9 h, see below): a clip past it returns `INPUT_TOO_LONG` (one-shot/batch), and a stream that reaches it flags `was_truncated`. |

Whisper slices audio into 30 s windows with prev-context stitching; parakeet's
conformer is effectively unbounded (the encoder positional table is recomputed
per run, not a fixed wall). This holds for every parakeet variant, including the
cache-aware streaming RNN-T member (`nemotron-3.5-asr-streaming-0.6b`): the
RNN-T transducer has no decoder context window, and the cache-aware path carries
constant-memory caches (`cache_last_channel` / `cache_last_time` + the decoder
LSTM state) rather than a growing KV, so it is unbounded. These families do not
need and do not have a length gate.

### 2. Hard context cap — reject up front

| Families | Limit source | Behavior |
| --- | --- | --- |
| qwen3_asr, canary_qwen, funasr_nano, granite, granite_nar, voxtral, cohere, canary | decoder context window (`dec_max_position_embeddings` / `dec_max_seq`), or the encoder positional table (`enc_pos_emb_max_len`, for cohere/canary) — all from GGUF | KV cache grows to fit, clamped to the model's true max. Over-length input is **rejected before the decode** (or before the encoder, where the encoder table is the binding limit) with `TRANSCRIBE_ERR_INPUT_TOO_LONG`. |

These families wrap an LLM-style decoder whose context window
(`audio_tokens + prompt + generation`) is the binding constraint. The number of
tokens a clip consumes is a deterministic function of its sample count
(`n_samples → mel frames → fixed subsampling → audio tokens`), so the library
computes the prefill size *before* running the encoder and rejects an
over-length clip immediately — the caller never pays for a compute pass that
cannot fit. The rejection goes through the log callback, not raw stderr.

The one case that cannot be predicted up front is the transcript itself running
long enough to exhaust the remaining budget mid-decode (rare — the output would
have to be very large for the audio length). There, the run returns the hard
status `TRANSCRIBE_ERR_OUTPUT_TRUNCATED` while keeping the partial transcript
readable (exactly like an aborted run); `transcribe_was_truncated(session)` is
also set, and a `WARN` is logged. A truncated transcript is never returned as
`TRANSCRIBE_OK` — a caller cannot mistake it for complete — and the partial
output is never discarded. In `transcribe_run_batch` this is a per-utterance
status (the whole-batch call still returns `TRANSCRIBE_OK`).

### 3. Soft window — warn and proceed

| Families | Window | Behavior |
| --- | --- | --- |
| gigaam (~25 s), sensevoice (~30 s), medasr (~400 s), moonshine (output-bound, ~48 s), moonshine_streaming (output-bound, ~17 min) | training / positional window | Any length is accepted; past the window the library emits a `WARN` (degraded accuracy is possible) and proceeds. `max_audio_ms` reports the window as advisory. |

These families have no hard architectural wall but were trained on a bounded
window; beyond it, accuracy degrades rather than failing. The library does not
reject (a caller may knowingly accept degradation) but it does warn so the
degradation is never silent. Upstream `gigaam` rejects over-length audio
outright; transcribe.cpp deliberately warns-and-proceeds instead so callers
keep control.

Moonshine is the honest edge case in this bucket: its cap is on *output*
(`max_length = 194` decode tokens, ≈ 48 s), not input, so it cannot be gated on
audio length — a dense short clip can hit it too. It is reported via
`transcribe_was_truncated()` and a `WARN` (and, offline, the hard
`TRANSCRIBE_ERR_OUTPUT_TRUNCATED` status) when the cap is reached.
`moonshine_streaming` has the same output-bound shape with a much larger window
(`dec_max_position_embeddings = 4096`, ≈ 17 min); because it also streams, its
truncation follows the streaming rule below — `stream_finalize` still returns
`TRANSCRIBE_OK` and the truncation surfaces only through
`transcribe_was_truncated()`.

## Context sizing and the `n_ctx` knob

For bucket-2 families the decoder KV cache **grows to fit the actual input**,
bucketed to avoid reallocation churn, and is **clamped to the model's true
maximum** read from GGUF metadata. This replaced earlier per-family hardcoded
walls (e.g. a fixed 2048 that ignored a much larger real context window) so the
GGUF metadata is the single source of truth for the ceiling.

`transcribe_session_params::n_ctx` lets a caller **lower** the ceiling to bound
memory:

- `0` (default): use the model's true maximum from GGUF.
- `> 0`: cap the context at this many tokens. Values above the model maximum are
  clamped down to it — the knob can only narrow, never extend past what the
  model supports.

For decoder-context-bound families, lowering `n_ctx` lowers the effective audio
limit and the upfront gate enforces the lowered value. Note that `max_audio_ms`
is a *model-level* capability, queried before any session exists, so it always
reports the model's default-context ceiling (`n_ctx == 0`); it is not re-derived
for a session that narrows `n_ctx`. A session that lowers `n_ctx` may therefore
reject audio shorter than the advertised `max_audio_ms`.

Encoder-bound families are different. For cohere and canary, the input-audio
limit is the encoder positional table, while `n_ctx` only bounds the decoder
self-KV / output budget. In those families `transcribe_session_get_limits()`
reports a smaller `effective_n_ctx` and `max_kv_bytes` when `n_ctx` is lowered,
but `effective_max_audio_ms` stays pinned to the encoder input bound.

**Chunked / unbounded families (bucket 1) ignore `n_ctx` entirely.** whisper,
parakeet, and voxtral_realtime have no lowerable context ceiling, so a non-zero
`n_ctx` is a documented no-op and `transcribe_session_get_limits()` keeps
reporting them unbounded. voxtral_realtime is the subtle case: it *does* have an
absolute decoder position cap (`dec_max_position`, ~2.9 h of audio), but that is
the model's true RoPE wall — not a memory ceiling a caller can lower — so
`n_ctx` does not narrow it (its decoder KV is a constant-memory sliding ring;
there is nothing for `n_ctx` to bound). A clip past the absolute cap is rejected
with `TRANSCRIBE_ERR_INPUT_TOO_LONG` (one-shot and batch) or surfaced via
`transcribe_was_truncated()` (streaming) regardless of `n_ctx`.

## What a caller observes — summary

| Situation | Status | Log | Result |
| --- | --- | --- | --- |
| Input within limit | `TRANSCRIBE_OK` | — | full transcript |
| Over-length, hard-cap family | `TRANSCRIBE_ERR_INPUT_TOO_LONG` | `ERROR` via callback | no transcript (rejected before the decode) |
| Generation ran long mid-decode | `TRANSCRIBE_ERR_OUTPUT_TRUNCATED` | `WARN` via callback | partial transcript readable; `transcribe_was_truncated() == true` |
| Over-window, soft-window family | `TRANSCRIBE_OK` | `WARN` via callback | full transcript (accuracy may be degraded) |
| Chunked / unbounded family | `TRANSCRIBE_OK` | — | full transcript |
| Cache/graph allocation failed | `TRANSCRIBE_ERR_OOM` | `ERROR` via callback | no transcript (no silent context shrink) |

In `transcribe_run_batch`, `INPUT_TOO_LONG` and `OUTPUT_TRUNCATED` are
per-utterance statuses (`transcribe_batch_status(session, i)`); the whole-batch
call returns `TRANSCRIBE_OK`.

`transcribe_was_truncated(session)` is reset at the top of every
`transcribe_run`, `transcribe_run_batch`, and `transcribe_stream_begin` (the same
lifecycle as `transcribe_was_aborted`).

## Streaming is the exception

`TRANSCRIBE_ERR_OUTPUT_TRUNCATED` is an **offline-only** status
(`transcribe_run` / `transcribe_run_batch`). An active stream is incremental
and has its own terminal-state machine (`transcribe_stream_*`,
IDLE/ACTIVE/FINISHED/FAILED), and `stream_feed` / `stream_finalize` return the
status of *that step*, not a verdict on the whole transcript. So when a
streaming decode reaches its context cap (e.g. `voxtral_realtime` at its
absolute position limit — hours of continuous audio, or `moonshine_streaming`
at its output window), the stream does **not** fail and `stream_finalize`
returns `TRANSCRIBE_OK`; the truncation is surfaced through
`transcribe_was_truncated(session)` and a `WARN`. This is deliberate:
forcing a stream into a failed terminal state on truncation would discard the
committed text the caller has been consuming. A streaming caller that needs to
detect truncation should check `transcribe_was_truncated()` after finalize.

## Design notes (for maintainers)

- The upfront gate and `max_audio_ms` share a shape for decoder-context-bound
  families but differ in precision:
  `max_audio_ms ≈ (ceiling − representative_prompt − generation_reserve) / tokens_per_ms`,
  where `ceiling` is the GGUF max. The gate uses the *exact* per-call prompt, so
  `max_audio_ms` is advisory/slightly-conservative, not the exact bound. The
  per-model rate constants live in `transcribe_model::LimitsBasis`, which
  `transcribe_session_get_limits()` reads to recompute the effective limit at a
  lowered `n_ctx`. For encoder-bound families, the session query keeps the audio
  bound from `max_audio_ms` while still reporting the lowered decoder KV budget.
- `n_ctx` is a session context/KV cap, never a way to extend the model:
  `effective_n_ctx = n_ctx == 0 ? model_max : min(n_ctx, model_max)`.
- Allocation failures (KV cache, compute graph) return `TRANSCRIBE_ERR_OOM`
  with a logged diagnostic, never `GGUF`/`BACKEND`, and the library does not
  silently shrink the context and retry.
- Length/context messages must go through the shared `transcribe_log` helpers,
  never `fprintf(stderr)` — a consumer that installed a log sink must see them.
- New families pick a bucket at port time and fill `max_audio_ms` accordingly
  (`0` for chunked/unbounded). The model-card template carries an "Input limits"
  block that must be populated with the real number.