# Higgs TTS 3 architecture

Facts verified against the tensor structure of `bosonai/higgs-tts-3-4b`
(`model.safetensors`, 929 tensors, all BF16) and the reference Python in
`sglang-omni/sglang_omni/models/higgs_tts/`.

## Talker

Qwen3-4B autoregressive decoder consuming interleaved text and audio
tokens.

| Field | Value |
|---|---|
| Layers | 36 (`body.layers.N.*`) |
| Hidden | 2560 |
| Attention | 32 Q heads / 8 KV heads, head dim 128, QK-norm |
| MLP | 9728, SwiGLU |
| Text vocab | 151936 (Qwen tokenizer), untied embedding and head |
| Context | 8192 |

Audio side, per frame (25 fps, 40 ms):

- Embedding: `tied.embedding.modality_embeddings.0.embedding` [8208, 2560],
  8 codebooks x 1026 fused in one tensor. Input frame = sum of 8 row
  lookups at offset `id + codebook * 1026`.
- Head: `tied.head.modality_heads.0` [8208, 2560], one projection sliced
  into 8 x 1026 logit groups per step.
- Delay pattern: MusicGen-style stagger, codebook i emits i steps after
  codebook 0.

## Codec

The Higgs Audio v2 tokenizer bundled in the same checkpoint under the
prefix `tied.embedding.modality_embeddings.0.model.*` (528 tensors,
201.4M params). xcodec family: acoustic path + semantic distillation
path fused before quantization.

| Module | Tensors | Params | Role |
|---|---|---|---|
| `semantic_model` | 211 | 94.4M | HuBERT-768 encoder, reference encoding only |
| `acoustic_encoder` | 110 | 51.3M | waveform -> latent, reference encoding only |
| `acoustic_decoder` | 110 | 20.2M | latent -> 24 kHz waveform, synthesis hot path |
| `decoder_semantic` / `encoder_semantic` | 27 | 31.3M | semantic branch glue |
| `quantizer` | 64 | 2.1M | 8 x RVQ, codebooks [1024, 64], project_in/out 1024 <-> 64 |
| `fc` / `fc1` / `fc2` | 6 | 2.1M | 1024 -> 1024 / 768 / 256 projections |

Notes:

- 1024 codes + 2 special tokens = the 1026 vocab per codebook in the
  talker head.
- Quantizer EMA buffers (`cluster_size`, `embed_avg`, `inited`) are
  training state, dropped at conversion.
- Weight norm is folded everywhere except `semantic_model` pos_conv
  (2 parametrization tensors), folded at conversion.

## Acoustic decoder

DAC decoder: input conv k7 (256 -> 1024 ch), 5 upsampling blocks, output
conv k7 to mono. Each block = ConvTranspose1d (kernel = 2 x stride) +
3 residual units (conv k7 dilated + conv k1) with Snake activations
(72 alpha tensors, shape [1, C, 1]).

| Block | Kernel | Stride |
|---|---|---|
| 0 | 16 | 8 |
| 1 | 10 | 5 |
| 2 | 8 | 4 |
| 3 | 4 | 2 |
| 4 | 6 | 3 |

Total upsampling 8 x 5 x 4 x 2 x 3 = 960 = 24000 Hz / 25 fps.

GGML mapping: ConvTranspose1d via COL2IM_1D, Snake via GGML_OP_SNAKE,
dilated convs via im2col. Same op set as qwentts.cpp / acestep.cpp.

## Synthesis hot path

talker (4.45B, quantizable) -> RVQ dequant -> fc2 (1024 -> 256) ->
acoustic_decoder (20M, high precision). `semantic_model` and
`acoustic_encoder` (145M) only serve voice-clone reference encoding and
load on demand.