# gguf : ACE-Step captioner and transcriber GGUF builder

Converts the ACE-Step v1.5 official captioner and transcriber models
(Qwen2.5-Omni-7B finetunes) from safetensors to GGUF, then produces
a full quant range for llama.cpp multimodal inference.

## Sources

- acestep-captioner  : https://huggingface.co/ACE-Step/acestep-captioner
- acestep-transcriber: https://huggingface.co/ACE-Step/acestep-transcriber

Both are Qwen2.5-Omni-7B conditional generation models finetuned for
music captioning (structured genre/instrument/structure descriptions)
and music transcription (time-aligned lyrics extraction).

## Outputs (in ./models/)

Per model :

- <name>-BF16.gguf           : text LM (thinker), 15.2 GB
- <name>-Q8_0.gguf           : Q8_0 quant, ~8 GB
- <name>-Q6_K.gguf           : Q6_K quant, ~6.2 GB
- <name>-Q5_K_M.gguf         : Q5_K_M quant, ~5.5 GB
- <name>-Q4_K_M.gguf         : Q4_K_M quant, ~4.7 GB
- mmproj-<name>-BF16.gguf    : vision + audio encoders, 1.3 GB

The mmproj stays in BF16 : it is a small encoder and audio feature
quality matters more than disk footprint.

## Build chain

1. venv : python3 -m venv .venv && source .venv/bin/activate
2. deps : pip install torch==2.11.0+cu130 --index-url https://download.pytorch.org/whl/cu130
          pip install -r /mnt/workspace/llama.cpp/requirements/requirements-convert_hf_to_gguf.txt
          pip install --force-reinstall torch==2.11.0+cu130 --index-url https://download.pytorch.org/whl/cu130
3. convert : source .venv/bin/activate && python convert.py
4. quantize: ./quantize.sh

Requires llama.cpp built at /mnt/workspace/llama.cpp/build/bin/ with
llama-quantize and llama-mtmd-cli targets :

    cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 \
        -DLLAMA_BUILD_TOOLS=ON -DCPPHTTPLIB_OPENSSL_SUPPORT=OFF
    cmake --build build -j --target llama-quantize llama-mtmd-cli llama-cli llama-server

## Inference

    /mnt/workspace/llama.cpp/build/bin/llama-mtmd-cli \
        -m models/acestep-captioner-Q5_K_M.gguf \
        --mmproj models/mmproj-acestep-captioner-BF16.gguf \
        --audio /path/to/track.wav \
        -p "Describe this music in detail."

## Notes

The llama.cpp converter needs a small compatibility patch to accept
the Qwen2_5OmniForConditionalGeneration architecture string used by
transformers 5.5+ (old name Qwen2_5OmniModel). The patch adds one
string to two @ModelBase.register decorators in convert_hf_to_gguf.py.