# sunoGPT - CLAUDE.md

This file provides guidance to Claude Code when working with the sunoGPT module.

## Overview

sunoGPT contains GPT-style language models for audio generation.

The model is a Block Causal Transformer (BCT) model. Core Concept: Unified transformer framework that processes sequences of "blocks" from left to right, where each block has context of all previous blocks. Each block has configurable properties (causality, discreteness, etc). See `utils/bct.py` for more details. This allows for flexible and modular model architecture design. A list of block types is defined in `block_types.py`.

## Testing Training

To test training locally with a specific GPU:

```bash
CUDA_VISIBLE_DEVICES=7 /home/victor/anaconda3/envs/suno_env/bin/python -u train.py \
    --debug_val_only=False \
    --master_port=12897 \
    --out_dir="/app2/suno/checkpoints" \
    --data_dir="/app2/suno/data/auk_v0" \
    --train_metas_filename="metas_v8_tr_mini.jsonl" \
    --val_metas_filename="metas_v8_val.jsonl" \
    --step_save_iters=2_000 \
    --eval_interval=1_000 \
    --eval_iters=5 \
    --batch_store_size=1 \
    \
    --n_layer=2 \
    --n_head=8 \
    --d_head=128 \
    --use_noncausal_input=False \
    \
    --learning_rate=5e-5 \
    --max_iters=28 \
    --warmup_iters=1_000 \
    --block_size=8192 \
    --batch_size=1 \
    --use_continuous_semantic_input=True \
    --use_block_type_embeddings=True \
    --output_distribution="semantic" \
    \
    --fsdp=True \
    --grad_checkpointing=True \
    --compile=False \
    \
    --wandb_log=False \
```