# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Neon is a collection of audio generation and processing tools developed by Suno, organized into several major components:

- most active
    - **sunoGPT**: GPT-style language models for audio generation
    - **sunoData**: Data processing scripts and notebooks
    - **sunoDiff**: Diffusion models upsampling from semantic tokens to codec latents
- less active
    - **suno-codec**: High-fidelity audio compression with DAC (Descript Audio Codec) 
    - **ear**: Audio quality evaluation models
    - **hoot**: Speech transcription using Fast Conformer CTC models
    - **ditto-training**: Audio similarity models
- deprecated
    - **stable-audio-tools**: PyTorch Lightning-based training and inference for audio generation models


## Code Architecture

### Multi-Project Structure

This repository contains multiple independent Python packages, each with their own dependencies and configuration. Each major component is self-contained with its own training scripts, configs, and models.

### Common Patterns
- **Configuration-driven**: YAML/JSON configs define model architectures and training parameters
- **Weights & Biases**: Experiment tracking and logging

### Data Handling
- **WebDataset**: Used for large-scale distributed training datasets
- **AudioTools**: Audio signal processing utilities
- **Memmap datasets**: For efficient large dataset access
- **S3 storage**: Models and datasets often stored in S3 buckets (or locally sometimes)

### Model Types
- **Transformers**: GPT-style language models for audio (sunoGPT)
- **Diffusion models**: Latent diffusion (sunoDiff)
- **Autoencoders**: VAE and RVQ-based audio compression (DAC, stable-audio codecs)
- **Conformers**: CTC-based speech recognition (hoot)
- **Audio encoders**: Feature extraction models (ear, ditto-training)

## Development Environment

### Python Version

Most modules require Python 3.10 (some specify <3.11 constraint).

## Loosely tracked, needs improvement

- The dependencies are loosely tracked because of the fast development, multiple developers/projects.
- Keep improving this by keeping pyproject.toml updated as much as possible.

### Key Dependencies
- PyTorch==2.6.0+cu124
- transformers==4.44.0 for sunoGPT
- PyTorch Lightning for training frameworks
- ruff 0.7.1+ for code formatting


### Installation Patterns
Most modules use editable pip installs:
```bash
pip install -e .
pip install -e ".[dev]"  # for development dependencies
```

## Configuration Management


### VSCode Settings
- Default Python formatter: ruff
- Format on save: enabled
- Language server: Pylance

### Git Workflow
- Main branch: `main`

### Use git worktree for launching multiple agents

- Create `~/projects/[repo name]-[feature name]` for each subtask, for example:

```bash
cd /home/vibert/projects/neon
git worktree add ../neon-feature -b feature-branch
```

## Training Infrastructure

- Cluster scripts: Training scripts for SLURM clusters stored in various `sruns/` directories and `slurm_scripts/` folders.
- `wandb` for tracking experiments


## Linting and Formatting
sunoGPT use `ruff` for formatting. Run formatting every edit. 

**Use the `suno_env` conda environment for ruff commands:**
```bash
# Format code
cd sunoGPT
/home/victor/anaconda3/envs/suno_env/bin/ruff format . --config ./pyproject.toml
```

## CHANGELOG

- Keep a changelog in each module, actively update it with major changes.