working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT working from /home/tony/Work/neon/sunoGPT [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: master_addr = h100-ord01-03-565 Overriding: out_dir = /app/suno/checkpoints Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: master_addr = h100-ord01-03-565 Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: master_addr = h100-ord01-03-565 Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: master_port = 12835 Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_metas_filename = meta_tr.jsonl Overriding: master_addr = h100-ord01-03-565 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: master_addr = h100-ord01-03-565 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: train_filename = data_tr.bin Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: semantic_codebook_weight = 4.0 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: sft_loss_scale = 0.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: train_info_filename = info_tr.json Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: sft_loss_scale = 0.0 Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: semantic_codebook_weight = 4.0 Overriding: wandb_project = chirp-v4-dpo Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn_3. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: train_metas_filename = meta_tr.jsonl Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: train_info_filename = info_tr.json Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: master_addr = h100-ord01-03-565 Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: val_info_filename = info_val.json Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 [2024-10-30_01:50:54]: Failed to import xformers. Overriding: master_addr = h100-ord01-03-565 [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 [2024-10-30_01:50:54]: Failed to import xformers. Overriding: master_port = 12835 [2024-10-30_01:50:54]: Failed to import xformers. Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 [2024-10-30_01:50:54]: Failed to import xformers. Overriding: train_filename = data_tr.bin [2024-10-30_01:50:54]: Failed to import xformers. [2024-10-30_01:50:54]: Failed to import xformers. Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: master_addr = h100-ord01-03-565 Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: master_addr = h100-ord01-03-565 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: sft_loss_scale = 0.0 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. [2024-10-30_01:50:54]: Failed to import flash_attn RMSNorm. Falling back to torch RMSNorm. Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: master_addr = h100-ord01-03-565 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: master_port = 12835 Overriding: train_info_filename = info_tr.json Overriding: out_dir = /app/suno/checkpoints Overriding: val_filename = data_val.bin Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: train_filename = data_tr.bin Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: train_metas_filename = meta_tr.jsonl Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: train_info_filename = info_tr.json Overriding: sft_loss_scale = 0.0 Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: out_dir = /app/suno/checkpoints Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: grad_clip = 0.1 Overriding: master_addr = h100-ord01-03-565 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: train_filename = data_tr.bin Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: train_metas_filename = meta_tr.jsonl Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: train_info_filename = info_tr.json Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: val_filename = data_val.bin Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: use_qk_norm = True Overriding: master_addr = h100-ord01-03-565 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: train_filename = data_tr.bin Overriding: master_addr = h100-ord01-03-565 Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: master_addr = h100-ord01-03-565 Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: sft_loss_scale = 0.0 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: wandb_project = chirp-v4-dpo Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: wandb_log = True Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: wandb_project = chirp-v4-dpo Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: semantic_codebook_weight = 4.0 Overriding: train_metas_filename = meta_tr.jsonl Overriding: master_addr = h100-ord01-03-565 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: master_addr = h100-ord01-03-565 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: sft_loss_scale = 0.0 Overriding: semantic_codebook_weight = 4.0 Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: max_iters = 915 Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: wandb_log = True Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: master_addr = h100-ord01-03-565 Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: semantic_codebook_weight = 4.0 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: activation_f = silu Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: master_addr = h100-ord01-03-565 Overriding: master_port = 12835 Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: out_dir = /app/suno/checkpoints Overriding: data_dir = /app/suno/data/dpo/30b_t5_v15 Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: train_filename = data_tr.bin Overriding: train_metas_filename = meta_tr.jsonl Overriding: train_info_filename = info_tr.json Overriding: val_filename = data_val.bin Overriding: val_metas_filename = meta_val.jsonl Overriding: val_info_filename = info_val.json Overriding: learning_rate = 5e-07 Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: min_lr = 1e-09 Overriding: do_ipo = True Overriding: dpo_beta = 40.0 Overriding: sft_loss_scale = 0.0 Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: semantic_codebook_weight = 4.0 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: last_codebook_weight = 0.5 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: semantic_codebook_weight = 4.0 Overriding: last_codebook_weight = 0.5 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: block_size = 8832 Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: warmup_iters = 50 Overriding: max_iters = 915 Overriding: grad_clip = 0.1 Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: rope_theta = 500000 Overriding: eval_interval = 4000 Overriding: eval_iters = 25 Overriding: step_save_iters = 4000 Overriding: block_size = 8832 Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: fsdp = True Overriding: use_qk_norm = True Overriding: activation_f = silu Overriding: t_text = 2560 Overriding: t_memmap = 6016 Overriding: t_audio = 6272 Overriding: use_rotary_pos_emb = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: rope_theta = 500000 Overriding: use_qk_norm = True Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: d_head = 128 Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: activation_f = silu Overriding: embed_scale_factor = 10.0 Overriding: global_every_n_layers = 1 Overriding: wandb_log = True Overriding: gradient_accumulation_steps = 1 Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: n_layer = 60 Overriding: n_head = 56 Overriding: d_head = 128 Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: grad_checkpointing = True Overriding: n_kv_head = 4 Overriding: attention_type = tao Overriding: gradient_accumulation_steps = 1 Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: batch_size = 2 Overriding: eval_loss_batch_size = 16 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: fsdp = True Overriding: sharding_strategy = full_shard Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: grad_checkpointing = True Overriding: shuffle_data = False Overriding: preload_checkpoint = /app/suno/data/dpo/models/model_30b_ft_t5.pt Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 Overriding: model_cache_loss_name = 30b_t5_bt16 Overriding: preload_strict = False Overriding: local_cache_dir = /mnt/localdisk/tmp_tony Overriding: wandb_log = True Overriding: wandb_project = chirp-v4-dpo Overriding: wandb_run_name = dpo_30b_t5_v15_beta40 NCCL version 2.21.5+cuda12.4 [2024-10-30_01:51:01]: ddp init, rank 13, local_rank 5 [2024-10-30_01:51:01]: ddp init, rank 11, local_rank 3 [2024-10-30_01:51:01]: ddp init, rank 15, local_rank 7 [2024-10-30_01:51:01]: ddp init, rank 10, local_rank 2 [2024-10-30_01:51:01]: ddp init, rank 14, local_rank 6 [2024-10-30_01:51:01]: ddp init, rank 9, local_rank 1 [2024-10-30_01:51:01]: ddp init, rank 12, local_rank 4 [2024-10-30_01:51:01]: ddp init, rank 8, local_rank 0 [2024-10-30_01:51:01]: ddp init, rank 7, local_rank 7 [2024-10-30_01:51:01]: ddp init, rank 6, local_rank 6 [2024-10-30_01:51:01]: ddp init, rank 30, local_rank 6 [2024-10-30_01:51:01]: ddp init, rank 5, local_rank 5 [2024-10-30_01:51:01]: ddp init, rank 1, local_rank 1 [2024-10-30_01:51:01]: ddp init, rank 29, local_rank 5 [2024-10-30_01:51:01]: ddp init, rank 28, local_rank 4 [2024-10-30_01:51:01]: ddp init, rank 31, local_rank 7 [2024-10-30_01:51:01]: ddp init, rank 24, local_rank 0 [2024-10-30_01:51:01]: ddp init, rank 2, local_rank 2 [2024-10-30_01:51:01]: ddp init, rank 3, local_rank 3 [2024-10-30_01:51:01]: ddp init, rank 4, local_rank 4 [2024-10-30_01:51:01]: ddp init, rank 26, local_rank 2 [2024-10-30_01:51:01]: ddp init, rank 25, local_rank 1 [2024-10-30_01:51:01]: ddp init, rank 27, local_rank 3 [2024-10-30_01:51:01]: ddp init, rank 17, local_rank 1 [2024-10-30_01:51:01]: ddp init, rank 0, local_rank 0 [2024-10-30_01:51:01]: ddp init, rank 23, local_rank 7 [2024-10-30_01:51:01]: ddp init, rank 21, local_rank 5 [2024-10-30_01:51:01]: ddp init, rank 19, local_rank 3 [2024-10-30_01:51:01]: ddp init, rank 22, local_rank 6 [2024-10-30_01:51:01]: ddp init, rank 18, local_rank 2 [2024-10-30_01:51:01]: ddp init, rank 20, local_rank 4 [2024-10-30_01:51:01]: ddp init, rank 16, local_rank 0 [2024-10-30_01:51:01]: ddp init: world size 32 ddp_rank 0. [2024-10-30_01:51:01]: loss discounts for codebooks: [0.308 0.077 0.073 0.07 0.066 0.063 0.059 0.056 0.052 0.049 0.045 0.042 0.038] [2024-10-30_01:51:06]: Total world size 32 [2024-10-30_01:51:06]: logging checkpoint here: /app/suno/checkpoints/2024-10-30_01-51-06 [2024-10-30_01:51:06]: loading data... [2024-10-30_01:51:07]: indexed 100.0% of data [2024-10-30_01:51:07]: 592 lines of data_val.bin loaded. [2024-10-30_01:51:08]: indexed 100.0% of data [2024-10-30_01:51:08]: 58,480 lines of data_tr.bin loaded. [2024-10-30_01:51:08]: train data weights: 50.0% perference_0 50.0% perference_1 [2024-10-30_01:51:08]: done loading data [2024-10-30_01:51:08]: GPU capacity: NVIDIA H100 80GB HBM3 (0) with 79.44GiB memory [2024-10-30_01:51:08]: Initializing train model from scratch trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 [2024-10-30_01:56:57]: number of parameters: 32020M [2024-10-30_01:56:57]: finish init train model trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 [2024-10-30_01:56:57]: not compiling model. [2024-10-30_01:56:57]: start loading state dict [2024-10-30_01:56:57]: verifying model args... trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 trainable params: 32,230,984,704 || all params: 32,230,984,704 || trainable%: 100.0000 [2024-10-30_01:57:04]: copying checkpoint file to local cachedir... [2024-10-30_02:00:01]: loading model state_dict on gpu 0 [2024-10-30_02:00:25]: loading model state_dict on gpu 1 [2024-10-30_02:00:50]: loading model state_dict on gpu 2 [2024-10-30_02:01:13]: loading model state_dict on gpu 3 [2024-10-30_02:01:35]: loading model state_dict on gpu 4 [2024-10-30_02:01:58]: loading model state_dict on gpu 5 [2024-10-30_02:02:21]: loading model state_dict on gpu 6 [2024-10-30_02:02:44]: loading model state_dict on gpu 7 [2024-10-30_02:03:06]: finish loading state dict [2024-10-30_02:03:06]: wrapping model in FSDP .... [2024-10-30_02:03:29]: GPU memory usage for model: 12.32GiB(15.51%) [2024-10-30_02:03:29]: applying fsdp activation checkpointing... [2024-10-30_02:03:29]: num decayed parameter tensors: 567, with 1,006,788,192 parameters [2024-10-30_02:03:29]: num non-decayed parameter tensors: 124, with 430,080 parameters [2024-10-30_02:03:29]: using fused Optimizer: False [2024-10-30_02:03:29]: model setup done [2024-10-30_02:03:29]: Validate random number: 0.9918216287004739 [2024-10-30_02:03:29]: Evaluating [2024-10-30_02:03:29]: Start the ref model loss eval loop. [2024-10-30_02:03:29]: Pre-compute cache loss [2024-10-30_02:03:29]: Start evaluating loss for slipt train [2024-10-30_02:03:29]: estimated total number of iterations 115,size of the data 58480,size of batch 16,ddp_rank is 0 [2024-10-30_02:03:49]: iter 0/115: step_time 20606.1ms, throughput 7k tok/s/node, [2024-10-30_02:12:08]: iter 25/115: step_time 19984.2ms, throughput 7k tok/s/node, [2024-10-30_02:20:27]: iter 50/115: step_time 19946.6ms, throughput 7k tok/s/node, [2024-10-30_02:28:47]: iter 75/115: step_time 20010.3ms, throughput 7k tok/s/node, [2024-10-30_02:37:06]: iter 100/115: step_time 19914.7ms, throughput 7k tok/s/node, [2024-10-30_02:41:46]: Start evaluating loss for slipt val [2024-10-30_02:41:46]: estimated total number of iterations 2,size of the data 592,size of batch 16,ddp_rank is 0 [2024-10-30_02:42:05]: iter 0/2: step_time 19351.4ms, throughput 7k tok/s/node, [2024-10-30_02:42:26]: 32 [2024-10-30_02:42:27]: Saving pre-computed cache loss: /app/suno/data/dpo/30b_t5_v15/30b_t5_bt16_cached_loss.json [2024-10-30_02:42:27]: Check if loaded correctly: train 58480 vs 58480 val 592 vs 592 [2024-10-30_02:42:27]: Validate random number: 0.014685461513078701 [2024-10-30_02:42:27]: training... [2024-10-30_02:42:29]: Eval -- [15186, 15187] Policy positive loss: tensor([2.3954]), torch.float32, Policy negative loss: tensor([2.2978]), torch.float32 [2024-10-30_02:42:29]: Eval -- Reference positive loss: tensor([2.3954]), torch.float32, Reference negative loss: tensor([2.2978]), torch.float32 [2024-10-30_02:42:29]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([-9.5367e-06]), torch.float32 [2024-10-30_02:42:34]: Eval -- [17398, 17399] Policy positive loss: tensor([3.6300]), torch.float32, Policy negative loss: tensor([3.5186]), torch.float32 [2024-10-30_02:42:34]: Eval -- Reference positive loss: tensor([3.6300]), torch.float32, Reference negative loss: tensor([3.5186]), torch.float32 [2024-10-30_02:42:34]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([1.9073e-05]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:42:39]: Eval -- [35594, 35595] Policy positive loss: tensor([2.3742]), torch.float32, Policy negative loss: tensor([2.3666]), torch.float32 [2024-10-30_02:42:39]: Eval -- Reference positive loss: tensor([2.3742]), torch.float32, Reference negative loss: tensor([2.3666]), torch.float32 [2024-10-30_02:42:39]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([-9.5367e-06]), torch.float32 [2024-10-30_02:42:44]: Eval -- [28142, 28143] Policy positive loss: tensor([2.6093]), torch.float32, Policy negative loss: tensor([2.2525]), torch.float32 [2024-10-30_02:42:44]: Eval -- Reference positive loss: tensor([2.6093]), torch.float32, Reference negative loss: tensor([2.2525]), torch.float32 [2024-10-30_02:42:44]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 WARNING: loss start index mismatch in DPO pair 13912 13912 27824 27825 3880 3842 [2024-10-30_02:42:49]: Eval -- [47844, 47845] Policy positive loss: tensor([4.1043]), torch.float32, Policy negative loss: tensor([3.9302]), torch.float32 [2024-10-30_02:42:49]: Eval -- Reference positive loss: tensor([4.1043]), torch.float32, Reference negative loss: tensor([3.9302]), torch.float32 [2024-10-30_02:42:49]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:42:54]: Eval -- [13276, 13277] Policy positive loss: tensor([3.5779]), torch.float32, Policy negative loss: tensor([2.5723]), torch.float32 [2024-10-30_02:42:54]: Eval -- Reference positive loss: tensor([3.5779]), torch.float32, Reference negative loss: tensor([2.5723]), torch.float32 [2024-10-30_02:42:54]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:42:59]: Eval -- [11148, 11149] Policy positive loss: tensor([3.5142]), torch.float32, Policy negative loss: tensor([3.4547]), torch.float32 [2024-10-30_02:42:59]: Eval -- Reference positive loss: tensor([3.5142]), torch.float32, Reference negative loss: tensor([3.4547]), torch.float32 [2024-10-30_02:42:59]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([1.9073e-05]), torch.float32, Rejected rewards: tensor([9.5367e-06]), torch.float32 [2024-10-30_02:43:05]: Eval -- [43898, 43899] Policy positive loss: tensor([3.4983]), torch.float32, Policy negative loss: tensor([3.9635]), torch.float32 [2024-10-30_02:43:05]: Eval -- Reference positive loss: tensor([3.4983]), torch.float32, Reference negative loss: tensor([3.9635]), torch.float32 [2024-10-30_02:43:05]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:43:10]: Eval -- [25866, 25867] Policy positive loss: tensor([3.7272]), torch.float32, Policy negative loss: tensor([3.8789]), torch.float32 [2024-10-30_02:43:10]: Eval -- Reference positive loss: tensor([3.7272]), torch.float32, Reference negative loss: tensor([3.8789]), torch.float32 [2024-10-30_02:43:10]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:43:15]: Eval -- [33578, 33579] Policy positive loss: tensor([2.9234]), torch.float32, Policy negative loss: tensor([3.0097]), torch.float32 [2024-10-30_02:43:15]: Eval -- Reference positive loss: tensor([2.9234]), torch.float32, Reference negative loss: tensor([3.0097]), torch.float32 [2024-10-30_02:43:15]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:43:20]: Eval -- [44504, 44505] Policy positive loss: tensor([4.1815]), torch.float32, Policy negative loss: tensor([4.3967]), torch.float32 [2024-10-30_02:43:20]: Eval -- Reference positive loss: tensor([4.1815]), torch.float32, Reference negative loss: tensor([4.3967]), torch.float32 [2024-10-30_02:43:20]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-1.9073e-05]), torch.float32, Rejected rewards: tensor([-1.9073e-05]), torch.float32 [2024-10-30_02:43:25]: Eval -- [56426, 56427] Policy positive loss: tensor([4.4424]), torch.float32, Policy negative loss: tensor([4.3527]), torch.float32 [2024-10-30_02:43:25]: Eval -- Reference positive loss: tensor([4.4424]), torch.float32, Reference negative loss: tensor([4.3527]), torch.float32 [2024-10-30_02:43:25]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([1.9073e-05]), torch.float32, Rejected rewards: tensor([-1.9073e-05]), torch.float32 [2024-10-30_02:43:30]: Eval -- [46, 47] Policy positive loss: tensor([4.0921]), torch.float32, Policy negative loss: tensor([4.7633]), torch.float32 [2024-10-30_02:43:30]: Eval -- Reference positive loss: tensor([4.0921]), torch.float32, Reference negative loss: tensor([4.7633]), torch.float32 [2024-10-30_02:43:30]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([1.9073e-05]), torch.float32, Rejected rewards: tensor([1.9073e-05]), torch.float32 [2024-10-30_02:43:35]: Eval -- [302, 303] Policy positive loss: tensor([2.8447]), torch.float32, Policy negative loss: tensor([2.6540]), torch.float32 [2024-10-30_02:43:35]: Eval -- Reference positive loss: tensor([2.8447]), torch.float32, Reference negative loss: tensor([2.6540]), torch.float32 [2024-10-30_02:43:35]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:43:40]: Eval -- [240, 241] Policy positive loss: tensor([3.9296]), torch.float32, Policy negative loss: tensor([2.6493]), torch.float32 [2024-10-30_02:43:40]: Eval -- Reference positive loss: tensor([3.9296]), torch.float32, Reference negative loss: tensor([2.6493]), torch.float32 [2024-10-30_02:43:40]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([9.5367e-06]), torch.float32 [2024-10-30_02:43:45]: Eval -- [146, 147] Policy positive loss: tensor([2.8221]), torch.float32, Policy negative loss: tensor([2.6995]), torch.float32 [2024-10-30_02:43:45]: Eval -- Reference positive loss: tensor([2.8221]), torch.float32, Reference negative loss: tensor([2.6995]), torch.float32 [2024-10-30_02:43:45]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:43:50]: Eval -- [152, 153] Policy positive loss: tensor([2.6807]), torch.float32, Policy negative loss: tensor([2.6267]), torch.float32 [2024-10-30_02:43:50]: Eval -- Reference positive loss: tensor([2.6807]), torch.float32, Reference negative loss: tensor([2.6267]), torch.float32 [2024-10-30_02:43:50]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:43:55]: Eval -- [566, 567] Policy positive loss: tensor([2.6070]), torch.float32, Policy negative loss: tensor([2.3618]), torch.float32 [2024-10-30_02:43:55]: Eval -- Reference positive loss: tensor([2.6070]), torch.float32, Reference negative loss: tensor([2.3618]), torch.float32 [2024-10-30_02:43:55]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:44:00]: Eval -- [514, 515] Policy positive loss: tensor([3.5247]), torch.float32, Policy negative loss: tensor([3.6247]), torch.float32 [2024-10-30_02:44:00]: Eval -- Reference positive loss: tensor([3.5247]), torch.float32, Reference negative loss: tensor([3.6247]), torch.float32 [2024-10-30_02:44:00]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:44:05]: Eval -- [432, 433] Policy positive loss: tensor([2.9964]), torch.float32, Policy negative loss: tensor([3.7812]), torch.float32 [2024-10-30_02:44:05]: Eval -- Reference positive loss: tensor([2.9964]), torch.float32, Reference negative loss: tensor([3.7812]), torch.float32 [2024-10-30_02:44:05]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:44:10]: Eval -- [454, 455] Policy positive loss: tensor([2.0015]), torch.float32, Policy negative loss: tensor([2.0051]), torch.float32 [2024-10-30_02:44:10]: Eval -- Reference positive loss: tensor([2.0015]), torch.float32, Reference negative loss: tensor([2.0051]), torch.float32 [2024-10-30_02:44:10]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([-9.5367e-06]), torch.float32 [2024-10-30_02:44:15]: Eval -- [492, 493] Policy positive loss: tensor([3.7076]), torch.float32, Policy negative loss: tensor([3.5377]), torch.float32 [2024-10-30_02:44:15]: Eval -- Reference positive loss: tensor([3.7076]), torch.float32, Reference negative loss: tensor([3.5377]), torch.float32 [2024-10-30_02:44:15]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([-9.5367e-06]), torch.float32 [2024-10-30_02:44:20]: Eval -- [22, 23] Policy positive loss: tensor([3.3191]), torch.float32, Policy negative loss: tensor([3.1166]), torch.float32 [2024-10-30_02:44:20]: Eval -- Reference positive loss: tensor([3.3191]), torch.float32, Reference negative loss: tensor([3.1166]), torch.float32 [2024-10-30_02:44:20]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([1.9073e-05]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:44:25]: Eval -- [486, 487] Policy positive loss: tensor([3.5860]), torch.float32, Policy negative loss: tensor([3.1908]), torch.float32 [2024-10-30_02:44:25]: Eval -- Reference positive loss: tensor([3.5860]), torch.float32, Reference negative loss: tensor([3.1908]), torch.float32 [2024-10-30_02:44:25]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_02:44:27]: loss estimation took 120.5 seconds. (100.0% of loop) [2024-10-30_02:44:27]: step 0: train loss 4.0120, val loss 3.9719 [2024-10-30_02:44:38]: iter 0: avg_loss 0.000, step_time 131307.9ms, mfu 0.0%, throughput 0k tok/s, total time 131s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 2746 2746 5492 5493 5856 5875 [2024-10-30_02:48:50]: iter 25: avg_loss 0.000, step_time 10103.7ms, mfu 133.2%, throughput 56k tok/s, total time 384s, memory 58.06GiB(73.09%) [2024-10-30_02:53:03]: iter 50: avg_loss 0.000, step_time 10074.8ms, mfu 133.6%, throughput 56k tok/s, total time 636s, memory 58.06GiB(73.09%) [2024-10-30_02:57:15]: iter 75: avg_loss 0.000, step_time 10118.4ms, mfu 133.0%, throughput 56k tok/s, total time 888s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 25670 25670 51340 51341 6008 5989 [2024-10-30_03:01:27]: iter 100: avg_loss 0.000, step_time 10084.3ms, mfu 133.5%, throughput 56k tok/s, total time 1140s, memory 58.06GiB(73.09%) [2024-10-30_03:05:39]: iter 125: avg_loss 0.000, step_time 10093.9ms, mfu 133.3%, throughput 56k tok/s, total time 1392s, memory 58.06GiB(73.09%) [2024-10-30_03:09:52]: iter 150: avg_loss 0.000, step_time 10084.2ms, mfu 133.5%, throughput 56k tok/s, total time 1645s, memory 58.06GiB(73.09%) [2024-10-30_03:14:04]: iter 175: avg_loss 0.000, step_time 10074.1ms, mfu 133.6%, throughput 56k tok/s, total time 1897s, memory 58.06GiB(73.09%) [2024-10-30_03:18:16]: iter 200: avg_loss 0.000, step_time 10105.7ms, mfu 133.2%, throughput 56k tok/s, total time 2149s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 13912 13912 27824 27825 3880 3842 [2024-10-30_03:22:31]: iter 225: avg_loss 0.000, step_time 10084.1ms, mfu 133.5%, throughput 56k tok/s, total time 2404s, memory 58.06GiB(73.09%) [2024-10-30_03:26:43]: iter 250: avg_loss 0.000, step_time 10095.6ms, mfu 133.3%, throughput 56k tok/s, total time 2656s, memory 58.06GiB(73.09%) [2024-10-30_03:30:55]: iter 275: avg_loss 0.000, step_time 10093.3ms, mfu 133.3%, throughput 56k tok/s, total time 2908s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 3020 3020 6040 6041 5648 5647 WARNING: loss start index mismatch in DPO pair 28635 28635 57270 57271 6298 6297 [2024-10-30_03:35:07]: iter 300: avg_loss 0.000, step_time 10087.8ms, mfu 133.4%, throughput 56k tok/s, total time 3161s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 13121 13121 26242 26243 5985 5995 [2024-10-30_03:39:20]: iter 325: avg_loss 0.000, step_time 10101.6ms, mfu 133.2%, throughput 56k tok/s, total time 3413s, memory 58.06GiB(73.09%) [2024-10-30_03:43:33]: iter 350: avg_loss 0.000, step_time 10095.8ms, mfu 133.3%, throughput 56k tok/s, total time 3666s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 8598 8598 17196 17197 5406 5363 [2024-10-30_03:47:45]: iter 375: avg_loss 0.000, step_time 10078.1ms, mfu 133.5%, throughput 56k tok/s, total time 3918s, memory 58.06GiB(73.09%) [2024-10-30_03:51:57]: iter 400: avg_loss 0.000, step_time 10083.0ms, mfu 133.5%, throughput 56k tok/s, total time 4170s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 10470 10470 20940 20941 5173 5120 [2024-10-30_03:56:09]: iter 425: avg_loss 0.000, step_time 10073.1ms, mfu 133.6%, throughput 56k tok/s, total time 4423s, memory 58.06GiB(73.09%) [2024-10-30_04:00:21]: iter 450: avg_loss 0.000, step_time 10096.0ms, mfu 133.3%, throughput 56k tok/s, total time 4675s, memory 58.06GiB(73.09%) [2024-10-30_04:04:34]: iter 475: avg_loss 0.000, step_time 10073.0ms, mfu 133.6%, throughput 56k tok/s, total time 4927s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 26987 26987 53974 53975 6659 6612 [2024-10-30_04:08:47]: iter 500: avg_loss 0.000, step_time 10051.2ms, mfu 133.9%, throughput 56k tok/s, total time 5180s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 6907 6907 13814 13815 5636 5642 [2024-10-30_04:12:59]: iter 525: avg_loss 0.000, step_time 10086.0ms, mfu 133.4%, throughput 56k tok/s, total time 5432s, memory 58.06GiB(73.09%) [2024-10-30_04:17:11]: iter 550: avg_loss 0.000, step_time 10097.8ms, mfu 133.3%, throughput 56k tok/s, total time 5685s, memory 58.06GiB(73.09%) [2024-10-30_04:21:24]: iter 575: avg_loss 0.000, step_time 10083.2ms, mfu 133.5%, throughput 56k tok/s, total time 5937s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 19775 19775 39550 39551 5720 5702 [2024-10-30_04:25:38]: iter 600: avg_loss 0.000, step_time 10108.4ms, mfu 133.1%, throughput 56k tok/s, total time 6192s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 22554 22554 45108 45109 5395 5331 [2024-10-30_04:29:51]: iter 625: avg_loss 0.000, step_time 10095.0ms, mfu 133.3%, throughput 56k tok/s, total time 6444s, memory 58.06GiB(73.09%) [2024-10-30_04:34:04]: iter 650: avg_loss 0.000, step_time 10079.2ms, mfu 133.5%, throughput 56k tok/s, total time 6697s, memory 58.06GiB(73.09%) [2024-10-30_04:38:16]: iter 675: avg_loss 0.000, step_time 10077.8ms, mfu 133.5%, throughput 56k tok/s, total time 6950s, memory 58.06GiB(73.09%) [2024-10-30_04:42:28]: iter 700: avg_loss 0.000, step_time 10080.5ms, mfu 133.5%, throughput 56k tok/s, total time 7202s, memory 58.06GiB(73.09%) [2024-10-30_04:46:41]: iter 725: avg_loss 0.000, step_time 10079.4ms, mfu 133.5%, throughput 56k tok/s, total time 7454s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 727 727 1454 1455 5569 5561 [2024-10-30_04:50:53]: iter 750: avg_loss 0.000, step_time 10067.4ms, mfu 133.7%, throughput 56k tok/s, total time 7706s, memory 58.06GiB(73.09%) WARNING: loss start index mismatch in DPO pair 24517 24517 49034 49035 6232 6231 [2024-10-30_04:55:05]: iter 775: avg_loss 0.000, step_time 10094.0ms, mfu 133.3%, throughput 56k tok/s, total time 7958s, memory 58.06GiB(73.09%) [2024-10-30_04:59:18]: iter 800: avg_loss 0.000, step_time 10098.6ms, mfu 133.3%, throughput 56k tok/s, total time 8211s, memory 58.06GiB(73.09%) [2024-10-30_05:03:30]: iter 825: avg_loss 0.000, step_time 10095.5ms, mfu 133.3%, throughput 56k tok/s, total time 8463s, memory 58.06GiB(73.09%) [2024-10-30_05:07:42]: iter 850: avg_loss 0.000, step_time 10095.9ms, mfu 133.3%, throughput 56k tok/s, total time 8715s, memory 58.06GiB(73.09%) [2024-10-30_05:11:54]: iter 875: avg_loss 0.000, step_time 10059.1ms, mfu 133.8%, throughput 56k tok/s, total time 8967s, memory 58.06GiB(73.09%) [2024-10-30_05:16:06]: iter 900: avg_loss 0.000, step_time 10076.4ms, mfu 133.6%, throughput 56k tok/s, total time 9219s, memory 58.06GiB(73.09%) [2024-10-30_05:18:20]: Eval -- [24352, 24353] Policy positive loss: tensor([2.8760]), torch.float32, Policy negative loss: tensor([2.3769]), torch.float32 [2024-10-30_05:18:20]: Eval -- Reference positive loss: tensor([2.8760]), torch.float32, Reference negative loss: tensor([2.3767]), torch.float32 [2024-10-30_05:18:20]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([9.5367e-06]), torch.float32, Rejected rewards: tensor([-0.0066]), torch.float32 [2024-10-30_05:18:25]: Eval -- [9114, 9115] Policy positive loss: tensor([2.3341]), torch.float32, Policy negative loss: tensor([2.2415]), torch.float32 [2024-10-30_05:18:25]: Eval -- Reference positive loss: tensor([2.3365]), torch.float32, Reference negative loss: tensor([2.2397]), torch.float32 [2024-10-30_05:18:25]: Eval -- Preference loss: tensor([6.7877e-05]), torch.float32, Chosen rewards: tensor([0.0961]), torch.float32, Rejected rewards: tensor([-0.0743]), torch.float32 [2024-10-30_05:18:30]: Eval -- [35076, 35077] Policy positive loss: tensor([2.9078]), torch.float32, Policy negative loss: tensor([2.9586]), torch.float32 [2024-10-30_05:18:30]: Eval -- Reference positive loss: tensor([2.9047]), torch.float32, Reference negative loss: tensor([2.9557]), torch.float32 [2024-10-30_05:18:30]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-0.1224]), torch.float32, Rejected rewards: tensor([-0.1158]), torch.float32 [2024-10-30_05:18:35]: Eval -- [48910, 48911] Policy positive loss: tensor([1.9385]), torch.float32, Policy negative loss: tensor([1.9495]), torch.float32 [2024-10-30_05:18:35]: Eval -- Reference positive loss: tensor([1.9365]), torch.float32, Reference negative loss: tensor([1.9488]), torch.float32 [2024-10-30_05:18:35]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-0.0798]), torch.float32, Rejected rewards: tensor([-0.0262]), torch.float32 [2024-10-30_05:18:40]: Eval -- [5636, 5637] Policy positive loss: tensor([2.8539]), torch.float32, Policy negative loss: tensor([2.8217]), torch.float32 [2024-10-30_05:18:40]: Eval -- Reference positive loss: tensor([2.8539]), torch.float32, Reference negative loss: tensor([2.8203]), torch.float32 [2024-10-30_05:18:40]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([1.9073e-05]), torch.float32, Rejected rewards: tensor([-0.0568]), torch.float32 [2024-10-30_05:18:45]: Eval -- [43634, 43635] Policy positive loss: tensor([2.3624]), torch.float32, Policy negative loss: tensor([2.0015]), torch.float32 [2024-10-30_05:18:45]: Eval -- Reference positive loss: tensor([2.3635]), torch.float32, Reference negative loss: tensor([2.0015]), torch.float32 [2024-10-30_05:18:45]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([0.0459]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_05:18:50]: Eval -- [43566, 43567] Policy positive loss: tensor([3.2611]), torch.float32, Policy negative loss: tensor([2.6236]), torch.float32 [2024-10-30_05:18:50]: Eval -- Reference positive loss: tensor([3.2634]), torch.float32, Reference negative loss: tensor([2.6222]), torch.float32 [2024-10-30_05:18:50]: Eval -- Preference loss: tensor([7.7180e-05]), torch.float32, Chosen rewards: tensor([0.0940]), torch.float32, Rejected rewards: tensor([-0.0546]), torch.float32 [2024-10-30_05:18:55]: Eval -- [31206, 31207] Policy positive loss: tensor([2.9605]), torch.float32, Policy negative loss: tensor([2.6688]), torch.float32 [2024-10-30_05:18:55]: Eval -- Reference positive loss: tensor([2.9609]), torch.float32, Reference negative loss: tensor([2.6641]), torch.float32 [2024-10-30_05:18:55]: Eval -- Preference loss: tensor([5.5042e-05]), torch.float32, Chosen rewards: tensor([0.0153]), torch.float32, Rejected rewards: tensor([-0.1879]), torch.float32 [2024-10-30_05:19:00]: Eval -- [29522, 29523] Policy positive loss: tensor([2.9387]), torch.float32, Policy negative loss: tensor([2.7367]), torch.float32 [2024-10-30_05:19:00]: Eval -- Reference positive loss: tensor([2.9381]), torch.float32, Reference negative loss: tensor([2.7367]), torch.float32 [2024-10-30_05:19:00]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-0.0240]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_05:19:05]: Eval -- [4616, 4617] Policy positive loss: tensor([2.8020]), torch.float32, Policy negative loss: tensor([2.8208]), torch.float32 [2024-10-30_05:19:05]: Eval -- Reference positive loss: tensor([2.8020]), torch.float32, Reference negative loss: tensor([2.8208]), torch.float32 [2024-10-30_05:19:05]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_05:19:10]: Eval -- [20906, 20907] Policy positive loss: tensor([3.6060]), torch.float32, Policy negative loss: tensor([3.7397]), torch.float32 [2024-10-30_05:19:10]: Eval -- Reference positive loss: tensor([3.6026]), torch.float32, Reference negative loss: tensor([3.7281]), torch.float32 [2024-10-30_05:19:10]: Eval -- Preference loss: tensor([1.8065e-05]), torch.float32, Chosen rewards: tensor([-0.1355]), torch.float32, Rejected rewards: tensor([-0.4655]), torch.float32 [2024-10-30_05:19:15]: Eval -- [49866, 49867] Policy positive loss: tensor([3.1923]), torch.float32, Policy negative loss: tensor([3.3054]), torch.float32 [2024-10-30_05:19:15]: Eval -- Reference positive loss: tensor([3.1923]), torch.float32, Reference negative loss: tensor([3.3038]), torch.float32 [2024-10-30_05:19:15]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([-0.0655]), torch.float32 [2024-10-30_05:19:20]: Eval -- [526, 527] Policy positive loss: tensor([3.2061]), torch.float32, Policy negative loss: tensor([3.3530]), torch.float32 [2024-10-30_05:19:20]: Eval -- Reference positive loss: tensor([3.2072]), torch.float32, Reference negative loss: tensor([3.3478]), torch.float32 [2024-10-30_05:19:20]: Eval -- Preference loss: tensor([3.8659e-05]), torch.float32, Chosen rewards: tensor([0.0459]), torch.float32, Rejected rewards: tensor([-0.2054]), torch.float32 [2024-10-30_05:19:25]: Eval -- [52, 53] Policy positive loss: tensor([4.2705]), torch.float32, Policy negative loss: tensor([4.1006]), torch.float32 [2024-10-30_05:19:25]: Eval -- Reference positive loss: tensor([4.2682]), torch.float32, Reference negative loss: tensor([4.0977]), torch.float32 [2024-10-30_05:19:25]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([-0.0918]), torch.float32, Rejected rewards: tensor([-0.1180]), torch.float32 [2024-10-30_05:19:30]: Eval -- [212, 213] Policy positive loss: tensor([2.4695]), torch.float32, Policy negative loss: tensor([2.5434]), torch.float32 [2024-10-30_05:19:30]: Eval -- Reference positive loss: tensor([2.4675]), torch.float32, Reference negative loss: tensor([2.5403]), torch.float32 [2024-10-30_05:19:30]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([-0.0809]), torch.float32, Rejected rewards: tensor([-0.1246]), torch.float32 [2024-10-30_05:19:35]: Eval -- [332, 333] Policy positive loss: tensor([2.4096]), torch.float32, Policy negative loss: tensor([2.9598]), torch.float32 [2024-10-30_05:19:35]: Eval -- Reference positive loss: tensor([2.4102]), torch.float32, Reference negative loss: tensor([2.9553]), torch.float32 [2024-10-30_05:19:35]: Eval -- Preference loss: tensor([5.4239e-05]), torch.float32, Chosen rewards: tensor([0.0240]), torch.float32, Rejected rewards: tensor([-0.1814]), torch.float32 [2024-10-30_05:19:40]: Eval -- [124, 125] Policy positive loss: tensor([2.7210]), torch.float32, Policy negative loss: tensor([3.0686]), torch.float32 [2024-10-30_05:19:40]: Eval -- Reference positive loss: tensor([2.7210]), torch.float32, Reference negative loss: tensor([3.0703]), torch.float32 [2024-10-30_05:19:40]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([0.]), torch.float32, Rejected rewards: tensor([0.0699]), torch.float32 [2024-10-30_05:19:45]: Eval -- [318, 319] Policy positive loss: tensor([2.0999]), torch.float32, Policy negative loss: tensor([2.3065]), torch.float32 [2024-10-30_05:19:45]: Eval -- Reference positive loss: tensor([2.1031]), torch.float32, Reference negative loss: tensor([2.3071]), torch.float32 [2024-10-30_05:19:45]: Eval -- Preference loss: tensor([9.8649e-05]), torch.float32, Chosen rewards: tensor([0.1268]), torch.float32, Rejected rewards: tensor([0.0240]), torch.float32 [2024-10-30_05:19:50]: Eval -- [16, 17] Policy positive loss: tensor([3.9718]), torch.float32, Policy negative loss: tensor([3.5809]), torch.float32 [2024-10-30_05:19:50]: Eval -- Reference positive loss: tensor([3.9706]), torch.float32, Reference negative loss: tensor([3.5797]), torch.float32 [2024-10-30_05:19:50]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-0.0481]), torch.float32, Rejected rewards: tensor([-0.0481]), torch.float32 [2024-10-30_05:19:55]: Eval -- [584, 585] Policy positive loss: tensor([3.1286]), torch.float32, Policy negative loss: tensor([3.4535]), torch.float32 [2024-10-30_05:19:55]: Eval -- Reference positive loss: tensor([3.1286]), torch.float32, Reference negative loss: tensor([3.4535]), torch.float32 [2024-10-30_05:19:55]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-9.5367e-06]), torch.float32, Rejected rewards: tensor([0.]), torch.float32 [2024-10-30_05:20:00]: Eval -- [90, 91] Policy positive loss: tensor([2.0568]), torch.float32, Policy negative loss: tensor([2.0808]), torch.float32 [2024-10-30_05:20:00]: Eval -- Reference positive loss: tensor([2.0564]), torch.float32, Reference negative loss: tensor([2.0813]), torch.float32 [2024-10-30_05:20:00]: Eval -- Preference loss: tensor([0.0002]), torch.float32, Chosen rewards: tensor([-0.0131]), torch.float32, Rejected rewards: tensor([0.0219]), torch.float32 [2024-10-30_05:20:05]: Eval -- [436, 437] Policy positive loss: tensor([2.0674]), torch.float32, Policy negative loss: tensor([1.9709]), torch.float32 [2024-10-30_05:20:05]: Eval -- Reference positive loss: tensor([2.0655]), torch.float32, Reference negative loss: tensor([1.9686]), torch.float32 [2024-10-30_05:20:05]: Eval -- Preference loss: tensor([0.0001]), torch.float32, Chosen rewards: tensor([-0.0765]), torch.float32, Rejected rewards: tensor([-0.0918]), torch.float32 [2024-10-30_05:20:10]: Eval -- [558, 559] Policy positive loss: tensor([2.6877]), torch.float32, Policy negative loss: tensor([2.0745]), torch.float32 [2024-10-30_05:20:10]: Eval -- Reference positive loss: tensor([2.6870]), torch.float32, Reference negative loss: tensor([2.0792]), torch.float32 [2024-10-30_05:20:10]: Eval -- Preference loss: tensor([0.0003]), torch.float32, Chosen rewards: tensor([-0.0284]), torch.float32, Rejected rewards: tensor([0.1879]), torch.float32 [2024-10-30_05:20:15]: Eval -- [414, 415] Policy positive loss: tensor([2.9550]), torch.float32, Policy negative loss: tensor([3.7837]), torch.float32 [2024-10-30_05:20:15]: Eval -- Reference positive loss: tensor([2.9601]), torch.float32, Reference negative loss: tensor([3.7769]), torch.float32 [2024-10-30_05:20:15]: Eval -- Preference loss: tensor([3.4815e-07]), torch.float32, Chosen rewards: tensor([0.2054]), torch.float32, Rejected rewards: tensor([-0.2710]), torch.float32 [2024-10-30_05:20:17]: loss estimation took 119.6 seconds. (1.3% of loop) [2024-10-30_05:20:17]: step 914: train loss 3.9820, val loss 3.9899 [2024-10-30_05:24:48]: saving checkpoint to /app/suno/checkpoints/2024-10-30_01-51-06 [2024-10-30_05:27:11]: saving took 413.8 seconds. (4.4% of loop) [2024-10-30_05:27:34]: iter 914: avg_loss 0.000, step_time 556793.7ms, mfu 2.4%, throughput 1k tok/s, total time 9908s, memory 58.06GiB(73.09%) [2024-10-30_05:27:34]: done. [2024-10-30_05:27:34]: all seen idxs: 58480 wandb: 🚀 View run dpo_30b_t5_v15_beta40 at: https://wandb.ai/suno/chirp-v4-dpo/runs/8xvta7dd wandb: Find logs at: wandb/run-20241030_015102-8xvta7dd/logs working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE working from /home/tony/Work/neon/sunoGPT DONE