xi=~dZddlmZmZddlmZeGddZeGddZeGdd Zy ) z\ Training Configuration Classes Contains dataclasses for LoRA and training configurations. ) dataclassfield)ListcteZdZUdZdZeed<dZeed<dZe ed<e d Z e e ed <d Ze ed <d Zy) LoRAConfigaConfiguration for LoRA (Low-Rank Adaptation) training. Attributes: r: LoRA rank (dimension of low-rank matrices) alpha: LoRA scaling factor (alpha/r determines the scaling) dropout: Dropout probability for LoRA layers target_modules: List of module names to apply LoRA to bias: Whether to train bias parameters ("none", "all", or "lora_only") ralphag?dropoutc gdSN)q_projk_projv_projo_projr7/mnt/workspace/ACE-Step-1.5/acestep/training/configs.pyzLoRAConfig.?rdefault_factorytarget_modulesnonebiascv|j|j|j|j|jdS)z&Convert to dictionary for PEFT config.)r lora_alpha lora_dropoutrr)r r r rrselfs rto_dictzLoRAConfig.to_dicts3** LL"11II   rN)__name__ __module__ __qualname____doc__r int__annotations__r r floatrrrstrrr"rrrrr sPAsJE3OGU %7!NDID# rrceZdZUdZdZeed<dZeed<dZeed<dZ e ed <dZ e ed <dZ e ed <dZ e ed <ed Zeeed<dZe ed<dZe ed<dZe ed<dZe ed<dZy) LoKRConfigz5Configuration for LoKr (Low-Rank Kronecker) training.@ linear_dim linear_alphafactorFdecompose_both use_tucker use_scalarweight_decomposec gdSrrrrrrzLoKRConfig.4rrrr full_matrix bypass_moders_loraunbalanced_factorizationc |j|j|j|j|j|j |j |j|j|j|j|jd S)z)Convert to dictionary for LyCORIS config. r.r0r2r3r4r5r6rr8r9r:r;r=r s rr"zLoKRConfig.to_dict<sp// --kk"11//// $ 5 5"11++++||(,(E(E  rN)r#r$r%r&r.r'r(r0r2r3boolr4r5r6rrrr*r8r9r:r;r"rrrr,r,)s?JL#FC ND JJ"d" %7!NDIKKGT%*d* rr,cTeZdZUdZdZeed<dZeed<dZ eed<dZ eed <d Z eed <d Z eed <dZ eed<d Zeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<dZeed<d Zeed<dZeed <d!Zeed"<dZeed#<d$Zeed%<dZeed&<d'Zeed(<d,d*Zd+Z y))-TrainingConfigapConfiguration for LoRA training process. Training uses: - Device-aware mixed precision (bf16 on CUDA/XPU, fp16 on MPS, fp32 on CPU) - Discrete timesteps from turbo shift=3.0 schedule (8 steps) - Randomly samples one of 8 timesteps per training step: [1.0, 0.9545, 0.9, 0.8333, 0.75, 0.6429, 0.5, 0.3] Attributes: shift: Timestep shift factor (fixed at 3.0 for turbo model) num_inference_steps: Number of inference steps (fixed at 8 for turbo) learning_rate: Initial learning rate batch_size: Training batch size gradient_accumulation_steps: Number of gradient accumulation steps max_epochs: Maximum number of training epochs save_every_n_epochs: Save checkpoint every N epochs warmup_steps: Number of warmup steps for learning rate scheduler weight_decay: Weight decay for optimizer max_grad_norm: Maximum gradient norm for clipping mixed_precision: Preferred precision mode for logging/config tracking seed: Random seed for reproducibility output_dir: Directory to save checkpoints and logs g@shiftrnum_inference_stepsg-C6? learning_rate batch_sizegradient_accumulation_stepsd max_epochs save_every_n_epochs warmup_stepsg{Gz? weight_decay? max_grad_normbf16mixed_precisionFuse_fp8gradient_checkpointing*seedz ./lora_output output_dir num_workersT pin_memoryprefetch_factorpersistent_workerspin_memory_devicelog_every_n_steps val_splitNc\d|jcxkrdkstdtdy)Nr_rNz val_split must be in [0.0, 1.0).)r` ValueErrorr s r __post_init__zTrainingConfig.__post_init__s5dnn*s*?@ @+?@ @+rc4id|jd|jd|jd|jd|jd|j d|j d|jd |jd |jd |jd |jd |jd|jd|jd|jd|j |j"|j$|j&|j(|j*dS)zConvert to dictionary.rArBrCrErGrIrKrLrMrOrQrRrSrUrVrWrX)rZr[r]r^r`)rArBrCrErGrIrKrLrMrOrQrRrSrUrVrWrXrZr[r]r^r`r s rr"zTrainingConfig.to_dicts^ TZZ !4#;#;  T//  $//  *4+K+K  $//   "4#;#;  D--  D--  T//  t33  t||  %d&A&A  DII  $// 4++! " $//# $ $33"&"9"9!%!7!7!%!7!7-  r)returnN)!r#r$r%r&rAr)r(rBr'rCrErGrIrKrLrMrOrQr*rRr>rSrUrVrWrXrZr[r]r^r`rcr"rrrr@r@Ns0E5  M5J'((J!!L#L%M5!OS!GT#(D(D#N%J%KJOS##s sIuA rr@N) r& dataclassesrrtypingrrr,r@rrrrhsa )     : ! !  ! H S S  S r