^ie:SrSSKJrJrJrJr SSKJr SSKrSSK J r SSK J r Sr SSKJrJrJrJr SrS \ R,S \ R,4S jrS \\4SjrSS\S S4SjjrS\ S \\\\\444Sjrg!\a S r\R*"S 5 Nbf=f)zs LoRA Injection Utilities for ACE-Step Provides functions for injecting LoRA adapters into the DiT decoder model. )ListTupleAnyDict)loggerN) LoRAConfigc[USS5nUbU"5nOSnSUlU$![a [R"SSS9 U$f=f![ a SUlO%![a [R"SSS9 Of=f[USS5(dD[R "S5 SUlO%![a [R"S SS9 Of=fgf=f) a%Safely call enable_input_require_grads on the decoder. This helper wraps the original enable_input_require_grads method, handling NotImplementedError gracefully and tracking whether the hook was successfully enabled. Args: self: The decoder module to call enable_input_require_grads on. (_acestep_orig_enable_input_require_gradsNTz/Failed to set _acestep_input_grads_hook_enabled)exc_infoF$_acestep_input_grads_warning_emittedzkSkipping enable_input_require_grads for decoder: get_input_embeddings is not implemented (expected for DiT)z2Failed to set _acestep_input_grads_warning_emitted)getattr!_acestep_input_grads_hook_enabled ExceptionrdebugNotImplementedErrorinfor )selforig_enable_input_require_gradsresults V/mnt/workspace/acestep.cpp/tests/../../ACE-Step-1.5/acestep/training/lora_injection.py _safe_enable_input_require_gradsrs'. 8$'# * 646FF 59D 2    LLAD     5:D 2  LLAD  tCUKK KKM  <@9  HSW %sy A &A A A A  A CACB>CB+C-B54C5CCCCC)get_peft_model LoraConfigTaskType PeftModelTFz@PEFT library not installed. LoRA training will not be available.modulereturnc|Un[U51n[USS5nUcO&[U5nXB;aOURU5 UnM7[USS5nUb4[USS5nUb"[U[R 5(aUnOUn[USS5nUb![U[R 5(aUnU$)aZUnwrap PEFT/Fabric wrappers from a model/decoder to retrieve the base DiT module. This internal helper walks the wrapper chain and returns the underlying ``nn.Module`` that can be passed to PEFT for adapter injection. Args: module: A model or decoder that may have PEFT/Fabric wrappers. Returns: The unwrapped base DiT decoder module. _forward_moduleN base_modelmodel)idr add isinstancennModule)rdecoderseen_ids next_decodernext_idr inner_model final_models r_unwrap_decoderr-MsG7 }H w(94@   \"    W ,5Jj'48  "z+ryy'I'I!G G'7D1K:k299#E#E Nc^/n[US5(arURR5HTumn[U4SjS55(dM"[ U[ R 5(dMCURT5 MV U$)zGet the list of module names in the DiT decoder that can have LoRA applied. Args: model: The AceStepConditionGenerationModel Returns: List of module names suitable for LoRA r'c3,># UH oT;v M g7fN).0projnames r )get_dit_target_modules..sU,TD4<,Ts)q_projk_projv_projo_proj)hasattrr' named_modulesanyr$r%Linearappend)r!target_modulesrr5s @rget_dit_target_modulesrBtskNui  !MM779LD&U,TUUUfbii00"))$/: r.freeze_encoderc^ SnUR5H_um nST ;nT RU5=(d [U 4SjU55nU(a SUlMHU(d U(aMXSUlMa SnSnUR5H<upXcR 5- nUR(dM*XsR 5- nM> [ R "SXg- S35 [ R "S US35 g ) zFreeze all non-LoRA parameters in the model. Args: model: The model to freeze parameters for freeze_encoder: Whether to freeze the encoder (condition encoder) )encoder text_encodervision_encoderz model.encoderlora_c3L># UHnTRUS35v M g7f).N) startswith)r3prefixr5s rr6-freeze_non_lora_parameters..s'> 8HfDOOvhaL ) )8Hs!$TFrzFrozen parameters: ,zTrainable parameters: N)named_parametersrKr> requires_gradnumelrr) r!rCencoder_prefixesparamis_lora is_encoder total_paramstrainable_params_r5s @rfreeze_non_lora_parametersrYsV--/ eT/__%56 #> 8H> ;  "&E  ::"'E 0L**, %      - -  KK%l&Ea%HIJ KK()9!(<=>r. lora_configc Z[(d [S5e[UR5nX l[ US5(aC[ US5(d2UR nX2l[R"[U5Ul[ US5(aSUl [URURURUR UR"[$R&S9n[)X$5nXPlUR+5HupgSU;dM SUlM [/SUR1555n[/S UR1555n UU US :aX- OS URURUR S .n [2R4"S 5 [2R4"S US35 [2R4"SU SSU SSS35 [2R4"SURSUR35 X 4$![a GNf=f)zInject LoRA adapters into the DiT decoder of the model. Args: model: The AceStepConditionGenerationModel lora_config: LoRA configuration Returns: Tuple of (peft_model, info_dict) zJPEFT library is required for LoRA training. Install with: pip install peftenable_input_require_gradsr is_gradient_checkpointingF)r lora_alpha lora_dropoutrAbias task_typerHc3@# UHoR5v M g7fr1)rQr3ps rr6'inject_lora_into_dit..s=* X  emm,GMw455g;??11;?8-2-=-= ,g. *w344 05G -" --$$ (("11   -- "'.>.@TT%,>JQ>N+:TU--!''%44  D KK12 KK&|A&678 KK "#3A"6b>O9PQT8UUVW KK- i 8I8I7JKL ;M   sH H*)H*)T)__doc__typingrrrrlogururrmtorch.nnr%acestep.training.configsrrpeftrrrrrkrlwarningr&r-strrBboolrYrwr2r.rrs *) /,^ WN $BII$"))$NT#Y(?d?d?@GG 3S#X GAWN NNUVWsA==BB