xiWQdZddlZddlZddlZddlmZmZmZmZm Z ddl m Z ddl m Z ddlZddlZddlmZmZ ddlmZdZGd deZdeedeeej4ffdZGddereneZGddeZdeedeeeffdZGddereneZ dede eeeefeeefffdZ!y#e$r!d Ze j.d Gd d ZYwxYw)z PyTorch Lightning DataModule for LoRA Training Handles data loading and preprocessing for training ACE-Step LoRA adapters. Supports both raw audio loading and preprocessed tensor loading. N)OptionalListDictAnyTuple)logger) safe_path)Dataset DataLoader)LightningDataModuleTFz?Lightning not installed. Training module will not be available.c eZdZy)r N)__name__ __module__ __qualname__;/mnt/workspace/ACE-Step-1.5/acestep/training/data_module.pyr r s rr cleZdZdZdefdZdedeefdZdefdZ dede ee jffd Z y ) PreprocessedTensorDatasetaDataset that loads preprocessed tensor files. This is the recommended dataset for training as all tensors are pre-computed: - target_latents: VAE-encoded audio [T, 64] - encoder_hidden_states: Condition encoder output [L, D] - encoder_attention_mask: Condition mask [L] - context_latents: Source context [T, 65] - attention_mask: Audio latent mask [T] No VAE/text encoder needed during training - just load tensors directly! tensor_dirct|}tjj|st d|||_g|_td|j }tjj|rst|d5}tj|}dddjdg}|D]1}|j|}||j j|3nltj|j D]J}|jds|dk7s|j jt||j L|j D cgc]$} tjj| s#| &c} |_t#|j t#|j k7rBt%j&dt#|j t#|j z d t%j(d t#|j d |j y#1swYxYwcc} w) a"Initialize from a directory of preprocessed .pt files. Args: tensor_dir: Directory containing preprocessed .pt files and manifest.json Raises: ValueError: If tensor_dir is not an existing directory or escapes safe root. zNot an existing directory: z manifest.jsonbaserNsamplesz.ptzSome tensor files not found: z missingzPreprocessedTensorDataset: z samples from )r ospathisdir ValueErrorr sample_pathsexistsopenjsonloadget_resolve_manifest_pathappendlistdirendswith valid_pathslenrwarninginfo) selfr validated_dir manifest_pathfmanifest raw_pathsrawresolvedps r__init__z"PreprocessedTensorDataset.__init__0s"*- ww}}]+::,GH H'')"/H 77>>- (mS)Q99Q<* Y3I 66s;'%%,,X6! ZZ0::e$o)=%%,,!!$//:1(,'8'8N'8!BGGNN1.>*?)@AOO$ & 1*) Os I*$III r4returncp t||j}tjj |r|S t|}tjj |rt jd||S t jd|y#t $rYjwxYw#t $rY3wxYw)u$Resolve a single manifest sample path to a validated absolute path. Tries ``base=tensor_dir`` first (correct for new manifests that store paths relative to the tensor directory). If the resulting path does not exist on disk, falls back to resolving against the global safe root (backward compat for legacy manifests that stored CWD-relative paths like ``./datasets/…/foo.pt``). Returns: Validated absolute path, or ``None`` if the path cannot be resolved safely. rz-Resolved legacy manifest path via safe root: z%Skipping unresolvable manifest path: N) r rrrr!rrdebugr,)r.r4childs rr&z0PreprocessedTensorDataset._resolve_manifest_path_s c8Eww~~e$ %  cNEww~~e$ CC5I % >seDE      s#7BAB) B&%B&) B54B5c,t|jSN)r+r*r.s r__len__z!PreprocessedTensorDataset.__len__s4##$$ridxc |j|}tj|dd}|d|d|d|d|d|jd id S) zLoad a preprocessed tensor file. Returns: Dictionary containing all pre-computed tensors for training cpuT) map_location weights_onlytarget_latentsattention_maskencoder_hidden_statesencoder_attention_maskcontext_latentsmetadatarErFrGrHrIrJ)r*torchr$r%)r.r@ tensor_pathdatas r __getitem__z%PreprocessedTensorDataset.__getitem__sm &&s+ zz+EM##34"#34%)*A%B&*+C&D#$56R0   rN)rrr__doc__strr7rr&intr?rrLTensorrOrrrrr#sY - 3- ^!#!(3-!F%% s tC,='> rrbatchr8c td|D}td|D}g}g}g}g}g}|D]}|d} | jd|krH| j|| jdz | jd} tj| | gd} |j | |d} | jd|kr:| j|| jdz } tj| | gd} |j | |d} | jd|krH| j|| jdz | jd} tj| | gd} |j | |d } | jd|krH| j|| jdz | jd} tj| | gd} |j | |d }|jd|kr:|j||jdz } tj|| gd}|j |tj |tj |tj |tj |tj ||Dcgc]}|d  c}d Scc}w) a%Collate function for preprocessed tensor batches. Handles variable-length tensors by padding to the longest in the batch. Args: batch: List of sample dictionaries with pre-computed tensors Returns: Batched dictionary with all tensors stacked c3@K|]}|djdyw)rErNshape.0ss r z-collate_preprocessed_batch..s"Eu!+,2215uc3@K|]}|djdyw)rGrNrWrYs rr\z-collate_preprocessed_batch..s"Mu!!34::1=ur]rEr)dimrFrIrGrHrJrK)maxrX new_zerosrLcatr'stack)rTmax_latent_lenmax_encoder_lenrEattention_masksrGencoder_attention_masksrIsampletlpadamclehseamr[s rcollate_preprocessed_batchrpsEuEENMuMMONO O $ % 88A; ',,~ ;RXXa[ICB9!,Bb!$ % 88A; ',,~ ; ! MC))S#JA.C$$S)-. 99Q?C))S#JA.C&&s+EJ ++n5++o6!&-B!C"'++.E"F ;;7,12EqQz]E2   3s4 KceZdZdZ ddededededededed effd Zdd e efd Z d e fdZ d e e fdZ xZS)PreprocessedDataModulezDataModule for preprocessed tensor files. This is the recommended DataModule for training. It loads pre-computed tensors directly without needing VAE, text encoder, or condition encoder at training time. r batch_size num_workers pin_memoryprefetch_factorpersistent_workerspin_memory_device val_splitc trt | ||_||_||_||_||_||_||_ ||_ d|_ d|_ y)atInitialize the data module. Args: tensor_dir: Directory containing preprocessed .pt files batch_size: Training batch size num_workers: Number of data loading workers pin_memory: Whether to pin memory for faster GPU transfer val_split: Fraction of data for validation (0 = no validation) N) LIGHTNING_AVAILABLEsuperr7rrsrtrurvrwrxry train_dataset val_dataset) r.rrsrtrurvrwrxry __class__s rr7zPreprocessedDataModule.__init__s_(  G  $$&$."4!2"!rstagecz|dk(s|t|j}|jdkDrt|dkDrst dt t||jz}t||z }t jjj|||g\|_ |_ y||_ d|_ yy)zSetup datasets.fitNrr_) rrryr+rarRrLutilsrN random_splitr}r~)r.r full_datasetn_valn_trains rsetupzPreprocessedDataModule.setups E>U]4T__EL~~!c,&7!&;As3|#4t~~#EFGl+e37<{{7G7G7T7T 7E"284"D$4&2"#' +rr8c B|jdk(rdn |j}|jdk(rdn |j}t|j|j d|j|j td|| }|jr|j|d<tdi|S)zCreate training dataloader.rNFT) datasetrsshufflertru collate_fn drop_lastrvrwrxr) rtrvrwdictr}rsrurprxr r.rvrwkwargss rtrain_dataloaderz'PreprocessedDataModule.train_dataloaders"&"2"2a"7$T=Q=Q&*&6&6!&;UAXAX&&((1+1    ! !*.*@*@F& '#F##rc Z|jy|jdk(rdn |j}|jdk(rdn |j}t |j|j d|j|j t||}|jr|j|d<tdi|S)zCreate validation dataloader.NrF)rrsrrtrurrvrwrxr) r~rtrvrwrrsrurprxr rs rval_dataloaderz%PreprocessedDataModule.val_dataloader+s    #"&"2"2a"7$T=Q=Q&*&6&6!&;UAXAX$$((1+1    ! !*.*@*@F& '#F##r)r_TTr=)rrrrPrQrRboolfloatr7rrr rr __classcell__rs@rrrrrs #'!#! ! !  !  !  ! !! ! ! F(8C=($$*$&$ 4$rrrceZdZdZ d deeeefdede fdZ deeeeffdZ de fdZ d e deee jffd Zy ) AceStepTrainingDatasetaDataset for ACE-Step LoRA training from raw audio. DEPRECATED: Use PreprocessedTensorDataset instead for better performance. Audio Format Requirements (handled automatically): - Sample rate: 48kHz (resampled if different) - Channels: Stereo (2 channels, mono is duplicated) - Max duration: 240 seconds (4 minutes) - Min duration: 5 seconds (padded if shorter) r max_durationtarget_sample_ratec||_||_||_||_|j |_t jdt|j dy)zInitialize the dataset.zDataset initialized with z valid samplesN) r dit_handlerrr_validate_samples valid_samplesrr-r+)r.rrrrs rr7zAceStepTrainingDataset.__init__PsV &("4!335 /D4F4F0G/HWXrr8c g}t|jD]\}}|jdd}|stjd|d4 t |}tjj|stjd|d|{|jdstjd|di|d|i}|j||S#t $rtjd|d|YwxYw) zAValidate and filter samples, resolving audio paths to safe paths. audio_pathrzSample z: Missing audio_pathz: Rejected unsafe path: z: Audio file not found: captionz: Missing caption) enumeraterr%rr,r rrrisfiler')r.validirir validateds rrz(AceStepTrainingDataset._validate_samples`s"4<<0IAvL"5J+?@A %j1 77>>),+CJ<PQ::i(+<=>98 i8F LL -10 ! +CJ<PQ s C$DDc,t|jSr=)r+rr>s rr?zAceStepTrainingDataset.__len__}s4%%&&rr@cB|j|}|d}tj|\}}||jk7r2tjj ||j}||}|j ddk(r|jdd}n|j ddkDr |ddddf}t|j|jz}|j d|kDr |ddd|f}td|jz}|j d|kr>||j dz } tjjj|d| f}||jdd|jd d |jdd|jd d |jd |jd d|jd d|jd|j d|jz |jdd|jddd|dS)zGet a single training sample.rrr_rNg@rrlyricsz[Instrumental]bpmkeyscale timesignaturedurationlanguageunknownis_instrumentalT)rrrrrrrr)audiorrrJr)r torchaudior$r transformsResamplerXrepeatrRrrLnn functionalrkr%) r.r@rirrsr resampler max_samples min_samplespaddings rrOz"AceStepTrainingDataset.__getitem__s##C(L) OOJ/ r (( ("--66r4;R;RSIe$E ;;q>Q LLA&E [[^a "1"a%LE$++d.E.EEF ;;q>K '!\k\/*E# 7 778 ;;q>K '!EKKN2GHH''++EAw<@Ezz)R0jj+;<!::i4 **X/?@zz%("JJz26!'OR!@"JJz5;;q>DD[D[3[\"JJz9=#)::.?#F %  rN)n@i)rrrrPrrrQrrrRr7rr?rLrSrOrrrrrDs $"' Yd38n%Y Y  Y 4S#X#7:''+ s+ tC,='>+ rrc `td|D}g}g}|D]}|d}|jd}||kr1||z }tjjj |d|f}|j |tj|}||krd||d|j |tj|tj||D cgc]} | d c} |D cgc]} | d c} |D cgc]} | d c} |D cgc]} | d  c} d Scc} wcc} wcc} wcc} w) z0Collate function for raw audio batches (legacy).c3@K|]}|djdyw)rr_NrW)rZris rr\z)collate_training_batch..s ?v&/''*r]rr_rNrrrJr)rrFcaptionsrrJ audio_paths) rarXrLrrrkr'onesrd) rTmax_len padded_audiorgrir audio_lenrmaskr[s rcollate_training_batchrs6???GLOwKKN w  )GHH''++EAw<@EE"zz'" w  D t$ \*++o6+015aQy\51(-.11X;.,12EqQz]E2167A,7  2.27s D' D! 9 D& D+c eZdZdZ ddeeeefdedede de de f fd Z dd e efd Z d efd Zd e efd ZxZS)AceStepDataModulezDataModule for raw audio loading (legacy). DEPRECATED: Use PreprocessedDataModule for better training performance. rrsrtrurryctrt| ||_||_||_||_||_||_||_ d|_ d|_ yr=) r{r|r7rrrsrtrurryr}r~) r.rrrsrtrurryrs rr7zAceStepDataModule.__init__sW  G   &$&$("!rrc|dk(s|a|jdkDrt|jdkDrtdt t|j|jz}t t t|j}tj||d|}||d}|Dcgc]}|j|}}|Dcgc]}|j|}}t||j|j|_ t||j|j|_ yt|j|j|j|_ d|_ yycc}wcc}w)Nrrr_)ryr+rrarRlistrangerandomrrrrr}r~) r.rrindices val_indices train_indicesr train_samples val_sampless rrzAceStepDataModule.setups> E>U]~~!c$,,&7!&;As3t||#4t~~#EFGuS%678w'%fuo ' :G H-Qa- H8CD 1t||A D%;!4#3#3T5F5F&"$:!1!143D3D$ &<LL$"2"2D4E4E&"$( /+!IDs 1E+ E0r8c ~t|j|jd|j|jt dS)NT)rsrrtrurr)r r}rsrtrurr>s rrz"AceStepDataModule.train_dataloaders8   ((-  rc|jyt|j|jd|j|jt S)NF)rsrrtrur)r~r rsrtrurr>s rrz AceStepDataModule.val_dataloadersD    #   ((-   r)r_rTrrr=)rrrrPrrrQrrRrrr7rrr rrrrs@rrrs# d38n%         0(8C=(4  *     4  rr json_pathc2t|}tjj|st d|t |dd5}t j|}dddjdi}|jdg}||fS#1swY1xYw)aLoad a dataset from JSON file. Args: json_path: Path to the JSON dataset file. Returns: Tuple of (samples list, metadata dict). Raises: ValueError: If json_path does not point to an existing file or escapes safe root. zDataset JSON file not found: rzutf-8)encodingNrJr) r rrrrr"r#r$r%)rrr1rNrJrs rload_dataset_from_jsonrs)$I 77>>) $8 DEE iw /1yy| 0xx B'Hhhy"%G H  0 /s B  B)"rPrr#rtypingrrrrrlogururacestep.training.path_safetyr rLrtorch.utils.datar r lightning.pytorchr r{ ImportErrorr,rrQrSrpobjectrrrrrrrrrrs- 332 0 5r r jAd4jAT#u||:K5LAHa$4G0Va$Pg Wg T$t*c3h@N /B+N bceDc3h4H$sTWx.4X.YQ FNNTU   sB;;#C! C!