hti  SSKrSSKJs Jr "SS5rS/4S\R S\R 4SjjrSSS S/4S \R S \R S \S \S\S\4 Sjjr Sr Sr Sr SS\R S\R S\R S\R S \S\S\ S\ 4SjjrSS\R S\R S\R S\S \S\4 SjjrS\R S\R S\R S\S \4 Sjrg) NcH\rSrSrSS\4SjjrS\R4SjrSr g) MomentumBuffermomentumcXlSUlg)Nrrrunning_average)selfrs Y/mnt/workspace/acestep.cpp/tests/../../ACE-Step-1.5/acestep/models/common/apg_guidance.py__init__MomentumBuffer.__init__s   update_valuecHURUR-nX-UlgNr)r r new_averages r updateMomentumBuffer.update s!mmd&:&:: +9rrN)g) __name__ __module__ __qualname____firstlineno__floatr torchTensorr__static_attributes__rr rrs!!:5<<:rrv0v1cURnURRnUS:XaUR5UR5pUR 5UR 5p[ R RRXS9nX-RUSS9U-nX- nURU5RU5URU5RU54$)NmpsdimTr$keepdim) dtypedevicetypecpudoublernn functional normalizesumto)rr dimsr' device_type v0_parallel v0_orthogonals r projectr5s HHE))..Ke2668B YY["))+    & &r & 4B7--D$-7">% # #K 0-2B2B52I2L2L[2Y YYrgg@ pred_cond pred_uncondguidance_scalemomentum_bufferetanorm_thresholdc X- nUbURU5 URnUS:aD[R"U5nUR SUSS9n [R "XU - 5n Xz-n[ XpU5upXU --n XS- U --nU$)NrT)pr$r&)rr r ones_likenormminimumr5)r6r7r8r9r:r;r1diffones diff_norm scale_factor diff_paralleldiff_orthogonalnormalized_update pred_guideds r apg_forwardrK!s  "D"t$..t$IItTI: }}TI+EF "%,Td%C"M' *==15FFFK rcXX- --$rr) cond_output uncond_output cfg_strengths r cfg_forwardrP;s ;+FG GGrcU[RRUSSS9- nU[RRUSSS9- n[R"X-SSS9nU$)z Calculate cosine similarity between two normalized tensors. Args: tensor1: First tensor [B, ...] tensor2: Second tensor [B, ...] Returns: Cosine similarity value [B, 1] r?Tr%)rlinalgrAr/)tensor1tensor2cosvalues r call_cos_tensorrV?s^ ))'q$)GGG ))'q$)GGGyy*4@H OrcURup#nURX#-U5R5nURX#-U5R5nUR5UR5:wa [ S5e[ R "X-SSS9n[ R "X-SSS9nXVS-- U-nX- nURX#U5URX#U54$)a< Decompose latent_diff into parallel and perpendicular components relative to latent_hat_uncond. Args: latent_diff: Difference tensor [B, C, ...] latent_hat_uncond: Unconditional prediction tensor [B, C, ...] Returns: projection: Parallel component perpendicular_component: Perpendicular component zBlatent_diff and latent_hat_uncond must have the same shape [n, d].r?Tr%g:0yE>)shapeviewrsize ValueErrorrr/reshape) latent_difflatent_hat_uncondntc dot_product norm_square projectionperpendicular_components r compute_perpendicular_componentrfPsGA!""15!,224K)..qua8>>@.3355]^^))K;DQK))-AqRVWKt!348IIJ)6 ??1 #%<%D%DQ1%M MMrlatentsnoise_pred_condnoise_pred_uncondsigma angle_clip apply_norm apply_clipcRURSURS:wa]URSURS-S:wa [S5eURSURS-nURUSS9nURSn Un U Rupn [U[[ 45(aN[ R"X0RURS9nURSSS5RU SS5nO[ R"U5(a|UR5S:Xa%URSSS5RU SS5nOZUR5U :XaURU SS5nO2[SU SUR35e[S[U535eUS- n XS:-S -n XU -- nXU-- nX- n[!URS U 5R#[ 5UR%S U 5R'5R#[ 55R)S S 5n[ R*"U5RXS5nU(a[ R,"U U-U*U5OU U-n[/UU5unn[ R0"U5U-nU[ R2"U5-[ R2"U5- [ R2"U5S :-UU -[ R2"U5S :*--nUU-nU(aBU[ R4R7USS S9-[ R4R7USS S9- nUU- U- nUR%XU 5R#UR5nU$)a ADG (Angle-based Dynamic Guidance) forward pass for Flow Matching. In flow matching (including SD3), sigma represents the current timestep t_curr. The predictions are velocity fields v(x_t, t). Args: latents: Current state x_t [N, T, d] where d=64 noise_pred_cond: Conditional velocity prediction v_cond [N, T, d] noise_pred_uncond: Unconditional velocity prediction v_uncond [N, T, d] sigma: Current timestep t_curr (not t_prev!) guidance_scale: Guidance strength angle_clip: Maximum angle for clipping (default: pi/6) apply_norm: Whether to normalize the result (ADG_w_norm variant) apply_clip: Whether to clip the angle (ADG_wo_clip when False) Returns: Guided velocity prediction [N, T, d] r?rzYnoise_pred_cond time dimension must be a whole-number multiple of latents time dimension.r#)r(r'z6sigma has incompatible shape. Expected scalar or size z, got z&sigma must be a number or tensor, got gMbP?rg !g !?Tr%)rXr[repeat_interleave isinstanceintrrtensorr(r'rYexpand is_tensornumel TypeErrorr)rVr0r\ contiguousclampacoscliprfcossinrRrA)rgrhrirjr8rkrlrmrepeatsr_noise_pred_textr`raweightlatent_hat_textr^r] cos_theta latent_thetalatent_theta_newprojperp latent_v_new latent_p_new latent_new noise_preds r adg_forwardrksv:}}Q?0033   #gmmA&6 6! ;xy y!''*gmmA.>>++G+; a A%O##GA!%#u&& U>>O 1a#**1a3    ;;=A JJq!Q'..q!Q7E [[]a JJq!Q'EUVWUXX^_d_j_j^klm m@e NOOa F z "T )F 77O*;";;!5K R#&&u-!!"a(33588? eK$::i(--aA6LU_uzz&<"7*jQeknzez0>OPJD$99-.@L%))$455 ,8OO ,$&(*.-599\;RVZ;Z*[\L ,J%,,"3"3OTX"3"YY\a\h\h\m\m At]n]-- J&%/J##A!,// >J rc "[UUUUUUSSS9$)z ADG with normalization - preserves the magnitude of latent predictions. This variant normalizes the final latent to maintain the same norm as the conditional prediction, which can help preserve image quality. T)rkrlrmr)rgrhrirjr8rks r adg_w_norm_forwardrs( w&(%","&"& ((rc [XX#USSS9$)z ADG without angle clipping - allows unbounded angle adjustments. This variant doesn't clip the angle, which may result in more aggressive guidance but could be less stable. F)rlrmr)rgrhrirjr8s r adg_wo_clip_forwardrs w1B>fkx} ~~r)gX%?FT)gX%?)rtorch.nn.functionalr,r-Frrr5rrKrPrVrfboolrrrrrr rs :: Z Z Z*'+ ||$     4H"NB!Q \\Q\\Q||Q << Q  Q  QQQt! ( \\(\\(||(  (  (  (0  \\ \\ ||     r