+ jcRt^RIt^RIHt^RIHt^RIt^RIHt^RI HuH t ^RI H t^RIHt^RIHt^RIHt^RIHtHtHtHt^R IHt^R IHt]!R R ]44t]!R R]44t]!RR]44t !RR]PB4t"!RR]PB4t#!RR]PB4t$!RR]PB4t%!RR]PB4t&!RR]PB4t'!RR]PB4t(!RR ]PB4t)]!R!R"]44t*]!R#R$7!R%R&]*44t+R&R".t,R#)'zTransformers Xcodec model.N) dataclass) lru_cache)initialization)conv1d_output_length)PreTrainedAudioTokenizerBase)Unpack) ModelOutputTransformersKwargsauto_docstringcan_return_tuple) AutoModel XcodecConfigc8a]tRt^&toRtRtRtV3RltRtVt R#) XcodecOutputaW Args: audio_codes (`torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)`, *optional*): Discrete code indices computed using `model.encode`. audio_values (`torch.FloatTensor` of shape `(batch_size, channels, num_samples)`, *optional*) Decoded audio values obtained using the decoder part of Xcodec. Ncv<V^8dQh/S[PR,;R&S[PR,;R&#)N audio_codes audio_values)torch LongTensor FloatTensor)format __classdict__s"k/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/xcodec/modeling_xcodec.py __annotate__XcodecOutput.__annotate__&s5!!D(/##d*1) __name__ __module__ __qualname____firstlineno____doc__rr__annotate_func____static_attributes____classdictcell__rs@rrr&s,0K-1Lrrc4a]tRt^4toRtRtV3RltRtVtR#)XcodecEncoderOutputz Args: audio_codes (`torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)`, *optional*): Discrete code indices computed using `model.encode`. NcH<V^8dQh/S[PR,;R&#)rNr)rr)rrs"rr XcodecEncoderOutput.__annotate__4s!!D(/rr) rr r!r"r#rr$r%r&r's@rr)r)4s ,0Krr)c4a]tRt^?toRtRtV3RltRtVtR#)XcodecDecoderOutputz Args: audio_values (`torch.FloatTensor` of shape `(batch_size, channels, num_samples)`, *optional*): Decoded audio values obtained using the decoder part of Xcodec. NcH<V^8dQh/S[PR,;R&#)rNrrr)rrs"rr XcodecDecoderOutput.__annotate__?s##d*1rr) rr r!r"r#rr$r%r&r's@rr-r-?s .2Lrr-cTaa]tRt^JtoRtV3RlV3RlltV3RlRltRtVtV;t #)XcodecResidualUnitzFResidual block for SemanticEncoder and SemanticDecoder used in Xcodec.c2<V^8dQhRS[RS[RS[RS[/#)rconfig in_channels out_channelsdilationrint)rrs"rrXcodecResidualUnit.__annotate__Ms0oo|o#oSo\_orc 0<\SV`4\P!4VnVP ^, ^,V,p\P !VVVP ^VV^RR7Vn\P !W3^RR7VnR#)F)stridepaddingr7groupsbias)r5r6 kernel_sizer@N) super__init__nnELU activationunit_kernel_sizeConv1dconv1conv2)selfr4r5r6r7r> __class__s&&&&& rrCXcodecResidualUnit.__init__Ms{ &&(++a/A5AYY    # #   YY<`ahmn rcN<V^8dQhRS[PRS[P/#r hidden_statereturnrTensor)rrs"rrr:]s#,,ELL,U\\,rcVPV4pVPV4pVPV4pVPV4pW,#NrFrIrJ)rKrP output_tensors&& rforwardXcodecResidualUnit.forward]sC 5  =1  6  =1 ++rrV) rr r!r"r#rCrXr%r& __classcell__rLrs@@rr2r2Js#Poo ,,,rr2cPaa]tRt^etoV3RlV3RlltV3RlRltRtVtV;t#)XcodecSemanticEncoderBlockc2<V^8dQhRS[RS[RS[RS[/#rr4r5r6r=r8)rrs"rr'XcodecSemanticEncoderBlock.__annotate__fs0 x x| x# xS xZ] xrc .<\SV`4\P!VPUu.uFp\ WW%4NK up4VnV^8Xd^M^V,pV^, ^,p\P!W#WdVRR7VnR#uupi)r<TrAr=r>r@N) rBrCrD ModuleListblock_dilationsr2 res_unitsrHconv) rKr4r5r6r=r7kernelr>rLs &&&&& rrC#XcodecSemanticEncoderBlock.__init__fs \b\r\r s\rPX [ K\r s  kF A:!#IIkVdkrvw tsBcN<V^8dQhRS[PRS[P/#rOrR)rrs"rrr`q#ELLU\\rc^VPF pV!V4pK VPV4pV#rU)rerfrKrPunits&& rrX"XcodecSemanticEncoderBlock.forwardqs.NND -L#yy. rrfre rr r!r"rCrXr%r&rZr[s@@rr]r]es  x xrr]cDaa]tRt^xtoV3RltV3RlRltRtVtV;t#)SemanticEncoderc H<\SV`4\VP4\VP48wd \ R4h\ P!VPVPVP^VP^,RR7Vn VPp.p\VP4FHwrE\VPVPV,,4pV\WWe4., pTpKJ \ P!V4VnR#)z:Number of strides must match the number of channel_ratios.Fr@N)rBrClenstrideschannel_ratios ValueErrorrDrHsemantic_hidden_sizerArf enumerater9r]rc conv_blocks)rKr4r5r{ir=r6rLs&& rrCSemanticEncoder.__init__ys  v~~ #f&;&;"< <YZ ZII  ' '  ' '      ! #   11  "6>>2IAv::V=R=RST=UUVL 6vLab bK&K3 ==5rcN<V^8dQhRS[PRS[P/#rOrR)rrs"rrSemanticEncoder.__annotate__rjrc^VPV4pVPF pV!V4pK V#rUrfr{rKrPblocks&& rrXSemanticEncoder.forwards0yy. %%E .L&rrrpr[s@@rrrrrxs6,rrrcPaa]tRt^toV3RlV3RlltV3RlRltRtVtV;t#)SemanticDecoderBlockc2<V^8dQhRS[RS[RS[RS[/#r_r8)rrs"rr!SemanticDecoderBlock.__annotate__s*  | # S Z] rc <\S V`4V^8Xd#\P!VV^^^RR7VnMJ^V,pV^,^,pV^,^8Xd^M^p\P !W#WTWgRR7Vn\P !VPUu.uFp\WW84NK up4Vn R#uupi)r<TrbFrtN) rBrCrDrHrfConvTranspose1drcrdr2re) rKr4r5r6r=rAr>output_paddingr7rLs &&&&& rrCSemanticDecoderBlock.__init__s  Q;  DIf*Kza'G"(1*/QqN**;^cDI^d^t^t u^tRZ l M^t u  us!CcN<V^8dQhRS[PRS[P/#rOrR)rrs"rrrrjrc^VPV4pVPF pV!V4pK V#rUrorls&& rrXSemanticDecoderBlock.forwards.yy. NND -L#rrorpr[s@@rrrs  .rrcDaa]tRt^toV3RltV3RlRltRtVtV;t#)SemanticDecoderc b<\SV`4\P!VP\ VPVP ^,,4VP^VP^,RR7Vn.p\VP4Fwr4\ VPVP V,,4pV\VP 4^, 8d6\ VPVP V^,,,4pM VPpV\WWd4., pK \P!V4Vn\P!VPVPVP^VP^,RR7VnR#)F)r5r6rAr=r>r@)r=r>r@N)rBrCrDrHryr9rwrArIrzrvrurrcr{rJ)rKr4r{r|r=r5r6rLs&& rrCSemanticDecoder.__init__sQ YY33V886;P;PQR;SST**&&!+    "6>>2IAf99F#>AVAVWX[\W\A]#]^ %:: 0l[\ \K3==5YY  ' '  ' '   &&!+   rcN<V^8dQhRS[PRS[P/#rOrR)rrs"rrSemanticDecoder.__annotate__s#ELLU\\rcVPV4pVPF pV!V4pK VPV4pV#rU)rIr{rJrs&& rrXSemanticDecoder.forwards>zz,/ %%E .L&zz,/ r)rIrJr{rpr[s@@rrrs >rrcHaa]tRt^toRtV3RltRtRtRtRt Vt V;t #)XcodecEuclideanCodebookz!Codebook with Euclidean distance.c<\SV`4\P!VPVP 4pVPVn\ P!\P!R.44Vn \ P!\P!VP44Vn \ P!V4Vn \ P!VP44Vn R#)TN)rBrCrzeros codebook_size codebook_dimrDBufferrSinited cluster_sizeembedclone embed_avg)rKr4rrLs&& rrC XcodecEuclideanCodebook.__init__s  F00&2E2EF#11ii dV 45 IIekk&2F2F&GHYYu% 5;;=1rc4VPP4pVP^4P^RR7pV^V,V,, VP^4P^RR7,)pVP RR7P pV#)rT)keepdimdim)rtpowsummaxindices)rK hidden_statesr scaled_statesdist embed_inds&& rquantize XcodecEuclideanCodebook.quantizes %))!,00D0A ]!2U!::UYYq\=M=MaY]=M=^^_HHH$,, rcVPpVPRVR,34pVPV4pVP!VRR!pV#)r<Nr)shapereshaperview)rKrrrs&& rencodeXcodecEuclideanCodebook.encodesM##%--r59o> MM-0 NNE#2J/ rc\P!VPVPP4VP4pV#rU)F embeddingtordevice)rKr quantizeds&& rdecodeXcodecEuclideanCodebook.decodes/KK TZZ->-> ?L r)rrrrr) rr r!r"r#rCrrrr%r&rZr[s@@rrrs!+2rrcNaa]tRt^toRtV3RlV3RlltRtRtRtVt V;t #)XcodecVectorQuantizationzQ Vector quantization implementation. Currently supports only euclidean distance. c <V^8dQhRS[/#rr4r )rrs"rr%XcodecVectorQuantization.__annotate__s88|8rcD<\SV`4\V4VnR#rU)rBrCrcodebook)rKr4rLs&&rrC!XcodecVectorQuantization.__init__s /7 rcbVP^^^4pVPPV4pV#r)permuterr)rKrembed_ins&& rrXcodecVectorQuantization.encodes/%--aA6 =='' 6rcbVPPV4pVP^^^4pV#r)rrr)rKrrs&& rrXcodecVectorQuantization.decode s/=='' 2##Aq!,r)r) rr r!r"r#rCrrr%r&rZr[s@@rrrs#88  rrcaa]tRtRtoRtV3RlV3RlltRtR V3RlRlltR V3RlR lltV3R lR lt R t Vt V;t #) XcodecResidualVectorQuantizationizn Residual vector quantization implementation. Follows Algorithm 1 in https://huggingface.co/papers/2107.03312 c <V^8dQhRS[/#rr )rrs"rr-XcodecResidualVectorQuantization.__annotate__s44|4rc"<\SV`4\P!\ VP 4Uu.uFp\ V4NK up4VnVPVnVPVn VP VnR#uupirU) rBrCrDrcrangenum_quantizersr quantizers frame_rater)rKr4_rLs&& rrC)XcodecResidualVectorQuantization.__init__sq --SXY_YnYnSo(pSoa)A&)ISo(pq ++#11$33)qsB c r\P!VP4VP,R, #)zReturn bandwidth per quantizer.i)mathlog2rr)rKs&rget_bandwidth_per_quantizerEErc <V^8dQhRS[/#)rrQ)r9)rrs"rrr#s#rc  VP4pVPpVe7VR8d0\\^\P !W, 444pV#)z:Return num_quantizers based on specified target bandwidth.)rrr9rrfloor)rK bandwidthbw_per_qrs&& r get_num_quantizers_for_bandwidthAXcodecResidualVectorQuantization.get_num_quantizers_for_bandwidth#sJ335,,  Y_ Q 93G(H!IJNrcN<V^8dQhRS[PRS[P/#)r embeddingsrQrR)rrs"rrr+s#%,,rc VPV4pTp.pVPRVF>pVPV4pVPV4pWH, pVP V4K@ \ P !V4p V #)z Encode the input tensor into discrete indices using RVQ, with the number of quantizers selected based on the given bandwidth. Each quantizer /codebook residually quantizes the input and returns the nearest indices in terms of Euclidean distance. N)rrrrappendrstack) rKrrrresidual all_indices quantizerrr out_indicess &&& rr'XcodecResidualVectorQuantization.encode+s >>yI .9I&&x0G!((1I+H   w ' : kk+. rcN<V^8dQhRS[PRS[P/#)rcodesrQrR)rrs"rrr;s#ELLU\\rc \P!RRVPR7p\V4FJwr4VPV,pVP V4pW&P VP4,pKL V#)z9Decode the given codes to their quantized representation.r)rr)rfullrrzrrr)rKr quantized_outr|rrrs&& rr'XcodecResidualVectorQuantization.decode;sd 2s5<<@ #E*JA*I!((1I)LL,FFM+r)rrrrrU) rr r!r"r#rCrrrrr%r&rZr[s@@rrrs=44F rrca]tRtRtoRt]tRtRtRt R.t ] P!4R4t RtR t]R 4tRR ltR tVtR #)XcodecPreTrainedModeliEzz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. xcodec input_valuesaudiorc \V\P4'dj\P!VP RVP PR7VPe#\P!VP4R#R#\V\P\P34'dC\P!VP4\P!VP 4R#\V\P4'd\P!VP 4VPep\P !VP"VP$VP&^,,, 4p\P(!VPV)VR7R#R#VP*P,R8Xd#\P!VP.4R#\V\P04'dVP34R#\V\P44'd&\P!VP RRR7R#\V\64'EdVP8P;4Fhp\V\P4'gK%\P<!VP RR7\P>!VP^4Kj VP@P;4Fhp\V\P4'gK%\P<!VP RR7\P>!VP^4Kj R#\V\B4'd\PD!VPF\HPJ!R.44\P!VPL4\P!VPN4\P!VPP4R#R#) zInitialize the weightsr)meanstdN)abSnake1dg{Gz?)rT)) isinstancerDLinearinitnormal_weightr4initializer_ranger@zeros_ LayerNorm GroupNormones_rHkaiming_normal_rsqrtr?r5rAuniform_rLralpharreset_parameters Embedding XcodecModelacoustic_encodermodules trunc_normal_ constant_acoustic_decoderrcopy_rrrSrrr)rKmodulek submodules&& r _init_weights#XcodecPreTrainedModel._init_weightsRs fbii ( ( LLSdkk6S6S T{{& FKK(' r|| < = = KK $ JJv}} %  * *   /{{&IIfmmv/A/AFDVDVWXDY/YZ[ fkkaR15'   & &) 3 JJv|| $  2 2 3 3  # # %  - - LLSd ;  , ,$44<<> i33&&y'7'7TBNN9>>15?$44<<> i33&&y'7'7TBNN9>>15? 7 8 8 JJv}}ellD6&: ; KK++ , KK % KK(( ) 9rc J\PPPPpV!VP P 4V!VP P4VP PFcpV!VP 4VPVPVP3F'pV!VP 4V!VP4K) Ke V!VPP RR7V!VPPRR7VPPFipV!VPRR7VPVPVP3F+pV!VP RR7V!VPRR7K- Kk R#)znApply weight norm in the acoustic encoder and decoder because the original checkpoint has weight norm applied.r nameN)rrDutilsparametrizations weight_normrrIrJr res_unit1 res_unit2 res_unit3rconv_t1)rKr'rres_units& rapply_weight_norm'XcodecPreTrainedModel.apply_weight_normxs'hhnn55AA D))//0D))//0**00E  $"__eoouOHNN+HNN+P1 D))//h?D))//h?**00E  H 5"__eoouOHNN:HNN:P1rc VPVP3FpVP4Fp\PP P VRR7\VR4'gKCRVP9gKV\PP PPVRRR7K K R# \\3dLzi;i)z=Remove the weight norm from the acoustic encoder and decoder.r r#r&T)leave_parametrizedN) rrrrrDr%remove_weight_normrxAttributeErrorhasattrr& parametrizeremove_parametrizations)rKrms& rr1(XcodecPreTrainedModel.remove_weight_norms,,d.C.CDF^^%HHNN55ah5G1011h!BTBT6THHNN..FFq(gkFl &E#N3s+CCCc 8aRV3Rllo\S!V44#)z1 Recursively iterate to fetch all Conv1d layers. c8V^8dQhR\P/#)rrrDModule)rs"rr>XcodecPreTrainedModel._get_conv1d_layers..__annotate__s   rc<.p\V\P4'dVPV4VP 4FpVP S!V44K V#rU)rrDrHrchildrenextend)r params_listchildget_conv1d_layers_recursives& rrBMXcodecPreTrainedModel._get_conv1d_layers..get_conv1d_layers_recursivesVK&")),,""6* *""#>u#EF+ r)tuple)rKrrBs&&@r_get_conv1d_layers(XcodecPreTrainedModel._get_conv1d_layerss  0899rNc \VfTpVPV4pVFp\WA4pK V#)z_ For a given module, compute the output length that would be obtained after all Conv1d layers. )rEr)rK input_lengthr conv1d_layerslayers&&& r_get_conv1d_output_lengths0XcodecPreTrainedModel._get_conv1d_output_lengthss: >F//7 "E/DL#rrrU)rr r!r"r#r config_classbase_model_prefixmain_input_nameinput_modalities_no_split_modulesrno_gradr r-r1rrErKr%r&r's@rrrEsm  L $O;< ]]_#*#*J;, m::&  rrz$The Xcodec neural audio codec model.) custom_introcaa]tRtRtoV3Rlt]V3RlRl4tV3RlRlt]RV3RlRll4t ]RV3R lR ll4t ]] RV3R lR ll44t R t VtV;t#)ricJ<\SV`V4WnVP^,Vn\ P !VP4pVPVn VPVn VPVP4\V4Vn\V4Vn\ P !VP"4P%4Vn\(P*!VP,VP,4Vn\(P*!VP,VP"P,4Vn\(P*!VP,VPP,4Vn\5V4VnVP94R#)rN)rBrCr4 hop_lengthpadr from_configacoustic_model_configencoderrdecoderr_adjust_dac_decoderrrencoder_semanticrdecoder_semanticsemantic_model_configevalsemantic_modelrDr hidden_sizefcfc1fc2rr post_init)rKr4acoustic_modelrLs&& rrCXcodecModel.__init__s'   $$)"..v/K/KL . 6 6 . 6 6   !6!67 / 7 / 7'33F4P4PQVVX))F..0B0BC99V//1M1M1Y1YZ99V//1M1M1Y1YZ9&A rc4<V^8dQhRS[P/#)rr[r:)rrs"rrXcodecModel.__annotate__s ) )RYY )rc VP4Fsp\V\P4'gK%\VP\ 4'dVP^,M VPpV^,3VnKu \VR4'dI\VP\P4'd\P!4VnR#R#R#)z DAC implemented in Xcodec is slightly different from the HF version. DAC in Xcodec adjusts the output padding in every ConvTranspose1d in the decoder and removes the final `nn.Tanh` activation function. tanhN) rrrDrr=rDrr3rlTanhIdentity)r[rr=s& rr\XcodecModel._adjust_dac_decodersoo'F&""4"455-7 u-M-Mq)SYS`S`)/! %( 7F # # 7<<(I(I;;=GL)J #rcN<V^8dQhRS[PRS[P/#)rrrQr/)rrs"rrrjs'##u7H7H#UM^M^#rcpVR,p\P!WPVP34p\P!4;_uu_4VP VRR7pVP pRRR4\P !X^R7pVP^R7# +'giL:;i)NNNT)output_hidden_statesNr)rrrrr)rrWrrRrarrr)rKroutputsrstackeds&& r_extract_semantic_features&XcodecModel._extract_semantic_featuress#G, uu\HHdhh+?@ ]]__)),T)RG#11M++m3|||"" _s  B%% B5 c <V^8dQhRS[PRS[R,RS[R,RS[PS[,/#)rrrN return_dictrQ)rrSfloatboolr))rrs"rrrjsI1010ll104<10D[ 10 + + 10rc :VeTMVPPpVP^,pV^8wd\RV 24hVfVPPR,pM@W PP9d'\RV RVPP R24hVP V4P 4pVPVP^^44pVPVP^,VP4VP^,8wd=VP\P!WPVP344pMVPV4p\P!VPVP 4V.^R7pVP#VP^^44P^^4pVP$P'W4p V P^^4p V'gV #\)V 4#)a input_values (`torch.FloatTensor` of shape `(batch_size, channels, num_samples)`): Float values of the input audio waveform. bandwidth (`float`, *optional*): The target bandwidth in (kbps) supports only values in `config.target_bandwidths`. Defaults to the highest available bandwidth `4.0` kbps. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`]. Returns: `torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)` containing the discrete encoded audio codes. zAudio must be mono, but got z)This model doesn't support the bandwidth z. Select one of .rr)r4ryrrxtarget_bandwidthsrvdetachr] transposerKrrrWrcatrrrcrrr)) rKrrrychannelse_semantic_input e_semantic e_acousticrrs &&&& rrXcodecModel.encodes&&1%>> from datasets import load_dataset >>> from transformers import AutoFeatureExtractor, XcodecModel >>> model_id = "hf-audio/xcodec-hubert-librispeech" >>> model = XcodecModel.from_pretrained(model_id) >>> feature_extractor = AutoFeatureExtractor.from_pretrained(model_id) >>> dataset = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> dataset = dataset.cast_column("audio", Audio(sampling_rate=feature_extractor.sampling_rate)) >>> audio_sample = dataset[0]['audio']['array'] >>> inputs = feature_extractor(raw_audio=audio_sample, return_tensors="pt") >>> outputs = model(**inputs) >>> audio_codes = outputs.audio_codes >>> audio_values = outputs.audio_values ``` NF)ryT.)rrr)rrrr)rKrrrrlengthrs&&&&, rrXXcodecModel.forward9s\Z##B'  ++l5+QK{{;D{A!DS'6'\R  OOr) rrr4r^r]rcrdrerWrra)NNrU)rr r!r"rC staticmethodr\rvr rrr rXr%r&rZr[s@@rrrs& ) )##101010f11162P2P2P2Prr)-r#r dataclassesr functoolsrrtorch.nnrDtorch.nn.functional functionalrrr audio_utilsrmodeling_utilsrprocessing_utilsrr%rr r r autor configuration_xcodecrrr)r-r;r2r]rrrrrrrrr__all__rrrrsh! ! &/:& .  2; 2  2 0+0 0 2+2 2,,6&bii<299>%bii%Pbii@ryy,/ryy/ds8sslGHpP'pPIpPf 1 2r