+ jSRt^RIt^RIt^RIHt^RIHt^RIt^RI t ^RI H t ^RI H t ^RIHt^RIHt^RIHt^R IHt^R IHt^R IHt^R IHt^R IHt^RIH t ^RI!H"t"H#t#H$t$H%t%H&t&H't'^RI(H)t)H*t*H+t+^RI,H-t-^RI.H/t/H0t0H1t1H2t2H3t3H4t4H5t5^RI6H7t7H8t8^RI9H:t:H;t;^RIRt?]5P!]A4tB^tC]1!RR7]!RR]/444tDRZRRlltER[RRlltF!R R!] 4tG!R"R#] 4tH!R$R%] 4tI!R&R'] P4tK!R(R)] P4tL!R*R+] P4tM!R,R-] P4tNR\R.R/lltO!R0R1] P4tP!R2R3] P4tQ!R4R5] 4tR!R6R7] 4tS!R8R9] P4tT!R:R;] P4tU!R<R=] P4tV!R>R?] P4tW!R@RA] P4tX!RBRC] P4tY]1!RDRE]*44tZ]1!RFRG]Z44t[]1!RHR7!RIRJ]Z44t\]1!RKR7!RLRM]Z44t]]1!RNR7!RORP]Z44t^]1!RQRR]Z44t_!RSRT] P4t`!RURV] P4ta]1!RWR7!RXRY]Z44tb.R]OtcR#)^zPyTorch Wav2Vec2 model.N)Callable) dataclass) load_file)nn)CrossEntropyLoss)initialization)ACT2FN)filter_output_hidden_states)is_deepspeed_zero3_enabled)is_fsdp_managed_module)create_bidirectional_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputCausalLMOutputSequenceClassifierOutputTokenClassifierOutputWav2Vec2BaseModelOutput XVectorOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel*get_torch_context_manager_or_global_device)Unpack) ModelOutputTransformersKwargsauto_docstring cached_filecheck_torch_load_is_safeis_peft_availablelogging)can_return_tuplemerge_with_config_defaults)OutputRecordercapture_outputsWav2Vec2Configzadapter.{}.binzadapter.{}.safetensorsza Output type of [`Wav2Vec2ForPreTraining`], with potential hidden states and attentions. ) custom_introcPa]tRt^EtoRtRtRtRtRtRt Rt Rt Rt V3Rlt RtVtR#)Wav2Vec2ForPreTrainingOutputa loss (*optional*, returned when `sample_negative_indices` are passed, `torch.FloatTensor` of shape `(1,)`): Total loss as the sum of the contrastive loss (L_m) and the diversity loss (L_d) as stated in the [official paper](https://huggingface.co/papers/2006.11477). projected_states (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.proj_codevector_dim)`): Hidden-states of the model projected to *config.proj_codevector_dim* that can be used to predict the masked projected quantized states. projected_quantized_states (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.proj_codevector_dim)`): Quantized extracted feature vectors projected to *config.proj_codevector_dim* representing the positive target vectors for contrastive loss. codevector_perplexity (`torch.FloatTensor` of shape `(1,)`): The perplexity of the codevector distribution, used to measure the diversity of the codebook. contrastive_loss (*optional*, returned when `sample_negative_indices` are passed, `torch.FloatTensor` of shape `(1,)`): The contrastive loss (L_m) as stated in the [official paper](https://huggingface.co/papers/2006.11477). diversity_loss (*optional*, returned when `sample_negative_indices` are passed, `torch.FloatTensor` of shape `(1,)`): The diversity loss (L_d) as stated in the [official paper](https://huggingface.co/papers/2006.11477). Nc<V^8dQh/S[PR,;R&S[PR,;R&S[PR,;R&S[PR,;R&S[S[P,R,;R&S[S[P,R,;R&S[PR,;R&S[PR,;R &#) Nlossprojected_statesprojected_quantized_statescodevector_perplexity hidden_states attentionscontrastive_lossdiversity_loss)torch FloatTensortuple)format __classdict__s"o/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/wav2vec2/modeling_wav2vec2.py __annotate__)Wav2Vec2ForPreTrainingOutput.__annotate__Es2   d ")34''$.556!& 1 1D 8?78!,,t3:9:**+d29;<e''(4/6=>''$.5?@%%,3A)__name__ __module__ __qualname____firstlineno____doc__r+r,r-r.r/r0r1r2__annotate_func____static_attributes____classdictcell__)r7s@r8r(r(Es@$&*D15;?6:59M26J15/3NAr;r(c V^8dQhR\\\3,R\R\R\PR,R\R\ P /#)r*shape mask_prob mask_lengthattention_maskN min_masksreturn)r5intfloatr3 LongTensornpndarray)r6s"r8r9r9hsctt c?ttt$$t+ t  t ZZ tr;c ^aaaaaVwpoS^8d \R4hSS8d\RS RS R24h\PP^4P 4oVVVVV3RlpVe.VP 4P R 4P4M\V4Uu.uFpSNK upp\P!VS3\R7p .p V!S4p V ^8XdV #VFp V!V 4p \PP\P!V S^, , 4V RR7p\V4^8Xd S^, pM V^,p\P!V\P!W, \P R7V,.4pV P#V4K \P$!V 4p \P&!V R ,W[S34p V P)W[S,4p \P!S4R ,p\P&!VW[S34P)W[S,4pV V,p V P+4S^, 8dS^, WS^, 8&\P,!W^R 4V #uupi) a2 Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://huggingface.co/papers/1904.08779). Note that this method is not optimized to run on TPU and should be run on CPU as part of the preprocessing during training. Args: shape: The shape for which to compute masks. This should be of a tuple of size 2 where the first element is the batch size and the second element is the length of the axis to span. mask_prob: The percentage of the whole axis (between 0 and 1) which will be masked. The number of independently generated mask spans of length `mask_length` is computed by `mask_prob*shape[1]/mask_length`. Note that due to overlaps, `mask_prob` is an upper bound and the actual percentage will be smaller. mask_length: size of the mask min_masks: minimum number of masked spans attention_mask: A (right-padded) attention mask which independently shortens the feature axis of each batch dimension. z&`mask_length` has to be bigger than 0.zO`mask_length` has to be smaller than `sequence_length`, but got `mask_length`: z and `sequence_length`: `c<\SV,S, S,4p\VS4pVS,S8d SS,pVS^, , V8d\VS^, , ^4pV#)z;Given input length, compute how many spans should be masked)rLmax) input_lengthnum_masked_spanepsilonrHrGrJsequence_lengths& r8compute_num_masked_span6_compute_mask_indices..compute_num_masked_spansoi,6DwNOoy9 [ (? :-r?r@rrrCrD __classcell__rr7s@@r8rrs Ar;rc<aa]tRtRtoRV3RlltRtRtVtV;t#)Wav2Vec2LayerNormConvLayeric<\SV`4V^8dVPV^, ,M^VnVPV,Vn\ P !VPVPVPV,VPV,VPR7Vn \ P!VPRR7Vn \VP,VnR#)rrT)elementwise_affineN)rrrrrrrrrrr LayerNorm layer_normrrrrs&&&r8r#Wav2Vec2LayerNormConvLayer.__init__s !#Car;cZVP^8dVRRRVP)13,pV#)rr_Nrrs&&r8rWav2Vec2SamePadLayer.forward{s6    ")!Q0F43F3F2F0F*FGMr;rrrs@@r8rrvsKr;rcBaa]tRtRtoRtV3RltRtRtRtVt V;t #)Wav2Vec2FeatureEncoderiz.Construct the features from raw audio waveformc <\SV`4VPR8XdR\V^R7.\ VP ^, 4Uu.uFp\ W^,R7NK up,pMYVPR8Xd0\ VP 4Uu.uFp\WR7NK ppM\RVP R24h\P!V4Vn RVn RVn R#uupiuupi)group)rlayerz`config.feat_extract_norm` is z), but has to be one of ['group', 'layer']FTN)rrfeat_extract_normrrgnum_feat_extract_layersrrr`r ModuleList conv_layersgradient_checkpointing_requires_grad)rrirrs&& r8rWav2Vec2FeatureEncoder.__init__s   # #w .5fqIJNSTZTrTruvTvNwNNw,V!eDNwNK % % 0HMfNlNlHmHm1*6>Hm K01I1I0JJst ==5&+#"Ns C6C;cPVP4F pRVnK RVnR#)FN) parameters requires_gradrrparams& r8_freeze_parameters)Wav2Vec2FeatureEncoder._freeze_parameterss#__&E"'E '#r;cVR,pVP'dVP'dRVnVPF pV!V4pK V#)r_Tr)rtrainingrr)r input_valuesr/ conv_layers&& r8rWav2Vec2FeatureEncoder.forwardsL$W-     4===*.M '**J&}5M+r;)rrr) r=r>r?r@rArrrrCrDrrs@@r8rrs8#&$  r;rc8aa]tRtRtoV3RltRtRtVtV;t#)Wav2Vec2FeatureProjectionicL<\SV`4\P!VPR,VP R7Vn\P!VPR,VP4Vn \P!VP4Vn R#)repsNr^) rrrrrlayer_norm_epsrLinearr projectionDropoutfeat_proj_dropoutdropoutrrrs&&r8r"Wav2Vec2FeatureProjection.__init__sf ,,vr':@U@UV))FOOB$79K9KLzz&":":; r;cnVPV4pVPV4pVPV4pW3#r)rrr)rr/norm_hidden_statess&& r8r!Wav2Vec2FeatureProjection.forwards7!__];(:;  ]3 00r;)rrrrrs@@r8rrs< 11r;rcV^8dQhR\PR\PR\PR\PR\PR,R\R,R\R \ \ ,/#) r*modulequerykeyvaluerINscalingrkwargs)rModuler3TensorrMrr)r6s"r8r9r9s{%% II% <<% % << % LL4' % T\ %%'(%r;c VfVPR4R,p\P!WP^^44V,pVe W,p\P P VRR7p\P PWVPR7p\P!W4p V P^^4P4p W3#)Nr)prr^) rr3matmulrr functionalsoftmaxrr contiguous) rrrrrIrrr attn_weights attn_outputs &&&&&&&, r8eager_attention_forwardr*s**R.D(<<}}Q':;gEL!#4 ==((2(>L==((6??([L,,|3K''1-88:K  $$r;c\aa]tRtRtoRtRV3RlV3RllltR V3RlRlltRtVtV;t #) Wav2Vec2Attentioniz=Multi-headed attention from 'Attention Is All You Need' papercR<V^8dQhRS[RS[RS[RS[RS[RS[RS[R,/#) r* embed_dim num_headsr is_decoderr is_causalrN)rLrMrir%)r6r7s"r8r9Wav2Vec2Attention.__annotate__s_CCCC C  C  CC%Cr;c<\SV`4WnW nW0nW,VnWpnVP V,VP8wd\RVP RV R24hVP R,VnW@n W`n \P!WVR7Vn \P!WVR7Vn\P!WVR7Vn\P!WVR7VnR#)z;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).)rNr#)rrr.r/rhead_dimrr`rr0r1rr k_projv_projq_projout_proj) rr.r/rr0rr1rrs &&&&&&&&r8rWav2Vec2Attention.__init__s "" !.  MMI %$.. 8MdnnM]$YKr3 }}d* $"ii 4@ ii 4@ ii 4@  )TB r;c<V^8dQhRS[PRS[PR,RS[PR,RS[R,RS[S[,RS[S[PS[PR,3,/#)r*r/key_value_statesNrIoutput_attentionsrrK)r3rrirr r5)r6r7s"r8r9r2s~0)0)||0) ,,-0) t+ 0)  $; 0)-.0) u||U\\D00 10)r;c VRJpVPRRp.VORNVPN5pVPV4PV4P ^^4p V'dTMTp .V PRRORNVPN5p VP V 4PV 4P ^^4p VP V 4PV 4P ^^4p \P!VPP\4pT!VV V V V3RVP'gRM VPRVPRV/VBwppVP!.VORN5!P!4pVP#V4pVV3#)z#Input shape: Batch x Time x ChannelNrrr<r^)rFr4r7viewrr5r6r get_interfacer_attn_implementationr*rrrrsr'r8)rr/r;rIr<ris_cross_attention input_shape hidden_shape query_statescurrent_stateskv_shape key_states value_statesattention_interfacer)r(s&&&&&, r8rWav2Vec2Attention.forwards.T9$))#2. 88b8$--8 {{=166|DNNqRST -?)]B^))#2.BBDMMB[[055h?II!QO {{>277AKKAqQ (?(M(M KK , ,.E) %8      % $}}}C$,, % LL % 0 %  % ! \"));;;;FFH mmK0 L((r;) rrr.r4r1r0r5r/r8r7rr6)r>FTFN)NNF) r=r>r?r@rArrrCrDrrs@@r8r,r,s#GCC>0)0)0)r;r,c8aa]tRtRtoV3RltRtRtVtV;t#)Wav2Vec2FeedForwardi*c<\SV`4\P!VP4Vn\P !VPVP4Vn \VP\4'd\VP,VnMVPVn\P !VPVP4Vn\P!VP 4VnR#r)rrrractivation_dropoutintermediate_dropoutr rintermediate_sizeintermediate_dense isinstance hidden_actstrrintermediate_act_fn output_densehidden_dropoutoutput_dropoutrs&&r8rWav2Vec2FeedForward.__init__+s $&JJv/H/H$I!"$))F,>,>@X@X"Y f'' - -'-f.?.?'@D $'-'8'8D $IIf&>&>@R@RS jj)>)>?r;cVPV4pVPV4pVPV4pVPV4pVP V4pV#r)rRrVrPrWrYrs&&r8rWav2Vec2FeedForward.forward8sX// > 00? 11-@ ))-8 ++M: r;)rVrRrPrWrYrrs@@r8rMrM*s @r;rMcHaa]tRtRtoV3RltRV3RlRlltRtVtV;t#)Wav2Vec2EncoderLayeriBc<\SV`4\VPVPVP RVR7Vn\P!VP4Vn \P!VPVPR7Vn \V4Vn\P!VPVPR7VnR#)Fr.r/rr0rr N)rrr,rnum_attention_headsattention_dropout attentionrrrXrrr rrM feed_forwardfinal_layer_normrs&&r8rWav2Vec2EncoderLayer.__init__Cs *((00,,  zz&"7"78 ,,v'9'9v?T?TU/7 " V-?-?VEZEZ [r;cr<V^8dQhRS[PRS[PR,RS[S[,/#r*r/rINrr3rrr)r6r7s"r8r9!Wav2Vec2EncoderLayer.__annotate__Rs<|| t++, r;c TpVP!V3RV/VBwrVPV4pWA,pVPV4pWPV4,pVP V4pV#rI)rcrrrdrerr/rIr attn_residualrvs&&&, r8rWav2Vec2EncoderLayer.forwardRsr & >>-aaZ`a  ]3 %5  6 %(9(9-(HH --m< r;)rcrrdrerrrrs@@r8r^r^Bs \r;r^cHaa]tRtRtoV3RltRV3RlRlltRtVtV;t#)#Wav2Vec2EncoderLayerStableLayerNormidc<\SV`4\VPVPVP RVR7Vn\P!VP4Vn \P!VPVPR7Vn \V4Vn\P!VPVPR7Vn\#VRR4e\%V4VnR#RVnR#)Fr`r adapter_attn_dimN)rrr,rrarbrcrrrXrrr rrMrdregetattrWav2Vec2AttnAdapterLayer adapter_layerrs&&r8r,Wav2Vec2EncoderLayerStableLayerNorm.__init__es *((00,,  zz&"7"78 ,,v'9'9v?T?TU/7 " V-?-?VEZEZ [ 6-t 4 @!9&!AD !%D r;cr<V^8dQhRS[PRS[PR,RS[S[,/#rhri)r6r7s"r8r90Wav2Vec2EncoderLayerStableLayerNorm.__annotate__xs<|| t++, r;c "TpVPV4pVP!V3RV/VBwrVPV4pWA,pWPVP V44,pVP eWP V4,pV#rl)rrcrrdrervrms&&&, r8r+Wav2Vec2EncoderLayerStableLayerNorm.forwardxs &  6 >>-aaZ`a  ]3 %5 %(9(9$:O:OP]:^(__    )),>,>},MMMr;)rvrcrrdrerrrrs@@r8rqrqds&&r;rqcHaa]tRtRtoV3RltRV3RlRlltRtVtV;t#)Wav2Vec2Encoderic<\SV`4Wn\V4Vn\ P !VPVPR7Vn \ P!VP4Vn \ P!\VP4Uu.uFp\!V4NK up4VnRVnR#uupir FN)rrrrpos_conv_embedrrrr rrrXrrrgnum_hidden_layersr^layersrrrrvrs&& r8rWav2Vec2Encoder.__init__s  =fE,,v'9'9v?T?TUzz&"7"78 mm5QWQiQiKj$kKja%9&%AKj$kl &+#%l#C cr<V^8dQhRS[PRS[PR,RS[S[,/#rh)r3tensorrrr)r6r7s"r8r9Wav2Vec2Encoder.__annotate__sA!@!@||!@ t+!@+, !@r;c Ve9VPR4P^^VP^,4p^W(&\VPVVR7pVP V4pWP VP4,pVPV4pVPV4p\4;'g \V4pVPFbp\P!.4pVP;'dWPP 8p V 'd V'gKWV!V3RV/VBpKd \#VR7#N)r inputs_embedsrIrI)last_hidden_stater^) unsqueezerepeatrFr rrtodevicerrr r rr3rbr layerdropr rr/rIrexpand_attention_maskposition_embeddings synced_gpusrdropout_probabilityskip_the_layers &&&, r8rWav2Vec2Encoder.forwards  %$2$<$(>}?S?S(TT  6  ]3 02RR6LT6R [[E"'**R. !]]ZZ/B[[EZEZ/ZN![[ %m ]N ]V\ ] !??r;rrrrrrrrrs@@r8r}r}s,!@!@!@r;r}cHaa]tRtRtoV3RltRV3RlRlltRtVtV;t#)Wav2Vec2EncoderStableLayerNormic<\SV`4Wn\V4Vn\ P !VPVPR7Vn \ P!VP4Vn \ P!\VP4Uu.uFp\!V4NK up4VnRVnR#uupir)rrrrrrrrr rrrXrrrgrrqrrrs&& r8r'Wav2Vec2EncoderStableLayerNorm.__init__s  =fE,,v'9'9v?T?TUzz&"7"78 mmBGH`H`Ba bBaQ 0 8Ba b  ',# crc0<V^8dQhRS[S[,/#)r*r)rr)r6r7s"r8r9+Wav2Vec2EncoderStableLayerNorm.__annotate__s #@#@+, #@r;c VVe9VPR4P^^VP^,4p^W(&\VPVVR7pVP V4pW,pVP V4p\4;'g \V4pVPFbp\P!.4pVP;'dWPP8p V 'd V'gKWV!V3RV/VBpKd VPV4p\VR7#r)rrrFr rrrr r rr3rbrrrrrs &&&, r8r&Wav2Vec2EncoderStableLayerNorm.forwards   %$2$<$AQAQ Q+001Mt`d`m`moqr !oob)..zBO &&r;)rrrrrr) r=r>r?r@rAr staticmethodrrrCrDrrs@@r8rrs- (  #'#'r;rc8aa]tRtRtoV3RltRtRtVtV;t#)Wav2Vec2Adapteri7c<a\SV`4SPSP8wdW\P !SPSP4Vn\P!SP4VnMR;VnVn\P!V3Rl\SP444Vn SPVn R#)Nc3:<"TFp\S4xK R#5ir)Wav2Vec2AdapterLayer).0rvrs& r8 +Wav2Vec2Adapter.__init__..Bs#kJjQ$8$@$@Jjs)rroutput_hidden_sizerrr projrproj_layer_normrrgnum_adapter_layersrrrs&fr8rWav2Vec2Adapter.__init__8s   $ $(:(: : &"4"4f6O6OPDI#%<<0I0I#JD /3 3DI,mm#k%PVPiPiJj#kk ))r;cVPe1VPe#VPV4pVPV4pVP^^4pVPFMp\P P 4pVP 'dW0P8gKEV!V4pKO VP^^4pV#r)rrrrrOrarr)rr/rlayerdrop_probs&& r8rWav2Vec2Adapter.forwardEs 99 T%9%9%E IIm4M 00?M%//15 [[EYY--/N===^nn%D %m 4 ! &//15 r;)rrrrrrs@@r8rr7s *r;rc8aa]tRtRtoV3RltRtRtVtV;t#)riVc<\SV`4\P!VP^VP,VP VP ^R7VnR#)r*)rrN)rrrrradapter_kernel_sizeadapter_striderrs&&r8rWav2Vec2AdapterLayer.__init__WsJ II  % % )) )  & &((   r;cjVPV4p\PPV^R7pV#)rr!)rrr%glurs&&r8rWav2Vec2AdapterLayer.forwardas/ -0  ))-Q)? r;)rrrs@@r8rrVs r;rcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)ruihc <\SV`4VPVnVPVn\ P!VP 4Vn\ P!VP VP4Vn \ P!4Vn \ P!VPVP 4Vn R#)z Implements adapter modules directly with 3D tensor weight as parameters and without using ModuleList to speed up training throughput. N)rrrs input_dimr hidden_dimrrnormr linear_1ReLUact_fnlinear_2rs&&r8r!Wav2Vec2AttnAdapterLayer.__init__is 00 ,,LL1  $//4>>B ggi  $..$//B r;c4<V^8dQhRS[P/#)r*r/)r3r4)r6r7s"r8r9%Wav2Vec2AttnAdapterLayer.__annotate__wsU%6%6r;cVPV4pVPV4pVPV4pVPV4pV#r)rrrrrs&&r8r Wav2Vec2AttnAdapterLayer.forwardws@ -0  m4  M2  m4 r;)rrrrrrrrs@@r8ruruhs Cr;ruc aa]tRtRtoRtRtRtRtRtRt Rt R] ] .R] !]^RR 7/t]P"!4V3R l4tRV3R lR lltRV3R lRlltRtRtRV3RlRlltV3RltRtVtV;t#)Wav2Vec2PreTrainedModeliwav2vec2raudioTr/r0encoder)index layer_namec  4<\SV`V4\V\4'd7VPP 4VP P 4R#\V\4'dz\P!VPPR^R7\P!VPP4\P!VP4R#\V\ 4'd\P!VP"P^^\$P&!^VP"P(^,VP"P*,, 4,R7\P,!VP"P^4R#\V\.4'd\$P&!^VP0P2, 4p\P!VP0PV)VR7\P!VP0PV)VR7R#\V\4P64'd\P8!VP4VPep\$P&!VP:VP*VP(^,,, 4p\P!VPV)VR7R#R#R#)zInitialize the weightsr>)rstd)abN)r _init_weightsrSWav2Vec2ForPreTraining project_hidreset_parameters project_qrinitnormal_rrzeros_runiform_rrrmathsqrtr in_channels constant_rr in_featuresrrkaiming_normal_r)rrkrs&& r8r%Wav2Vec2PreTrainedModel._init_weightss f% f4 5 5    / / 1    - - /  = > > LL++22! D KK**// 0 MM&,, -  ? @ @ LL "" !v{{'>'>q'AFKKD[D['["\]]  NN6;;++Q /  9 : : !f//;;; MM&++00QB! <  * *   /{{&IIfmmv/A/AFDVDVWXDY/YZ[ fkkaR15'+r;cX<V^8dQhRS[PS[,RS[R,/#)r*rw add_adapterN)r3rNrLri)r6r7s"r8r9$Wav2Vec2PreTrainedModel.__annotate__s.e>N>NQT>Tcgjncnr;c hVfVPPMTpRp\VPPVPP4FwrEV!WV4pK V'dE\ VPP 4F!pV!V^VPP4pK# V#)z8 Computes the output length of the convolutional layers cN\P!W, VRR7^,#)floor) rounding_mode)r3divrUrrs&&&r8_conv_out_lengthRWav2Vec2PreTrainedModel._get_feat_extract_output_lengths.._conv_out_lengths 99\7wWZ[[ [r;)rrziprrrgrr)rrwrrrrrvs&&& r8 _get_feat_extract_output_lengths8Wav2Vec2PreTrainedModel._get_feat_extract_output_lengthss 2=1Ddkk--+  \ $'t{{'>'> @W@W#X K,]PM$Y 4;;99: 04;;C]C] ^ ;r;c:<V^8dQhRS[RS[P/#)r*feature_vector_lengthrI)rLr3rN)r6r7s"r8r9rs#%(:?:J:Jr;cVPRR7R,pVPWCR7pVP\P4pVP ^,p\P !Wa3VPVPR7p^V\P!VP ^,VPR7V^, 3&VPR.4PR4PR.4P4pV#)rr!r)r\r)rr^)r_r^) cumsumrrr3longrFrhr\rrkflipri)rrrIrnon_padded_lengthsoutput_lengthsrus&&&& r8"_get_feature_vector_attention_mask:Wav2Vec2PreTrainedModel._get_feature_vector_attention_masks ,22r2:5A>>?Q>k'**5::6#))!,   /~7K7KTbTiTi uv ^%9%9!%<^EZEZ[]kno]opq',,bT299"=BBB4HMMOr;cVPPf\VP R24h/pVP 4FJwr#\ V\ 4'gKVP4FwrEWQRPW$.4&K KL \ V\4'd9VPP4Fwr%WQRPRV.4&K V#)NzF has no adapter layers. Make sure to define `config.adapter_attn_dim`..lm_head) rrsr`r named_modulesrSrunamed_parametersjoinWav2Vec2ForCTCr()radapter_weightsrr param_namers& r8 _get_adapters%Wav2Vec2PreTrainedModel._get_adapterss ;; ' ' ///uvw w ..0LD&":;;)/)@)@)B%JDICHHd-?$@A*C1 dN + +#||<<> ?D)T): ;< ?r;c VP4F,p\V\4'gKVPV4K. \V\4'dVPVP 4R#R#)zS (Re-)initialize attention adapter layers and lm head for adapter-only fine-tuning N)modulesrSrurr,r()rrs& r8init_adapter_layers+Wav2Vec2PreTrainedModel.init_adapter_layerssV llnF&":;;""6*% dN + +   t|| , ,r;c <V^8dQhRS[/#)r* target_langrU)r6r7s"r8r9rsm'm'm'r;c VPPf\RV R24hWP8Xd$V'g\P RV R24R#VP RR4pVP RR4pVP R R4pVP R R4pVP R R4pVP R R4p VP R R4p VPPp Rp V RJd5\PV4p \V V VVVVV VR7p\V4p V fM\PV4p \V V VVVVV VR7p\4\ P"!VRRR7p VP%4p\'V P)44\'VP)44, p\'VP)44\'V P)44, p\+V4^8d"\RX RRP-V4 R24h\+V4^8d"\RX RRP-V4 R24hV R,P.^,pVVPP08wdY\2P4!VPP6VVP8VP:R7VnVVPnV P?4UUu/uFwppVVPAVV,4bK! p ppVPCV RR7WnR# \dT 'dhEL*\d"T 'd\RT RT RT R24hELTi;i \dh\dh\d\RT RT RT R24hi;iuuppi)a Load a language adapter model from a pre-trained adapter model. Parameters: target_lang (`str`): Has to be a language id of an existing adapter weight. Adapter weights are stored in the format adapter..safetensors or adapter..bin force_load (`bool`, defaults to `True`): Whether the weights shall be loaded even if `target_lang` matches `self.target_lang`. cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. local_files_only(`bool`, *optional*, defaults to `False`): Whether or not to only look at local files (i.e., do not try to download the model). token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. To test a pull request you made on the Hub, you can pass `revision="refs/pr/"`. mirror (`str`, *optional*): Mirror source to accelerate downloads in China. If you are from China and have an accessibility problem, you can set this option to resolve it. Note that we do not guarantee the timeliness or safety. Please refer to the mirror site for more information. Activate the special ["offline-mode"](https://huggingface.co/transformers/installation.html#offline-mode) to use this method in a firewalled environment. Examples: ```python >>> from transformers import Wav2Vec2ForCTC, AutoProcessor >>> ckpt = "facebook/mms-1b-all" >>> processor = AutoProcessor.from_pretrained(ckpt) >>> model = Wav2Vec2ForCTC.from_pretrained(ckpt, target_lang="eng") >>> # set specific language >>> processor.tokenizer.set_target_lang("spa") >>> model.load_adapter("spa") ``` NzCannot load_adapter for - if `config.adapter_attn_dim` is not defined.z#Adapter weights are already set to r' cache_dirforce_downloadFproxieslocal_files_onlytokenrevisionuse_safetensors)filenamer;r<r=r>r?r:zCan't load the model for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z=' is the correct path to a directory containing a file named cpuT) map_location weights_onlyzThe adapter weights z has unexpected keys: z, z has missing keys: zlm_head.weightrr\)strict)"rrsr`r6loggerwarningpop _name_or_pathWAV2VEC2_ADAPTER_SAFE_FILEr6rsafe_load_fileOSError ExceptionWAV2VEC2_ADAPTER_PT_FILErr3loadr/setkeysrlr+rF vocab_sizerr rrr\r(itemsrload_state_dict)rr6 force_loadrr:r;r<r=r>r?r@model_path_or_id state_dictfilepath weight_pathr-unexpected_keys missing_keystarget_vocab_sizer vs&&&, r8 load_adapter$Wav2Vec2PreTrainedModel.load_adaptersx ;; ' ' /7 }Dqrs s ** *: NN@ QO P JJ{D1 $4e<**Y-!::&8%@ 7D)::j$/ **%6=;;44  % '188EH )$%#1#%5%'  ,K8 &  /66{CH" )$%#1#%5%'  )*"ZZ!&!% .,,.joo/037K7K7M3NN?//12S9J5KK  ! #3K=@VW[W`W`apWqVrrstu u   "3K=@STXT]T]^jTkSllmno o''78>>qA  6 6 699 ..0A$++]a]g]gDL&7DKK "?I>N>N>PQ>Pdaaoa011>P Q Z6'] "#  "!45E4FG==M>FZqJ# B   01A0BC99I8JK::B1F 6RsH L>6N;%N=>NNNN'N?NN: N:#N:c&<V^8dQh/S[;R&#r*rr$)r6r7s"r8r9rs r;)r(r6r)T)r=r>r?r@base_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnr^rqr"r,_can_record_outputsr3no_gradrrr$r/r3r_rBrCrDrrs@@r8rrs#$O&*#N.0STn%6aIV  ]]_66<*( -m'm'ir;rcaa]tRtRtoV3RlV3RlltRtR V3RlRllt]]!RR7] R V3R lR ll444t R t Vt V;t #) Wav2Vec2Modelic <V^8dQhRS[/#rbr$)r6r7s"r8r9Wav2Vec2Model.__annotate__s~r;c<\SV`V4Wn\V4Vn\ V4VnVPR8gVPR8dH\P!\P!VP4P44VnVP 'd\#V4VnM\'V4VnVP('d \+V4MRVnVP/4R#)r>N)rrrrfeature_extractorrfeature_projectionmask_time_probmask_feature_probrrr3rrrmasked_spec_embeddo_stable_layer_normrrr}rradapter post_initrs&&r8rWav2Vec2Model.__init__s   !7!?";F"C  3 &&*B*BS*H%'\\%,,v?Q?Q2R2[2[2]%^D "  & & &9&ADL*62DL282D2D2Dv.$  r;c :VPP4R#z Calling this function will disable the gradient computation for the feature encoder so that its parameter will not be updated during training. N)rqrrs&r8freeze_feature_encoder$Wav2Vec2Model.freeze_feature_encoders 113r;c<V^8dQhRS[PRS[PR,RS[PR,/#)r*r/rNrI)r3r4rN)r6r7s"r8r9rosE,,((,!,,t3,((4/ ,r;c \VPRR4'gV#VP4wrEpVe)VPP VP 4W&MVPP ^8dVP'd\WE3VPP VPPVVPPR7p\P!W!P\PR7pVPP VP 4W&VPP^8dVP'd\WF3VPPVPP VPP"R7p\P!WqP\PR7pVR,P%RVR4p^W&V#)z Masks extracted features along time axis and/or along feature axis according to [SpecAugment](https://huggingface.co/papers/1904.08779). apply_spec_augmentT)rGrHrIrJrE)rGrHrJrr^)rtrrrurr\rsrr~mask_time_lengthmask_time_min_masksr3rrrirtmask_feature_lengthmask_feature_min_masksr)rr/rrIrurXrmask_feature_indicess&&&& r8_mask_hidden_states!Wav2Vec2Model._mask_hidden_statesst{{$8$?? 4A3E3E3G0 [  (/3/E/E/H/HI\I\/]M , [[ ' '! + 5-++44 KK88-++99 ! !& ->G[G[chcmcm n /3/E/E/H/HI\I\/]M , ;; ( (1 ,#8)++77 KK;;++<< $ $)<<0DMaMainisis#t #7#@#G#GO]_#` 23M /r;F)tie_last_hidden_statesc <V^8dQhRS[PR,RS[PR,RS[PR,RS[S[,RS[S[,/#)r*rNrIrrrK)r3rr4rrr5r)r6r7s"r8r9rosf( ( llT)(  t+( !,,t3 ( +, ( ( ( ( r;c VPV4pVP^^4pVe&VPVP^,VRR7pVP V4wreVP WcVR7pVP !V3RV/VBpVPpVPeVPV4p\VVR7#)a mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Fr)rrIrI)rextract_features) rqrr$rFrrrrrrwr)rrrIrrrr/encoder_outputss&&&&, r8rWav2Vec2Model.forwards 11,?+55a;  %!DD &&q)>uEN+/*A*ABR*S' 00 ~1 ,,  )   (99 << # LL7M&+-  r;)rwrrrqrrruNN)r=r>r?r@rr}rr!r#rrrCrDrrs@@r8rmrmsR(4,,\ E2( ( 3 ( ( r;rmz? Wav2Vec2 Model with a quantizer and `VQ` head on top. caa]tRtRtoV3RlV3RlltV3RlRltRt]R V3RlRll4t] ] R V3R lR ll44t R t Vt V;t#)ric <V^8dQhRS[/#rbr$)r6r7s"r8r9#Wav2Vec2ForPreTraining.__annotate__$s  ~ r;c<\SV`V4\V4Vn\P !VP 4Vn\V4Vn \P!VPVP4Vn \P!VPVP4VnVP!4R#r)rrrmrrrfeat_quantizer_dropoutdropout_featuresr quantizerr rproj_codevector_dimrrrrxrs&&r8rWav2Vec2ForPreTraining.__init__$s  %f- " 6+H+H I6v>99V%7%79S9ST6#8#8&:T:TU r;c <V^8dQhRS[/#)r*r)rL)r6r7s"r8r9r1s11#1r;c &WPnR#)zR Set the Gumbel softmax temperature to a given value. Only necessary for training N)rr)rrs&&r8set_gumbel_temperature-Wav2Vec2ForPreTraining.set_gumbel_temperature1s&1"r;c NVPPP4R#r{rrqrr|s&r8r}-Wav2Vec2ForPreTraining.freeze_feature_encoder7 ''::s\ ))_$HaP(();)A)A)C_EZEZE\bdemm   % r;c <V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[S[,RS[S[,/#)r*rNrIrrrrK)r3r BoolTensorrrr5r()r6r7s"r8r9rUsQ Q llT)Q  t+Q !++d2 Q #("2"2T"9 Q +, Q  - -Q r;c Ve VP\P4pVP!V3RVRV/VBpVP VP 4pVP VP4pVe&VPVP^,VRR7pVPWR7wrV PVPPP4p VPV 4p R;p ;rVEeV PwrpV PR V4VP4PR 4,pVPWR V4P!^^^^4pVP#V R ,VVVP$P&4pV V8HP)R 4pVP+4'd\-R4VR,V&VP/^^4P1R VP3^44p^VP4, R,P/^^4P54p\6P8P;VP-4VR R 7p VP$P<VP$P>,pVV , V, VPA4,p WP$PBV ,,p \EV VV V VPFVPHV V R 7#)aE mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. sampled_negative_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length, num_negatives)`, *optional*): Indices indicating which quantized target vectors are used as negative sampled vectors in contrastive loss. Required input for pre-training. Example: ```python >>> import torch >>> from transformers import AutoFeatureExtractor, Wav2Vec2ForPreTraining >>> from transformers.models.wav2vec2.modeling_wav2vec2 import _compute_mask_indices, _sample_negative_indices >>> from datasets import load_dataset >>> feature_extractor = AutoFeatureExtractor.from_pretrained("facebook/wav2vec2-base") >>> model = Wav2Vec2ForPreTraining.from_pretrained("facebook/wav2vec2-base") >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> input_values = feature_extractor(ds[0]["audio"]["array"], return_tensors="pt").input_values # Batch size 1 >>> # compute masked indices >>> batch_size, raw_sequence_length = input_values.shape >>> sequence_length = model._get_feat_extract_output_lengths(raw_sequence_length).item() >>> mask_time_indices = _compute_mask_indices( ... shape=(batch_size, sequence_length), mask_prob=0.2, mask_length=2 ... ) >>> sampled_negative_indices = _sample_negative_indices( ... features_shape=(batch_size, sequence_length), ... num_negatives=model.config.num_negatives, ... mask_time_indices=mask_time_indices, ... ) >>> mask_time_indices = torch.tensor(data=mask_time_indices, device=input_values.device, dtype=torch.long) >>> sampled_negative_indices = torch.tensor( ... data=sampled_negative_indices, device=input_values.device, dtype=torch.long ... ) >>> with torch.no_grad(): ... outputs = model(input_values, mask_time_indices=mask_time_indices) >>> # compute cosine similarity between predicted (=projected_states) and target (=projected_quantized_states) >>> cosine_sim = torch.cosine_similarity(outputs.projected_states, outputs.projected_quantized_states, dim=-1) >>> # show that cosine similarity is much higher than random >>> cosine_sim[mask_time_indices.to(torch.bool)].mean() > 0.5 tensor(True) >>> # for contrastive loss training model should be put into train mode >>> model = model.train() >>> loss = model( ... input_values, mask_time_indices=mask_time_indices, sampled_negative_indices=sampled_negative_indices ... ).loss ```NrIrFr)rz-inf:rNNre) reduction)r+r,r-r.r/r0r1r2r^)Nr_i)%rr3rirrrrrr$rFrrrr\r?r permuterrcontrastive_logits_temperatureallanyrMrrsrrrr% cross_entropyrrrediversity_loss_weightr(r/r0)rrrIrrroutputstransformer_featuresrquantized_featuresr.r+r1r2rurXrnegative_quantized_featuresr neg_is_postargetnum_codevectorss&&&&&, r8rWav2Vec2ForPreTraining.forwardSsB  ( 1 4 4UZZ @ --  ) 0    $//0I0IJ 001I1IJ  %!DD &&q)>uEN59NN 5C5 10224>>3H3H3N3NO!^^,>?3777 # /7I7O7O 4J +=*A*A"k*R(--/44R8+ '+F*J*JR+gaAq! ( 44"7++$ :: F-0KKPPQSTJ~~).vr :&%%a+33B AGF,1133t;FFq!LTTVF!}}::6<<>6]b:c "kkCCdkkFgFggO.1FF/Y]n]r]r]ttN$kk&G&G.&XXD+1'9"7!//))-)  r;)rrrrr)g?)NNN)r=r>r?r@rrr}rrr rrrCrDrrs@@r8rrsa  11 =(Q Q Q Q r;rzp Wav2Vec2 Model with a `language modeling` head on top for Connectionist Temporal Classification (CTC). c~aa]tRtRtoR V3RlV3RllltRtRtRt]] R V3RlRll44t R t Vt V;t #) r,ic.<V^8dQhRS[R,/#)r*r6Nr7)r6r7s"r8r9Wav2Vec2ForCTC.__annotate__sC$Jr;c <\SV`V4\V4Vn\P !VP 4VnW nVPf\RVP R24h\VR4'dVP'd VPM VPp\P !W1P4VnVP%4R#)a  target_lang (`str`, *optional*): Language id of adapter weights. Adapter weights are stored in the format adapter..safetensors or adapter..bin. Only relevant when using an instance of [`Wav2Vec2ForCTC`] with adapters. Uses 'eng' by default. NzYou are trying to instantiate z with a configuration that does not define the vocabulary size of the language model head. Please instantiate the model as follows: `Wav2Vec2ForCTC.from_pretrained(..., vocab_size=vocab_size)`. or define `vocab_size` of your model's configuration.r)rrrmrrr final_dropoutrr6rSr`rrrrrr r(rx)rrr6rrs&&& r8rWav2Vec2ForCTC.__init__s  %f- zz&"6"67 &    $00@AHH *1)G)GFL^L^L^F % %djdvdv yy!35F5FG  r;c Z\4\P!R48XdR#VPpVe)\ VP RR4f\ RV R24hVf1\ VP RR4e\PR4R#VeVPVRR7R#R#) a This method overwrites [`~PreTrainedModel.tie_weights`] so that adapter weights can be correctly loaded when passing `target_lang=...` to `from_pretrained(...)`. This method is **not** supposed to be called by the user and is prone to be changed in the future. metaNrszCannot pass `target_lang`: r9z)By default `target_lang` is set to 'eng'.T)rV) rr3rr6rtrr`rGinfor_)rrr6s&, r8 tie_weightsWav2Vec2ForCTC.tie_weights s 6 75<<;O O  &&  "wt{{r?r@rrr}rr rrrCrDrrs@@r8r,r,sD :<0=(: : : : r;r,z Wav2Vec2 Model with a sequence classification head on top (a linear layer over the pooled output) for tasks like SUPERB Keyword Spotting. craa]tRtRtoV3RltRtRt]]] RV3RlRll444t Rt Vt V;t #) !Wav2Vec2ForSequenceClassificationirc:<\SV`V4\VR4'dVP'd \ R4h\ V4VnVP^,pVP'd7\P!\P!V4V, 4Vn \P!VPVP 4Vn\P!VP VP$4VnVP)4R#)rz_Sequence classification does not support the use of Wav2Vec2 adapters (config.add_adapter=True)N)rrrrr`rmrruse_weighted_layer_sumrrr3rn layer_weightsr rclassifier_proj_size projector num_labels classifierrxrr num_layersrs&& r8r*Wav2Vec2ForSequenceClassification.__init__ys  6= ) )f.@.@.@q &f- --1  ( ( (!#ejj.Dz.Q!RD 6#5#5v7R7RS))F$?$?ARARS r;c NVPPP4R#r{rr|s&r8r}8Wav2Vec2ForSequenceClassification.freeze_feature_encoderrr;c VVPP4F pRVnK R#rrrs& r8r3Wav2Vec2ForSequenceClassification.freeze_base_modelrr;c <V^8dQhRS[PR,RS[PR,RS[PR,RS[S[,RS[S[,/#r)r3rrrr5r)r6r7s"r8r9.Wav2Vec2ForSequenceClassification.__annotate__sd9 9 llT)9  t+9  t# 9 +, 9 ) ) 9 r;c (VPP'dRVR&VP!V3RV/VBpVPP'dzVPp\P !V^R7p\ PPVPRR7pWgPR^^4,P^R7pM VPpVPV4pVfVP^R7pMVPVP ^,V4p V P#R4P%^^VP ^,4p RWj(&VP^R7V P^R7PR^4, pVP'V4p Rp VeG\)4p V !V PRVPP*4VPR44p \-V V VPVP.R7#)  input_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Float values of input raw speech waveform. Values can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoProcessor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. See [`Wav2Vec2Processor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Toutput_hidden_statesrIr!Nr>rr^)rrrr/r3stackrr%r&rr?rerrrr$rFrrrrrrr0)rrrIrrrr/ norm_weights pooled_output padding_maskexpand_padding_maskrr+loss_fcts&&&&, r8r)Wav2Vec2ForSequenceClassification.forwards, ;; - - --1F) *--  )   ;; - - -#11M!KK 1=M==001C1C0LL*->->r1a-HHMMRSMTM#55M}5  !)..1.5MBB=CVCVWXCY[ijL"."8"8"<"C"CAq-J]J]^_J`"a 25M. /)--!-4|7G7GA7G7N7S7STVXY7ZZM/  ')HFKKDKK,B,BCV[[QS_UD'!//))   r;)rrrrrr=r>r?r@rr}rr r rrrCrDrrs@@r8rrrsB"=( 9 9 !9 9 r;rcraa]tRtRtoV3RltRtRt]]] RV3RlRll444t Rt Vt V;t #) #Wav2Vec2ForAudioFrameClassificationic<\SV`V4\VR4'dVP'd \ R4h\ V4VnVP^,pVP'd7\P!\P!V4V, 4Vn \P!VPVP 4VnVP VnVP%4R#)rzbAudio frame classification does not support the use of Wav2Vec2 adapters (config.add_adapter=True)N)rrrrr`rmrrrrrr3rnrr rrrrxrs&& r8r,Wav2Vec2ForAudioFrameClassification.__init__s  6= ) )f.@.@.@t &f- --1  ( ( (!#ejj.Dz.Q!RD ))F$6$68I8IJ ++ r;c NVPPP4R#r{rr|s&r8r}:Wav2Vec2ForAudioFrameClassification.freeze_feature_encoderrr;c VVPP4F pRVnK R#rrrs& r8r5Wav2Vec2ForAudioFrameClassification.freeze_base_modelrr;c <V^8dQhRS[PR,RS[PR,RS[PR,RS[S[,RS[S[,/#r)r3rrrr5r)r6r7s"r8r90Wav2Vec2ForAudioFrameClassification.__annotate__sd0 0 llT)0  t+0  t# 0 +, 0 & & 0 r;c VPP'dRVR&VP!V3RV/VBpVPP'dzVPp\P !V^R7p\ PPVPRR7pWgPR^^4,P^R7pM VPpVPV4pRp Ve^\4p V !VPRVP4\P !VPRVP4^R74p \#V VVPVP$R7#) rTrrIr!N)axisrr^)rrrr/r3rrr%r&rr?rerrrrrrr0) rrrIrrrr/rrr+rs &&&&, r8r+Wav2Vec2ForAudioFrameClassification.forwards?, ;; - - --1F) *--  )   ;; - - -#11M!KK 1=M==001C1C0LL*->->r1a-HHMMRSMTM#55M/  ')HFKKDOOtuD$!//))   r;)rrrrrrrs@@r8rrsB =( 0 0 !0 0 r;rc<aa]tRtRtoRV3RlltRtRtVtV;t#) AMSoftmaxLossi/c<\SV`4W0nW@nW n\ P !\P!W4RR7Vn \ P!4Vn R#)T)rN) rrscalemarginrrrr3randnrrr+)rrrrrrs&&&&&r8rAMSoftmaxLoss.__init__0sI   $ll5;;y#EUYZ '') r;cVP4p\PPVP^R7p\PPV^R7p\ P !W4pW@P, p\PPW P4pVP\ P!VP4WT4,pVPWr4pV#)rr!)rrr% normalizerr3mmrone_hotrrrrir+) rr/rr cos_thetapsionehotrr+s &&& r8rAMSoftmaxLoss.forward8s!((!(< // 1/E HH]3 ++%&&v?ekk&++-HHyy( r;)r+rrrr)g>@g?rrs@@r8rr/s*  r;rcHaa]tRtRtoRV3RlltV3RlRltRtVtV;t#) TDNNLayeriFc<\SV`4V^8dVPV^, ,MVPV,VnVPV,VnVP V,VnVPV,Vn\P!VPVP ,VP4Vn \P!4Vn R#)rN)rrtdnn_dimrr tdnn_kernelr tdnn_dilationdilationrr kernelrrrs&&&r8rTDNNLayer.__init__Gs +BFD]D]!^))F$=$=v?X?XY&v'@'@&BSBST RsF)c NVPPP4R#r{rr|s&r8r})Wav2Vec2ForXVector.freeze_feature_encoderrr;c VVPP4F pRVnK R#rrrs& r8r$Wav2Vec2ForXVector.freeze_base_modelrr;cD<V^8dQhRS[PS[,/#)r*rw)r3rNrL)r6r7s"r8r9Wav2Vec2ForXVector.__annotate__s  e6F6F6L r;c XRpVPPF pV!W^4pK V#)z/ Computes the output length of the TDNN layers c.W, V,^,#rr<rs&&&r8rEWav2Vec2ForXVector._get_tdnn_output_lengths.._conv_out_lengths!.69A= =r;)rr%)rrwrrs&& r8_get_tdnn_output_lengths+Wav2Vec2ForXVector._get_tdnn_output_lengthss1  >  ;;22K,]KM3r;c <V^8dQhRS[PR,RS[PR,RS[PR,RS[S[,RS[S[,/#r)r3rrrr5r)r6r7s"r8r9r@sjF F llT)F  t+F  t# F +, F   F r;c VPP'dRVR&VP!V3RV/VBpVPP'dzVPp\P !V^R7p\ PPVPRR7pWgPR^^4,P^R7pM VPpVPV4pVPF pV!V4pK Vf&VP^R7p VP!^R7p MVP#VP^R74p VP%V 4p .p .p \'V 4F[wrV P)WmRV13,P^R74V P)WmRV13,P!^R74K] \P !V 4p \P !V 4p \P*!W.RR7pVP-V4pVP/V4pRpVeVP1VV4p\3VVVVPVP4R7#)rTrrIr!N)r+r embeddingsr/r0r^)rrrr/r3rrr%r&rr?rerrr6rrrrD enumeraterprrqrr8rr0)rrrIrrrr/r tdnn_layer mean_features std_featuresfeat_extract_output_lengthstdnn_output_lengthsrlengthstatistic_poolingoutput_embeddingsrr+s&&&&, r8rWav2Vec2ForXVector.forwardsF, ;; - - --1F) *--  )   ;; - - -#11M!KK 1=M==001C1C0LL*->->r1a-HHMMRSMTM#55M}5 ))J&}5M$  !)..1.5M(,,,3L*.*O*OP^PbPbghPbPi*j '"&"?"?@["\ ML&':; $$]gvg:%>%C%C%C%JK##MWfW*$=$A$Aa$A$HI<"KK 6M ;;|4L!II}&CL 223DE!23  >>&&1D(!//))   r;)rrqrr8rr6rr)r=r>r?r@rr}rrDr r rrrCrDrrs@@r8r4r4fsP &=(   F F !F F r;r4)Nrr)Nr>)rr,rrr4rmr)drArr/collections.abcr dataclassesrnumpyrOr3safetensors.torchrrLrtorch.nnrrr activationsrbackbone_utilsr integrations.deepspeedr integrations.fsdpr masking_utilsr modeling_flash_attention_utilsr modeling_layersrmodeling_outputsrrrrrrmodeling_utilsrrrprocessing_utilsrrrrrrrrr utils.genericr r!utils.output_capturingr"r#configuration_wav2vec2r%rOrK get_loggerr=rG_HIDDEN_STATES_START_POSITIONr(r~rrrrrrrrrr*r,rMr^rqr}rrrrrurrmrr,rrrr"r4__all__r<r;r8ris $! 9%&!9@76B9sr&JE2,5   H %!"  4;4  4:tn$D#=*!;6!;0*bii*Z299%RYY%P 1 1 %8R) R)j"))05D$*D$N+@bii+@\/@RYY/@dI'BIII'Xbii>299$ryy2`'o`'`'F u +u u p C 4C  C L A ,A  A H ] (?]  ] @S *AS S lBII. @ { 0{  { | r;