+ j-Rt^RIt^RIHt^RIHt^RIHt^RI H t ^RI H t ^RI Ht^RIHt^RIHt^R IHtHtHtHtHtHtHt^R IHt^t!R R ]P<4t!R R]4t !RR]4t!!RR]P<4t"!RR]4t#!RR]4t$]!RR]44t%!RR]]%4t&!RR]4t'!RR]4t(.ROt)R#) zPyTorch Hubert model.N)initialization)ACT2FN)is_deepspeed_zero3_enabled)BaseModelOutput)PreTrainedModel)auto_docstring)OutputRecorder)Wav2Vec2EncoderWav2Vec2EncoderStableLayerNormWav2Vec2FeatureEncoderWav2Vec2ForCTC!Wav2Vec2ForSequenceClassification Wav2Vec2ModelWav2Vec2SamePadLayer HubertConfigc8aa]tRt^)toV3RltRtRtVtV;t#)HubertPositionalConvEmbeddingcz<\SV`4\P!VPVPVP VP ^,VP R7VnRVn\VRR4'd(\P!VP4VnEM\PPp\\PPR4'd%\PPPp\4'Ed5^RIpVP"P%VPP&^R7;_uu_4V!VPR^R7VnRRR4\VPR 4'dVVPPP&P(pVPPP&P*pM,VPP,pVPP.pVP"P1W4VP"P1W4MV!VPR^R7Vn\3VP 4Vn\6VP8,VnR# +'giEL7;i) ) kernel_sizepaddinggroupsNconv_pos_batch_normF weight_norm modifier_rankweight)namedimparametrizations)super__init__nnConv1d hidden_sizenum_conv_pos_embeddingsnum_conv_pos_embedding_groupsconv batch_normgetattr BatchNorm1dutilsrhasattrr r deepspeedzeroGatheredParametersr original0 original1weight_gweight_vregister_external_parameterHubertSamePadLayerrrfeat_extract_activation activation)selfconfigrr.r3r4 __class__s&& j/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/hubert/modular_hubert.pyr"&HubertPositionalConvEmbedding.__init__*s II      6622a777    60% 8 8 nnV-?-?@DO((..Krxx00-@@ hh77CC )++ ^^66tyy7G7GWX6YY +DIIH! LDIZ499&899#yy99@@JJH#yy99@@JJH#yy11H#yy11H::4J::4J' aH )&*H*HI  !?!?@ZYYs J)) J: cVP^^4pVPeVPV4pVPV4pVPV4pVP V4pVP^^4pV#)) transposer)r(rr8r9 hidden_statess&&r<forward%HubertPositionalConvEmbedding.forwardOsn%//15 ?? & OOM:M -0  ]3  6 %//15 )r8r)r(r __name__ __module__ __qualname____firstlineno__r"rC__static_attributes____classdictcell__ __classcell__r; __classdict__s@@r<rr)s#AJ  rErc]tRt^[tRtR#)r6NrGrHrIrJrKrQrEr<r6r6[rEr6c]tRt^_tRtR#)HubertFeatureEncoderrQNrRrQrEr<rUrU_rSrErUc8aa]tRt^ctoV3RltRtRtVtV;t#)HubertFeatureProjectionc<\SV`4VPVnVP'd9\P!VP R,VP R7Vn\P!VP R,VP4Vn \P!VP4Vn R#)r?)epsN)r!r"feat_proj_layer_normr# LayerNormconv_dimlayer_norm_eps layer_normLinearr% projectionDropoutfeat_proj_dropoutdropoutr9r:r;s&&r<r" HubertFeatureProjection.__init__ds $*$?$?!  $ $ $ ll6??2+>FDYDYZDO))FOOB$79K9KLzz&":":; rEcVP'dVPV4pVPV4pVPV4pV#)N)r[r_rardrAs&&r<rCHubertFeatureProjection.forwardls>  $ $ $ OOM:M 6  ]3 rE)rdr[r_rarFrNs@@r<rWrWcs<rErWc]tRt^utRtR#) HubertEncoderrQNrRrQrEr<rjrjurSrErjc]tRt^ytRtR#)HubertEncoderStableLayerNormrQNrRrQrEr<rlrlyrSrErlc aa]tRt^}toRtRtRtRR.tRtRt Rt Rt R] ] .R]!]^R R 7/t]P$!4V3R l4tV3R lR ltV3RlRltV3RltRtVtV;t#)HubertPreTrainedModelhubert input_valuesaudioHubertEncoderLayerParametrizedConv1dTrB attentionsencoder)index layer_namec <\SV`V4\V\P4'EdI\ 4'd^RIp\VR4'dx\VR4'dfVPPVPVP.^R7;_uu_4\P!VP4RRR4MyVPPVP^R7;_uu_4\P!VP4RRR4M \P!VP4VPe#\P !VP4R#R#\V\"4'd7\VR4'd#\P$!VP&4R#R#\V\(4'dZ\VR4'dF\P*!VP,RVP.P0^,, 4R#R#R# +'giL;i +'giEL;i)zInitialize the weightsNr4r3rmasked_spec_embed layer_weightsg?)r! _init_weights isinstancer#r$rr.r-r/r0r4r3initkaiming_normal_rbiaszeros_ HubertModeluniform_ryHubertForSequenceClassification constant_rzr:num_hidden_layers)r9moduler.r;s&& r<r{#HubertPreTrainedModel._init_weightss f% fbii ( ()++ 6:..76:3N3N"::FOOV__;]mn:oo,,V]];po#::6==XY:ZZ,,V]];[Z$$V]]3{{& FKK('  , ,v233 f6674  ? @ @v//v33SDKK!I 7 I I cD<V^8dQhRS[PS[,/#)r input_lengths)torch LongTensorint)formatrOs"r< __annotate__"HubertPreTrainedModel.__annotate__s  e>N>NQT>T rEc Rp\VPPVPP4Fwr4V!WV4pK V#)z8 Computes the output length of the convolutional layers cN\P!W, VRR7^,#)floor) rounding_mode)rdiv) input_lengthrstrides&&&r<_conv_out_lengthPHubertPreTrainedModel._get_feat_extract_output_lengths.._conv_out_lengths 99\7wWZ[[ [rE)zipr: conv_kernel conv_stride)r9rrrrs&& r< _get_feat_extract_output_lengths6HubertPreTrainedModel._get_feat_extract_output_lengthssG  \ $'t{{'>'> @W@W#X K,]PM$YrEc:<V^8dQhRS[RS[P/#)rfeature_vector_lengthattention_mask)rrr)rrOs"r<rrs"   ]b]m]m rEcVPVPR44P\P4pVP ^,p\P !WA3VPVPR7p^V\P!VP ^,VPR7V^, 3&VPR.4PR4PR.4P4pV#)r?)dtypedevice)rrZ) rsumtorlongshapezerosrrarangeflipcumsumbool)r9rroutput_lengths batch_sizes&&& r<"_get_feature_vector_attention_mask8HubertPreTrainedModel._get_feature_vector_attention_masks>>~?Q?QRT?UVYYZ_ZdZde#))!,   /~7K7KTbTiTi uv ^%9%9!%<^EZEZ[]kno]opq',,bT299"=BBB4HMMOrEc&<V^8dQh/S[;R&#rr:r)rrOs"r<rr}s rErQ)rGrHrIrJbase_model_prefixmain_input_nameinput_modalities_no_split_modulessupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnrr!HubertEncoderLayerStableLayerNormrHubertAttention_can_record_outputsrno_gradr{rr__annotate_func__rKrLrMrNs@@r<rnrn}s!$O-/CD&*#N,.OPn_A)T  ]]_``2    srErncZaa]tRt^toV3RlV3RlltRtRV3RlRlltRtVtV;t #)rc <V^8dQhRS[/#rr)rrOs"r<rHubertModel.__annotate__s|rEc<\SV`V4Wn\V4Vn\ V4VnVPR8gVPR8dH\P!\P!VP4P44VnVP 'd\#V4VnM\'V4VnVP)4V=R#)gN)r!r"r:rUfeature_extractorrWfeature_projectionmask_time_probmask_feature_probr# ParameterrTensorr%rrydo_stable_layer_normrlrurj post_initadapterres&&r<r"HubertModel.__init__s   !5f!="9&"A  3 &&*B*BS*H%'\\%,,v?Q?Q2R2[2[2]%^D "  & & &7?DL(0DL  LrEc\R4h)zNot needed for Hubert)AttributeError)r9s&r<freeze_feature_encoder"HubertModel.freeze_feature_encoders455rEc<V^8dQhRS[PR,RS[PR,RS[PR,RS[R,RS[R,RS[R,RS[S[,/#) rrpNrmask_time_indicesoutput_attentionsoutput_hidden_states return_dictreturn)rr FloatTensorrtupler)rrOs"r<rrsE E llT)E  t+E !,,t3 E  $; E #Tk E D[E   E rEc 8VeTMVPPpVeTMVPPpVeTMVPPpVP V4pVP ^^4pVe$VP VP^,V4pVPV4p VPWR7p VPV VVVVR7p V ^,p V'gV 3V R,,#\V V PV PR7#)a mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Example: ```python >>> from transformers import AutoProcessor, HubertModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft") >>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft") >>> def map_to_array(example): ... example["speech"] = example["audio"]["array"] ... return example >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.map(map_to_array) >>> input_values = processor(ds["speech"][0], return_tensors="pt").input_values # Batch size 1 >>> hidden_states = model(input_values).last_hidden_state ```)r)rrrr:r?NN)last_hidden_staterBrt)r:rrrrr@rrr_mask_hidden_statesrurrBrt) r9rprrrrrkwargsextract_featuresrBencoder_outputss &&&&&&&, r<rCHubertModel.forwards'H2C1N-TXT_T_TqTq$8$D $++JjJj &1%rs &!@/-#4/!"/BII/d -  1 bii$ O  #A BOBBJ\ -!6\ ~ >  &G  frE