+ jW(^RIHt^RIHt^RIt^RIHt^RIHt^RI H t ^RI H t H t Ht^RIHt^RIHt^R IHtHt^R IHt^R IHt^R IHtHtHtHtHt^R I H!t!^RI"H#t#H$t$^RI%H&t&^RI'H(t(H)t)H*t*H+t+^RI,H-t-^RI.H/t/H0t0^RI1H2t2H3t3H4t4^RI5H6t6H7t7H8t8^RI9H:t:H;t;]+Px!]=4t>])!RR7]!RR]444t?])!RR7]!RR]444t@!RR]P4tB!R R!]P4tC!R"R#]34tD!R$R%]24tE!R&R']64tF!R(R)]4tG])!R*R+]$44tH!R,R-]H4tI!R.R/]74tJ!R0R1]:4tK])!R2R7!R3R4]H]44tL.R5OtMR#)6)Callable) dataclassN)strict)ACT2FN)Cache DynamicCacheEncoderDecoderCache)PreTrainedConfig)GenerationMixin)create_bidirectional_maskcreate_causal_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPast)BaseModelOutputWithPastAndCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput)RopeParameters)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tuplelogging)merge_with_config_defaults)OutputRecordercapture_outputs) GlmAttentionGlmRotaryEmbeddingapply_rotary_pos_emb)LlamaDecoderLayer LlamaModeleager_attention_forward) WhisperModelshift_tokens_rightzUsefulSensors/moonshine-tiny) checkpointcaa]tRt^2toRtRtR.tRRRRRR R R /tR tR t Rt ^t ^t ^t ^tRtRtRtRtRtRtRt^tRtRtRtRtRt^t^tRtRtV3Rlt V3Rlt!Rt"Vt#V;t$#)MoonshineConfiga encoder_num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `encoder_num_key_value_heads=encoder_num_attention_heads`, the model will use Multi Head Attention (MHA), if `encoder_num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `num_attention_heads`. decoder_num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `decoder_num_key_value_heads=decoder_num_attention_heads`, the model will use Multi Head Attention (MHA), if `decoder_num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `decoder_num_attention_heads`. pad_head_dim_to_multiple_of (`int`, *optional*): Pad head dimension in encoder and decoder to the next multiple of this value. Necessary for using certain optimized attention implementations. encoder_hidden_act (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder. decoder_hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the decoder. Example: ```python >>> from transformers import MoonshineModel, MoonshineConfig >>> # Initializing a Moonshine style configuration >>> configuration = MoonshineConfig().from_pretrained("UsefulSensors/moonshine-tiny") >>> # Initializing a model from the configuration >>> model = MoonshineModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` moonshinepast_key_valuesnum_key_value_headsdecoder_num_key_value_headsnum_attention_headsdecoder_num_attention_headsnum_hidden_layersdecoder_num_hidden_layers hidden_actdecoder_hidden_actii iNgelusiluig{Gz?TFc <VPfVPVnVPfVPVnVP RR4\ SV`!R/VBR#)Npartial_rotary_factorg?)encoder_num_key_value_headsencoder_num_attention_headsr.r0 setdefaultsuper __post_init__)selfkwargs __class__s&,p/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/moonshine/modular_moonshine.pyr?MoonshineConfig.__post_init__sX  + + 3/3/O/OD ,  + + 3/3/O/OD ,137 ''c<V^8dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[R,;R &S[R,;R &S[R,;R &S[;R &S[;R &S[;R&S[;R&S[;R&S[;R&S[S[,R,;R&S[;R&S[;R&S[S[,;R&S[R,;R&S[S[S[,,R,;R&S[R,;R&S[;R&#) vocab_size hidden_sizeintermediate_sizeencoder_num_hidden_layersr2r<r0Nr;r.pad_head_dim_to_multiple_ofencoder_hidden_actr4max_position_embeddingsinitializer_rangedecoder_start_token_id use_cacherope_parametersis_encoder_decoderattention_biasattention_dropout bos_token_id eos_token_id pad_token_idtie_word_embeddings)intstrfloatboolrdictlist)format __classdict__s"rC __annotate__MoonshineConfig.__annotate__2shijkl!mn #&op #&qr"%(st"%(uv"%t2wx"%t2yz"%t2{|$}~$@!&AB#CD #EFGH$d*T18IJ#KL MNs{(OP* QRS /D(,ST*#UV$WrE)r.r;)%__name__ __module__ __qualname____firstlineno____doc__ model_typekeys_to_ignore_at_inference attribute_maprHrIrJrKr2r<r0r;r.rLrMr4rNrOrPrQrRrSrTrUrVrWrXrYr?__annotate_func____static_attributes____classdictcell__ __classcell__rBras@@rCr*r*2s& PJ#4"5<<8* MJK!%&%&'('(.2.2.2$$#&#"#I48O# N%( L+,L#L $([rEr*z Extends [~modeling_outputs.BaseModelOutput] to include the output attention mask since sequence length is not preserved in the model's forward. ) custom_introc4a]tRt^toRtRtV3RltRtVtR#)MoonshineEncoderModelOutputa~ attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices after sequence compression. Returned because the sequence length may differ from the input sequence length. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. NcH<V^8dQh/S[PR,;R&#)rGNattention_masktorchTensor)r`ras"rCrb(MoonshineEncoderModelOutput.__annotate__s LL4'.!rEr:) rdrerfrgrhrurlrmrnras@rCrsrss+/N!rErscDaa]tRt^toV3RltV3RlRltRtVtV;t#)MoonshineEncoderMLPc<\SV`4Wn\V,Vn\ P !VPVP4Vn \ P !VPVP4Vn R#N r>__init__configr activation_fnnnLinearrIrJfc1fc2r@rr3rBs&&&rCrMoonshineEncoderMLP.__init__s\  #J/99V//1I1IJ99V55v7I7IJrEcN<V^8dQhRS[PRS[P/#rG hidden_statesreturnrv)r`ras"rCrb MoonshineEncoderMLP.__annotate__s#U\\ellrEclVPV4pVPV4pVPV4pV#r~)rrr)r@rs&&rCforwardMoonshineEncoderMLP.forwards4/ **=9 / rErrrr rdrerfrgrrrmrnrorps@@rCr|r|sKrEr|cDaa]tRt^toV3RltV3RlRltRtVtV;t#)MoonshineDecoderMLPc"<\SV`4Wn\V,Vn\ P !VPVP^,4Vn \ P !VPVP4Vn R#)rGNrrs&&&rCrMoonshineDecoderMLP.__init__sa  #J/99V//1I1IA1MN99V55v7I7IJrEcN<V^8dQhRS[PRS[P/#rrv)r`ras"rCrb MoonshineDecoderMLP.__annotate__s#U\\ellrEcVPV4pVP^RR7wrVPV4V,pVPV4pV#)rG)dim)rchunkrr)r@rgates&& rCrMoonshineDecoderMLP.forwardsQ/ +11!1< **40=@ / rErrrps@@rCrrsKrErc]tRt^tRtR#)MoonshineRotaryEmbeddingr:N)rdrerfrgrmr:rErCrrsrErcTaa]tRt^toV3RlV3RlltRV3RlRlltRtVtV;t#)MoonshineAttentionc 8<V^8dQhRS[RS[RS[RS[RS[/#)rGr layer_idx is_causalr/r-)r*rZr])r`ras"rCrbMoonshineAttention.__annotate__s=&&&& & ! & ! &rEc<VPRVRV/4\SV` W4W0n\ VRVP VP ,4VnVPPeWVPPpW`PV,^, V,,pWpP, Vn R#^Vn R#)r/r-head_dimN) updater>rrgetattrrIr/rrrLhead_dim_padding) r@rrrr/r-target_multipletarget_head_dimrBs &&&&&& rCrMoonshineAttention.__init__s  ,.ACXZmno +" F4F4F&JdJd4de  ;; 2 2 >"kkEEO---/2QTU2UZi1ijO$3mm$CD !$%D !rEc<V^8dQhRS[PRS[S[PS[P3,R,RS[PR,RS[R,RS[PR,RS[S[,RS[S[PS[PR,S[S[P,R,3,/#) rGrposition_embeddingsNrur,key_value_statesrAr)rwrxtuplerrr)r`ras"rCrbrsO)O)||O)#5<<#=>EO) t+ O)  O)  ,,- O)-.O) u||U\\D0% 2E2LL MO)rEc VPRRwrxVPV4PWxVPPVP 4P ^^4p VRJp Ve`VPPVP4p V 'd'RVPVP&VPpM VPpVeTMTp V 'd`V'dXX 'dPVPVP,Pp VPVP,PpMVPV 4PVRVPPVP 4P ^^4p VP!V 4PVRVPPVP 4P ^^4pV 'd#VeVP#WVP4wrV 'g7Vwpp\%WVV4wrVeVP#WVP4wr\&P(!VPP*\,4pVP.;'dVRJ;'dV^8pVP0^8d\2P4P6P9V ^VP034p \2P4P6P9V ^VP034p \2P4P6P9V^VP034pT!VV V VV3RVP:'gRM VP<RVP>RV/VBwppVP0^8dVRRVP0)13,pVPAWxR4PC4pVPEV4pVV3#)NTdropoutr7scalingr.r)#shapeq_projviewrr-r transpose is_updatedgetrcross_attention_cacheself_attention_cachelayerskeysvaluesk_projv_projrr"r get_interface_attn_implementationr%rrrwr functionalpadtrainingrUrreshape contiguouso_proj)r@rrrur,rrAbszq_len query_statesis_cross_attentionrcurrent_states key_states value_statescossinattention_interfacer attn_output attn_weightss&&&&&&, rCrMoonshineAttention.forwardsa#(("-  KK & + +C 8W8WY]YfYf g q qrsuv w .T9  &(3377GJ!=A**4>>:"1"G"G"1"F"F.>-I)} /j(//?DDJ*11$..AHHL N+c2t{{>> N1a   N+c2t{{>> N1a  "o&A+:+A+A*\`\j\j+k( !*HC';LVY[^'_ $L*+:+A+A*\`\j\j+k( (?(M(M KK , ,.E) NNKK~'=KK%!)  1 $ 88..22aa]tRtRtoV3RlV3RlltRtVtV;t#)MoonshineEncoderLayeri(c&<V^8dQhRS[RS[/#)rGrrr*rZ)r`ras"rCrb"MoonshineEncoderLayer.__annotate__)s U U U3 UrEcH<\SV`W4\VVRVPVPR7Vn\ WP4Vn\P!VPRR7Vn \P!VPRR7Vn R#)Frrrr/r-biasN)r>rrr<r; self_attnr|rMmlpr LayerNormrIinput_layernormpost_attention_layernormr@rrrBs&&&rCrMoonshineEncoderLayer.__init__)s} ++ & B B & B B  'v/H/HI!||F,>,>UK(* V5G5Ge(T%rE)rrrr)rdrerfrgrrmrnrorps@@rCrr(s U U UrErcXaa]tRtRtoRV3RlV3RllltRV3RlRlltRtVtV;t#) MoonshineDecoderLayeri9c4<V^8dQhRS[RS[R,/#)rGrrNr)r`ras"rCrb"MoonshineDecoderLayer.__annotate__:s#LLL3:LrEc<\SV`4VPVn\VVRVPVP R7Vn\VVRVPVP R7Vn\WP4Vn \P!VPRR7Vn \P!VPRR7Vn\P!VPRR7VnR#)TrFrN)r>rrIrr/r-r encoder_attnrr3rrrrrfinal_layernormrs&&&rCrMoonshineDecoderLayer.__init__:s !--+ & : : & : :  / & : : & : :  'v/@/@A!||F,>,>UK(* V5G5Ge(T%!||F,>,>UKrEcZ<V^8dQhRS[PRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[R,R S[R,R S[S[PS[P3,R,R S[S[PS[P3,R,R S[S[,R S[S[PS[S[PS[P3,R,3,/ #)rGrruNencoder_hidden_statesencoder_attention_mask position_idsencoder_position_idsr,rQrencoder_position_embeddingsrAr) rwrx LongTensorrr]rrr FloatTensor)r`ras"rCrbrRs ,,||, t+, %||d2 , !& t 3 , &&- ,$..5,,$;,#5<<#=>E,&+5<<+E%F%M,+,, u  %(9(95;L;L(L"MPT"TT U,rEc  BTp VPV4pVP!RRVRVRVRVRVRV /V BwrW,pVe4Tp VPV4pVPVVVVVR7wrW,pTp VP V4pVP V4pW,pV#)rrurr,rQr)rrrur,rQr:)rrrrrr)r@rrurrrrr,rQrrrAresidual_s&&&&&&&&&&&, rCrMoonshineDecoderLayer.forwardRs!,,]; >> ' ) & ,    !4    !0 ,$H 99-HM#00+!65 /# 1  M%4M ,,]; /  0 rE)rrrIrrrrr~) NNNNNNFNNrrps@@rCrr9s LL0,,,rErcba]tRtRtoRtRtRtRtRR.tRt Rt Rt V3RlR lt V3R lt R tVtR #) MoonshinePreTrainedModelimodel input_valuesaudioTrrc4<V^8dQhRS[P/#)rG input_lengths)rwr)r`ras"rCrb%MoonshinePreTrainedModel.__annotate__s##e>N>N#rEc \V^, ^@, ^,4p\V^, ^, ^,4p\V^, ^, ^,4pV#)z8 Computes the output length of the convolutional layers )rZ)r@routput_conv1_lengthoutput_conv2_lengthoutput_conv3_lengths&& rC _get_feat_extract_output_lengths9MoonshinePreTrainedModel._get_feat_extract_output_lengthssZ"=3#6"") num_groups num_channelsepsrrN)r>rrrIrConv1dconv1conv2conv3 GroupNorm groupnorm ModuleListrangerKrrr layer_normr rotary_embgradient_checkpointing post_init)r@r embed_dimidxrBs&& rCrMoonshineEncoder.__init__s   && YYq)ReT YYy!i-QqQ YYq9}iQqQ PTUmm;@AaAa;b c;bC "6 /;b c  ,,yu=2&A&+#  dsD;c4<V^8dQhRS[P/#rGrrModule)r`ras"rCrbrsbiirEcVP#r~r#r@s&rCget_input_embeddings%MoonshineEncoder.get_input_embeddingss zzrEc4<V^8dQhRS[P/#)rGvaluer3)r`ras"rCrbrs"))rEcWnR#r~r6)r@r;s&&rCset_input_embeddings%MoonshineEncoder.set_input_embeddingss rEc <V^8dQhRS[PRS[PR,RS[S[,RS[S[,/#)rGrruNrAr)rwrrxrrrr)r`ras"rCrbrsN< < ''<  t+< +, < ( ( < rEc VP^4p\PPVP V44pVP V4p\PP VPV44p\PP VPV44pVP^^^4pRpVe@VPVPR ,4pR pVRRRV13,RRV13,pTp\VPVVVR7p\P!^VP^,VP R7P^4pVP#WHR7p VP$Fp V !V3RVRVRV /VBpK VP'V4p\)TVeVP+4R 7#RR 7#) a Args: input_values (`torch.FloatTensor` of shape `(batch_size, audio_length)`): Float values of the raw speech waveform. Raw speech waveform can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoFeatureExtractor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding indices in `input_values`. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) N.r inputs_embedsrurdevicerrurr)last_hidden_stateruri) unsqueezerrtanhr#r'r5r$r%permuterrr rrwarangerDr+rr*rsrZ) r@rrurAroutput_attention_maskmask_lendownsample_striderr encoder_layers &&&, rCrMoonshineEncoder.forwards.$--a0  **4::l+CD }5  **4::m+DE  **4::m+DE %--aA6 !%  %<<^=Q=QRT=UVH * +C1D3D1D,DEc9H9nUN$2 !2;;')"/  ||A}':':1'=mFZFZ[eefgh "oomoW![[M)-*%8   M) 6 *+:O:[0446  ae  rE) rr#r$r%r,r'r*rr+r~)rdrerfrgrhrrr_can_record_outputsrr8r=rrrrmrnrorps@@rCrrsi%O(. $ < <  < < rErc aa]tRtRtoRtR]!]^RR7R]R]!]^RR7/tV3R lV3R llt ] ] RV3R lR ll44t R t VtV;t#)MoonshineDecoderi input_idsrr)index layer_namercross_attentionsrc <V^8dQhRS[/#r r )r`ras"rCrbMoonshineDecoder.__annotate__ suuurEc  <\SV`V4\P!VPRR7Vn\P !\VP4Uu.uFp\W4NK up4Vn R#uupiFrN) r>rrrrInormr(r)r1rr)r@rr/rBs&& rCrMoonshineDecoder.__init__ s_  LL!3!3%@ mmSXY_YqYqSr$sSrC%:6%GSr$st $ssBc^<V^8dQhRS[PR,RS[PR,RS[PR,RS[R,RS[PR,RS[R,RS[PR,R S[PR,R S[S[,R S[S[ ,/ #) rGrSNrurr,rBrQrrrAr) rwrrxrrr]rrrr)r`ras"rCrbrXsG G ##d*G  t+G &&- G  G ((4/ G $;G  %0047G !& t 3G +,G  ( (G rEc  VRJVRJ, 'd \R4hVfVPV4pV'd9Vf5\\VPR7\VPR74pVfbVeVP 4M^p \ P!VP^,VPR7V ,pVP^4p\VPVVVVR7p \VPVVVR7pTp VPWR7p VPFpV!V V V3RVR VR VR VR V /V Bp K VPV 4p \!T V'dVR 7#RR 7#)a\ encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding indices in `encoder_hidden_states`. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) Nz:You must specify exactly one of input_ids or inputs_embedsr!rC)rrBrur,rrArErrr,rQr)rFr,) ValueError embed_tokensr rrget_seq_lengthrwrJrrDrGr r r+rr[r)r@rSrurr,rBrQrrrApast_seen_tokens causal_maskrr decoder_layers&&&&&&&&&, rCrMoonshineDecoder.forwards0  -t"; < <YZ Z   --i8M 01,dkk2RT`hlhshsTtuO  CRC^==?de  << (;(;A(>}G[G[\_ooL'11!4L(;;')+%  ";;;'1"7 " & "oomoW![[M)% (>  *  !0 $ %8  M) -0 8+/8O  >B  rE)rr[)NNNNNNNN)rdrerfrgrrrrrPrrrrrmrnrorps@@rCrRrRsm!On%7q[Y.N+=QSab uu  G G  G G rErRcPa]tRtRtoRt]]RV3RlRll44tRtVt R#)MoonshineModeliZc\R4h)zNot needed for Moonshine)AttributeErrorr7s&rC_mask_input_features#MoonshineModel._mask_input_features[s788rENc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[S[S[P,,R,RS[R,RS[S[P,R,R S[S[P,R,R S[R,R S[S[,R S[/ #) rGrNrudecoder_input_idsdecoder_attention_maskencoder_outputsr,decoder_inputs_embedsdecoder_position_idsrQrAr) rwrrrr r]rrr)r`ras"rCrbMoonshineModel.__annotate__`sC C ''$.C ((4/C !++d2 C !& 0 04 7 C uU%6%6784? C -t3C  %U%6%67$>C $E$4$45<C $;C +,C  C rEc  fVfVP!V3RV/V BpVP!R RVRVRVPRVPRVRVRVRV /V Bp \ V PV P V P V PV PVPVP VPR 7#) a: input_values (`torch.FloatTensor` of shape `(batch_size, audio_length)`): Float values of the raw speech waveform. Raw speech waveform can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoFeatureExtractor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. decoder_position_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`): Indices of positions of each input sequence tokens in the position embeddings. Used to calculate the position embeddings up to `config.decoder_config.max_position_embeddings` Example: ```python >>> import torch >>> from transformers import AutoFeatureExtractor, MoonshineModel >>> from datasets import load_dataset >>> model = MoonshineModel.from_pretrained("UsefulSensors/moonshine-tiny") >>> feature_extractor = AutoFeatureExtractor.from_pretrained("UsefulSensors/moonshine-tiny") >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> inputs = feature_extractor(ds[0]["audio"]["array"], return_tensors="pt") >>> input_values = inputs.input_values >>> decoder_input_ids = torch.tensor([[1, 1]]) * model.config.decoder_start_token_id >>> last_hidden_state = model(input_values, decoder_input_ids=decoder_input_ids).last_hidden_state >>> list(last_hidden_state.shape) [1, 2, 288] ``` rurSrrr,rBrrQ)rFr,decoder_hidden_statesdecoder_attentionsrVencoder_last_hidden_staterencoder_attentionsr:) encoderdecoderrFrurr,rrrV) r@rrurmrnror,rprqrQrAdecoder_outputss &&&&&&&&&&, rCrMoonshineModel.forward^sZ  "/3||L/rYg/rkq/rOEI\\ F ' F 1 F #2"C"C F $3#A#A F , F 0 F . F   F  F "-??+;;"1"?"?.99,==&5&G&G"1"?"?.99  rEr:) NNNNNNNNN) rdrerfrgrjrrrrmrnrzs@rCrgrgZs.9C C C rErgzj The Moonshine Model with a language modeling head. Can be used for automatic speech recognition. caa]tRtRtoRR/tV3RlV3RlltRtRtV3RlR lt] ] R V3R lR ll44t R t Vt V;t#)!MoonshineForConditionalGenerationizproj_out.weightz!model.decoder.embed_tokens.weightc <V^8dQhRS[/#r r )r`ras"rCrb.MoonshineForConditionalGeneration.__annotate__srEc<\SV`V4\V4Vn\P !VP VPRR7VnVP4R#rZ) r>rrgrrrrIrHproj_outr-)r@rrBs&&rCr*MoonshineForConditionalGeneration.__init__sH  #F+  &"4"4f6G6GeT  rEcVP#r~rr7s&rCget_output_embeddings7MoonshineForConditionalGeneration.get_output_embeddingss }}rEcWnR#r~r)r@new_embeddingss&&rCset_output_embeddings7MoonshineForConditionalGeneration.set_output_embeddingss& rEc4<V^8dQhRS[P/#r2r3)r`ras"rCrbrs11bii1rEc6VPP4#r~)rr8r7s&rCr86MoonshineForConditionalGeneration.get_input_embeddingsszz..00rEc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[S[S[P,,R,RS[R,RS[S[P,R,R S[S[P,R,R S[R,R S[PR,R S[S[,R S[/ #)rGrNrurmrnror,rprqrQlabelsrAr) rwrrrr r]rrr)r`ras"rCrbrs ` ` ''$.` ((4/` !++d2 ` !& 0 04 7 ` uU%6%6784? ` -t3`  %U%6%67$>` $E$4$45<` $;`   4'` +,`  ` rEc  JV PRR4p V e=Vf9Vf5\WPPVPP4pVP !V3RVRVRVRVRVRVR VR V /V Bp VP V P4pRpV e5VP!RR TR RR VPPRV eT MT /V Bp\VVV PV PV PV PV PV P V P"R7 #)ah input_values (`torch.FloatTensor` of shape `(batch_size, audio_length)`): Float values of the raw speech waveform. Raw speech waveform can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoFeatureExtractor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. decoder_position_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`): Indices of positions of each input sequence tokens in the position embeddings. Used to calculate the position embeddings up to `config.decoder_config.max_position_embeddings` Example: ```python >>> import torch >>> from transformers import AutoProcessor, MoonshineForConditionalGeneration >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("UsefulSensors/moonshine-tiny") >>> model = MoonshineForConditionalGeneration.from_pretrained("UsefulSensors/moonshine-tiny") >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> inputs = processor(ds[0]["audio"]["array"], return_tensors="pt") >>> input_values = inputs.input_values >>> generated_ids = model.generate(input_values, max_new_tokens=100) >>> transcription = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] >>> transcription 'Mr. Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.' ``` shift_labelsNrurmrornr,rprqrQlogitsrrH) lossrr,rtrurVrvrrwr:)popr'rrXrPrrrF loss_functionrHrr,rtrurVrvrrw)r@rrurmrnror,rprqrQrrAroutputsrrs&&&&&&&&&&&, rCr)MoonshineForConditionalGeneration.forwards|hzz.$7   (-B-J$6KK44dkk6X6X%!'+jj  ' ) ' 0 ' , ' $: ' , ' #8 ' "6 '   '  ' w889   %% ;;11.:-E\6   D#33")"?"?&99$55&-&G&G")"?"?&99  rE)rr) NNNNNNNNNN)rdrerfrg_tied_weights_keysrrrr8rrrrmrnrorps@@rCr}r}s[ ,-PQ'11` ` ` ` rEr})r*rgrr})Ncollections.abcr dataclassesrrwtorch.nnrhuggingface_hub.dataclassesr activationsr cache_utilsrrr configuration_utilsr generationr masking_utilsr r modeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_rope_utilsrmodeling_utilsrrprocessing_utilsrutilsrrrr utils.genericrutils.output_capturingrrglm.modeling_glmr r!r"llama.modeling_llamar#r$r%whisper.modeling_whisperr&r' get_loggerrdloggerr*rsr4r|rrrrrrrrRrgr}__all__r:rErCrs%! .!CC3)JB92F&RR7EUUYYG   H %9:S(&S(;S(l   // /  / "))  "))   1 e)e)PU-U"E6EP###0d /d NV zV rI \I X v (@/v  v r rE