+ jRt^RIHt^RIHt^RIHt^RIt^RIH uH t ^RIH t ^RI H t^RIHt^RIHtHt^R IHt^R IHt^R IHt^R IHt^R IHt^RIHtH t ^RI!H"t"H#t#H$t$^RI%H&t&^RI'H(t(H)t)H*t*H+t+^RI,H-t-^RI.H/t/H0t0^RI1H2t2H3t3^RI4H5t5^RI6H7t7^RI8H9t9H:t:]+Pv!]<4t=])!RR7]!RR]444t>])!RR7]!RR]444t?RR+l4tJ!R,R-] P4tKR?R.R/lltL!R0R1] P4tM!R2R3]4tN!R4R5]4tO])!R6R7]$44tP])!R8R9]P44tQ!R:R;]P]4tR.R@OtSR#)AzPyTorch Idefics model.)Callable) dataclass)AnyN)nn)initialization)ACT2FN)Cache DynamicCache)GenerationMixin)use_kernel_forward_from_hub)create_causal_mask)GradientCheckpointingLayer) ModelOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedConfigPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tuplelogging)deprecate_kwarg)maybe_autocastmerge_with_config_defaults)OutputRecordercapture_outputs IdeficsConfig)IdeficsPerceiverResampler)IdeficsVisionEmbeddingsIdeficsVisionTransformerz{ Base class for Idefics model's outputs that may also contain a past key/values (to speed up sequential decoding). ) custom_introcDa]tRt^4toRtRtRtRtRtRt V3Rlt Rt Vt R#)IdeficsBaseModelOutputWithPastaa last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1, hidden_size)` is output. image_hidden_states (`tuple(torch.FloatTensor)`, *optional*): Tuple of `torch.FloatTensor` (one for the output of the image embeddings, `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder, and optionally by the perceiver Nc<V^8dQh/S[PR,;R&S[R,;R&S[S[P,R,;R&S[S[P,R,;R&S[S[P,R,;R&#)Nlast_hidden_statepast_key_values hidden_states attentionsimage_hidden_statestorch FloatTensorrtuple)format __classdict__s"m/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/idefics/modeling_idefics.py __annotate__+IdeficsBaseModelOutputWithPast.__annotate__4s(((4/6)*T\(+,**+d29-.e''(4/6/0u001D8?1) __name__ __module__ __qualname____firstlineno____doc__r(r)r*r+r,__annotate_func____static_attributes____classdictcell__r2s@r3r%r%4s. 37$(O59M26J;?1r6r%zS Base class for Idefics causal language model (or autoregressive) outputs. cHa]tRt^OtoRtRtRtRtRtRt Rt V3Rlt Rt Vt R#)IdeficsCausalLMOutputWithPasta1 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`tuple(torch.FloatTensor)`, *optional*): Tuple of `torch.FloatTensor` (one for the output of the image embeddings, `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder, and optionally by the perceiver NcJ<V^8dQh/S[PR,;R&S[PR,;R&S[R,;R&S[S[P,R,;R&S[S[P,R,;R&S[S[P,R,;R&#)r'Nlosslogitsr)r*r+r,r-)r1r2s"r3r4*IdeficsCausalLMOutputWithPast.__annotate__Os0   d ")12    $+34T\(56**+d2978e''(4/69:u001D8?;r6r7)r8r9r:r;r<rDrEr)r*r+r,r=r>r?r@s@r3rBrBOs2"&*D'+F$(O59M26J;?;r6rBc j\P!VP^,4PR^4P ^V4PR4P VP 4pVP^V4pVPR4VR&VPR4VR&VPR4VR&VPR4VR&RV9dVR,pVP^V4VR&VeVP^V4VR&VR,eVR,P^V4VR&VR,e VR,P^V4VR&W3#VR,e VR,P^V4VR&W3#VR,eVR,P^V4VR&W3#) pixel_valuesimage_encoder_embeddingsperceiver_embeddingsimage_attention_masktoken_type_idsattention_mask) r.arangeshapeviewrepeattodevice index_selectget) input_ids expand_sizeis_encoder_decoderrNencoder_outputs model_kwargsexpanded_return_idxrMs&&&&&, r3expand_inputs_for_generationr^os  Y__Q'(--b!4;;A{KPPQSTWWXaXhXhi&&q*=>I#/#3#3N#CL /;/?/?@Z/[L+,+7+;+;freeze_model..s$]D\qZ%:%:D\sTF)rr`rarbmodulesanyrequires_grad_)modelmodule_exceptionsmappingmmodule_exceptions_mappedrhs&& @r3 freeze_modelrssR\\"))R\\G 5FF4Eq 4EF--/ $]D\$]$]D\$]!]!]  ! !$ '  ! !% ( " L GsCc^aa]tRt^toRtRV3RlV3RllltRtV3RlRltRtVt V;t #) IdeficsDecoupledEmbeddinga Implements a decoupling of parameters to allow freezing (or not) a subset of the embeddings. In practise, the regular `weight` can be trained or frozen (i.e. `partially_freeze=True`), and if `num_additional_embeddings` > 0, then it will create `num_additional_embeddings` additional parameters that are always trained. If `num_additional_embeddings=0`, then the module defaults back to the regular behavior of `nn.Embedding`. c2<V^8dQhRS[R,RR/#)r'partially_freezeNreturn)bool)r1r2s"r3r4&IdeficsDecoupledEmbedding.__annotate__s%33 + 3 3r6c `<VeWq8d\RV RV 24h\S V`!R RVRVRVRVRV/VBWnWpnW nW@nV'dVPPR 4VP ^8d,\P!VP VVVR 7Vn R#R#) a Args: num_embeddings (`int`): Size of the dictionary of embeddings num_additional_embeddings (`int`): Number of additional embeddings. Only useful when you `partially_freeze=True`. embedding_dim (`int`): The size of each embedding vector partially_freeze: (`bool`, *optional*, defaults to `False`): If `True`, the regular `weight` will be frozen. `additional_weight` is never frozen. padding_idx (`int`, *optional*): The padding index (needs to be less than num_embeddings) Note: there are a lot of other parameters to initialize a standard `nn.Embedding` such as `padding_idx`, `max_norm` or `norm_type`. We are not supporting these. Nz/padding_idx must be within num_embeddings. Got z and num_embeddings embedding_dimrUdtype padding_idxF)r|r}rUr~r7) ValueErrorsuper__init__r|rnum_additional_embeddingsrwweightrmrrbadditional_embedding) selfr|rr}rwrUr~rkwargs __class__s &&&&&&&&,r3r"IdeficsDecoupledEmbedding.__init__s6  "{'CN{m[`ao`pqr r  ) '    $    -&)B& 0  KK & &u -  ) )A -(* #==+ )D % .r6c nVP^8Xd!\P!WP4#VP 4p\ P !WP84pW,pVPW0P, 4p^W&\P!WP4pWEV&V#)a{ we have 2 embeddings, with different indices - one pretrained self.weight and another self.additional_embedding.weight that is being trained. in order to make a lookup of the input ids, we: 1. find out the indices of the entries belonging to the 2nd embedding 2. extract those values while subtracting the size of the first embedding (num_embeddings), since the 2nd embedding starts from 0 and not num_embeddings 3. perform the 2nd embedding lookup 4. now we handle the 1st embedding, we overwrite indices belonging to the 2nd embedding with a padding index 5. perform the 1st embedding lookup 6. now we overwrite the values in the 1st embedding lookup with the values of the 2nd embedding lookup note: for the 1st embedding lookup we could have looked up only the low indices and not do the padding, but then we have to create a new tensor and populate it with 2 tensors that are spread out across various indices - i.e. not a simple concat - I haven't benchmarked the complex case if it's any faster, given that seqlens are usually relatively short it's probably not faster or if faster not by much - but might be a good idea to measure. ) rF embeddingrcloner.wherer|r)rrXadditional_vocab_indicesinput_ids_additional_vocabadditional_embeddings full_vectors&& r3forward!IdeficsDecoupledEmbedding.forwards*  ) )Q .;;y++6 6OO% #(;;yr? __classcell__rr2s@@r3rurus-33j%NAAAr6rucjaa]tRtRtoRtR V3RlV3RllltV3RlRltV3RlRltR tVt V;t #) IdeficsDecoupledLinearia Implements a decoupling of parameters to allow freezing (or not) a subset of the parameters. In practise, the regular `weight` can be trained or frozen (i.e. `partially_freeze=True`), and if `out_additional_features` > 0, then it will create `out_additional_features * in_features` additional parameters that are always trained. If `out_additional_features=0`, then the module defaults back to the regular behavior of `nn.Linear`. c <<V^8dQhRS[RS[RS[RS[RS[RR/#)r' in_features out_featuresout_additional_featuresbiasrwrxN)intry)r1r2s"r3r4#IdeficsDecoupledLinear.__annotate__sG"""""% "  "  " "r6c 8<\SV`WWFV4W0nWPnWnW nV'd?VP PR4V'dVPPR4V^8d#\P!VVVVVR7Vn R#R#)a' out_additional_features: int. Number of additional trainable dimensions. Only makes sense when `partially_freeze=True`. partially_freeze: bool. If True, the regular `weight` will be frozen and extra parameters (if any) will be trainable. If False, default to the regular behavior of nn.Linear. F)rrrrUr~N) rrrrwrrrrmrrra additional_fc) rrrrrrwrUr~rs &&&&&&&&r3rIdeficsDecoupledLinear.__init__s D%H'>$ 0&(  KK & &u - ((/ "Q &!#'4 "D  'r6cN<V^8dQhRS[PRS[P/#)r'inputrx)r.Tensor)r1r2s"r3r4rAs#U\\ellr6c\P!WPVP4pVP^8d*VP V4p\ P!W#3R4pV#)rHrO)rlinearrrrrr.cat)rroutputadditional_featuress&& r3rIdeficsDecoupledLinear.forwardAsQ%dii8  ' '! +"&"4"4U"; YYQQklpmImIlJJQRVR[R[cgRgQhh{|@|Q|Q{RS Sr6)rrrrrw)rHTTNNrrs@@r3rrs1""HSSSr6rcBaa]tRtRtoRV3RlltRtRtRtVtV;t #)IdeficsRMSNormiPc <\SV`4\P!\P !V44VnW nR#)z- IdeficsRMSNorm is equivalent to T5LayerNorm N)rrr Parameterr.onesrvariance_epsilon)r hidden_sizeepsrs&&&r3rIdeficsRMSNorm.__init__Qs/ ll5::k#:; #r6cVP\P4P^4P RRR7pV\P !W P ,4,pVPP\P\P39d&VPVPP4pVPV,#)r'T)keepdimrO) rTr.float32powmeanrsqrtrrr~float16bfloat16)rr*variances&& r3rIdeficsRMSNorm.forwardYs ##EMM266q9>>r4>P% H?T?T4T(UU  ;;   ? ?),,T[[->->?M{{]**r6c^\VPP4 RVP 2#)z, eps=)r0rrQrrs&r3rIdeficsRMSNorm.extra_reprcs* ))*+6$2G2G1HIIr6)rr)gư>) r8r9r:r;rrrr>r?rrs@@r3rrPs$+JJr6rcaa]tRtRto]!RRR7R V3RlV3Rlll4t]]!RRR7R V3RlRll44t]P!4] R 44t R t Vt V;t#) IdeficsRotaryEmbeddingihrUz5.18)versionc <V^8dQhRS[/#r'configr)r1r2s"r3r4#IdeficsRotaryEmbedding.__annotate__jsOO}Or6c<\SV`4VPVnVPVnWnVP P R,VnVPpVPR8wd\VP,pV!VP V4wq@n \P!VRR7Vn \P!VP4RR7VnR#) rope_typedefaultF) persistentN)rrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenrrope_parametersrcompute_default_rope_parametersrattention_scalingrBufferinv_freqroriginal_inv_freq)rrrU rope_init_fnrrs&&& r3rIdeficsRotaryEmbedding.__init__is "("@"@$*$B$B! 44[A!%!E!E >>Y &.t~~>L+7 V+L(( (u= !#8>>+;!Nr6cP<V^8dQhRS[RS[S[PS[3,/#)r'rrx)rr0r.rfloat)r1r2s"r3r4r|s.55 5Y^_d_k_kmr_rYs5r6c 0VPR,p\VRR4;'gVPVP,pRpRV\P !^V^\P R7V, ,, pVPV4V3#)a~ Computes the inverse frequencies according to the original RoPE implementation Args: config ([`~transformers.PreTrainedConfig`]): The model configuration. Returns: Tuple of (`torch.Tensor`, `float`), containing the inverse frequencies for the RoPE embeddings and the post-processing scaling factor applied to the computed cos/sin (unused in this type of RoPE). rope_thetahead_dimNg?r~)rgetattrrnum_attention_headsr.rPrrT)rrUrbasedimattention_factorrs&&, r3r6IdeficsRotaryEmbedding.compute_default_rope_parameterszs~%%l3fj$/cc63E3EIcIc3c$5<<3#MPS#STU{{6"$444r6c^VPR,PVP^,R ^4P\P VP R7pVR ,P 4p\VP P\4'd2VP PR8wdVP PMRp\VRR7;_uu_4W4,P^^4p\P!Wf3R R7pVP4VP,pVP4VP,p RRR4XPVP R7X PVP R73# +'giLH;i) Nr~rUmpscpuF) device_typeenabledrr)NNNNNrO)rNr)rexpandrQrTr.rrUretyperr transposercosrsinr~) rx position_idsinv_freq_expandedposition_ids_expandedrfreqsembrrs &&& r3rIdeficsRotaryEmbedding.forwards2 MM- ( / / 0B0B10Er1 M P PW\WbWbklksks P t !-Z 8 > > @'1!((--'E'E!((--[`J`ahhmmfk  U C C&>II!QOE))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs (A4F F, )rrrrrrrrd)r8r9r:r;rr staticmethodrr.no_gradrrr>r?rrs@@r3rrhsuXv.OO/O Xv.55/5$ ]]_<<