+ jW^RIHt^RIt^RIHt^RIHt^RIHtHt^RI H t ^RI H t H t ^RIHt^R IHt^R IHtHt^R IHtHt^R IHtHt^R IHt^RIHtHtH t ^RI!H"t"^RI#H$t$H%t%^RI&H't'^RI(H)t)Rt*] !R4R+Rl4t+RRlt,R,RRllt-] !]+4!RR]P\44t/] !R4!RR]P\44t0!RR ]P\4t1!R!R"]4t2]!R#R$]44t3!R%R&]P\4t4]!R'R(]344t5]!R)R*]3] 44t6.R-Ot7R#).)CallableN)nn)ACT2FN)Cache DynamicCache)GenerationMixin)use_kernel_forward_from_hubuse_kernelized_func)create_causal_mask)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPast)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tuple)deprecate_kwarg)maybe_autocastmerge_with_config_defaults)capture_outputs GraniteConfigcVRRVPR,^,13,pVRVPR,^,R13,p\P!V)V3RR7#)z*Rotates half the hidden dims of the input..Ndim)shapetorchcat)xx1x2s& m/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/granite/modeling_granite.py rotate_halfr(,sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''rotary_pos_embcVPV4pVPV4pW,\V4V,,pW,\V4V,,pWV3#)aIApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezer()qkcossin unsqueeze_dimq_embedk_embeds&&&&& r'apply_rotary_pos_embr43sS& -- &C -- &Cw;q>C/0Gw;q>C/0G  r)cdV^8dQhR\PR\R\P/#) hidden_statesn_repreturn)r"Tensorint)formats"r' __annotate__r=Ms. U UU\\ U# U%,, Ur)cVPwr#rEV^8XdV#VR,PW#WV4pVPW#V,WE4#)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) )NNNr?Nr?r?)r!expandreshape)r7r8batchnum_key_value_headsslenhead_dims&& r' repeat_kvrFMsU 2?1D1D.E z!"23::5W\dlmM  e(CT TTr)cV^8dQhR\PR\PR\PR\PR\PR,R\R\R \ \ ,/#) r6modulequerykeyvalueattention_maskNscalingdropoutkwargs)rModuler"r:floatrr)r<s"r'r=r=Ysw%% II% <<% % << % LL4' %  %%'(%r)c "\W P4p\W0P4p \P!WP ^^44V,p Ve W,p \ P PV R\PR7PVP4p \ P PWVPR7p \P!W4p V P ^^4P4p W3#)r6)rdtype)ptrainingr )rFnum_key_value_groupsr"matmul transposer functionalsoftmaxfloat32torSrNrU contiguous) rHrIrJrKrLrMrNrO key_states value_states attn_weights attn_outputs &&&&&&&, r'eager_attention_forwardrbYs3 ; ;$>&B\B\$\!22 !'!9!9ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii  & & 68J8JQWQfQf  r)c<V^8dQhRS[PRS[S[PS[P3,R,RS[PR,RS[R,RS[S[,RS[S[PS[P3,/#)r6r7position_embeddingsNrLpast_key_valuesrOr9)r"r:tuplerrr)r<ris"r'r=rjs&)&)||&)#5<<#=>E&) t+ &)  &) +, &) u||U\\) *&)r)c VPRRp.VORNVPN5pVPV4PV4P ^^4pVP V4PV4P ^^4p VP V4PV4P ^^4p Vwr\WW4wrVeVPWVP4wr\P!VPP\4p T !VVV V V3RVP'gRM VP RVP"/VBwrVP$!.VORN5!P'4pVP)V4pW3#)NrNrMr )r!rErxviewrXryrzr4updatergr get_interfacerf_attn_implementationrbrUrtrMrAr]r{)r}r7rrLrrO input_shape hidden_shape query_statesr^r_r/r0attention_interfacerar`s&&&&&, r'forwardGraniteAttention.forwards$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &#7 RU#[  &'6'='=jX\XfXf'g $J(?(M(M KK , ,.E) %8      % $}}}C$2H2H % LL %  % ! "));;;;FFH kk+. ((r)) rtrfrEruryrgrVr{rxrMrzN)NNN) __name__ __module__ __qualname____firstlineno____doc__ror__static_attributes____classdictcell__ __classcell__r~ris@@r'rdrdrs!G  .&)&)&)r)rdRMSNormcZaa]tRt^toRV3RlV3RllltV3RlRltRtRtVtV;t #)GraniteRMSNormc$<V^8dQhRS[RR/#)r6epsr9N)rQ)r<ris"r'r=GraniteRMSNorm.__annotate__s$$$$$r)c <\SV`4\P!\P !V44VnW nR#)z- GraniteRMSNorm is equivalent to T5LayerNorm N)rnror Parameterr"onesweightvariance_epsilon)r}rqrr~s&&&r'roGraniteRMSNorm.__init__s/ ll5::k#:; #r)cN<V^8dQhRS[PRS[P/#)r6r7r9)r"r:)r<ris"r'r=rs#;;U\\;ell;r)c:VPpVP\P4pVP ^4P RRR7pV\P !W0P,4,pVPVPV4,#)r6T)keepdimr ) rSr\r"r[powmeanrsqrtrr)r}r7 input_dtypevariances&& r'rGraniteRMSNorm.forwardsw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r)c^\VPP4 RVP 2#)z, eps=)rrr!r)r}s&r' extra_reprGraniteRMSNorm.extra_reprs* ))*+6$2G2G1HIIr))rr)gư>) rrrrrorrrrrrs@@r'rrs%$$;;JJr)rc8aa]tRt^toV3RltRtRtVtV;t#) GraniteMLPc<\SV`4WnVPVnVPVn\ P !VPVPVPR7Vn\ P !VPVPVPR7Vn \ P !VPVPVPR7Vn \VP,Vn R#)rlN)rnrorfrqintermediate_sizerrvmlp_bias gate_projup_proj down_projr hidden_actact_fnr}rfr~s&&r'roGraniteMLP.__init__s  !--!'!9!94#3#3T5K5KRXRaRabyy!1!143I3IPVP_P_` 4#9#94;K;KRXRaRabV../ r)cVPVPVPV44VPV4,4pV#r)rrrr)r}r$rs&& r'rGraniteMLP.forwards6NN4;;t~~a/@#ADLLQRO#ST r))rrfrrrqrr rrrrrorrrrrs@@r'rrs0r)rcTaa]tRt^toV3RlV3RlltRV3RlRlltRtVtV;t#)GraniteDecoderLayerc&<V^8dQhRS[RS[/#)r6rfrgrh)r<ris"r'r= GraniteDecoderLayer.__annotate__s>>}>>r)cB<\SV`4VPVn\WR7Vn\ V4Vn\VPVPR7Vn \VPVPR7Vn VPVn R#))rfrgrN) rnrorqrd self_attnrmlpr rms_norm_epsinput_layernormpost_attention_layernormresidual_multiplierr|s&&&r'roGraniteDecoderLayer.__init__sx !--)Mf%-f.@.@fFYFYZ(6v7I7IvObOb(c%#)#=#= r)c.<V^8dQhRS[PRS[PR,RS[PR,RS[R,RS[R,RS[S[PS[P3,R,RS[S[,R S[P/#) r6r7rLN position_idsr use_cacherrOr9)r"r: LongTensorrboolrrr)r<ris"r'r=rs22||2 t+2&&- 2  2 $; 2#5<<#=>E2+,2 2r)c TpVPV4pVP!RRVRVRVRVRVRV/VBwrWVP,,pTpVPV4pVP V4pWVP,,pV#)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch_size, sequence_length)` if flash attention is used or `(batch_size, 1, query_sequence_length, key_sequence_length)` if default attention is used. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_values (`Cache`, *optional*): cached past key and value projection states position_embeddings (`tuple[torch.FloatTensor, torch.FloatTensor]`, *optional*): Tuple containing the cosine and sine positional embeddings of shape `(batch_size, seq_len, head_dim)`, with `head_dim` being the embedding dimension of each attention head. kwargs (`dict`, *optional*): Arbitrary kwargs to be ignored, used for FSDP and other methods that injects code into the model r7rLrrrr)rrrrr) r}r7rLrrrrrOresidual_s &&&&&&&, r'rGraniteDecoderLayer.forwards<!,,]; >> ' ) & ,    !4    !43K3K#KK  55mD /  43K3K#KK r))rqrrrrr)NNNFNrrs@@r'rrs>>222r)rc`a]tRtRtoRtRtR.tR.tRtRt Rt Rt Rt R] R]/tV3RltR tVtR #) GranitePreTrainedModelimodelTrrr7 attentionsc&<V^8dQh/S[;R&#r6rfr)r<ris"r'r=#GranitePreTrainedModel.__annotate__s r)rN)rrrrbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraph_supports_attention_backendrrd_can_record_outputs__annotate_func__rr)ris@r'rrs] &*#./#4"5N!"&,&r)rcaa]tRtRto]!RRR7R V3RlV3Rlll4t]]!RRR7R V3RlRll44t]P!4] R 44t R t Vt V;t#) GraniteRotaryEmbeddingi.devicez5.18)versionc <V^8dQhRS[/#rr)r<ris"r'r=#GraniteRotaryEmbedding.__annotate__0sOO}Or)c<\SV`4VPVnVPVnWnVP P R,VnVPpVPR8wd\VP,pV!VP V4wq@n \P!VRR7Vn \P!VP4RR7VnR#) rope_typedefaultF) persistentN)rnromax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenrfrope_parametersrcompute_default_rope_parametersrattention_scalingrBufferinv_freqcloneoriginal_inv_freq)r}rfr rope_init_fnrr~s&&& r'roGraniteRotaryEmbedding.__init__/s "("@"@$*$B$B! 44[A!%!E!E >>Y &.t~~>L+7 V+L(( (u= !#8>>+;!Nr)cP<V^8dQhRS[RS[S[PS[3,/#)r6rfr9)rrr"r:rQ)r<ris"r'r=rBs.55 5Y^_d_k_kmr_rYs5r)c 0VPR,p\VRR4;'gVPVP,pRpRV\P !^V^\P R7V, ,, pVPV4V3#)a~ Computes the inverse frequencies according to the original RoPE implementation Args: config ([`~transformers.PreTrainedConfig`]): The model configuration. Returns: Tuple of (`torch.Tensor`, `float`), containing the inverse frequencies for the RoPE embeddings and the post-processing scaling factor applied to the computed cos/sin (unused in this type of RoPE). rope_thetarENg?rS)rrprqrrr"arangerQr\)rfrrObaserattention_factorrs&&, r'r6GraniteRotaryEmbedding.compute_default_rope_parameters@s~%%l3fj$/cc63E3EIcIc3c$5<<3#MPS#STU{{6"$444r)c^VPR,PVP^,R ^4P\P VP R7pVR ,P 4p\VP P\4'd2VP PR8wdVP PMRp\VRR7;_uu_4W4,P^^4p\P!Wf3R R7pVP4VP,pVP4VP,p RRR4XPVP R7X PVP R73# +'giLH;i) N)rSrmpscpuF) device_typeenabledrr)Nr?Nr )r?Nr?)rr@r!r\r"rQr isinstancetypestrrrXr#r/rr0rS) r}r$rinv_freq_expandedposition_ids_expandedr freqsembr/r0s &&& r'rGraniteRotaryEmbedding.forwardTs2 MM- ( / / 0B0B10Er1 M P PW\WbWbklksks P t !-Z 8 > > @'1!((--'E'E!((--[`J`ahhmmfk  U C C&>II!QOE))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs (A4F F, )rrfrrrrrr)rrrrrro staticmethodrr"no_gradrrrrrrs@@r'rr.suXv.OO/O Xv.55/5$ ]]_<<O  CRC^==?de  << (;(;A(>}G[G[\_ooL'11!4L(;;')+%  & "oomoW![[)H4;;+H+HIM)**!0  $  %8 MJ -0 &++  r))r!r)r(r%r&rr'r)NNNNNN) rrrrrorrrrrrrrs@@r'rrgs=" 5 5  5 5 r)rcaa]tRtRtoRR/tRR/tRR.R.3/tRR/tV3R lt] ] R V3R lR ll44t R t Vt V;t#)GraniteForCausalLMizlm_head.weightzmodel.embed_tokens.weightlm_headcolwise_gather_outputr7logitskeep_full_weightc<\SV`V4\V4VnVPVn\ P !VPVPRR7VnVP4R#)FrlN) rnrorrrrrvrqr:r*rs&&r'roGraniteForCausalLM.__init__sU  !&)  ++yy!3!3V5F5FUS  r)cP<V^8dQhRS[PR,RS[PR,RS[PR,RS[R,RS[PR,RS[PR,RS[R,R S[S[P,R S[S[,R S[ / #) r6r-NrLrrr.labelsrlogits_to_keeprOr9) r"rr:rr/rr;rrr)r<ris"r'r=GraniteForCausalLM.__annotate__s6 6 ##d*6  t+6 &&- 6  6 ((4/ 6   4'6 $;6 ell*6 +,6  6 r)c  VP!R RVRVRVRVRVRV/V Bp V Pp \V\4'd\ V)R4MTp VP V RV R3,4p WP P, p RpVe-VP!R R V R VR VP P/V Bp\VV V PV PV PR 7#)a{ Example: ```python >>> from transformers import AutoTokenizer, GraniteForCausalLM >>> model = GraniteForCausalLM.from_pretrained("meta-granite/Granite-2-7b-hf") >>> tokenizer = AutoTokenizer.from_pretrained("meta-granite/Granite-2-7b-hf") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```r-rLrrr.rNr?r<rAr)lossr<rr7rr)rr1r r;slicer:rflogits_scaling loss_functionrrrr7r)r}r-rLrrr.rArrBrOoutputsr7 slice_indicesr<rEs&&&&&&&&&, r'rGraniteForCausalLM.forwards>,0::, , ), &, , , ( ,  , ,  11 8B>SV8W8W~ot4]k mA}a,?@A++444  %%pVpFpt{{OeOepiopD%#33!//))   r))r:rr)NNNNNNNr)rrrr_tied_weights_keys_tp_plan_pp_plan _fsdp_planrorrrrrrrs@@r'r9r9sf*,GH23H_-z:;H/0J6 6 6 6 r)r9))r)r9rr)8collections.abcrr"r activationsr cache_utilsrr generationr integrationsr r masking_utilsr modeling_layersr modeling_outputsr rmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrutils.deprecationr utils.genericrrutils.output_capturingrconfiguration_graniterr(r4rFrbrPrdrrrrrrr9__all__rr)r'rbsN,% !.)L/9OKF&II0G50(-./2 U%2)*@)ryy@)+@)FY'JRYYJ(J(   =4=@_$6