+ j*RRt^RIHt^RIt^RIHt^RIHt^RIH t ^RI H t ^RI H t ^RIHt^R IHt^R IHtHtHtHtHtHtHt^R IHt]P8!]4t!R R ]4t!RR]4t !RR]4t!!RR]4t"!RR]4t#!RR]4t$.ROt%R#)zPyTorch BitNet model.)CallableN)Cache)FlashAttentionKwargsCausalLMOutputWithPast)ALL_ATTENTION_FUNCTIONS)Unpack)logging)GemmaMLP)LlamaAttentionLlamaDecoderLayerLlamaForCausalLM LlamaModel LlamaRMSNormapply_rotary_pos_embeager_attention_forward BitNetConfigc]tRt^)tRtR#) BitNetRMSNormN__name__ __module__ __qualname____firstlineno____static_attributes__rj/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/bitnet/modular_bitnet.pyrr)rrcDaa]tRt^-toV3RlV3RlltRtRtVtV;t#) BitNetMLPc <V^8dQhRS[/#)configr)format __classdict__s"r __annotate__BitNetMLP.__annotate__.s]]|]rcr<\SV`V4\VPVPR7VnR#)epsN)super__init__rintermediate_size rms_norm_eps ffn_sub_norm)selfr$ __class__s&&rr-BitNetMLP.__init__.s+  )&*B*BH[H[\rc VPVPVPVPV44VP V4,44pV#N) down_projr0act_fn gate_projup_proj)r1xr6s&& rforwardBitNetMLP.forward2sFNN4#4#4T[[PQAR5SVZVbVbcdVe5e#fg r)r0 rrrrr-r;r__classdictcell__ __classcell__r2r&s@@rr!r!-s]]rr!cTaa]tRt^7toV3RlV3RlltRV3RlRlltRtVtV;t#)BitNetAttentionc&<V^8dQhRS[RS[/#)r#r$ layer_idx)rint)r%r&s"rr'BitNetAttention.__annotate__8sXX|XXrcr<\SV`W4\VPVPR7VnR#r*)r,r-r hidden_sizer/ attn_sub_norm)r1r$rDr2s&&&rr-BitNetAttention.__init__8s+ +*6+=+=6CVCVWrc<V^8dQhRS[PRS[S[PS[P3,RS[PR,RS[R,RS[S[,RS[S[PS[PR,3,/#)r# hidden_statesposition_embeddingsattention_maskNpast_key_valueskwargsreturn)torchTensortuplerrr)r%r&s"rr'rF<s')')||')#5<<#=>') t+ ')  ') -. ') u||U\\D00 1')rc 2VPRRp.VORNVPN5pVPV4PV4P ^^4pVP V4PV4P ^^4p VP V4PV4P ^^4p Vwr\WW4wrVeVPWVP4wr\P!VPP\4p T !VVV V V3RVP'gRM VP RVP"/VBwrVP$!.VORN5!P'4pVP)V4pVP+V4pW3#)Ndropoutgscaling)shapehead_dimq_projview transposek_projv_projrupdaterDr get_interfacer$_attn_implementationrtrainingattention_dropoutrWreshape contiguousrIo_proj)r1rLrMrNrOrP input_shape hidden_shape query_states key_states value_statescossinattention_interface attn_output attn_weightss&&&&&, rr;BitNetAttention.forward<s$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &#7 RU#[  &'6'='=jX\XfXf'g $J(?(M(M KK , ,.E) %8      % $}}}C$2H2H % LL %  % ! "));;;;FFH ((5 kk+. ((r)rIr5r=r@s@@rrBrB7s XX')')')rrBc]tRt^ftRtR#)BitNetDecoderLayerrNrrrrrtrtfrrrtc]tRt^jtRtR#) BitNetModelrNrrrrrvrvjrrrvcNaa]tRt^ntoRR/tRtRtV3RlV3RlltRtVt V;t #)BitNetForCausalLMzlm_head.weightzmodel.embed_tokens.weightNc <V^8dQhRS[/#)r#rQr)r%r&s"rr'BitNetForCausalLM.__annotate__ss// /rc $<\SV`!R/VB#)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, transformers., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, transformers., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, BitNetForCausalLM >>> model = BitNetForCausalLM.from_pretrained("microsoft/bitnet-b1.58-2B-4T") >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/bitnet-b1.58-2B-4T") >>> prompt = f'<|begin_of_text|>User: Hey, are you conscious? Can you talk to me?<|eot_id|>Assistant: ' >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=100) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "User: Hey, are you conscious? Can you talk to me?Assistant: No, I'm not conscious. I'm an artificial intelligence designed to assist with information and tasks. How can I help you today?" ```r)r,r;)r1 super_kwargsr2s&,rr;BitNetForCausalLM.forwardss4w...rr) rrrr_tied_weights_keys_tp_plan_pp_planr;rr>r?r@s@@rrxrxns**,GHHH///rrx)rxrvBitNetPreTrainedModel)&__doc__collections.abcrrR cache_utilsrmodeling_flash_attention_utilsrmodeling_outputsrmodeling_utilsrprocessing_utilsrutilsr gemma.modeling_gemmar llama.modeling_llamar r r rrrrconfiguration_bitnetr get_loggerrloggerrr!rBrtrvrx__all__rrrrs$ B65&+/   H % L ,)n,)^ *  * /(/D r