+ jW^RIt^RIHt^RIHt^RIHt^RIHt^RI t ^RI H t ^RI H t ^RIHt^R IHt^R IHtHt^R IHt^R IHtHtHtHtHtHtHtH t ^R I!H"t"^RI#H$t$H%t%^RI&H't'^RI(H)t)H*t*^RI+H,t,^RI-H.t.H/t/H0t0H1t1H2t2^RI3H4t4H5t5H6t6^RI7H8t8H9t9^RI:H;t;^RIt>H?t?H@t@HAtAHBtBHCtC^RIDHEtEHFtFHGtGHHtHHItIHJtJHKtKHLtLHMtM^RINHOtO^RIPHQtQ^RIRHStS^RITHUtUHVtVHWtWHXtX]1P!]Z4t[RRlt\!RR]H4t]!R R!]E4t^]!R"R#]$44t_]/]!R$R%]%444t`!R&R'] P4tb!R(R)]K4tc!R*R+] P4td!R,R-] P4te!R.R/] P4tf!R0R1] P4tg!R2R3] P4th!R4R5] P4tj!R6R7] P4tk!R8R9] P4tl!R:R;] P4tm!R<R=] P4tn!R>R?]@4toRqR@RAlltp!RBRC]S4tq]6!RDRE]=44tr!RFRG]>4ts!RHRI] P4tt!RJRK]@4tu!RLRM]A4tv!RNRO] P4tw!RPRQ]O4tx!RRRS] P4ty!RTRU]>4tz!RVRW]C4t{!RXRY]J4t|]/!RZR[7!R\R]]B44t}]/!R^R[7!R_R`]?44t~!RaRb]|4t!RcRd]|4t!ReRf]I4tRgRhltRiRjlt]/!RkR[7!RlRm]G44t]/!RnR[7!RoRp]F44t.RrOtR#)sN)UserDict)Callable) dataclass)cached_property)nn) functional)initialization)ACT2FN)Cache DynamicCache)PreTrainedConfig)_preprocess_mask_argumentsblockwise_overlaycreate_bidirectional_maskcreate_causal_maskcreate_masks_for_generate!create_sliding_window_causal_maskmaybe_pad_block_sequence_idssliding_window_overlay)FlashAttentionKwargs)BaseModelOutputWithPastBaseModelOutputWithPooling)ROPE_INIT_FUNCTIONS)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleloggingtorch_compilable_check)maybe_autocastmerge_with_config_defaultsno_inherit_decorator)OutputRecordercapture_outputs) AutoModel)Gemma3AttentionGemma3DecoderLayerGemma3ForCausalLM Gemma3MLPGemma3RotaryEmbeddingGemma3TextModelGemma3TextScaledWordEmbedding) Gemma3nCausalLMOutputWithPastGemma3nForConditionalGeneration Gemma3nModelGemma3nModelOutputWithPastGemma3nMultimodalEmbedderGemma3nPreTrainedModelGemma3nRMSNormapply_rotary_pos_embeager_attention_forward)MixtralExperts)sliding_window_mask_function)Sam3ViTRotaryEmbedding)Gemma4AudioConfig Gemma4ConfigGemma4TextConfigGemma4VisionConfigcV^8dQhR\R\PR\PR,R\R,R\PR,R\PR\/#) config inputs_embedsattention_maskNpast_key_values position_idsblock_sequence_idsreturn)r torchTensorr dict)formats"j/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/gemma4/modular_gemma4.py __annotate__rMSsl<< <<<<LL4'<T\ < ,,% <  < <c vRVRVRVRVRV/p\R/VBp\VRR4'd.RVP9dVPPR4pM^p\ R/VBR V/Bwp rrV 'dTp M \ WRW4p \R/VBR \ V 4R \VP4R V/BpR VR V/#)awCreate full_attention and sliding_attention masks with correct composition. For global (full attention) layers: causal only (no bidirectional) For local (sliding window) layers: AND(sliding_window, OR(causal, blockwise)) Unlike Gemma 3 (which applies bidirectional attention on all layers), Gemma 4 explicitly disables bidirectional attention on global attention layers. rArBrCrDrE is_slidingNT layer_idxor_mask_functionand_mask_functionfull_attentionsliding_attention) rgetattrrPindexrrrrsliding_window)rArBrCrDrErF mask_kwargs full_masksliding_layer_idx early_exit_ kv_length kv_offsetpadded_block_sequence_ids sliding_masks&&&&&& rLcreate_masks_for_vision_modelrcSs" &.? K#1[1I d33@Z@Z8Z+66<