+ jSURt^RIt^RIHt^RIHt^RIt^RIHtH t Rt ]!RR44t RRR llt RR R llt RR R lltRRRlltRRRlltR#)a IMPORTANT NOTICE: Every class and function in this file is deprecated in favor of using the much more general `masking_utils.py` primitives. New code should not rely on it, it is only kept for backward compatibility for now, and will be removed in the future. N) dataclass)Union)is_torchdynamo_compiling is_tracingzThe attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.ca]tRt^#toRtRV3RlRlltRV3RlRlltRV3RlRllt]RV3R lR ll4t ]RV3R lR ll4t ]V3R lRl4t ]RV3RlRll4t V3Rlt RtVtR#)AttentionMaskConvertera A utility attention mask class that allows one to: - Create a causal 4d mask - Create a causal 4d mask with slided window - Convert a 2d attention mask (batch_size, query_length) to a 4d attention mask (batch_size, 1, query_length, key_value_length) that can be multiplied with attention scores Examples: ```python >>> import torch >>> from transformers.modeling_attn_mask_utils import AttentionMaskConverter >>> converter = AttentionMaskConverter(True) >>> converter.to_4d(torch.tensor([[0, 0, 0, 1, 1]]), 5, key_value_length=5, dtype=torch.float32) tensor([[[[-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, 0.0000e+00, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, 0.0000e+00, 0.0000e+00]]]]) ``` Parameters: is_causal (`bool`): Whether the attention mask should be a uni-directional (causal) or bi-directional mask. sliding_window (`int`, *optional*): Optionally, the sliding window masks can be created if `sliding_window` is defined to a positive integer. Nc4<V^8dQhRS[RS[R,/#) is_causalsliding_windowNboolint)format __classdict__s"f/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/modeling_attn_mask_utils.py __annotate__#AttentionMaskConverter.__annotate__Fs  $ d c\P!\\4WnW nVP e-VP ^8:d\ RVP R24hR#R#)NzaMake sure that when passing `sliding_window` that its value is a strictly positive integer, not ``)warningswarnDEPRECATION_MESSAGE FutureWarningr r ValueError)selfr r s&&&r__init__AttentionMaskConverter.__init__Fss )=9",    *t/B/Ba/GstxuHuHtIIJK 0H *rc <V^8dQhRS[RS[RS[RS[PRS[S[PR3,RS[P R,/#) r batch_size query_lengthkey_value_lengthdtypedevicestrreturnN)rtorchr#rr$Tensor)rrs"rrrQsb  {{  ellE)*   rc VP'g\RVP R24hW3pW2, pRpVR,^8gVPe!VP VVVVVPR7pV#)z Creates a causal 4D mask of (bsz, head_dim=1, query_length, key_value_length) shape and adds large negative bias to upper right hand triangular matrix (causal mask). z"Please use `to_causal_4d` only if z has `is_causal` set to True.Nr$past_key_values_lengthr )r r __class__r _make_causal_mask) rr r!r"r#r$ input_shaper+causal_4d_masks &&&&&& r to_causal_4d#AttentionMaskConverter.to_causal_4dQs~~~A$..AQQnop p"0 !1!@ r?Q $"5"5"A!33'=#22 4Nrc <V^8dQhRS[PRS[RS[PRS[R,RS[P/#)r attention_mask_2dr!r#r"Nr&)r'r(rr#)rrs"rrrrsM- -  <<- - {{ - * -  - rc 4VP^,V3pRpVR,^8gVPeUVP'dCVf \R4hWB, pVP VVVP VVPR7pMVPe \ R4hVPWVR,R7PVP 4pVe?VPVP4\P!V4P4pTp V #)z Converts 2D attention mask to 4D attention mask by expanding mask to (bsz, head_dim=1, query_length, key_value_length) shape and by adding a large negative bias to not-attended positions. If attention_mask is causal, a causal mask will be added. NzpThis attention mask converter is causal. Make sure to pass `key_value_length` to correctly create a causal mask.r*z?Sliding window is currently only implemented for causal masking)tgt_lenr,)shaper r rr.r$NotImplementedError _expand_maskto masked_fillr r'finfomin) rr4r!r#r"r/r0r+expanded_attn_maskexpanded_4d_masks &&&&& rto_4dAttentionMaskConverter.to_4drs')..q1<@  Oa 4#6#6#B' G&6%D "!33(//'=#22 4N ,%&gh h"../@Q\]_Q`.add  $ $   %!/!;!;  )1}&:&:499R=!&LLaPwwu~ !A %99ekk'QVfhlmsuvD  %->BH ::eood%**&MX`aL())zz|   lEKK,>,B,B C$%,,S!WH^>^__rcb<V^8dQhRS[PRS[PRS[R,/#r rWr#r6Nr'r(r#r)rrs"rrrs3 _ _5<< _ _cDj _rc \P!\\4VP 4wr4VeTMTpVR,P V^W$4P V4p\P!RVR7V, pVPVP \P4\P!V4P4#)zW Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. ?rH)rJNNrJ) rrrrrMrUr:r'tensorr;r r<r=)rWr#r6rVsrc_len expanded_mask inverted_masks&&& rr9#AttentionMaskConverter._expand_masks  )=9yy{ $0'g-.55c1gORRSXY  S6F (()9)9%**)Eu{{SXGYG]G]^^rc:<V^8dQhRS[PRS[/#)r rb min_dtype)r' FloatTensorfloat)rrs"rrrs'0_0_((0_0_rc \P!\\4VP\ P 8Xd \R4hVP\ P!W8HRRR7(4#)a Attend to all tokens in masked rows from the expanded attention mask, for example the relevant first rows when using left padding. This is required by F.scaled_dot_product_attention memory-efficient attention path. Details: https://github.com/pytorch/pytorch/issues/110213 `expanded_mask` is [bsz, num_masks, tgt_seq_len, src_seq_len] or [bsz, tgt_seq_len, src_seq_len]. `attention_mask` is [bsz, src_seq_len]. The dimension num_masks of `expanded_mask` is most often 1, but it can also be the number of heads in the case of alibi attention bias. For example, if `expanded_mask` is (e.g. here left-padding case) ``` [[[[0, 0, 0], [0, 0, 0], [0, 0, 1]]], [[[1, 0, 0], [1, 1, 0], [1, 1, 1]]], [[[0, 0, 0], [0, 1, 0], [0, 1, 1]]]] ``` then the modified `expanded_mask` will be ``` [[[[1, 1, 1], <-- modified [1, 1, 1], <-- modified [0, 0, 1]]], [[[1, 0, 0], [1, 1, 0], [1, 1, 1]]], [[[1, 1, 1], <-- modified [0, 1, 0], [0, 1, 1]]]] ``` z\AttentionMaskConverter._unmask_unattended expects a float `expanded_mask`, got a BoolTensor.T)rGkeepdimr,) rrrrr#r'r rmulall)rbrfs&&r_unmask_unattended)AttentionMaskConverter._unmask_unattendeds_R  )=9   %** ,n   %))M,FBX\"]!]^^rc <V^8dQhRS[PR,RS[PRS[RS[R,RS[RS[/#)r attention_maskN inputs_embedsr+r is_trainingr&)r'r(rr )rrs"rrr sV8"8" t+8"||8"!$8"d 8"  8"  8"rc \P!\\4VP^,VP^,reWb,p\ V4pRp Vf,V'g V'gV^8XgWv8Xd VeWs8dRp V #VeWs8dS\ VP4^8XdR#V'g/\P!V^8H4'dV^8XgWv8XdRp V #)a Detects whether the optional user-specified attention_mask & the automatically created causal mask can be ignored in case PyTorch's SDPA is used, rather relying on SDPA's `is_causal` argument. In case no token is masked in the `attention_mask` argument, if `query_length == 1` or `key_value_length == query_length`, we rather rely on SDPA `is_causal` argument to use causal/non-causal masks, allowing to dispatch to the flash attention kernel (that can otherwise not be used if a custom `attn_mask` is passed). FT) rrrrr7rlenr'rl) rprqr+r rr_r!r" is_tracing_ignore_causal_masks &&&&& r_ignore_causal_mask_sdpa/AttentionMaskConverter._ignore_causal_mask_sdpas"  )=9'--a0-2E2Ea2H<'@ / "  !K!Q&*:*J#+/?/P%)""! #'7'H>''(A- UYY~/B%C%C1$(8(H)-&"!rc2<V^8dQh/S[;R&S[;R&#)r r r r )rrs"rrr#s"@OABCrr r N)cpuN)NF)__name__ __module__ __qualname____firstlineno____doc__rr1r@ staticmethodr.r9rmrx__annotate_func____static_attributes____classdictcell__)rs@rrr#sB  B- - ^!`!`!`F _ _ _0_0_d8"8"8"Mrrc V^8dQhR\PR,R\P\,\,R\PR\ R\ R,/#r rpNr/rqr+r r'r(rDtuplelistr)rs"rrrDsY11LL4'1e#d*1<<1 1 $J 1rc\RVR7pVR ,V,pVeA\VP4^8Xd'VPWR ,WbPR7pV#Ve\VP4^8XdV^,^V^,V3p\ VP4V8wd&\ R\ VP4 RV R24hRV, pVPVP\P4\P!VP4P4pV#VPV^,VR ,WbPVPR7pV#) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)` Args: attention_mask (`torch.Tensor` or `None`): A 2D attention mask of shape `(batch_size, key_value_length)` input_shape (`tuple(int)` or `list(int)` or `torch.Size`): The input shape should be a tuple that defines `(batch_size, query_length)`. inputs_embeds (`torch.Tensor`): The embedded inputs as a torch Tensor. past_key_values_length (`int`): The length of the key value cache. sliding_window (`int`, *optional*): If the model uses windowed attention, a sliding window should be passed. Tr{)r"r#z#Incorrect 4D attention_mask shape: z ; expected: .r_rFr,)rrtr7r@r#rrr;r:r'r r<r=r1r$) rpr/rqr+r attn_mask_converterr"expected_shapercs &&&&& r!_prepare_4d_causal_attention_maskrDsg.14P^_"2)??!c.*>*>&?1&D,22 O>NViVi3 ( #  #N,@,@(AQ(F%a.![^=MN %% &. 85eNrse ! Dw  ]"]" ]"@ 1j7t X \2r