+ jJRt^RIHt^RIt^RIHuHt^RIH t ^RI H t ^RI H t ] P!]4tRRRlltRRR lltRR R lltRR R lltRRRlltRRRlltRRRlltRRRlltRRRlltR#)uVision utility functions for pre-computing very dynamic and data-dependent tensors that can break model graph capturing. All functions are standalone (no model weights) and compute tensors from `grid_thw` + config scalars. They are used by vision encoders and can be precomputed before `torch.compile` / `torch.export` tracing since they use untraceable ops (`repeat_interleave`, `.tolist()`, `nonzero()`, loops). Each `get_*` accepts an optional `kwargs` dict; if it contains the precomputed tensor under the natural key (`"cu_seqlens"`, `"position_ids"`, …), the function pops and returns it instead of computing. Vision encoders write `x = get_vision_x(..., kwargs=kwargs)` and the matching key is removed from the caller's kwargs as a side-effect of the pop. ) annotationsN)PreTrainedConfig)logging)get_max_seqlenc(V^8dQhRRRRRRRR/#)grid_thw torch.Tensormerge_temporalboolkwargs dict | Nonereturn)formats"Z/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/vision_utils.py __annotate__r*s2FFF,0FBMFFcVeVPRR4;peV#\PP4'd VPM\P pV'd'VR,VR,,VR,,pM3\P !VR,VR,,VR,4p\P!VP^VR7R^R7#) uGet cumulative sequence lengths from vision grid info, or pop from `kwargs` if precomputed. Args: grid_thw: `(num_images_or_videos, 3)` — temporal, height, width per entry. merge_temporal: when `False` (default), each frame is its own attention segment (`h * w` per frame, `t` segments per entry — the qwen2_vl / glm4v convention). When `True`, the whole clip is a single segment (`t * h * w`), i.e. attention spans all frames jointly (the kimi_k25 convention). kwargs: optional caller kwargs — if it contains `"cu_seqlens"` it is popped and returned. Returns: `cu_seqlens`: `(num_segments + 1,)` int32 cumulative sequence boundaries. N cu_seqlens)dimdtype)valueNNNrrrrr) poptorchjit is_tracingrint32repeat_interleaveFpadcumsum)rr r rrseqlenss&&& rget_vision_cu_seqlensr**s VZZ d-KKzX#ii2244HNN%++E4.8D>1HTNB))(4.8D>*I8TX>Z 55AU3V1 EErc ,V^8dQhRRRRRRRRR R /#) rrr configrr r r r rztuple[torch.Tensor, int | None]r)rs"rrrDs: " " "  " "  " % "rc<\WVR7p\WAVR7pWE3#)zGet cumulative and maximum sequence lengths for packed vision attention. ``merge_temporal`` is forwarded to [`get_vision_cu_seqlens`] (``True`` for clip-level attention, e.g. kimi_k25).)r r )r )r*r)rr,r r r max_seqlens&&&& rget_vision_attention_seqlensr/Ds&'xW]^J 6BJ  !!rc ,V^8dQhRRRRRRRRR R/#) rrr spatial_merge_sizezint | torch.Tensorinclude_temporalr r r rr)rs"rrrQs:.*.*.**.*.*  .*  .*rc VVeVPRR4;peV#VPp\V\4'd2\P !V.VR7P \V44p.p\VP4VP44EFtwwrgr\P!\P!WuR7\P!WR7RR7wrWy,WV ,V 3p V PV 4P^^4P4p V PV 4P^^4P4p V'dw\P!WeR7PWx,4p VP!\P"!WP%V4V P%V4.RR74EK<VP!\P"!W.RR7P%V^44EKw \P&!V^R7#)uGet position IDs for vision rotary embeddings, or pop from `kwargs` if precomputed. Args: grid_thw: `(num_images_or_videos, 3)` spatial_merge_size: merge block size — either a single `int` (same for all images) or a `(num_images_or_videos,)` tensor (per-image). kwargs: optional caller kwargs — if it contains `"position_ids"` it is popped and returned. include_temporal: when ``True``, prepend a temporal-index column and return `(total_tokens, 3)` — for encoders whose rotary embedding rotates T/H/W axes (minimax_m3_vl). When ``False`` (default), return `(total_tokens, 2)` for the 2-axis (h, w) case (qwen2_5_vl / qwen3_vl / glm4v / paddleocr_vl); the h/w indices are still repeated ``t`` times for video inputs. Returns: `position_ids`: `(total_tokens, 3)` long if ``include_temporal`` else `(total_tokens, 2)`, with the spatial indices laid out block-major over ``m×m`` spatial-merge blocks. N position_idsdeviceij)indexingr)r r6 isinstanceintr!tensorexpandlenziptolistmeshgridarangereshape transposeflattenr%appendstackrepeatcat)rr1r2r r4r6thw merge_sizehpos_idswpos_ids block_shapetpos_idss&&&& rget_vision_position_idsrSQs.vzz.$/OO|\ __F$c**"\\+=*>vNUUVYZbVcdL!$X__%68J8Q8Q8S!T q"^^ LL * LL *   OZP ##K0::1a@HHJ##K0::1a@HHJ ||A5GGNH    Xq7I8??[\K],^df g h    X,@b I P PQRTU V W"U 99\q ))rc 0V^8dQhRRRRRRRRRRR R /#) rrr r1r< window_size patch_sizer r r!tuple[torch.Tensor, torch.Tensor]r)rs"rrrsD:+:+:+:+:+ :+  :+ ' :+rc Ve0VPRR4pVPRR4pVeVeWV3#.p^.p^pW!,V,pV^,p VP4EFwrp \V 4\V 4\V 4rp W,p W,p\P!W,V,4P WV4pWV,, pWV,, pV V,V,pVV,V,p\ P!V^V^V3RR4pVP V VVVV4pVP^^^^^4P V VV,W4pVR8gP^^.4P R4pVP R4pVVR8g,pVPVV,4VP^4V ,VR,,pVPVP44WzV ,V,, pEK \P!V^R7p\P!W`P\P R7p\P"!V4pWV3#) uxGet window attention indices, or pop `"window_index"`/`"cu_window_seqlens"` from `kwargs` if both precomputed. Args: grid_thw: `(num_images_or_videos, 3)` spatial_merge_size: merge block size from vision config. window_size: window size from vision config. patch_size: patch size from vision config. kwargs: optional caller kwargs — if it contains both `"window_index"` and `"cu_window_seqlens"` they are popped and returned. Returns: `window_index`: `(total_tokens,)` long — reorder indices for windowed attention. `cu_window_seqlens`: `(num_windows + 1,)` int32 — cumulative window boundaries. N window_indexcu_window_seqlensconstantr9)r6rir:)r rAr<r!rCrDr&r'permutesumrGr(extendrJr=r6r$unique_consecutive)rr1rUrVr rYrZwindow_index_idvit_merger_window_sizespatial_merge_unitgrid_tgrid_hgrid_w llm_grid_h llm_grid_windexpad_hpad_w num_windows_h num_windows_w index_paddedr) index_newcu_seqlens_tmps&&&&& rget_vision_window_indexrps`(zz.$7 "JJ':DA  #(9(E2 2L cO(>*L+Q."*//"3!$Vc&k3v;1 1  V0:=>FFv[ef&6L)LL&6L)LL#e+0FF #e+0FF uuUQq%$8*dK #++ M#9=J` $++Aq!Q:BB MM13I  4',,aV4<A%RV%ad%&%rc\VP\P4pV'd5W^, ,\P!V^, ^R7, pMVR,V,V, R, p\P!V4pVR8Xd%\P !^^VP R7pM;VR8Xd%\P !R ^VP R7pM\RV: R24hVP4R ,V,p V P^V^, 4p VR ,VR ,, V, P4p VR8Xd^V , P^R7p MR p V ^,V ,V ^,, V ,V ,^,pW,^V ,, V ,^V ,,V ,^V ,, p\P!V ^8*W4p VR8XdW^8W^, 8*,,p W3#) uPer-axis interpolation taps into a `side`-length source table and their weights, for `index` target positions on an axis of length `size`. `mode` selects the kernel width — 2 taps (`"bilinear"`) or 4 taps (`"bicubic"`, Keys convolution kernel with `a=-0.75`). `size` may be a scalar or a per-element tensor (ragged batches). `padding` controls out-of-range taps: `"border"` clamps to the edge value (like `F.interpolate`), `"zeros"` gives them zero weight (like `F.grid_sample(padding_mode="zeros")`).)ming?bilinearr5bicubiczUnsupported interpolation mode z$ (expected 'bilinear' or 'bicubic').zerosr:rNg) tor!float32clampfloorrCr6 ValueErrorlongabswhere)rhrrrsrtrvrwsrcroffsetsraw_tapstapsdistanceweightsanearfars&&&&&& r _interpolation_axis_taps_weightsrs HHU]] #Eax 5;;taxQ#??s{d"T)C/ KK E z,,q!ELL9  ,,r1U\\::4(Bfghhzz|G$w.H >>!TAX &DG uW~-7<<>H zx<&&1&- Q("a!e,88CaG q1u$01q58HDq1uL++h!mT7'!m1H0DEF =rc8V^8dQhRRRRRRRRR RR RR R R R/#)rrr num_grid_per_sider<rtrurvr r1rwr r rrWr)rs"rrrsbBBBB B B  B  B B'BrcLVe0VPRR4pVPRR4pVeVeWx3#Tp Tp VPp VR,VR,,VR,,p \P!VR,V 4p \P!VR,V 4p\P!\P !V P ^4RRR 4V 4p\P!V P4V R7V, W,,pW,pVV ,pVV ,V ,pVW,,V,pVW,V,,pVV ,V,pVV ,V,p\VWW#V4wpp\VWW#V4wppVP^,pVR ,V ,VR ,,PRVV,4pVR ,VR ,,PRVV,4pVV3#) uPer-patch gather indices/weights that resample a square learned `(num_grid_per_side, num_grid_per_side)` position-embedding table to each image's `(h, w)` grid, or pop `"interp_indices"`/`"interp_weights"` from `kwargs` if both precomputed. Reproduces `F.interpolate(mode=mode, align_corners=align_corners)` as `(total_patches, n_taps)` indices + weights (`n_taps` = 4 for `"bilinear"`, 16 for `"bicubic"` — the Keys kernel with `a=-0.75`), consumed by a fused `F.embedding_bag` or a weighted `embedding` sum. Fully vectorised over packed patches (ragged `(h, w)` handled with `repeat_interleave`, no per-image loop), so it supports dynamic shapes like the other grid_thw precompute helpers. `spatial_merge_size > 1` emits patches in spatial-merge-block order (for encoders that consume merged tokens); `1` keeps raster order. Args: grid_thw: `(num_images_or_videos, 3)` — temporal, height, width per entry. num_grid_per_side: `int(num_position_embeddings ** 0.5)` from the vision config. mode: `"bilinear"` or `"bicubic"`. align_corners: matches the corresponding `F.interpolate` flag. spatial_merge_size: merge block size; `1` keeps raster patch order. kwargs: optional caller kwargs — if it contains both `"interp_indices"` and `"interp_weights"` they are popped and returned. Returns: `indices`: `(total_thw, n_taps)` long — gather indices into the flattened pos_embed table. `weights`: `(total_thw, n_taps)` float — interpolation weights. Ninterp_indicesinterp_weightsr5rrrr:r)rrN)rNr) r r6r!r%r&r'r(rCr]rshaperD)rrrtrvr1rwr rrrsmerger6countsheightswidthsstartswithinblocks_win_colin_row block_col block_rowrowcolh_taps h_weightsw_taps w_weightsnindicesrs&&&&&&& r,get_vision_interpolation_indices_and_weightsrsD$4d;$4d;  %.*D!1 1 D E __F d^htn ,x~ =F%%htnf=G  $ $Xd^V rsn # 1)   H %F4 ".*b:+z%PBJ<4#Bbbr