+ j<^RIHt^RIt^RIHt^RIHt^RIHt ^RI H t ^RI H t ^RIHt^RIHt^R IHtHtHt^R IHtHtHtHtHt^R IHt^R IHtH t ^R I!H"t"^RI#H$t$H%t%^RI&H't'H(t(H)t)^RI*H+t+H,t,H-t-H.t.H/t/H0t0H1t1^RI2H3t3^RI4H5t5^RI6H7t7H8t8^RI9H:t:H;t;^RIH?t?^RI@HAtAHBtBHCtCHDtDHEtE^RIFHGtG^RIHHItIHJtJHKtKHLtLHMtM^RINHOtO^RIPHQtQ^RIRHStS]/P!]U4tV.RQOtW]-!R'R(7]!R)R"]Q444tX]-!R'R(7]!R*R#]G444tY]-!R'R(7]!R+R$] 444tZ!R,R-]'R.R/7t[]-!R0R%]44t\!R1R2]?4t]!R3R4]M4t^!R5R6]L4t_!R7R8]4t`!R9R:]S4ta]-!R;R]%44tb]-!R<R!]b44tc!R=R>]C4td!R?R@]I4te!RARB];4tf]0!RCRD]:44tg!RERF]K4th!RGRH]J4ti!RIRJ]A4tj!RKRL]B4tk!RMR ]=4tl!RNR]84tm!ROR]74tn]-!RPR&](44toR#)R)CallableN)strict)initialization)Cache)PreTrainedConfig)TorchvisionBackend) BatchFeature)divide_to_patchesgroup_images_by_shapereorder_images)OPENAI_CLIP_MEANOPENAI_CLIP_STD ImageInputPILImageResamplingSizeDict)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPooling)ROPE_INIT_FUNCTIONS)ALL_ATTENTION_FUNCTIONSPreTrainedModel) ImagesKwargsProcessorMixinUnpack) TensorTypeTransformersKwargsauto_docstringcan_return_tupleloggingno_inherit_decorator torch_int)capture_outputs)get_vision_position_ids)$DeepseekOcr2ForConditionalGenerationDeepseekOcr2Model)DeepseekV4Experts DeepseekV4MLP)Gemma3TextModel)Kimi_K25VisionRotaryEmbedding)LagunaAttentionLagunaDecoderLayerLagunaRotaryEmbeddingapply_rotary_pos_embeager_attention_forward)MiniMaxM3VLTextConfig)MiniMaxM3VLRMSNormMiniMaxM3VLSparseMoeBlockMiniMaxM3VLTopKRouterMiniMaxM3VLVisionAttentionMiniMaxM3VLVisionMLP)apply_rotary_pos_emb_vision)SiglipVisionConfig)SiglipVisionEmbeddingsStep3p7ForConditionalGeneration Step3p7ModelStep3p7PreTrainedModelStep3p7TextModelStep3p7VisionModelStep3p7VisionConfigStep3p7TextConfig Step3p7ConfigStep3p7ImageProcessorStep3p7Processorzstepfun-ai/Step-3.7-Flash) checkpointcja]tRt^XtoRtRtRtRt^/t^t Rt ^t Rt Rt RtRtR tR tR tV3R ltR tVtR #)r=step3p5_vision vision_configaxiali quick_geluh㈵>g?Ni c VPRVP4VnVPRVP4VnVPRVP4VnVPRVP4Vn\ P !V3/VB\VPVP,4Vn R#)widthlayersheads ls_init_valueN) pop hidden_sizenum_hidden_layersnum_attention_headslayer_scale_init_valuer __post_init__int mlp_ratiointermediate_size)selfkwargss&,l/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/step3p7/modular_step3p7.pyrT!Step3p7VisionConfig.__post_init__ms!::gt/?/?@!'Hd6L6L!M#)::gt7O7O#P &,jj$B]B]&^#&&t6v6!$T%5%5%F!Gc<V^8dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R &S[R ,;R &S[;R &#) rPrQrR image_size patch_size hidden_actlayer_norm_epsrVrSNrope_parametersmax_position_embeddings)rUstrfloatdict)format __classdict__s"rZ __annotate__ Step3p7VisionConfig.__annotate__Xs!"  "!""'#$D['%&!''r\)rPrWrSrRrQgUUUUUU@)__name__ __module__ __qualname____firstlineno__ model_typebase_config_keydefault_rope_typerPrQrRr_r`rarbrVrSrcrdrT__annotate_func____static_attributes____classdictcell__ris@rZr=r=Xsa"J%OK!JJ"J N"I$'#'O#'H+r\caa]tRt^vtoRtRtRRRRRRRR R R R R /tRtRtRt RRRRRRRRRRRRRRRR/t ] !4t ] !4t ] !4t] !4t] !4t] !4t] !4t] !4t] !4t] !4t] !4t] !4t] !4t] !4tRtRt^t^-tR tR!tR"t R#t!R$t"^t#R#t$R%t%R%t&^t'R%t(R%t)Rt*R%t+R&t,Rt-R%t.R%t/R%t0R%t1V3R'lt2V3R(lt3R)t4Vt5V;t6#)*r>a mlp_layer_types (`list[str]`, *optional*): Per-layer MLP type: `"sparse"` (MoE) or `"dense"`. If not provided, derived from the legacy `moe_layers_enum` hub-config kwarg (comma-separated string or list of MoE layer indices), defaulting to all layers from index 3 onward being MoE. n_routed_experts (`int`, *optional*, defaults to 288): Total number of routed experts. Accessible as `num_local_experts` via `attribute_map`. share_expert_dim (`int`, *optional*, defaults to 1280): Intermediate size of the always-active shared expert. num_sliding_attention_heads (`int`, *optional*): Attention head count for `"sliding_attention"` layers, if different from `num_attention_heads`. Defaults to the legacy `attention_other_setting` hub-config kwarg's `num_attention_heads` entry. Applied via a `per_layer_config` override (see `PreTrainedConfig`), not a per-layer list field. query_pre_attn_scalar (`int` or `float`, *optional*): `Step3p7Attention.__init__` hook point: defaults to `head_dim`, giving standard `head_dim ** -0.5` scaling; overridable per released checkpoint variant. moe_router_scaling_factor (`float`, *optional*, defaults to 1.0): Scaling factor applied to the MoE block's routed-expert output (`routed_scaling_factor` in `Step3p7SparseMoeBlock`). swiglu_limits (`list[float | None]`, *optional*): Per-layer gate/up clamping bound; `None` means no clamping. swiglu_limits_shared (`list[float | int | None]`, *optional*): Per-layer gate/up clamping bound for the always-active shared expert; `None` means no clamping. mtp_layer_types (`list[str]`, *optional*): Per-MTP-layer attention type; split off `layer_types`'s legacy trailing pad instead of being discarded, so `Step3p7Config.get_mtp_config()` can build the MTP layers for `generate(use_mtp=True)`. mtp_mlp_layer_types (`list[str]`, *optional*): Per-MTP-layer MLP type, analogous to `mtp_layer_types` for `mlp_layer_types`. step3p5num_local_expertsn_routed_expertsnum_attention_groupsnum_key_value_headsmoe_num_experts moe_top_knum_experts_per_tokshare_expert_dimsshare_expert_dimnum_mtp_layersnum_nextn_predict_layersg@TFzlayers.*.self_attn.q_projcolwise_gather_outputzlayers.*.self_attn.k_projzlayers.*.self_attn.v_projzlayers.*.self_attn.g_projzlayers.*.self_attn.o_projrowwise_split_inputz!layers.*.mlp.experts.gate_up_projpacked_colwisezlayers.*.mlp.experts.down_projrowwisezlayers.*.mlp.expertsmoe_tp_expertsii,ii/rIii Ng?c  <aVPR^4pW nVPVPV,rCV'dRFWwrV\W4p\ V\ 4'gK(\ V4V8XgK:\WVRV4\WWsV4KY RFGp\W4p\ V\ 4'gK&\ V4V8XgK8\WVRV4KI RFEpVPV4p\ V\ 4'gK,\ V4V8XgK>VRVW&KG VPf'R.V,Vn V'dR.V,Vn VPfVPRR4p V ef\ V \4'dV PR4MT p V U u0uF/p \V 4P4'gK$\V 4kK1 p p M\!\#^V44p \#V4U u.uF qV 9dRMRNK p p V RVVn V'd WVVnVP&fNVPR R4pV'd#VPR VP(4VnMVP(VnTP+R \-VP4UUu/uFwppVR 8XgKVR VP&/bK upp4VP.fVP0VnVPRVP24pVPRR4pVPR R4p\4SV`l!R/VB\!VP4pVP8;'g/o\:;QJdV3RlV4F 'dK RM RM!V3RlV44'd"VUu/uF pVSV,bK upVnR#\ V\ 4'gV.\ VP4,p/VnVFNpVPP=V4p RRRVV ,/pV'd VV ,VR&VVP8V&KP V'd8RVP89d%VP8R,P?V4R#R#R#uup iuup iuuppiuupi)rN rope_thetapartial_rotary_factorsfull_attentionmoe_layers_enum,sparsedenseattention_other_settingrRper_layer_configsliding_attention rope_scalingc3b<"TF$p\SPV4\4xK& R#5iN) isinstancegetrg).0 layer_typerps& rZ 2Step3p7TextConfig.__post_init__..,s$Un z"&&,d33ns,/FT rope_typedefaultpartial_rotary_factor)) layer_typesmtp_layer_types)mlp_layer_typesmtp_mlp_layer_types) swiglu_limitsswiglu_limits_shared)rr) rOrrQgetattrrlistlensetattrrrrrresplitstriprUsetrangernum_sliding_attention_headsrR setdefault enumeratequery_pre_attn_scalarhead_dim default_thetasuperrTrcallindexupdate)rXrYrnpaddedfield mtp_fieldvaluekeyritemsimoe_setrr layer_idxrrrrlayer_type_setparamsr __class__s&, @rZrTStep3p7TextConfig.__post_init__sB $*::.H!#L (@%**D,B,BE],]6 #%  ,eT**s5zV/CDr3DUV_=%C,eT**s5zV/CDr3C@ 3eT**s5zV/C"')FK@    # 01A5D '(8'9Q_#`D j$//(\C0@0@,AA #%D , $$**:6%y, 1 N)6LQ6OF2339$$Z0 -  0D4H4H H$$%56==lK!I|sD]& 0$as$;!R:!R:R? S S S cd<V^8dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R &S[;R &S[;R &S[R ,;R &S[R ,;R&S[;R&S[R ,;R&S[S[S[,,R ,;R&S[;R&S[S[,R ,;R&S[;R&S[;R&S[S[S[,R ,,R ,;R&S[S[S[,R ,,R ,;R&S[S[,R ,;R&S[S[,R ,;R&#)r^rPrWr|rQrd vocab_size rms_norm_epsmoe_intermediate_sizerzrrNsliding_windowr pad_token_id bos_token_id eos_token_idattention_biasrmoe_router_scaling_factormlp_biasrrrr)rUrfrboolre)rhris"rZrjStep3p7TextConfig.__annotate__vsVWX"YZ [\]^!)_`abcd%efgh ij kl$J%mn"%t2opqr*#stS /D(/uv wx;-4yz %*{|}~ d*+d29@us{T12T9@AB#Y%,CDcT)0Er\)rrrrrrrrc)7rlrmrnro__doc__rp attribute_maprgatinguse_bidirectional_attentionbase_model_tp_planAttributeErrordense_intermediate_sizeshared_intermediate_size rotary_dim swiglu_alpha swiglu_limit index_n_headsindex_head_dimindex_block_sizeindex_topk_blocksindex_local_blocksoutput_router_logitsrouted_scaling_factorrouter_aux_loss_coefrouter_jitter_noiserPrWr|rQrdrrrrzrrrrrrrrrrrrrrrrTrsrtru __classcell__rris@@rZr>r>vs<J/ 5-*/4 MM F"' $%<#%<#%<#%<#%:+-=() 0 -.-/!J!#L!#L"$M#%N%'&('))+*,)+(*K" #)JL!%  !%N.2L#L+/L N04'*H59M<@(,O,0_LIr\c^aa]tRtRtoRtR]R]/tRtRt Rt Rt V3Rlt V3R lt R tVtV;t#) r?i<step3p7rE text_configNFiPc L<VPf\4VnMe\VP\4'dF\R/VPP 4UUu/uFwr#VR8wgKW#bK uppBVnVP f\ 4VnMe\VP \4'dF\ R/VP P 4UUu/uFwr#VR8wgKW#bK uppBVn\SV` !R/VBR#uuppiuuppi)Nrpr) rEr=rrgrrr>rrT)rXrYkvrs&, rZrTStep3p7Config.__post_init__Gs    %!4!6D  **D 1 1!4"$($6$6$<$<$>T$>DA!|BS414$>T"D     #02D  (($ / /0tTEUEUE[E[E]3sE]TQabfrarDADE]3stD  ''U 4ts$ D4D' D 7D c<V^8dQh/S[S[,R,;R&S[S[,R,;R&S[;R&S[;R&#)r^NrErprojector_biasimage_token_id)rgrrrU)rhris"rZrjStep3p7Config.__annotate__<sO **T18 ((4/6  r\)rrE)rlrmrnrorpr=r> sub_configsrErrrrTrsrtrurrs@@rZr?r?<s<J"$7HYZK48M26K N N (r\c0a]tRtRtoRtV3RltRtVtR#)Step3p7ImageProcessorKwargsiWa  patch_size (`int`, *optional*, defaults to 504): Target size (height = width) for each local patch crop. max_image_size (`int`, *optional*, defaults to 3024): Images larger than this (on their longest side) are scaled down uniformly before patch planning. c2<V^8dQh/S[;R&S[;R&#)r^r`max_image_sizerU)rhris"rZrj(Step3p7ImageProcessorKwargs.__annotate__WsOr\rN)rlrmrnrorrsrtrurvs@rZrrWsr\rF)totalcaa]tRtRtoRt]P tRRRR/tRt Rt Rt ] t ]tRt]t.ROtRt^t^t]V3R lR l4tV3R lR ltRV3R lRlltV3RlRltV3RlRlt]V3RlV3Rll4tV3RltRt Vt!V;t"#)r@idz Image processor for Step-3.7-Flash. Each input image is split into a global down-scaled view plus zero or more local patch crops via a sliding-window strategy, then every sub-image is resized and normalised independently. heightrGrKiTi c,<V^8dQhRS[RS[RS[/#)r^rKrreturn)rUr)rhris"rZrj"Step3p7ImageProcessor.__annotate__s'SS#SsStSr\c d\W4^ 8;'d\W, W, 4^8#)zH`True` for near-degenerate images (min side < 32px, aspect ratio > 4:1).minmax)rKrs&&rZ_is_extreme_aspect(Step3p7ImageProcessor._is_extreme_aspects-5!B&RR3u~v~+NQR+RRr\c<V^8dQhRS[RS[RS[RS[RS[S[S[S[3,S[S[S[3,S[S[S[S[3,/#)r^rKrr_r`r)rUtupler)rhris"rZrjrs^>w>w>w"%>w36>wDG>w uS#Xc3hc3D E>wr\c *a VPW4pV'd \W4;r\W!4VP8d>VP\W!4, p\W,4\W&,4r!\ W!4\W!4rW8:dW, R8dTM^o MW, ^8d \ Wt4o MVo S ^8Xd W3W3^^^V3#V 3RlW34wr\^V S ,4p \^V S ,4p W3W3S WV3#)u%Compute the sliding-window patch layout for one image. Unlike models with a fixed tile size (Idefics3, LLaVA-NeXT), Step3p7 adapts the window size to the image's aspect ratio, so this cannot be reduced to a simple ``ceil(h / tile) × ceil(w / tile)`` formula. Step 1 = normalise extreme inputs: - extreme-aspect images (min_side < 32, ratio > 4) are squared - images larger than ``max_image_size`` are scaled down uniformly Step 2 — choose window size from the normalised aspect ratio: - fits in global view (long_side ≤ image_size): tile only if elongated (long_side / short_side > 1.5), using short_side as the window - very elongated (ratio > 4): ``min(short_side, patch_size)`` - standard case: ``patch_size`` Step 3 — snap each dimension to the nearest window multiple (snap up when the remainder exceeds 20 % of the window size). Returns: global_width_height: ``(width, height)`` to resize the global view to before squaring crop_width_height: ``(crop_width, crop_height)`` snapped dimensions for patch extraction window_size: tile side length (``0`` → no local patches) num_patches_x: number of patches along the width num_patches_y: number of patches along the height needs_square_pad: whether the raw image must be zero-padded to a square before resizing g?c3<"TF6pVS8d)SVS,VS,RS,8,,MTxK8 R#5i)g?Nr)rdim window_sizes& rZr6Step3p7ImageProcessor._plan_patches..sL# '+%sk1S;5F{IZ5Z[\'s>A)rrrrUr)rXrKrr_r`needs_square_padscale short_side long_side crop_width crop_height num_patches_x num_patches_yrs&&&&& @rZ _plan_patches#Step3p7ImageProcessor._plan_patchess> 225A  / /E v  3 3 3''#f*<(D*!K  #a 'j5K$K ! ?UOQ1>NN N#  #  Az[89 A{k9:  9; fvvvr\c,<V^8dQhRS[RS[RS[/#)r^rrKrr)rhris"rZrjrs#--#-c-Z]-r\c T;'g/pVPRVP4pVR,pVPRVP4pVPW!WV4R,wrxWx,#)zBReturn the number of local patches for an image of the given size.sizerr`:N)rrr`r) rXrrK images_kwargsrr_r`rrs &&&& rZget_number_of_image_patches1Step3p7ImageProcessor.get_number_of_image_patchessh%++   3(^ "&&|T__E '+'9'9%'`ad'e$ ,,r\c<V^8dQhRS[PRS[RS[RRRS[S[PS[S[P,S[S[3,/#)r^imgr_r`resamplerr)torchTensorrUr r)rhris"rZrjrsc+A+A \\+A+A +A ' +A u||T%,,/c9 : +Ar\c VPwrVpVPWvW#4wwrwrp p ppV'd2\Wv4pVPV.\ VVR7R7^,pVP ^4pVP V\ WR7VR7pVP V\ W"R7VR7P^4pV ^8XdV.W3#VP V\ WR7VR7P^4p\VV R7pVVW3#)aJStep3p7-specific cropping: square-pad extreme aspect ratios, resize the global view, and slice out raw (pre-final-resize) local-patch tiles per `_plan_patches`'s layout. Returns the resized global view, the list of raw local-patch tensors (still at `window_size`, not yet resized to `patch_size`), and the patch grid dimensions. rrK)pad_sizer$)r`) shaperrpadr unsqueezeresizesqueezer )rXr#r_r`r$_rrK global_width global_heightrrrrrrside img_batch global_img img_for_croppatchess&&&&& rZ_get_image_patches(Step3p7ImageProcessor._get_image_patchess995   uj E )\ %Z     u%D((C584t+L(MaPCMM!$ [[HM,^iq[r [[XZ-Zem[nvv   ! r=? ?{{ x{EPX# '!* $L[I7M@@r\c<V^8dQhRS[R,RS[RS[RS[RS[S[S[,,R,RS[S[S[,,R,R R R S[R S[R S[R,RRRS[/ #)r^imagesz torch.Tensor do_rescalerescale_factor do_normalize image_meanN image_stdr$rrr`disable_groupingreturn_tensorszstr | TensorType | Noner)rrrfrrUr )rhris"rZrjrsII^$II I  I DK'$. I4;&-I'III+I2I Ir\c VPp ....3wrppVFpVPVWV4wppppVPV4VP\V44VPV4TP\ V4UUu.uF5p\ V4F#pVV^, 8H;'d VV^, 8NK% K7 upp4K VP \ P!V4W#WEV4pRVRV/p\VV R7p\V^R7pV'd\VRV R7wppVP4F;wpp VPV \WR7VR7p!VP V!W#WEV4VV&K= \VVRR 7p"\ P!V"U#U$u.uFp#V#Fp$V$NK K up$p#4VR &VU%u.uF$p%V%R .V\V%4, ,,NK& up%VR &V#uuppiuup$p#iuup%i)  pixel_valuesnum_local_patches)data tensor_type)rT) is_nestedrAr(r*)rIpixel_values_localFpatch_newline_masks)rr8appendrrrescale_and_normalizer%stackr rr rr.rr )&rXr;r<r=r>r?r@r$rr`rArBrYr_ global_imagesnested_patchesrFrKr#r5r7rrrowcol global_stackrGresult max_patchesgrouped_patches grouped_indexr+stacked_patchesresizednested_pixel_values_localper_image_patchespatchmasks&&&&&&&&&&&&&, rZ _preprocess!Step3p7ImageProcessor._preprocesssM[[ PRTVXZ\^P^M '8:MC@D@W@WZXA =J    ,  $ $S\ 2  ! !' *  & & %]33$]3=1,,HH}q7H1HH3I3 "11 KK & L^g L !2 4^D+Q7 .C$AQ. *O]+:*?*?*A&++#XZ%R]e&*.)C)CZS\*& +B)7 ae(f %,1;;0Ii0I,WheWh0Ii,F' ( H[-GZtw+D "9:::GZ-F( ) ]Pj-s %G8 /G8 #G> *Hc<<V^8dQhRS[RS[S[,RS[/#)r^r;rYr)rrrr )rhris"rZrjrGs)444v>Y7Z4_k4r\c &<\SV`!V3/VB#r)r preprocess)rXr;rYrs&&,rZrb Step3p7ImageProcessor.preprocessFsw!&3F33r\c<V^8dQh/S[;R&S[S[,;R&S[S[,;R&S[;R&S[;R&S[;R&#)r^r`r?r@rvision_patch_sizedownsampler_stride)rUrrf)rhris"rZrjrds[U . E{,!*+2345r\r)rErJrFr)#rlrmrnrorrBILINEARr$rr`r<r>r r?rr@do_convert_rgbr valid_kwargsmodel_input_namesrrerf staticmethodrrr r8r^rrbrsrtrurrs@@rZr@r@ds"**H c7C (DJJL.J,IN.LSN SS>w>w@--+A+AZIIV444Gr\c&a]tRtRtoRtRtVtR#)Step3p7VisionRotaryEmbeddingiNc VVP^4p\P!W"3RR7#)zM Recompose the frequencies into the final spatial layout used per each grid. r )flattenr%cat)rXfreqfreq_hws&& rZrecomposition_frequencies6Step3p7VisionRotaryEmbedding.recomposition_frequenciesOs%,,q/yy'+44r\rN)rlrmrnrorurtrurvs@rZrmrmNs55r\rmc]tRtRtRtR#)Step3p7VisionMLPiWrNrlrmrnrortrr\rZrxrxWr\rxc]tRtRtRtR#)Step3p7VisionAttentioni[rNryrr\rZr|r|[rzr\r|cTaa]tRtRtoV3RlV3RlltRV3RlRlltRtVtV;t#)Step3p7VisionEncoderLayeri_c <V^8dQhRS[/#r^configr=)rhris"rZrj&Step3p7VisionEncoderLayer.__annotate__`see2er\c\<\SV`4Wn\V4Vn\ V4Vn\P!VPVPR7Vn \P!VPVPR7Vn \P!VP\P !VP4,4Vn\P!VP\P !VP4,4VnR#)epsN)r__init__rr| self_attnrxmlpnn LayerNormrPrblayernorm_beforelayernorm_after ParameterrSr%oneslambda_1lambda_2rXrrs&&rZr"Step3p7VisionEncoderLayer.__init__`s  /7#F+ " V-?-?VEZEZ [!||F,>,>FDYDYZ V%B%BUZZPVPbPbEc%cd  V%B%BUZZPVPbPbEc%cd r\c<V^8dQhRS[PRS[PR,RS[S[PS[P3,R,RS[P/#)r^ hidden_statesattention_maskNposition_embeddingsr)r%r&r )rhris"rZrjrjs[|| t+#5<<#=>E   r\c TpVPV4pVP!V3RVRV/VBwrVPV,pW,pTpVPV4pVP V4pVP V,pW,pV#)rr)rrrrrr)rXrrrrYresidualr0s&&&&, rZforward!Step3p7VisionEncoderLayer.forwardjs!--m< >>  ) !4     5 %0  ,,]; /  5 %0 r\)rrrrrrrNN rlrmrnrorrrtrurrs@@rZr~r~_s eer\r~cPaa]tRtRtoV3RlV3RlltV3RlRltRtVtV;t#)Step3p7VisionEmbeddingsic <V^8dQhRS[/#rr)rhris"rZrj$Step3p7VisionEmbeddings.__annotate__s  2 r\c<\SV`V4\P!VPVP VP VP RR7VnR#)F) in_channels out_channels kernel_sizestridebiasN)rrrConv2d num_channelsrPr`patch_embeddingrs&&rZr Step3p7VisionEmbeddings.__init__sH  !yy++++))$$  r\cZ<V^8dQhRS[PRS[RS[RS[P/#)r^ embeddingsrrKr)r%r&rU)rhris"rZrjrs55<<UX]b]i]ir\cVPPP^,pW P,pW0P,p\ VR,4p\ P P4'g2WW8Xd,Wg8Xd&VPPP^4#VPPP^4pVPR,p VP^WwV 4pVP^^^^4p\PPVWV3RRR7pVP^^^^4P^RV 4pV#)r?bilinearF)rmode align_cornersrp)position_embeddingweightr+r`r!r%jit is_tracingr-reshapepermuter functional interpolateview) rXrrrK num_positions new_height new_widthsqrt_num_positionspatch_pos_embedr s &&&& rZinterpolate_pos_encoding0Step3p7VisionEmbeddings.interpolate_pos_encodings-//66< >1188BB1Er")11!5G]`a)11!Q1=--33 ( 4  *11!Q1=BB1b#Nr\)r) rlrmrnrorrrtrurrs@@rZrrs  r\rcaa]tRtRtoRtRtRR.tR.tRtRt Rt Rt Rt ] P!4V3Rl4tV3R ltR tVtV;t#) r:imodelTr~Step3p7DecoderLayerpast_key_valuesFc<\SV`V4\V\4'dQ\P !VP \P!VP4PR 44R #\V\4'd\PPVPVPP 4\PPVP"VPP 4R #\V\$4'dVP&FpVP(pVP*V,R8wd\,VP*V,,pV!VPVR7wrE\P !\/W R24V4\P !\/W R24V4K R #\V\04'd`\/VPRR4p\P2!VP4RVR7\P2!VP6RVR7R #\V\84'd]\/VPRR4p\P2!VP:RVR7\P<!VP>4R #\V\@4'd#\P<!VP:4R #R #) rDr)r _inv_freq_original_inv_freqinitializer_rangeg{Gz?)meanstdN)rDrp)!r _init_weightsrrinitcopy_ position_idsr%arangerexpandr~r constant_rrrSrStep3p7RotaryEmbeddingrcompute_default_rope_parametersrrrStep3p7Expertsnormal_ gate_up_proj down_projStep3p7TopKRouterrzeros_e_score_correction_biasStep3p7RMSNorm)rXmoduler rope_init_fn curr_inv_freqr0rrs&& rZr$Step3p7PreTrainedModel._init_weightss f% f5 6 6 JJv**ELL9M9M,N,U,UV],^ _  9 : : GG  foov}}/S/S T GG  foov}}/S/S T  6 7 7$00 %EE ##J/9<#6v7G7G 7S#TL#/ *#U  76\+CDmT 76\9K+LM}] 1 / /$++':DAC LL,,3C @ LL)) =  1 2 2$++':DAC LLSc : KK66 7  / / KK &0r\c&<V^8dQh/S[;R&#rr?)rhris"rZrj#Step3p7PreTrainedModel.__annotate__s r\r)rlrmrnrobase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_supports_sdpa_supports_flex_attn_supports_static_cache_supports_attention_backendr%no_gradrrsrtrurrs@@rZr:r:sd &*#46KL#4"5 N!"& ]]_''r\caa]tRtRtoRtR]R]/tV3RlV3Rllt] !RR7] V3R lR l44t V3R lt R t VtV;t#) r<iu!Vision encoder: patch embeddings → 2-D RoPE transformer layers → conv downsampler. The rotary embedding (``self.rotary_emb``) and layer stack (``self.layers``) are held directly on this module, following the Gemma4 convention of not wrapping them in a separate ``Encoder`` submodule. r attentionsc <V^8dQhRS[/#rr)rhris"rZrjStep3p7VisionModel.__annotate__s  2 r\cz<\SV`V4\V4Vn\P !VP VPR7Vn\V4Vn \P!\VP4Uu.uFp\V4NK up4Vn\P !VP VP ^,^^^R7Vn\P !VP ^,VP ^,^^^R7VnVP'4R#uupi)r)rrpaddingN)rrrrrrrPrb pre_layernormrm rotary_emb ModuleListrrQr~rLr downsampler1 downsampler2 post_init)rXrr0rs&& rZrStep3p7VisionModel.__init__s  1&9\\&*<*<&BWBWX6v>mmPUV\VnVnPo$pPo1%>v%FPo$pq IIf&8&8&:L:Lq:P^_histuII    "F$6$6$:RS]^   %qs D8F)tie_last_hidden_statescP<V^8dQhRS[PRS[S[,RS[/#)r^rErYr)r%r&rrr)rhris"rZrjrs2[[ELL[FCUWx4p VPV 4p VPV 4p \V P^4P^^^4R7#)T)r)device)spatial_merge_sizer)last_hidden_staterp)r+rr`rr%tensorrr#rrLrrrrrrq)rXrErY batch_sizer0rrKgrid_hgrid_w hidden_stategrid_thwrrlayerchannelss&&, rZrStep3p7VisionModel.forwards5(4'9'9$ v??555//444|dS )),7 <Q>QR.xAN "oolI[[E aCVaZ`aL! %%b)#++Aq!499*PV_ ((6 ((6 1E1Ea1H1P1PQRTUWX1YZZr\c&<V^8dQh/S[;R&#rr)rhris"rZrjrs r\)rrrrLrr)rlrmrnrorr~r|_can_record_outputsrr"rrrsrtrurrs@@rZr<r<sT 2,  E2[3[;r\c]tRtRtRtR#)rirNryrr\rZrrs r\rc]tRtRtRtR#)ri rNryrr\rZrr rzr\rc<aa]tRtRtoRV3RlltRtRtVtV;t#) Step3p7MLPi c<\SV`V4V'd VPM VPVnVP'dVPV,M^;'g \ R4VnR#)rinfN)rrrrWrrflimit)rXrris_shared_expertrs&&&&rZrStep3p7MLP.__init__s\  >!,-33 3C \\!_ " " { " C~~di((r\)rWr)Frrs@@rZrr st))r\rcLaa]tRtRtoRtRV3RlltV3RlRltRtVtV;t #)riTc<\SV`V4VPVnVf\ R4VnR#TVnR#)Nr)rrrintermediate_dimrfr)rXrrrs&&&rZrStep3p7Experts.__init__s4  & < <%1%9U5\ | r\cN<V^8dQhRS[PRS[P/#)r^gate_upr)r%r&)rhris"rZrjStep3p7Experts.__annotate__$s#5<<ELLr\cVP^RR7wr#VPV4PVPR7pVPVP)VPR7pW#,#)r^rorrrp)chunkrrr)rXr&rrs&& rZ _apply_gateStep3p7Experts._apply_gate$sZ===+{{4 &&4::&6 XX4::+4::X 6yr\)r#rr) rlrmrnro#_fp8_experts_clamp_after_activationrr*rtrurrs@@rZrrs *.'L r\rc]tRtRtRtR#)ri+rNryrr\rZrr+rzr\rc&a]tRtRtoRtRtVtR#)Step3p7SparseMoeBlocki/c.\PPV4VP'dVPV,;'gRMRp\ V4Vn\ WR7Vn\WRR7Vn VPVn R#)N)rT)r) rModulerrrrrexpertsrshared_expertsrr)rXrrrs&&& rZrStep3p7SparseMoeBlock.__init__0st 4 EKDXDXDX,,Y7??4^b %f- %fH (TR%+%E%E"r\)r2rrr3Nrlrmrnrorrtrurvs@rZr/r//sFFr\r/cTaa]tRtRtoV3RlV3RlltRV3RlRlltRtVtV;t#)Step3p7Attentioni;c&<V^8dQhRS[RS[/#)r^rr)r>rU)rhris"rZrjStep3p7Attention.__annotate__<s::0:S:r\cp<VPp\SV` WV4VPR,VnR#)rNg)rRrrrscaling)rXrr num_headsrs&&& rZrStep3p7Attention.__init__<s/..  I633T9 r\c<V^8dQhRS[PRS[S[PS[P3,RS[PR,RS[R,RS[S[,RS[S[PS[PR,3,/#)r^rrrNrrYr)r%r&r rrr)rhris"rZrjr9As*)*)||*)#5<<#=>*) t+ *)  *) +, *) u||U\\D00 1*)r\c LVPRRp.VORNVPN5pVPVPV4P V44P ^^4pVP VPV4P V44P ^^4p VPV4P V4P ^^4p VPV4p Vwr\WW4wrVeVPWVP4wr\P!VPP \"4pT!VVV V V3RVP$'gRM VP&RVP(RVP*/VBwppVP,!.VORN5!pVPpVP!.VRROVP.NVPN5!pWP1R4P34,pVP!V!pVP5V4pVV3#)Ndropoutrr;rrp)r+rq_normq_projr transposek_normk_projv_projg_projr-rrr get_interfacer_attn_implementationr.trainingattention_dropoutr;rrr<r-sigmoido_proj)rXrrrrrY input_shape hidden_shape query_states key_states value_states gate_statescossinattention_interface attn_output attn_weights output_shapes&&&&&, rZrStep3p7Attention.forwardAs$))#2. 88b8$--8 {{4;;}#=#B#B<#PQ[[\]_`a [[]!;!@!@!NOYYZ[]^_ {{=166|DNNqRST kk-0 &#7 RU#[  &'6'='=jX\XfXf'g $J(?(M(M KK , ,.E) %8      % $}}}C$2H2H % LL %  .. %  % ! \"));;;; "(( !&&Y Sb(9Y4>>Y4==Y !$9$9"$=$E$E$GG !&& 5 kk+. L((r\)r;rrrs@@rZr7r7;s:: *)*)*)r\r7c&a]tRtRtoRtRtVtR#)rinc\PPV4VPVnVPV,p\ W24VnWP nVPV,Vn VPV,R8Xd\M\pV!W4Vn \VPVPR7Vn\VPVPR7VnR#)rrN)rr1rrPrr7rrrattention_typerr/rrrrinput_layernormpost_attention_layernorm)rXrr layer_config mlp_classs&&& rZrStep3p7DecoderLayer.__init__os 4 !--..y9 ),B &$00;.4-C-CI-NRZ-Z)`j V/-f.@.@fFYFYZ(6v7I7IvObOb(c%r\)r]rPr^rr_rNr5rvs@rZrrnsddr\rcVaa]tRtRtoR]R]/tV3RlV3RlltV3RltRt Vt V;t #)r;irrc <V^8dQhRS[/#rr>)rhris"rZrjStep3p7TextModel.__annotate__s0r\c <\SV`V4\P!VPVP VP 4Vn\P!\VP4Uu.uFp\W4NK up4Vn \VP VPR7Vn\!VR7VnVP$'dr\VPVPVP$,4pVUu0uF pRV R2kK pp\'VP(;'g.4V,VnVP+4R#uupiuupi)r)rz(^|\.)layers\.z\.N)rrr EmbeddingrrP padding_idx embed_tokensrrrQrrLrrnormrrrr"_keys_to_ignore_on_load_unexpectedr)rXrr mtp_layerspatternsrs&& rZrStep3p7TextModel.__init__s  LL):):F!B/HD69$:a:a:g:geg6hks6sD 3 %nEs 3EE$c&<V^8dQh/S[;R&#rre)rhris"rZrjrfs r\)rlrjrLrkr) rlrmrnrorr7r rrsrtrurrs@@rZr;r;s, ,& r\cha]tRtRtoV3RlRlt]]R V3RlRll44tV3RltRt Vt R#) r9ic <V^8dQhRS[/#rr)rhris"rZrjStep3p7Model.__annotate__s  } r\c\PW4\VP4Vn\ VP 4VnVP PVn\P!VPP^,VP PVPR7Vn VPVnVP!4R#))rN)r:rr<rE vision_modelr;rlanguage_modelrrLinearrPrmulti_modal_projectorrimage_placeholder_token_idr)rXrs&&rZrStep3p7Model.__init__s''5.v/C/CD.v/A/AB ,,77&(YY  , ,q 0&2D2D2P2PW]WlWl& "+1*?*?' r\Nc <V^8dQhRS[PRS[PR,RS[S[,R,RS[S[,/#)r^rErJNrFr)r%r&rrUr r)rhris"rZrjrssO# # ll# "LL4/#  9t+ # + + # r\c VP!V3/VBpVPVP4pVe3VPVP!V3//VCRR/CBP4MRpVf^.VP^,,p.p^p \ V4Fwr.p V ^8d=WyW,p V P V P RV PR,44V P Wj,P RVPR,44W, p TP \V 4^8d\P!V 4MV ^,4K \VPVVPVPR7#)Noutput_hidden_statesF)r pooler_outputrrrp) rvryrr+rrLrrr%rrrrr)rXrErJrFrY vision_outputimage_featurespatch_image_featuresmerged cur_patch_idxr num_patch cur_feature patch_slices&&&&, rZget_image_featuresStep3p7Model.get_image_featuress{)),A&A 33M4S4ST "-  & &!!"4b8a68aCY[`8abtt    $!"n&:&:1&= =  %&78LAK1}2=C\] "";#3#3B 8I8I"8M#NO   ~055b.:N:Nr:RS T  &M MMC 4Dq4H%))K0kZ[n ]9*+== '55$//   r\c&<V^8dQh/S[;R&#rr)rhris"rZrjrs r\)rzrwryrvrr) rlrmrnrorrrrrsrtrurvs@rZr9r9s7  # # # %r\c,a]tRtRtoV3RltRtVtR#)r8ic&<V^8dQh/S[;R&#rr)rhris"rZrj,Step3p7ForConditionalGeneration.__annotate__rr\rN)rlrmrnrorsrtrurvs@rZr8r8sr\ctaa]tRtRtoRtR V3RlV3Rlllt]V3RlRl4tV3RlRltR t Vt V;t #) rAiaAProcessor for Step-3.7-Flash. Uses :class:`ProcessorMixin.__call__` for the standard image-token expansion flow: the image processor splits each image into global + local patch crops, then :meth:`replace_image_token` builds the per-image replacement string that ``get_text_with_replacements`` substitutes into the text. c<V^8dQhRR/#)r^rNr)rhris"rZrjStep3p7Processor.__annotate__svvY]vr\c <RVnVeVPVP4MRVnVPVP,pVP R,V,^,VnVPV,^,VnVPVP ,Vn VPVP,Vn \SV`0!RRVRVRV/VBR#)z Nrimage_processor tokenizer chat_templater) image_tokenconvert_tokens_to_idsrrerfrnum_image_feature_sizer`num_patch_feature_sizeimage_feature_placeholderpatch_feature_placeholderrr)rXrrrrYrrs&&&&, rZrStep3p7Processor.__init__s'S\Shi==d>N>NOnr 22_5W5WW'6';';H'E'OTU&U#'6'A'AV'KPQ&Q#)-)9)9Dz zz z)rrrr) rXrrrY num_patchesrKpatch_newline_maskreplrs &&&, rZreplace_image_token$Step3p7Processor.replace_image_tokens"#67 B *../DE?R?^0;dh{#A mD$B$B#C;O OD!!&8&;&;))$ *T;;r?r@rA)pcollections.abcrr%torch.nnrhuggingface_hub.dataclassesrrrr cache_utilsrconfiguration_utilsrimage_processing_backendsrimage_processing_utilsr image_transformsr r r image_utilsr rrrrmodeling_layersrmodeling_outputsrrmodeling_rope_utilsrmodeling_utilsrrprocessing_utilsrrrutilsrrrrrr r!utils.output_capturingr" vision_utilsr#$deepseek_ocr2.modeling_deepseek_ocr2r$r% deepseek_v4.modeling_deepseek_v4r&r'gemma3.modeling_gemma3r(kimi_k25.modeling_kimi_k25r)laguna.modeling_lagunar*r+r,r-r.)minimax_m3_vl.configuration_minimax_m3_vlr/$minimax_m3_vl.modeling_minimax_m3_vlr0r1r2r3r4qwen2_vl.modeling_qwen2_vlr5siglip.configuration_siglipr6siglip.modeling_siglipr7 get_loggerrllogger__all__r=r>r?rr@rmrxr|r~rr:r<rrrrrr/r7rr;r9r8rArr\rZrs% .& 3;2XXff9K6FDD63jO4FNE<;   H % 67H,H8H867AL-AL8ALH67($(8(2 ,e c4.c4c4R5#@5 +  7 " :"J4@('_('('V/[//[/[d 2  '  ) ) &   -  F5 F0)0)fd,d*45 $5 p&J"~""r\