+ j^RIt^RIt^RIHt^RIHt^RIt^RIHt^RIH t ^RI H t ^RI HtHt^RIHt^R IHt^R IHtHt^R IHt^R IHtHtHtHt^R IHtH t ^RI!H"t"H#t#^RI$H%t%!RR]PL4t']!RR7]!RR]444t(]!RR7]!RR]444t)]!RR7]!RR]444t*]!RR7]!RR]444t+!RR ]PL4t,!R!R"]PL4t-!R#R$]PL4t.!R%R&]PL4t/RBR'R(llt0!R)R*]PL4t1!R+R,]PL4t2R-t3R.t4!R/R0]4t5!R1R2]4t6]!R3R4]44t7!R5R6]74t8]!R7R8]744t9]!R9R7!R:R;]744t:]!R]744t;]!R?R7!R@RA]]744t<.RCOt=R#)DN)Callable) dataclass)nn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)GradientCheckpointingLayer)BackboneOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstring torch_int)can_return_tuplemerge_with_config_defaults)OutputRecordercapture_outputs SwinConfigcjaa]tRt^*toRtR V3RlV3RllltV3RlRltV3RlRltRtVt V;t #) SwinDropPathzStochastic depth (DropPath) per sample, for residual blocks. Identity when ``drop_prob`` is 0 or outside training. See `Deep Networks with Stochastic Depth `_. c$<V^8dQhRS[RR/#) drop_probreturnN)float)format __classdict__s"g/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/swin/modeling_swin.py __annotate__SwinDropPath.__annotate__1s##%#$#c0<\SV`4WnR#N)super__init__r)selfr __class__s&&r"r)SwinDropPath.__init__1s "r%cN<V^8dQhRS[PRS[P/#r hidden_statesrtorchTensor)r r!s"r"r#r$5s#<>S   & $$Q')DM4F4F4J,KK 50C0CML`L`a  M$=>   +m;;r%c <V^8dQhRS[/#rr)str)r r!s"r"r#r$>s%%C%r%c RVP 2#)zp=r)r*s&r" extra_reprSwinDropPath.extra_repr>sDNN#$$r%rG)r4) __name__ __module__ __qualname____firstlineno____doc__r)rArH__static_attributes____classdictcell__ __classcell__r+r!s@@r"rr*s- ##<<%%%r%rzN Swin encoder's outputs, with potential hidden states and attentions. ) custom_introc@a]tRt^BtoRtRtRtRtRtV3Rlt Rt Vt R#)SwinEncoderOutputa reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nc<V^8dQh/S[PR,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&#)rNlast_hidden_state.r/ attentionsreshaped_hidden_statesr1 FloatTensortuple)r r!s"r"r#SwinEncoderOutput.__annotate__Bs ((4/6!"**C/047>#$e'',-4;%&"%"3"3S"89D@G'r%) rJrKrLrMrNrWr/rXrY__annotate_func__rOrPr!s@r"rUrUBs)37:>M7;JCG'r%rUzV Swin model's outputs that also contains a pooling of the last hidden states. cDa]tRt^XtoRtRtRtRtRtRt V3Rlt Rt Vt R#)SwinModelOutputa pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`, *optional*, returned when `add_pooling_layer=True` is passed): Average pooling of the last layer hidden-state. reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nc<<V^8dQh/S[PR,;R&S[PR,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&#)rNrW pooler_output.r/rXrYrZ)r r!s"r"r#SwinModelOutput.__annotate__Xs$((4/6%&$$t+2'(**C/047>)*e'',-4;+,"%"3"3S"89D@G-r%r^) rJrKrLrMrNrWrdr/rXrYr_rOrPr`s@r"rbrbXs. 37.2M:>M7;JCG-r%rbz* Swin masked image model outputs. cDa]tRt^qtoRtRtRtRtRtRt V3Rlt Rt Vt R#)SwinMaskedImageModelingOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `bool_masked_pos` is provided): Masked image modeling (MLM) loss. reconstruction (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Reconstructed pixel values. reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nc<<V^8dQh/S[PR,;R&S[PR,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&#)rNlossreconstruction.r/rXrYrZ)r r!s"r"r#*SwinMaskedImageModelingOutput.__annotate__qs(   d "))*%%,3+,**C/047>-.e'',-4;/0"%"3"3S"89D@G1r%r^) rJrKrLrMrNrirjr/rXrYr_rOrPr`s@r"rgrgqs- &*D/3N:>M7;JCG1r%rgz0 Swin outputs for image classification. cDa]tRt^toRtRtRtRtRtRt V3Rlt Rt Vt R#)SwinImageClassifierOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Classification (or regression if config.num_labels==1) loss. logits (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`): Classification (or regression if config.num_labels==1) scores (before SoftMax). reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nc<<V^8dQh/S[PR,;R&S[PR,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&S[S[PR3,R,;R&#)rNrilogits.r/rXrYrZ)r r!s"r"r#&SwinImageClassifierOutput.__annotate__s(   d "))*    $++,**C/047>-.e'',-4;/0"%"3"3S"89D@G1r%r^) rJrKrLrMrNriror/rXrYr_rOrPr`s@r"rmrms- &*D'+F:>M7;JCG1r%rmcbaa]tRt^toRtRV3RlltV3RlRltR V3RlRlltRtVt V;t #) SwinEmbeddingszO Construct the patch and position embeddings. Optionally, also the mask token. c<\SV`4\V4VnVPPpVPP VnV'd7\P!\P!^^VP44MRVn VP'd6\P!\P!^W1P44MRVn\P!VP4Vn\P"!VP$4VnVP(VnWnR#)r8N)r(r)SwinPatchEmbeddingspatch_embeddings num_patches grid_size patch_gridr Parameterr1zeros embed_dim mask_tokenuse_absolute_embeddingsposition_embeddings LayerNormnormDropouthidden_dropout_probdropout patch_sizeconfig)r*ruse_mask_tokenrvr+s&&& r"r)SwinEmbeddings.__init__s  3F ;++77 //99O]",,u{{1a9I9I'JKcgLRKiKiKiBLLQ 5E5EF Gos  LL!1!12 zz&"<"<=  ++ r%cZ<V^8dQhRS[PRS[RS[RS[P/#)r embeddingsheightwidthrr1r2int)r r!s"r"r#SwinEmbeddings.__annotate__s;DD5<<DDUXD]b]i]iDr%c `VP^,pVPP^,p\PP 4'gWE8XdW#8Xd VP#VPR,pW P ,pW0P ,p\ VR,4p VPP^WV4p V P^^^^4p \PPV Wx3RRR7p V P^^^^4P^RV4#)z Interpolate pre-trained position encodings to support higher-resolution images at inference. Unlike ViT, Swin has no CLS token, so position embeddings cover patch positions only. ?bicubicF)sizemode align_corners) r:r~r1jit is_tracingrrreshapepermuter functional interpolateview) r*rrrrv num_positionsdim new_height new_widthsqrt_num_positionspatch_pos_embeds &&&& r"interpolate_pos_encoding'SwinEmbeddings.interpolate_pos_encodings !&&q) 0066q9 yy##%%+*F6?++ +r". __, &}c'9:22::1>Pfij)11!Q1=--33 ( 4 &&q!Q2772sCCr%c <V^8dQhRS[PR,RS[PR,RS[RS[S[P ,/#)r pixel_valuesNbool_masked_posrr)r1r[ BoolTensorboolr\r2)r r!s"r"r#rsR--''$.-))D0-#' - u||  -r%cVPwrErgVPV4wrVPV4pVP4wrpVeZVPP WR4p VP R4PV 4p VRV , ,W,,pVPe4V'dWPWV4,pMWP,pVPV4pW3#)Ng?r) r:rurrr|expand unsqueezetype_asr~rr)r*rrr_ num_channelsrrroutput_dimensions batch_sizeseq_len mask_tokensmasks&&&& r"rASwinEmbeddings.forwards *6););&(,(=(=l(K% YYz* !+!2 Q  &//00bIK",,R088ED#sTz2[5GGJ  # # /''*G*G \a*bb '*B*BB \\*- ,,r%)rrr|rrurxrr~F)NF) rJrKrLrMrNr)rrArOrPrQrRs@@r"rrrrs*"DD<---r%rrcNaa]tRt^toRtV3RltRtV3RlRltRtVt V;t #)rtz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, seq_length, hidden_size)` to be consumed by a Transformer. cZ<\SV`4VPVPr2VPVP rT\ V\PP4'dTMW"3p\ V\PP4'dTMW33pV^,V^,,V^,V^,,,pW0nW`n V^,V^,,V^,V^,,3Vn \P!WEW3R7VnR#)r8) kernel_sizestrideN)r(r) image_sizerrr{ isinstance collectionsabcIterablervrwrConv2d projection)r*rrrr hidden_sizervr+s&& r"r)SwinPatchEmbeddings.__init__s !'!2!2F4E4EJ$*$7$79I9Ik#-j+//:R:R#S#SZZdYq #-j+//:R:R#S#SZZdYq !!} 1 5*Q-:VW=:XY $&$Q-:a=8*Q-:VW=:XY))L:ir%c W0P^,,^8wdS^VP^,W0P^,,, 3p\PPW4pW P^,,^8wdU^^^VP^,W P^,,, 3p\PPW4pV#)z9Pad pixel_values to be divisible by patch_size if needed.)rrrpad)r*rrr pad_valuess&&&& r" maybe_padSwinPatchEmbeddings.maybe_pad s ??1% % *T__Q/%//!:L2LLMJ==,,\FL OOA& &! +Q4??1#5QRAS8S#STJ==,,\FLr%c<V^8dQhRS[PR,RS[S[PS[S[,3,/#)rrNr)r1r[r\r2r)r r!s"r"r# SwinPatchEmbeddings.__annotate__s; - -E$5$5$< -u||UZ[^U_G_A` -r%cVPwr#rEVPWV4pVPV4pVPwr$pWE3pVP^4P ^^4pWg3#)r)r:rrflatten transpose)r*rrrrrrrs&& r"rASwinPatchEmbeddings.forwardsp)5););&~~lEB __\2 (..1e#O''*44Q: ,,r%)rwrvrr rJrKrLrMrNr)rrArOrPrQrRs@@r"rtrts$ j - - -r%rtcfaa]tRtRtoRtV3RlV3RlltV3RlRltV3RlRltR tVt V;t #) SwinPatchMergingi!zP Patch Merging Layer. Args: dim (`int`): Number of input channels. c$<V^8dQhRS[RR/#)rrrN)r)r r!s"r"r#SwinPatchMerging.__annotate__*s**C*D*r%c<\SV`4\P!^V,^V,RR7Vn\P !^V,4VnR#)FbiasN)r(r)rLinear reductionrr)r*rr+s&&r"r)SwinPatchMerging.__init__*s> 1s7AG%@LLS) r%cZ<V^8dQhRS[PRS[RS[RS[P/#)r input_featurerrrr)r r!s"r"r#r/s4u||SQVQ]Q]r%c  V^,^8XgV^,^8Xd5\PPV^^^V^,^V^,34pV#)zPPad input feature map to be divisible by 2 in both spatial dimensions if needed.)rrr)r*rrrs&&&&r"rSwinPatchMerging.maybe_pad/sL QJ!OaMM--maAuqyRSU[^_U_=`aMr%cj<V^8dQhRS[PRS[S[S[3,RS[P/#)rrinput_dimensionsr)r1r2r\r)r r!s"r"r#r5s6U\\U3PS8_Y^YeYer%c Vwr4VPwrVpVPWSWG4pVPWV4p\P!\ ^4UU u.uF'p\ ^4FqRV R^1VR^1R3,NK K) up pRR7pVPVR^V,4pVP V4pVPV4pV#uup pi)rNNNNrr)r:rrr1catrangerr) r*rrrrrrrcolrows &&& r"rASwinPatchMerging.forward5s( (5(;(;% %**:uS }eD  KN:,.LL KK[^+a/A A4F4JKY W- ) (*yy1U1U1W1\1\]_1`mr's$r%c4<V^8dQhRS[P/#rDr0)r r!s"r"r#r^s'''r%c\P!VP^,4p\P!VP^,4p\P!\P!W.RR74p\P !V^4pVR,VR,, pVP ^^^4P4pVR;;,VP^,^, , uu&VR;;,VP^,^, , uu&VR;;,^VP^,,^, ,uu&VPR4#)rij)indexing)rrN)rNr)rrr)rrr8r) r1arangerstackmeshgridrr contiguoussum)r*coords_hcoords_wcoordscoords_flattenrelative_coordss& r"r8SwinRelativePositionBias._create_relative_position_index^s << 0 0 34<< 0 0 34U^^X,@4PQvq1(4~j7QQ)11!Q:EEG  D$4$4Q$7!$;;  D$4$4Q$7!$;;  A(8(8(;$;a$?? ""2&&r%c4<V^8dQhRS[P/#rDr0)r r!s"r"r#rosQQQr%cVPVP,pVPVPVPR4pVP ^^^4P 4P ^4#)r8r)rrrrrrr)r*relative_position_biass& r"rA SwinRelativePositionBias.forwardosd!%!B!B4C_C_!`!7!?RbmTn&o#r%c <V^8dQhRS[PRS[PR,RS[S[,RS[S[PS[P3,/#)rr/rNrr)r1r2r[rrr\)r r!s"r"r#r"sZ2)2)||2)))D02)+, 2) u||U\\) * 2)r%c VPRRp.VORNVPN5pVPV4PV4P ^^4pVP V4PV4P ^^4pVP V4PV4P ^^4pVP4p VezVP^,p V^,V ,p V^,p VP^4P^4PV RRRR4PR^W4pW,p MT p \P!VPP\4pT!VVVVV 3RVP 'gRM VP"RVP$/VBwppVP!.VORN5!P'4pVP)V4pVV3#)Nrr4rr)r:r$r)rrr*r+rrrrr get_interfacer_attn_implementationrr9r&rrr,)r*r/rr input_shape hidden_shape query_states key_states value_statesr num_windowsrr combined_maskattention_interfacerrs&&&, r"rASwinAttention.forwards$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST "&!  %(..q1K$Q;6J!!nG((+1 BB3Q1  3CM2M(?(M(M KK , ,.E) %8      % $}}}C$2H2H % LL %  % ! \"));;;;FFH kk+. L((r%) r&rr$r'r*r r,r)rrr+r' rJrKrLrMr)rArOrPrQrRs@@r"rrs pp 2)2)2)r%rcPaa]tRtRtoV3RlV3RlltV3RlRltRtVtV;t#)SwinMLPic&<V^8dQhRS[RS[/#)rrrr!)r r!s"r"r#SwinMLP.__annotate__s??z??r%c4<\SV`4\VP,Vn\ P !V\VPV,44Vn \ P !\VPV,4V4Vn R#r') r(r)r hidden_act activation_fnrrr mlp_ratiofc1fc2)r*rrr+s&&&r"r)SwinMLP.__init__se #F$5$5699S#f&6&6&<"=>99S!1!1C!78#>r%cN<V^8dQhRS[PRS[P/#r.r0)r r!s"r"r#r?s#U\\ellr%clVPV4pVPV4pVPV4pV#r')rDrBrE)r*r/s&&r"rASwinMLP.forwards4/ **=9 / r%)rBrDrEr;rRs@@r"r=r=s?? r%r=cVPwr#rEVPW#V,WV,W4pVP^^4P4PRWV4pV#)z* Partitions the given input into windows. rr:rrr)rrrrrrwindowss&& r"window_partitionrMsg/<.A.A+J!&&k);8LkM%%a+668==b+\hiG Nr%cVPR,pVPRW!,W1,WV4pVP^^4P4PRW#V4pV#)z7 Merges windows to produce higher resolution features. rrK)rLrrrrs&&&& r"window_reverserOs^==$Lll2v4e6JKfrsG1%00277F<XG Nr%caa]tRtRtoRV3RlV3RllltRV3RlRlltV3RlRltV3RlR ltV3R lR ltRV3R lR llt Rt Vt V;t #) SwinLayeric T<V^8dQhRS[RS[RS[S[S[3,RS[RS[RS[/#)rrrinput_resolutionrdrop_path_rate shift_size)rrr\r)r r!s"r"r#SwinLayer.__annotate__sWaaaa S/ a  a  aar%c<\SV`4\WWAPR7Vn\ P !W!PR7Vn\ P !W!PR7Vn \W4Vn \ P!VP4VnVPVnW`nW0nVR8d\#V4VnR#\ P$!4VnR#))repsr4N)r(r)rr attentionrrlayer_norm_epslayernorm_beforelayernorm_afterr=mlprrrrUrSrIdentity drop_path)r*rrrSrrTrUr+s&&&&&&&r"r)SwinLayer.__init__s &vIK]K]^ " S6K6K L!||C5J5JK6'zz&"<"<= !--$ 09G#9Mn5SUS^S^S`r%c <V^8dQhRS[PRS[S[S[3,RS[RS[S[,RS[P/#)rr/ralways_partitionrrr1r2r\rrrr)r r!s"r"r#rVsS-+-+||-+ S/-+ -+ +, -+  -+r%c ,V'gVPV4VwrVVP4wrxp Tp VPV4pVPWuWi4pVP WV4wrVP wrr\ VPV4VP4pVPRVPVP,V 4pVPWVPVPR7pVP!W3/VBwppVPV4pVPRVPVPV 4pVP\VVPW4RR7pV ^,^8gV ^,^8d VRRV1RV1R3,P4pVPWuV,V 4pWP!V4,pTpVP#V4pVP%V4pVPV4V,pVV3#)r8r5T)reverserNr)set_shift_and_window_sizerr\rrr:rM cyclic_shiftr get_attn_maskr6r7rZrrOrr`r]r^)r*r/rrcrrrrrchannelsshortcutr height_pad width_padhidden_states_windows attn_maskattention_outputrattention_windowsresiduals&&&&, r"rASwinLayer.forwards  * *+; <( "/"4"4"6 x --m< %**:uO %)NN=%$P! &3&9&9#y 01B1B=1QSWScSc d 5 : :2t?O?ORVRbRb?bdl m&& )<).> V`d.  a=1  1  1 1!WfWfufa2G H S S U -22:~xX >>2C#DD  ,,]; /  ]3h> l**r%c:<V^8dQhRS[S[S[3,RR/#)rrSrN)r\r)r r!s"r"r#rV@s#%S/dr%c \V4VP8:dq\^4Vn\P P 4'd+\P!\P!V44M \V4VnR#R#)zQClamp window and shift sizes when the window is larger than the input resolution.N)minrrrUr1rrtensor)r*rSs&&r"rg#SwinLayer.set_shift_and_window_size@s_  D$4$4 4'lDO=BYY=Q=Q=S=S %,,'789Y\]mYn   5r%c <V^8dQhRS[RS[RS[PRS[PRS[PR,/#)rrrr6r7rN)rr1r6r7r2)r r!s"r"r#rVHsHCEKKQVQ]Q]bgbnbnqubur%c 0VP^8:dR#\P!WR7p\P!W$R7pWQVP, 8P 4WQVP, 8P 4,pWbVP, 8P 4WbVP, 8P 4,pVR,^,VR,,P V4p \ WP4p V PRVPVP,4p V P^4V P^4, p V PV ^8gR4PV ^8HR4p V #)umBuild the cyclic-shift attention mask for shifted-window MSA; returns None when shift_size is 0. Each (h, w) position belongs to one of 9 cyclic-shift regions (3 along each axis), encoded as ``h_region * 3 + w_region``. Regions per axis: - 0: indices ``[0, axis - window_size)`` - 1: indices ``[axis - window_size, axis - shift_size)`` - 2: indices ``[axis - shift_size, axis)`` Implementation note: a single arithmetic pass on `torch.arange` (two comparisons + broadcast add) replaces the original 9-iteration nested-Python-loop slice-assignment — fully vectorised, no per-cell host-side scatter, no GPU↔host sync. N)r7r4)NrNN)NNrNrgY) rUr1rrlongrrMrr masked_fill) r*rrr6r7h_idxw_idxh_regionw_regionimg_mask mask_windowsros &&&&& r"riSwinLayer.get_attn_maskHsJ ??a  V3 U2d&6&666<<>%TXTcTcKcBcAiAiAkkT%5%555;;=RVRaRaJaAa@g@g@ii01A5AT8UUYYZ_`'2B2BC #((T-=-=@P@P-PQ  **1- 0F0Fq0II )))q.&AMMi[\n^ab r%c <V^8dQhRS[PRS[RS[RS[S[PS[S[R3,3,/#)rr/rrr.)r1r2rr\)r r!s"r"r#rVasL))u||)S))QVW\WcWcejknpsksetWtQu)r%c &VPW0P,, VP,pVPW P,, VP,p^^^V^V3p\PPW4pW3#)zHPad feature map so both spatial dimensions are divisible by window_size.)rrrr)r*r/rr pad_right pad_bottomrs&&&& r"rSwinLayer.maybe_padasy%%0@0@(@@DDTDTT &&2B2B)BBdFVFVV Ay!Z8  ))-D ((r%cT<V^8dQhRS[PRS[RS[P/#)rr/rfr)r1r2r)r r!s"r"r#rVis-  %,,  RWR^R^ r%c VP^8dHV'd^MRp\P!VW0P,W0P,3RR7pV#)zOApply a cyclic shift along the spatial dimensions for shifted-window attention.)shiftsdimsr)r8r)rUr1roll)r*r/rf directions&&& r"rhSwinLayer.cyclic_shiftisI ??Q $"I!JJ!OO3Y5PQM r%) rZr`rrSr]r\r^rUr)r4rr) rJrKrLrMr)rArgrirrhrOrPrQrRs@@r"rQrQsIaa(-+-+^2))   r%rQcfaa]tRtRtoV3RlV3RlltV3RlRltR V3RlRlltRtVtV;t #) SwinStageiuc d<V^8dQhRS[RS[RS[S[S[3,RS[RS[RS[S[,/#)rrrrSdepthrr`)rrr\listr)r r!s"r"r#SwinStage.__annotate__vs[RRRR S/ R  R  R;Rr%cD<\S V`4Wn\P!\ V4Uu.uF;p\ TTTTWh,V^,^8Xd^MVP^,R7NK= up4VnVeV!VR7Vn R#RVn R#uupi)r)rrrSrrTrUNr) r(r)rr ModuleListrrQrblocks downsample) r*rrrSrrr`rir+s &&&&&&&& r"r)SwinStage.__init__vs  mmu &A!%5'#,<%&UaZqf6H6HA6M &    2<1G*-T sABc z<V^8dQhRS[PRS[PRS[RS[RS[RS[P/#)rr/!hidden_states_before_downsamplingrr(output_hidden_states_before_downsamplingr)r1r2rr)r r!s"r"r#rs[bb||b,1<<b b  b 37 b br%c V'dY#TrpM3VPe"Y^,^,V^,^,rpMYTrpVPwrp VPWW4P^^^^4P 4#)u^ Select the spatial hidden states for this stage and reshape from (B, L, C) to (B, C, H, W). The chosen state and its resolution depend on output_hidden_states_before_downsampling: - True → pre-downsampling states at (height, width) — used by the backbone. - False → post-downsampling states at half the resolution (if a downsampler exists). )rr:rrr) r*r/rrrr spatial_statehwrrrs &&&&&& r"get_reshaped_hidden_states$SwinStage.get_reshaped_hidden_statess 4"CUaMa __ ("/1*1BUQYSTDTaMa"/aM%2%8%8" {!!*@HHAqRST__aar%c<V^8dQhRS[PRS[S[S[3,RS[RS[RS[S[,RS[S[PS[PS[PR,3,/#)rr/rrcrrrNrd)r r!s"r"r#rs~HH||H S/H H 37 H +, H u||U\\5<<$+>> ?Hr%c VwrgRpVPFp V !W3RV/VBwrK Tp VPeVPW4pVPWWgV4p WV3#)Nrc)rrr) r*r/rrcrrrrlast_attn_weights layer_modulerrYs &&&&&, r"rASwinStage.forwards)   KKL/;0BR0V\0 ,M,( -:) ?? & OO,M`M!%!@!@ fMu" 6GGGr%)rrr)FF) rJrKrLrMr)rrArOrPrQrRs@@r"rrus/RR8bb2HHHr%rc aa]tRtRtoRtRtRtRtR.tRt Rt Rt Rt Rt R]!]^RR7R ]!]^RR7/tR tR R .t]P(!4V3R l4tV3RltRtVtV;t#)SwinPreTrainedModeliswinrTrFr/)indexcapture_initial_hidden_staterXruz(attention\.self\.relative_position_indexz:attention\.relative_position_bias\.relative_position_indexc <\SV`V4\V\4'daVPe!\ P !VP4VPe#\ P !VP4R#R#\V\4'da\ P !VP4\ P!VPVP4PR44R#R#)zInitialize the weightsNr)r( _init_weightsrrrr|initzeros_r~rrcopy_rrr)r*r r+s&&r"r!SwinPreTrainedModel._init_weightss f% fn - -  , F--.))5 F6676  8 9 9 KK;; < JJv55v7]7]7_7d7deg7h i:r%c&<V^8dQh/S[;R&#rrr)r r!s"r"r# SwinPreTrainedModel.__annotate__s r%r^)image)rJrKrLrMbase_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_no_split_modules_supports_sdpa_supports_flash_attn_supports_flex_attn_supports_attention_backend_can_compile_fullgraphrr_can_record_outputs_input_embed_layer"_keys_to_ignore_on_load_unexpectedr1no_gradrr_rOrPrQrRs@@r"rrs$O!&*#$ N "&!  Y]^ nYaV[\ , 4E*&  ]]_ j j?r%rcaa]tRtRtoV3RlV3Rllt]]!RR7]R V3RlRll444tRt Vt V;t #) SwinEncoderic<<V^8dQhRS[RS[S[S[3,/#)rrrw)rr\r)r r!s"r"r#SwinEncoder.__annotate__s#zeCHor%c<\SV`V4\VP4VnWn\ \VP44Uu.uF@q1PV,\\VP4^, ^4, NKB pp\P!\ VP4Uu.uFp\T\VP^V,,4V^,^V,,V^,^V,,3VPV,VPV,V\VPRV4\VPRV^,4WPP^, 8d\ MRR7NK up4VnVP%4R#uupiuupi)r8N)rrrSrrr`r)r(r)lendepths num_layersrrrrTmaxrrrrr{rrlayers post_init)r*rrwrdpr layer_idxr+s&&& r"r)SwinEncoder.__init__sk  fmm, SXY\]c]j]jYkSlmSla$$q(3s6==/AA/Eq+IIISlmmm"'t!7 "8I!F,,q)|;<&/lq)|&DiPQlWXZcWcFd%e -- 2$..y9!#fmmJY&?"@3v}}UdW`cdWdGeCfg4=RS@S4S/Z^"8     n sAF;CGF)tie_last_hidden_statesc~<V^8dQhRS[PRS[S[S[3,RS[RS[RS[RS[S[,RS[/#)rr/rrcoutput_hidden_statesrrr)r1r2r\rrrrrU)r r!s"r"r#r sc+ + ||+  S/+  + # + 37 + +,+  + r%c RpV'dIVPwrp VP!V.VOV N5!P^^^^4P4p V 3pVPFcp V !VV3RVRV/VBwrp V'd W}3, pV P fK5V^,^,^,V^,^,^,3pKe \ VVR7#)a input_dimensions (`tuple[int, int]`): Spatial `(height, width)` of the patch grid entering the encoder. always_partition (`bool`, *optional*, defaults to `False`): If `True`, always apply window partitioning regardless of input resolution. output_hidden_states_before_downsampling (`bool`, *optional*, defaults to `False`): If `True`, `reshaped_hidden_states` contains pre-downsampling feature maps. Nrcr)rWrY)r:rrrrrrU)r*r/rrcrrrall_reshaped_hidden_statesrrr stem_spatialrreshaped_hidden_states&&&&&&, r"rASwinEncoder.forwards(&*" *7)<)< &J;"":N0@N+NVVWXZ[]^`abmmo +7 & KKL6B 7"27:b 7  7 3M!$*.FF*&&2%5a%81%<$BEUVWEX[\E\abDb#c (!+#=  r%)rrr)FFF) rJrKrLrMr)rrrrArOrPrQrRs@@r"rrsB* E2+ + 3 + + r%rc`aa]tRtRtoRV3Rllt]]RV3RlRll44tRtVt V;t #) SwinModeli8c <\SV`V4Wn\VP4Vn\ VP^VP ^, ,,4Vn\WR7Vn \WPP4Vn \P!VPVP R7VnV'd\P$!^4MRVnVP)4R#)z add_pooling_layer (`bool`, *optional*, defaults to `True`): Whether or not to apply pooling layer. use_mask_token (`bool`, *optional*, defaults to `False`): Whether or not to create and apply mask tokens in the embedding layer. )rrXN)r(r)rrrrrr{ num_featuresrrrrrxencoderrrr[ layernormAdaptiveAvgPool1dpoolerr)r*radd_pooling_layerrr+s&&&&r"r)SwinModel.__init__:s   fmm, 0 0119L3M MN(O"6??+E+EF d&7&7V=R=RS1Bb**1-  r%c <V^8dQhRS[PR,RS[PR,RS[RS[S[,RS[/#rrNrrrr)r1r[rrrrrb)r r!s"r"r#SwinModel.__annotate__QsY( ( ''$.( ))D0( #' ( +, (  ( r%c VPRVPP4pVPWVR7wrgVP!VV3RV/VBpVP p VP V 4p Rp VPe9VPV P^^44p \P!V ^4p \V V VPVPVPR7#)z bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`, *optional*): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). r)rrN)rWrdr/rXrY)poprrrrrWrrrr1rrbr/rXrY) r*rrrrrembedding_outputrencoder_outputssequence_output pooled_outputs &&&&, r"rASwinModel.forwardOs &zz*@$++BbBbc-1__ Tl.=. *,,   "6   *;;..9 ;; " KK(A(A!Q(GHM!MM-;M-')77&11#2#I#I   r%)rrrrrrr)TFNNF rJrKrLrMr)rrrArOrPrQrRs@@r"rr8s0*( ( ( ( r%rad Swin Model with a decoder on top for masked image modeling, as proposed in [SimMIM](https://huggingface.co/papers/2111.09886). Note that we provide a script to pre-train this model on custom data in our [examples directory](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-pretraining). c\aa]tRtRtoV3Rlt]]RV3RlRll44tRtVt V;t #)SwinForMaskedImageModelingi|c<\SV`V4\VRRR7Vn\ VP ^VP ^, ,,4p\P!\P!W!P^,VP,^R7\P!VP44Vn VP4R#)FT)rr) in_channels out_channelsrN)r(r)rrrr{rr Sequentialrencoder_strider PixelShuffledecoderr)r*rrr+s&& r"r)#SwinForMaskedImageModeling.__init__s  fdS 6++aF4E4E4I.JJK }} II(7L7La7ORXReRe7est  OOF11 2    r%c <V^8dQhRS[PR,RS[PR,RS[RS[S[,RS[/#r)r1r[rrrrrg)r r!s"r"r#'SwinForMaskedImageModeling.__annotate__s`H H ''$.H ))D0H #' H +, H ' H r%c VP!V3RVRV/VBpVPpVP^^4pVPwrxp \P !V R,4;rVP WxW4pVPV4p Rp VEeVPPVPP,pVP R W4pVPVPP^4PVPP^4P^4P4p\PP!WRR7pVV,P#4VP#4R,, VPP$, p \'V V VP(VP*VP,R7#) a bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). Examples: ```python >>> from transformers import AutoImageProcessor, SwinForMaskedImageModeling >>> import torch >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> image_processor = AutoImageProcessor.from_pretrained("microsoft/swin-base-simmim-window6-192") >>> model = SwinForMaskedImageModeling.from_pretrained("microsoft/swin-base-simmim-window6-192") >>> num_patches = (model.config.image_size // model.config.patch_size) ** 2 >>> pixel_values = image_processor(images=image, return_tensors="pt").pixel_values >>> # create random boolean mask of shape (batch_size, num_patches) >>> bool_masked_pos = torch.randint(low=0, high=2, size=(1, num_patches)).bool() >>> outputs = model(pixel_values, bool_masked_pos=bool_masked_pos) >>> loss, reconstructed_pixel_values = outputs.loss, outputs.reconstruction >>> list(reconstructed_pixel_values.shape) [1, 3, 192, 192] ```rrrNnone)rgh㈵>)rirjr/rXrYr)rrWrr:mathr=rrrrrrepeat_interleaverrrrl1_lossrrrgr/rXrY)r*rrrroutputsrrrsequence_lengthrrreconstructed_pixel_valuesmasked_im_lossrrreconstruction_losss&&&&, r"rA"SwinForMaskedImageModeling.forwardsL))  + &>   "33)33Aq94C4I4I1 /OS$899)11*FZ&*\\/%B"  &;;))T[[-C-CCD-55b$EO11$++2H2H!L""4;;#9#91=1  #%--"7"7 lr"7"s 1D8==?488:PTCTUX\XcXcXpXppN,5!//))#*#A#A   r%)rrrrrRs@@r"rr|s4 H H H H r%ra Swin Model transformer with an image classification head on top (a linear layer on top of the final hidden state of the [CLS] token) e.g. for ImageNet. Note that it's possible to fine-tune Swin on higher resolution images than the ones it has been trained on, by setting `interpolate_pos_encoding` to `True` in the forward of the model. This will interpolate the pre-trained position embeddings to the higher resolution. c\aa]tRtRtoV3Rlt]]RV3RlRll44tRtVt V;t #)SwinForImageClassificationicH<\SV`V4VPVn\V4VnVP^8d6\ P !VPPVP4M\ P!4Vn VP4R#)rN) r(r) num_labelsrrrrrr_ classifierr)r*rr+s&&r"r)#SwinForImageClassification.__init__sx   ++f% EKDUDUXYDYBIIdii,,f.?.? @_a_j_j_l  r%c <V^8dQhRS[PR,RS[PR,RS[RS[S[,RS[/#)rrNlabelsrrr)r1r[ LongTensorrrrrm)r r!s"r"r#'SwinForImageClassification.__annotate__sY! ! ''$.!   4'! #' ! +, ! # ! r%c VP!V3RV/VBpVPpVPV4pRpVeVP!W'VP3/VBp\ VVVP VPVPR7#)ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rN)riror/rXrY) rrdr loss_functionrrmr/rXrY) r*rrrrrrroris &&&&, r"rA"SwinForImageClassification.forwards))  %=    -- /  %%fdkkLVLD(!//))#*#A#A   r%)rr rrrrRs@@r"r r s0 ! ! ! ! r%r zM Swin backbone, to be used with frameworks like DETR and MaskFormer. ctaa]tRtRtoR.tV3RlV3Rllt]]]V3RlRl444t Rt Vt V;t #) SwinBackbonei)zswin.layernorm.*c <V^8dQhRS[/#rr)r r!s"r"r#SwinBackbone.__annotate__1s  z r%c <\SV`V4VP.\\ VP 44Uu.uF&p\ VP^V,,4NK( up,Vn\VRR7Vn /p\VPVP4FwrE\P!V4W4&K \P!V4VnVP#4R#uupi)rF)rN)r(r)r{rrrrrrrzip out_featuresrjrr ModuleDicthidden_states_normsr)r*rrrstagerr+s&& r"r)SwinBackbone.__init__1s  #--.X]^abhbobo^pXq1rXqST#f6F6FA6M2NXq1rrf> !#&t'8'8$--#H E)+l)C  &$I#%==1D#E  2ss,C9cP<V^8dQhRS[PRS[S[,RS[/#)rrrr)r1r2rrr )r r!s"r"r#rCs37 7 ll7 +,7   7 r%c .VP!V3RRRR/VBpRp\VPVP4FwrVWPP9gKVP wrxrVP ^^^^4P4pVPWyV ,V4pVPV,!V4pVPWyW4pVP ^^^^4P4pWF3, pK \VVPVPR7#)a Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> processor = AutoImageProcessor.from_pretrained("shi-labs/nat-mini-in1k-224") >>> model = AutoBackbone.from_pretrained( ... "microsoft/swin-tiny-patch4-window7-224", out_features=["stage1", "stage2", "stage3", "stage4"] ... ) >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 7, 7] ``` rcTr) feature_mapsr/rXr^) rr stage_namesrYrr:rrrrr rX) r*rrrr$r  hidden_staterrrrs &&, r"rASwinBackbone.forward@sH))  ! 6:    #&t'7'79W9W#X E))):F:L:L7 &+33Aq!Q?JJL +00e^\Z #77>|L +00UY +33Aq!Q?JJL / $Y%!88))  r%)rrr) rJrKrLrM_keys_to_ignore_on_load_missingr)rr rrArOrPrQrRs@@r"rr)sC (;&;#   7 !7 7 r%r)Nr4)r rrrr)>collections.abcrrr dataclassesrr1rrr activationsrbackbone_utilsrr modeling_layersr modeling_outputsr modeling_utilsr r processing_utilsrutilsrrrr utils.genericrrutils.output_capturingrrconfiguration_swinrrrrUrbrgrmrrrtrrrrr=rMrOrQrrrrrr r__all__r^r%r"r7s* $! &!H9.F&OOIE*%299%0   H H  H   HkH  H&  HKH  H*  H H  H*M-RYYM-`'-"))'-T%ryy%P)Qryy)QX%8C)BIIC)L bii  v*vrMH*MH`(j/(j(jVD %D N@ #@ @ F  [ !4[  [ |  2 !42  2 j L ="5L  L ^ r%