+ jP^RIt^RIHt^RIHtHtHt^RIt^RIH uH t ^RIH t H t ^RI HtHt^RIHt^RIHtHt^RIHt^R IHtHtHt^R IHt^R IHtHt.RCOt RRlt!]PDPGR4RRlt$]PDPGR4!RR] PJ4t&!RR] PJ4t'RDRRllt(]PDPGR 4!R!R"] PJ4t)!R#R$])4t*!R%R&] PJ4t+!R'R(]+4t,!R)R ] PJ4t-R*R+lt.R,]/t/!R-R ]4t0!R.R]4t1!R/R]4t2!R0R]4t3!R1R]4t4!R2R]4t5]!4]!R3]0Pl3R47R5RR6R/R7R8ll44t7]!4]!R3]1Pl3R47R5RR6R/R9R:ll44t8]!4]!R3]2Pl3R47R5RR6R/R;R<ll44t9]!4]!R3]3Pl3R47R5RR6R/R=R>ll44t:]!4]!R3]4Pl3R47R5RR6R/R?R@ll44t;]!4]!R3]5Pl3R47R5RR6R/RARBll44t"""',,"IN"dhildm" \\"r&cV^,V^,,pW,pVPW3R4pVP^^^4P4P^4pV#)rr.)viewpermute contiguous unsqueeze)r<r=r>Nrelative_position_biass&&& r$_get_relative_position_biasrH1sb AQ'A9R388rB3;;Aq!DOOQ[[\]^ !!r&rHcnaa]tRt^>toRt]P 3V3RlV3RllltV3RlRltRt Vt V;t #) PatchMergingzPatch Merging Layer. Args: dim (int): Number of input channels. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. cN<V^8dQhRS[RS[RS[P3,/#rdim norm_layer.r@rrModule)r# __classdict__s"r$r%PatchMerging.__annotate__E(((C(Xc299n-E(r&c<\SV`4\V4Wn\P !^V,^V,RR7VnV!^V,4VnR#FbiasNsuper__init__rrMrLinear reductionnormselfrMrN __class__s&&&r$r[PatchMerging.__init__EG D!1s7AG%@q3w' r&c <V^8dQhRS[/#rrr)r#rQs"r$r%rRL   r&c `\V4pVPV4pVPV4pV#z Args: x (Tensor): input tensor with expected layout of [..., H, W, C] Returns: Tensor with layout of [..., H/2, W/2, 2*C] )r:r^r]r`rs&&r$forwardPatchMerging.forwardLs. q ! IIaL NN1 r&rMr^r] __name__ __module__ __qualname____firstlineno____doc__r LayerNormr[rk__static_attributes____classdictcell__ __classcell__rarQs@@r$rJrJ>. IK ((   r&rJcnaa]tRt^YtoRt]P 3V3RlV3RllltV3RlRltRt Vt V;t #)PatchMergingV2zPatch Merging Layer for Swin Transformer V2. Args: dim (int): Number of input channels. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. cN<V^8dQhRS[RS[RS[P3,/#rLrO)r#rQs"r$r%PatchMergingV2.__annotate__`rSr&c<\SV`4\V4Wn\P !^V,^V,RR7VnV!^V,4VnR#rUrYr_s&&&r$r[PatchMergingV2.__init__`rcr&c <V^8dQhRS[/#rerf)r#rQs"r$r%r}grgr&c `\V4pVPV4pVPV4pV#ri)r:r]r^rjs&&r$rkPatchMergingV2.forwardgs. q ! NN1  IIaLr&rmrnrxs@@r$r{r{Yryr&r{TcBV^8dQhR\R\R\R\R\\,R\R\\,R\R \R \\,R \\,R \\ P,R \ R\/#)rinput qkv_weight proj_weightrGr> num_heads shift_sizeattention_dropoutdropoutqkv_bias proj_bias logit_scaletrainingr )rr?r@floatrr"bool)r#s"r$r%r%tsp p p p p # p c p  p S p p p vp p %,,'p p  p r&c  VPwrppV^,W^,,, V^,,pV^,W^,,, V^,,p\P!V^^^V^V34pVPwppppVP4pV^,V8d^V^&V^,V8d^V^&\ V4^8d,\ P !VV^,)V^,)3R R7pVV^,,VV^,,,pVPV VV^,,V^,VV^,,V^,V4pVP^^^^^^4PV V,V^,V^,,V4pV eFV eBV P4p V P4^,pV V^V,P4\P!VW4pVPVP^4VP^4^VVV,4P^^^^^4pV^,V^,V^,pppV e\P!VR R7\P!VR R7P!R R 4,p\ P"!V \$P&!R4R7P)4p VV ,pM8VVV,R ,,pVP+VP!R R 44pVV,p\ V4^8Ed(VP-VV34p^V^,)3V^,)V^,)3V^,)R33p^V^,)3V^,)V^,)3V^,)R33p ^p!VF<p"V F3p#V!VV"^,V"^,1V#^,V#^,13&V!^, p!K5 K> VPVV^,,V^,VV^,,V^,4pVP^^^^4PVV^,V^,,4pVP/^4VP/^4, pVP1V^8g\3R 44P1V^8H\3R44pVPVP^4V,VVVP^4VP^44pVVP/^4P/^4,pVPR VVP^4VP^44p\P4!VR R7p\P6!VW|R7pVP+V4P!^^4PVP^4VP^4V4p\P!VW*4p\P6!VWR7pVPV VV^,,VV^,,V^,V^,V4pVP^^^^^^4PV VVV4p\ V4^8d*\ P !VV^,V^,3R R7pVRRV1RV1R3,P94pV#)a Window based multi-head self attention (W-MSA) module with relative position bias. It supports both of shifted and non-shifted window. Args: input (Tensor[N, H, W, C]): The input tensor or 4-dimensions. qkv_weight (Tensor[in_dim, out_dim]): The weight tensor of query, key, value. proj_weight (Tensor[out_dim, out_dim]): The weight tensor of projection. relative_position_bias (Tensor): The learned relative position bias added to attention. window_size (List[int]): Window size. num_heads (int): Number of attention heads. shift_size (List[int]): Shift size for shifted window attention. attention_dropout (float): Dropout ratio of attention weight. Default: 0.0. dropout (float): Dropout ratio of output. Default: 0.0. qkv_bias (Tensor[out_dim], optional): The bias tensor of query, key, value. Default: None. proj_bias (Tensor[out_dim], optional): The bias tensor of projection. Default: None. logit_scale (Tensor[out_dim], optional): Logit scale of cosine attention for Swin Transformer V2. Default: None. training (bool, optional): Training flag used by the dropout parameters. Default: True. Returns: Tensor[N, H, W, C]: The output tensor after shifted window attention. )shiftsdimsN)rMgY@)max)prr+)r-rr.ggY)r/r0r1copysumr"rollrBrCreshapeclonenumelzero_linearsize normalize transposeclampmathlogexpmatmul new_zerosrE masked_fillrsoftmaxrrD)$rrrrGr>rrrrrrrrBr3r4Cpad_rpad_brr5pad_Hpad_W num_windowslengthqkvqkvattn attn_maskh_slicesw_slicescounthws$&&&&&&&&&&&&& r$shifted_window_attentionrtsFJA!Q ^aa.0 0KN BE ^aa.0 0KN BE eaAua/0AAueQ"J1~ 1 1~ 1  : JJq:a=.:a=.!A OKN*u A/FGK q%;q>);q>5KPQN;RT_`aTbdefA !Q1a#++A O[^kZ[n=\^_`A8#7>>#!Q&!f*%++- ((1j +C ++affQiAy!y. I Q QRSUVXY[\^_ `C!fc!fc!f!qA{{1"% A2(>(H(HR(PPkk+488E?CGGI k! iD( (xx B+, ( (D :KK/ Q(KN?Z]N*KzZ[}n^bMcdQ(KN?Z]N*KzZ[}n^bMcdA6; !A$1+qtad{23 NN5KN#:KNEU`abUcLcepqrest %%aAq199+{ST~XcdeXfGfg ''*Y-@-@-CC )))q.%-HTTU^bcUcejkneop yyk1; 166RS9VWV\V\]^V_`i))!,66q99yyYq 166!9= 99Tr "D 99T. BD A  A&..qvvay!&&)QGA K+A !w2A q%;q>)5KN+BKPQNT_`aTbdefA !Q1a#++AueQ?A : JJq*Q-A!?f M !RaR!Q,""$A Hr&rcvaa]tRt^toRtR V3RlV3RllltRtRtV3RlRltV3RlR lt R t Vt V;t #) ShiftedWindowAttentionz' See :func:`shifted_window_attention`. cj<V^8dQhRS[RS[S[,RS[S[,RS[RS[RS[RS[RS[/# rrMr>rrrrrrr@r?rr)r#rQs"r$r%#ShiftedWindowAttention.__annotate__sc.. .#Y.I .  .  ..!..r&c x<\S V`4\V4^8wg\V4^8wd \R4hW nW0nW@nWpnWn\P!W^,VR7Vn \P!WVR7Vn VP4VP4R#)rz.window_size and shift_size must be of length 2rWN)rZr[len ValueErrorr>rrrrrr\rproj#define_relative_position_bias_tabledefine_relative_position_index) r`rMr>rrrrrrras &&&&&&&&&r$r[ShiftedWindowAttention.__init__s  { q C Oq$8MN N&$"!2 99S'9IIcY7  002 ++-r&cZ\P!\P!^VP^,,^, ^VP^,,^, ,VP 44Vn\PPVP RR7R#)r{Gz?stdN) r Parameterr"zerosr>rr<init trunc_normal_r`s&r$r:ShiftedWindowAttention.define_relative_position_bias_table sw,.LL KKT--a0014T=M=Ma=P9PST9TUW[WeWe f- ) d??TJr&c\P!VP^,4p\P!VP^,4p\P!\P!WRR74p\P !V^4pVR,VR,, pVP ^^^4P4pVR;;,VP^,^, , uu&VR;;,VP^,^, , uu&VR;;,^VP^,,^, ,uu&VPR 4P 4pVPRV4R#) rijindexingNr=)r+r+N)r+Nr+)r+r+r)r+r+r-r.) r"aranger>stackmeshgridflattenrCrDrregister_buffer)r`coords_hcoords_wcoordscoords_flattenrelative_coordsr=s& r$r5ShiftedWindowAttention.define_relative_position_indexs<< 0 0 34<< 0 0 34U^^HNOvq1(4~j7QQ)11!Q:EEG D$4$4Q$7!$;;  D$4$4Q$7!$;;  A(8(8(;$;a$?? "1"5"5b"9"A"A"C 68OPr&c4<V^8dQhRS[P/#rr r!)r#rQs"r$r%rs  ELL r&cX\VPVPVP4#N)rHr<r=r>rs&r$get_relative_position_bias1ShiftedWindowAttention.get_relative_position_biass(*  - -t/K/KTM]M]  r&c&<V^8dQhRS[RS[/#rrf)r#rQs"r$r%r#s   F r&c hVP4p\VVPPVPPVVP VP VPVPVPVPPVPPVPR7 #){ Args: x (Tensor): Tensor with layout of [B, H, W, C] Returns: Tensor with same layout as input, i.e. [B, H, W, C] )rrrrrr) rrrweightrr>rrrrrXrr`rrGs&& r$rkShiftedWindowAttention.forward#s"&!@!@!B' HHOO II   "    NN"44LLXX]]iinn]]  r&)rrrrrr<rr>TTrr) rorprqrrrsr[rrrrkrurvrwrxs@@r$rrs9..4K Q     r&rcpaa]tRtRtoRtR V3RlV3RllltRtV3RlRltV3RlR ltR t Vt V;t #) ShiftedWindowAttentionV2i;z* See :func:`shifted_window_attention_v2`. cj<V^8dQhRS[RS[S[,RS[S[,RS[RS[RS[RS[RS[/#rr)r#rQs"r$r%%ShiftedWindowAttentionV2.__annotate__@sc<< <#Y<I <  <  <<!<rrrrrrrras &&&&&&&&& r$r[!ShiftedWindowAttentionV2.__init__@s     /  << "uzz9aQRBS7T2T(UV}} IIa4 ("''$*?3PY`eAf   XX]]((*a/F HHMM&1v: . 3 3 9 9 ; r&c\P!VP^,^, )VP^,\PR7p\P!VP^,^, )VP^,\PR7p\P!\P !W.RR74pVP ^^^4P4P^4pVR;;,VP^,^, ,uu&VR ;;,VP^,^, ,uu&V^,p\P!V4\P!\P!V4R,4,R, pVPRV4R#) r)dtyperrg?g@relative_coords_tableN)r+r+r+r)r+r+r+r-) r"rr>float32rrrCrDrEsignlog2absr)r`relative_coords_hrelative_coords_wrs& r$r+B)CTEUEUVWEX`e`m`mn!LL4+;+;A+>+B)CTEUEUVWEX`e`m`mn % ENNr"sigmoid)r`rGs& r$r3ShiftedWindowAttentionV2.get_relative_position_biasos_!< LL33 4 9 9"dnn M  ( (   "  "$emm4J&K!K%%r&c <V^8dQhRS[/#rerf)r#rQs"r$r%rxs   r&c ~VP4p\VVPPVPPVVP VP VPVPVPVPPVPPVPVPR7 #)r)rrrrrrr) rrrrrr>rrrrrXrrrs&& r$rk ShiftedWindowAttentionV2.forwardxs"&!@!@!B' HHOO II   "    NN"44LLXX]]iinn((]]  r&)rrr) rorprqrrrsr[rrrkrurvrwrxs@@r$rr;s3<<>M &&   r&rcxaa]tRtRtoRtRRRR]P ]3V3RlV3RllltV3RlRlt R t Vt V;t #) SwinTransformerBlockia Swin Transformer Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_size (List[int]): Window size. shift_size (List[int]): Shift size for shifted window attention. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob: (float): Stochastic depth rate. Default: 0.0. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. attn_layer (nn.Module): Attention layer. Default: ShiftedWindowAttention @rc<V^8dQhRS[RS[RS[S[,RS[S[,RS[RS[RS[RS[R S[R S[P 3,R S[R S[P 3,/ # rrMrr>r mlp_ratiorrstochastic_depth_probrN. attn_layerr@r?rrrrP)r#rQs"r$r%!SwinTransformerBlock.__annotate__s!6!6 !6!6#Y !6 I !6  !6!6!!6 %!6S"))^,!6S"))^,!6r&c l<\S V`4\V4V !V4VnV !VVVVVVR7Vn\ VR4VnV !V4Vn\V\W,4V.\PRVR7Vn VPP4Fp \V \P4'gK%\P P#V P$4V P&fK^\P P)V P&RR7K R#))rrrowN)activation_layerrrgư>r)rZr[rnorm1rr stochastic_depthnorm2r r@rGELUmlpmodules isinstancer\rxavier_uniform_rrXnormal_) r`rMrr>rrrrrrNrmras &&&&&&&&&&& r$r[SwinTransformerBlock.__init__s D!_     /   !00Eu M_ sS137"''[_ipq!!#A!RYY''''166%GGOOAFFO5 $r&c <V^8dQhRS[/#rerf)r#rQs"r$r%rsr&cWPVPVPV444,pWPVPVP V444,pV#r)rrrr rrjs&&r$rkSwinTransformerBlock.forwardsO %%dii 1 &>? ? %%dhhtzz!}&=> >r&)rr rrr) rorprqrrrsrrtrr[rkrurvrwrxs@@r$rrs< *#&'*/1||/E!6!6Fr&rcxaa]tRtRtoRtRRRR]P ]3V3RlV3RllltV3RlRlt R t Vt V;t #) SwinTransformerBlockV2ia Swin Transformer V2 Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_size (List[int]): Window size. shift_size (List[int]): Shift size for shifted window attention. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob: (float): Stochastic depth rate. Default: 0.0. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. attn_layer (nn.Module): Attention layer. Default: ShiftedWindowAttentionV2. rrc<V^8dQhRS[RS[RS[S[,RS[S[,RS[RS[RS[RS[R S[R S[P 3,R S[R S[P 3,/ #rr)r#rQs"r$r%#SwinTransformerBlockV2.__annotate__s    #Y  I     !  % S"))^, S"))^, r&c :<\S V`VVVVVVVVV V R7 R#))rrrrrNrN)rZr[) r`rMrr>rrrrrrNrras &&&&&&&&&&&r$r[SwinTransformerBlockV2.__init__s5     /"7!!  r&c <V^8dQhRS[/#rerf)r#rQs"r$r%r-sr&cWPVPVPV444,pWPVPVP V444,pV#r)rrrrr rjs&&r$rkSwinTransformerBlockV2.forwardsQ %%djj1&>? ? %%djj!&=> >r&) rorprqrrrsrrtrr[rkrurvrwrxs@@r$r+r+s; *#&'*/1||/G  4r&r+c\aa]tRtRtoRtRRRRRRR]3V3RlV3R llltR tR tVt V;t #) ria Implements Swin Transformer from the `"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" `_ paper. Args: patch_size (List[int]): Patch size. embed_dim (int): Patch embedding dimension. depths (List(int)): Depth of each Swin Transformer layer. num_heads (List(int)): Number of attention heads in different layers. window_size (List[int]): Window size. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob (float): Stochastic depth rate. Default: 0.1. num_classes (int): Number of classes for classification head. Default: 1000. block (nn.Module, optional): SwinTransformer Block. Default: None. norm_layer (nn.Module, optional): Normalization layer. Default: None. downsample_layer (nn.Module): Downsample layer (patch merging). Default: PatchMerging. rrg?iNc@<V^8dQhRS[S[,RS[RS[S[,RS[S[,RS[S[,RS[RS[RS[R S[R S[R S[S[R S[P 3,,R S[S[R S[P 3,,RS[R S[P 3,/ #)r patch_size embed_dimdepthsrr>rrrr num_classesrN.blockdownsample_layer)r?r@rrrrrP)r#rQs"r$r%SwinTransformer.__annotate__sM+M+IM+M+S M+ 9 M+ #Y M+M+M+!M+ %M+M+Xc299n56M+bii01M+#3 >2M+r&cN<\SV`4\V4WnV f\p V f\ \ PRR7p .pVP\ P!\ P!^W!^,V^,3V^,V^,3R7\.RO4V !V444\V4p^p\\V44Fp.pV^V,,p\VV,4FzpV \V4,V^, , pTPT !TVV,TVUu.uFpV^,^8Xd^MV^,NK upVVVVV R7 4V^, pK| VP\ P!V!4V\V4^, 8gKVPV !VV 44K \ P!V!VnV^\V4^, ,,pV !V4Vn\.RO4Vn\ P&!^4Vn\ P*!^4Vn\ P.!VV 4VnVP34Fp\5V\ P.4'gK%\ P6P9VP:RR7VP<fK`\ P6P?VP<4K R#uupi) Ngh㈵>)eps) kernel_sizestride)r>rrrrrrNrr)rrr(r-)rr(r-r) rZr[rr9rrrrtappendrConv2dr rrangerrfeaturesr^rCAdaptiveAvgPool2davgpoolFlattenrr\headr!r"rrrrXzeros_)r`r6r7r8rr>rrrrr9rNr:r;layerstotal_stage_blocksstage_block_idi_stagestagerMi_layersd_probr num_featuresr%ras&&&&&&&&&&&&&& r$r[SwinTransformer.__init__sw D!& =(E   48J"$ MM y!}jm.LV`abVceopqerUs %9%   ![S[)G%'Eaj(C 1/%2GGK]`aKab !'*$/OZ#[{!1)9AqAv$E{#["+ '*;.5#-  !#!2" MM"--/ 0#f+/* .sJ?@/*0 v.  1Vq#99 |, |, ++A. zz!} IIlK8 A!RYY''%%ahhD%966%GGNN166*  -$\s"L"cVPV4pVPV4pVPV4pVPV4pVP V4pVP V4pV#r)rDr^rCrFrrHrjs&&r$rkSwinTransformer.forward_sV MM!  IIaL LLO LLO LLO IIaLr&)rFrDrrHr^r9rC) rorprqrrrsrJr[rkrurvrwrxs@@r$rrs;4#&'*9=485AM+M+^r&cV^8dQhR\\,R\R\\,R\\,R\\,R\R\\,R\ R \ R \/ #) rr6r7r8rr>rweightsprogresskwargsr )r?r@rrrrrr)r#s"r$r%r%isS  ICy  c  ! k "r&c Ve(\VR\VPR,44\R RVRVRVRVRVRV/VBp Ve#V P VP VR R 74V #) Nr9 categoriesr6r7r8rr>rT)rW check_hashr3)rrmetarload_state_dictget_state_dict) r6r7r8rr>rrVrWrXmodels &&&&&&&&, r$_swin_transformerr`isfmSl9S5TU           4    E g44hSW4XY Lr&rZc]tRtRt]!R]!]^^]PR7/] CRRRRRRR R R R R R//RRRRRR/CR7t ] t Rt R#)riz7https://download.pytorch.org/models/swin_t-704ceda3.pth crop_size resize_size interpolation num_paramsibmin_sizerecipeUhttps://github.com/pytorch/vision/tree/main/references/classification#swintransformer_metrics ImageNet-1Kacc@1guV^T@acc@5glW@_opsgX9@ _file_sizeg\( [@_docsYThese weights reproduce closely the results of the paper using a similar training recipe.url transformsr\r3Nrv rorprqrrrrr r BICUBIC _COMMON_META IMAGENET1K_V1DEFAULTrur3r&r$rrs| E 3CO`OhOh   (  m VV  E & t M*Gr&c]tRtRt]!R]!]^^]PR7/] CRRRRRRR R R R R R//RRRRRR/CR7t ] t Rt R#)riz7https://download.pytorch.org/models/swin_s-5e29d889.pthrbrfirrgrhrirjrkrlgCT@rmgףp= X@rngZd{!@rogx&g@rprqrrr3Nrurwr3r&r$rrs| E 3CO`OhOh   (  m VV  E ' t M*Gr&c]tRtRt]!R]!]^^]PR7/] CRRRRRRR R R R R R//RRRRRR/CR7t ] t Rt R#)riz7https://download.pytorch.org/models/swin_b-68c6b09e.pthrbrfi<;rgrhrirjrkrlgh|?T@rmg)\(X@rng&1.@rogt@rprqrrr3Nrurwr3r&r$rrs| E 3CO`OhOh   (  m VV  F ' t M*Gr&c]tRtRt]!R]!]RR]PR7/] CRRRRR R R R R RRR//RRRRRR/CR7t ] t Rt R#)riz:https://download.pytorch.org/models/swin_v2_t-b137f0e2.pthrbrfiRrgrhXhttps://github.com/pytorch/vision/tree/main/references/classification#swintransformer-v2rjrkrlgS㥛T@rmg rX@rng(\@rogMb([@rprqrrr3Nrrrwr3r&r$rrs| H 3CO`OhOh   (  p VV  D ' t M*Gr&c]tRtRt]!R]!]RR]PR7/] CRRRRR R R R R RRR//RRRRRR/CR7t ] t Rt R#)riz:https://download.pytorch.org/models/swin_v2_s-637d8ceb.pthrrrbrfirgrhrrjrkrlg!rhT@rmgNbX94X@rngd;O'@rogg@rprqrrr3Nrrwr3r&r$rr| H 3CO`OhOh   (  p VV  F ' t M*Gr&c]tRtRt]!R]!]RR]PR7/] CRRRRR R R R R RRR//RRRRRR/CR7t ] t Rt R#)ri z:https://download.pytorch.org/models/swin_v2_b-781e5279.pthrirbrfi=rgrhrrjrkrlgI +U@rmgK7X@rng33333S4@rogˡEu@rprqrrr3Nrrwr3r&r$rr rr& pretrained)rVrVrWc^V^8dQhR\\,R\R\R\/#rrVrWrXr )rrrrr)r#s"r$r%r%$/!!x/!$!Y\!ap!r&c r\PV4p\R R^^.R^`R.R OR.R OR^^.RRRVR V/VB#) a Constructs a swin_tiny architecture from `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows `_. Args: weights (:class:`~torchvision.models.Swin_T_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_T_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_T_Weights :members: r6r7r8rr>r皙?rVrWr3rrrr(r )rverifyr`rVrWrXs$$,r$swin_tr"s.##G,G  q6   !  F  "      r&c^V^8dQhR\\,R\R\R\/#r)rrrrr)r#s"r$r%r%Jrr&c r\PV4p\R R^^.R^`R.R OR.R OR^^.RRRVR V/VB#) a Constructs a swin_small architecture from `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows `_. Args: weights (:class:`~torchvision.models.Swin_S_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_S_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_S_Weights :members: r6r7r8rr>r333333?rVrWr3rrrr)rrr`rs$$,r$swin_srHs.##G,G  q6   !  F  "      r&c^V^8dQhR\\,R\R\R\/#r)rrrrr)r#s"r$r%r%prr&c r\PV4p\R R^^.R^R.R OR.R OR^^.RRRVR V/VB#) a Constructs a swin_base architecture from `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows `_. Args: weights (:class:`~torchvision.models.Swin_B_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_B_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_B_Weights :members: r6r7r8rr>r?rVrWr3rrV )rrr`rs$$,r$swin_brns.##G,G  q6   !  F  "      r&c^V^8dQhR\\,R\R\R\/#r)rrrrr)r#s"r$r%r%0##(#45##_b#gv#r&c \PV4p\R R^^.R^`R.R OR.ROR^^.RRRVR VR \R \/ VB#)a| Constructs a swin_v2_tiny architecture from `Swin Transformer V2: Scaling Up Capacity and Resolution `_. Args: weights (:class:`~torchvision.models.Swin_V2_T_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_V2_T_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_V2_T_Weights :members: r6r7r8rr>rrrVrWr:r;r3rr)rrr`r+r{rs$$,r$ swin_v2_trs. &&w/G  q6   !  F  "   % (    r&c^V^8dQhR\\,R\R\R\/#r)rrrrr)r#s"r$r%r%rr&c \PV4p\R R^^.R^`R.R OR.ROR^^.RRRVR VR \R \/ VB#)a} Constructs a swin_v2_small architecture from `Swin Transformer V2: Scaling Up Capacity and Resolution `_. Args: weights (:class:`~torchvision.models.Swin_V2_S_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_V2_S_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_V2_S_Weights :members: r6r7r8rr>rrrVrWr:r;r3rr)rrr`r+r{rs$$,r$ swin_v2_srs. &&w/G  q6   !  F  "   % (    r&c^V^8dQhR\\,R\R\R\/#r)rrrrr)r#s"r$r%r%rr&c \PV4p\R R^^.R^R.R OR.ROR^^.RRRVR VR \R \/ VB#)a| Constructs a swin_v2_base architecture from `Swin Transformer V2: Scaling Up Capacity and Resolution `_. Args: weights (:class:`~torchvision.models.Swin_V2_B_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.Swin_V2_B_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.Swin_V2_B_Weights :members: r6r7r8rr>rrrVrWr:r;r3rr)rrr`r+r{rs$$,r$ swin_v2_brs. &&w/G  q6   !  F  "   % (    r&) rrrrrrrrrrrrr)rrNNNT)=r functoolsrtypingrrrr"torch.nn.functionalr functionalr0rops.miscr r ops.stochastic_depthr transforms._presetsr r utilsr_apirrr_metar_utilsrr__all__r:fxwraprHrPrJr{rrrrr+rr`ryrrrrrrrzrrrrrrr3r&r$rs ** #2H'66'B "  "#" +,2996RYY6p f ()N RYYN bS 5S l62996r/1/djbiijZ>& [2[2[2 2 2 2,0L0L!MN!$!!O!H,0L0L!MN!$!!O!H,0L0L!MN!$!!O!H,0A0O0O!PQ##t#R#L,0A0O0O!PQ##t#R#L,0A0O0O!PQ##t#R#r&