+ j)Rt^RIt^RIHt^RIt^RIHt^RIHt^RI H t ^RI H t ^RI Ht^RIHt^R IHtHtHt^R IHt]P.!]4t]!R R 7]!R R]444t!RR]P64t!RR]P64t!RR]P64t!RR]P64t!RR]P64t !RR]P64t!!RR]P64t"R=Rlt#!RR]P64t$!R R!]P64t%!R"R#]P64t&]PNPPR>R$R%ll4t)!R&R']P64t*!R(R)]P64t+!R*R+]P64t,!R,R-]P64t-!R.R/]P64t.!R0R1]P64t/!R2R3]P64t0!R4R5]P64t1!R6R7]P64t2]!R8R9]44t3]!R:R 7!R;R<]344t4R#$e'',-4;%) __name__ __module__ __qualname____firstlineno____doc__rrrrr__annotate_func____static_attributes____classdictcell__)rs@rrr"s,&*D04O.2M:>M7;J%r rcHaa]tRt^7toRtV3RltV3RlRltRtVtV;t #)ZoeDepthReassembleStagea This class reassembles the hidden states of the backbone into image-like feature representations at various resolutions. This happens in 3 stages: 1. Map the N + 1 tokens to a set of N tokens, by taking into account the readout ([CLS]) token according to `config.readout_type`. 2. Project the channel dimension of the hidden states according to `config.neck_hidden_sizes`. 3. Resizing the spatial dimensions (height, width). Args: config (`[ZoeDepthConfig]`): Model configuration class defining the model architecture. c r<\SV`4VPVn\P!4Vn\ VPVP4F+wr#VP P\WVR74K- VPR8Xd\P!4Vn VPpVPFdpVPP\P!\P!^V,V4\VP ,44Kf R#R#))channelsfactorprojectN)super__init__ readout_typer ModuleListlayerszipneck_hidden_sizesreassemble_factorsappendZoeDepthReassembleLayerreadout_projectsbackbone_hidden_size SequentialLinearr hidden_act)selfconfigneck_hidden_sizer. hidden_size_ __class__s&& rr1 ZoeDepthReassembleStage.__init__Gs "//mmo (+F,D,DfF_F_(` $  KK  6vagh i)a   ) +$&MMOD ! 55K--%%,,MM"))A O["I6RXRcRcKde. ,r cn<V^8dQhRS[S[P,RS[S[P,/#rrreturnlistrTensor)rrs"rr$ZoeDepthReassembleStage.__annotate__Xs1&&T%,,%7&W[\a\h\hWi&r c vV^,P^,p\P!V^R7pVR,VR,rVPwrgpVPWbW84pVP ^^^^4P 4pVP R8Xd[VP^4P R4pVP^R7PV4p \P!W3R4pM(VP R8XdWPR4,p.p \VPV^R74F}wrVP R8XdVPV ,!V 4p V P ^^^4PVRW#4p VPV ,!V 4p V PV 4K V #)z Args: hidden_states (`list[torch.FloatTensor]`, each of shape `(batch_size, sequence_length + 1, hidden_size)`): List of hidden states from the backbone. dimr/add)NNN)rQNN)rRrrT)shapercatreshapepermute contiguousr2flatten unsqueeze expand_as enumeratesplitr:r4r8) r?r patch_height patch_width batch_size cls_tokentotal_batch_sizesequence_length num_channelsreadoutout stage_idx hidden_states &&&& rforwardZoeDepthReassembleStage.forwardXs #1%++A.  -Q7 #0#6 e8L=:G:M:M7<%--.>kh %--aAq9DDF    ))11!4<CM   % '),?,?,CCM'01D1DZUV1D1W'X #I  I-#44Y? M (//1a8@@RQ]kL;;y1,?L JJ| $(Y r )r4r:r2 r"r#r$r%r&r1rkr(r) __classcell__rDrs@@rr+r+7s "&&&r r+c8aa]tRt^toV3RltRtRtVtV;t#)r9c z<\SV`4VPp\P!WB^R7VnV^8d!\P !W"W3^R7VnR#V^8Xd\P!4VnR#V^8d2\P!W"^\^V, 4^R7VnR#R#)rT) in_channels out_channels kernel_sizertstridepaddingN) r0r1r;rConv2d projectionConvTranspose2dresizeIdentityint)r?r@r-r.rBrDs&&&& rr1 ZoeDepthReassembleLayer.__init__s 11 )) `ab A:,,XVlmnDK q[++-DK aZ))HAcRSV\R\oghiDKr cJVPV4pVPV4pV#Nryr{r?rjs&&rrkZoeDepthReassembleLayer.forwards$|4 {{<0 r r r"r#r$r%r1rkr(r)rnros@@rr9r9s j r r9cDaa]tRt^toV3RlV3RlltRtRtVtV;t#)ZoeDepthFeatureFusionStagec <V^8dQhRS[/#rr@r )rrs"rr'ZoeDepthFeatureFusionStage.__annotate__sCC~Cr c<\SV`4\P!4Vn\ \ VP44F'pVPP\V44K) R#r) r0r1rr3r4rangelenr6r8ZoeDepthFeatureFusionLayer)r?r@rCrDs&& rr1#ZoeDepthFeatureFusionStage.__init__sM mmo s63345A KK  9&A B6r cVRRR1,p.pRp\WP4F+wrEVf V!V4pMV!W44pVPV4K- V#)NrU)r5r4r8)r?rfused_hidden_statesfused_hidden_staterjlayers&& rrk"ZoeDepthFeatureFusionStage.forwardsg%dd+  !#&}kk#B L!)%*<%8"%*+=%L"  & &'9 : $C#"r )r4rros@@rrrsCC ##r rcHaa]tRt^toRtV3RltV3RlRltRtVtV;t #)ZoeDepthPreActResidualLayerz ResidualConvUnit, pre-activate residual unit. Args: config (`[ZoeDepthConfig]`): Model configuration class defining the model architecture. c <\SV`4VPVnVPe VPMVP'*p\ P !4Vn\ P!VPVP^^^VR7Vn \ P !4Vn \ P!VPVP^^^VR7Vn VP'de\ P!VPVPR7Vn\ P!VPVPR7VnR#R#)N)rtrvrwbias)eps)r0r1!use_batch_norm_in_fusion_residualuse_batch_normuse_bias_in_fusion_residualrReLU activation1rxfusion_hidden_size convolution1 activation2 convolution2 BatchNorm2dbatch_norm_eps batch_norm1 batch_norm2)r?r@rrDs&& rr1$ZoeDepthPreActResidualLayer.__init__s $FF11=  . .((( $ 779II  % %  % %,  779II  % %  % %,      !~~f.G.GVMbMbcD !~~f.G.GVMbMbcD  r cN<V^8dQhRS[PRS[P/#rrjrHrrK)rrs"rr(ZoeDepthPreActResidualLayer.__annotate__s#''ELL'U\\'r c*TpVPV4pVPV4pVP'dVPV4pVP V4pVP V4pVP'dVP V4pW,#r)rrrrrrrr?rjresiduals&& rrk#ZoeDepthPreActResidualLayer.forwards'' 5 ((6    ++L9L'' 5 ((6    ++L9L&&r )rrrrrrrrmros@@rrrs  dD'''r rc\aa]tRt^toRtRV3RlV3RllltRV3RlRlltRtVtV;t #) ra Feature fusion layer, merges feature maps from different stages. Args: config (`[ZoeDepthConfig]`): Model configuration class defining the model architecture. align_corners (`bool`, *optional*, defaults to `True`): The align_corner setting for bilinear upsample. c&<V^8dQhRS[RS[/#)rr@ align_corners)r bool)rrs"rr'ZoeDepthFeatureFusionLayer.__annotate__sCC~CdCr c<\SV`4W n\P!VP VP ^RR7Vn\V4Vn\V4Vn R#)rTT)rtrN) r0r1rrrxrryrresidual_layer1residual_layer2)r?r@rrDs&&&rr1#ZoeDepthFeatureFusionLayer.__init__sR *))F$=$=v?X?Xfgnrs:6B:6Br cv<V^8dQhRS[PRS[PR,RS[P/#)rrjrNrHr)rrs"rrrs6ELLELL4*>x*HHL++L9 }}00 qzI[I[1 |4 r )rryrr)Trrmros@@rrrs%CCr rcTaa]tRtRtoRtV3RlV3RlltV3RlRltRtVtV;t #) ZoeDepthNeckia3 ZoeDepthNeck. A neck is a module that is normally used between the backbone and the head. It takes a list of tensors as input and produces another list of tensors as output. For ZoeDepth, it includes 2 stages: * ZoeDepthReassembleStage * ZoeDepthFeatureFusionStage. Args: config (dict): config dict. c <V^8dQhRS[/#rr )rrs"rrZoeDepthNeck.__annotate__#s??~?r c <\SV`4WnVPe$VPPR8Xd RVnM\ V4Vn\P!4Vn VPF@pVPP\P!W!P^^RR74KB \V4VnR#)Nswinv2F)rtrwr)r0r1r@backbone_config model_typereassemble_stager+rr3convsr6r8rxrr fusion_stage)r?r@channelrDs&& rr1ZoeDepthNeck.__init__#s    ! ! -&2H2H2S2SW_2_$(D !$;F$CD !]]_ //G JJ  bii1J1JXYcdkpq r07v>r cn<V^8dQhRS[S[P,RS[S[P,/#rGrI)rrs"rrr4s1$$T%,,%7$W[\a\h\hWi$r c \V\\34'g \R4h\ V4\ VP P 48wd \R4hVPeVPWV4p\V4UUu.uFwrEVPV,!V4NK pppVPV4pWvR,3#uuppi)z Args: hidden_states (`list[torch.FloatTensor]`, each of shape `(batch_size, sequence_length, hidden_size)` or `(batch_size, hidden_size, height, width)`): List of hidden states from the backbone. z2hidden_states should be a tuple or list of tensorszOThe number of hidden states should be equal to the number of neck hidden sizes.rU) isinstancerrJ TypeErrorrr@r6 ValueErrorrr^rr)r?rr`raifeaturefeaturesoutputs&&&& rrkZoeDepthNeck.forward4s -%77PQ Q } T[[%B%B!C Cno o  , 11-{[M=F}=UV=UzqDJJqM'*=UV""8,|## Ws$C)r@rrrrmros@@rrrs# ??"$$$r rcHaa]tRtRtoRtV3RltV3RlRltRtVtV;t #)#ZoeDepthRelativeDepthEstimationHeadiLa Relative depth estimation head consisting of 3 convolutional layers. It progressively halves the feature dimension and upsamples the predictions to the input resolution after the first convolutional layer (details can be found in DPT's paper's supplementary material). c<\SV`4VPVnRVnVP'd!\ P !RRRRRR7VnVPp\ P !W"^,^^^R7Vn\ P!^RRR7Vn \ P !V^,VP^^^R7Vn \ P !VP^^^^R7Vn R#)NrurTr)r)rTrT)r0r1 head_in_indexryadd_projectionrrxrconv1Upsampleupsamplenum_relative_featuresconv2conv3)r?r@rrDs&& rr1,ZoeDepthRelativeDepthEstimationHead.__init__Ss #11   iiSfV]cdDO,,YYxQAaYZ[  SWX YYx1}f.J.JXYbcmno YYv;;QAVWabc r c^<V^8dQhRS[S[P,RS[P/#rGrI)rrs"rr0ZoeDepthRelativeDepthEstimationHead.__annotate__bs())T%,,%7)ELL)r cWP,pVPe-VPV4p\P!4!V4pVP V4pVP V4pVP V4p\P!4!V4pTpVPV4p\P!4!V4pVP^R7pW23#)NrN) rryrrrrrrsqueeze)r?rrrs&& rrk+ZoeDepthRelativeDepthEstimationHead.forwardbs%&8&89 ?? & OOM:MGGIm4M =1  m4  =1  -0   =1  -0 '//A/6((r )rrrrryrrmros@@rrrLs d)))r rc W,pW,pV\P!V4,V\P!V4,, W, \P!W, V,4,, #)z%log(nCk) using stirling approximation)rlog)nkrs&&&r log_binomrxsP A A uyy| a%))A,. .!%599QUS[;Q1Q QQr cXaa]tRtRtoR]P 3V3RlltRRltRtVt V;t #)LogBinomialSoftmaxirc n<\SV`4WnW n\P !\ P!^V4P^R^^4RR7Vn \P !\ P!VP^, .4P^R^^4RR7Vn R#)aCompute log binomial distribution for n_classes Args: n_classes (`int`, *optional*, defaults to 256): Number of output classes. act (`torch.nn.Module`, *optional*, defaults to `torch.softmax`): Activation function to apply to the output. F) persistentNrU) r0r1ractrBufferrarangeviewk_idxtensor k_minus_1)r? n_classesrrDs&&&rr1LogBinomialSoftmax.__init__s YYu||Ay9>>q"aKX]^ 5<<! #=#B#B1b!Q#O\abr c VP^8XdVP^4p^V, PVRR7pVPVRR7p\VPVP 4VP \ P!V4,,VPVP , \ P!V4,,pVPWR, ^R7#)aXCompute the log binomial distribution for probabilities. Args: probabilities (`torch.Tensor` of shape `(batch_size, num_channels, height, width)`): Tensor containing probabilities of each class. temperature (`float` or `torch.Tensor` of shape `(batch_size, num_channels, height, width)`, *optional*, defaults to 1): Temperature of distribution. eps (`float`, *optional*, defaults to 1e-4): Small number for numerical stability. Returns: `torch.Tensor` of shape `(batch_size, num_channels, height, width)`: Log binomial distribution logbinomial(p;t). ?)minmaxrN) ndimr\clamprrrrrr)r? probabilities temperaturerone_minus_probabilitiesys&&&& rrkLogBinomialSoftmax.forwards    ")33A6M#$}#4";";";"M%+++= dnndjj 1jj599]33 4~~ *eii8O.PP Q xxQx//r )rrrr)r-C6?) r"r#r$r%rsoftmaxr1rkr(r)rnros@@rrrs!$%-- c00r rc<aa]tRtRtoRV3RlltRtRtVtV;t#)%ZoeDepthConditionalLogBinomialSoftmaxic  <\SV`4W#,V,p\P!\P!W#,V^^^R7\P !4\P!V^^^^R7\P !44VnRVnVPVn VPVn \V\PR7VnR#)aPer-pixel MLP followed by a Conditional Log Binomial softmax. Args: in_features (`int`): Number of input channels in the main feature. condition_dim (`int`): Number of input channels in the condition feature. n_classes (`int`, *optional*, defaults to 256): Number of classes. bottleneck_factor (`int`, *optional*, defaults to 2): Hidden dim factor. rur )rN)r0r1rr<rxGELUSoftplusmlpp_epsmax_tempmin_temprrr log_binomial_transform)r?r@ in_features condition_dimrbottleneck_factor bottleneckrDs&&&&&& rr1.ZoeDepthConditionalLogBinomialSoftmax.__init__s* !16GG == IIk1:1UV`a b GGI IIj%Qq! L KKM     &8 &V#r c VP\P!W3^R74pVR,VR,rTW@P,pVR,VR,VR,,, pWPP,pVR,VR,VR,,, pVP ^4pVP VP , V,VP ,pVPWE4#)a2 Args: main_feature (`torch.Tensor` of shape `(batch_size, num_channels, height, width)`): Main feature. condition_feature (torch.Tensor of shape `(batch_size, num_channels, height, width)`): Condition feature. Returns: `torch.Tensor`: Output log binomial distribution rN)rQ:NrN.)rQrNN.)rQrR.)rQrT.)rrconcatrr\rrr)r? main_featurecondition_featureprobabilities_and_temperaturerrs&&& rrk-ZoeDepthConditionalLogBinomialSoftmax.forwards)-|>_ef1g(h% )* 5 )* 5# & 2 %i0M)4L}]fOg4gh !JJ. !), I0FU^I_0_` !++A. }}t}}4 CdmmS **=FFr )rrrrr)rrrros@@rr r s#WJGGr r c<aa]tRtRtoRV3RlltRtRtVtV;t#)ZoeDepthSeedBinRegressoric <\SV`4VPVnVPVnW@nWPn\P!VPV^^^4Vn \P!RR7Vn \P!W2^^^4Vn VPR8Xd\P!RR7VnR#\P!4VnR#)aBin center regressor network. Can be "normed" or "unnormed". If "normed", bin centers are bounded on the (min_depth, max_depth) interval. Args: config (`int`): Model configuration. n_bins (`int`, *optional*, defaults to 16): Number of bin centers. mlp_dim (`int`, *optional*, defaults to 256): Hidden dimension. min_depth (`float`, *optional*, defaults to 1e-3): Min depth value. max_depth (`float`, *optional*, defaults to 10): Max depth value. TinplacenormedN)r0r1bottleneck_featuresrbin_centers_type min_depth max_depthrrxrract1rract2)r?r@n_binsmlp_dimr*r+rDs&&&&&&rr1!ZoeDepthSeedBinRegressor.__init__s" !55 & 7 7""YYt//!QB GGD) YYw1a8 -1-B-Bh-NBGGD) TVT_T_Ta r c VPV4pVPV4pVPV4pVPV4pVPR8XdVR,pW"P ^RR7, pVP VP, V,p\PPVR RVPR7p\P!V^R7pRVR R R 1R 3,VR,,,pW23#W"3#)zM Returns tensor of bin_width vectors (centers). One vector b for every pixel r'MbP?TrOkeepdimconstant)rvaluerNg?rQN.)rRrRrRrRrTrRrU)rQrS.) rr,rr-r)sumr+r*rrpadrcumsum)r?x bin_centersbin_widths_normed bin_widths bin_edgess&& rrk ZoeDepthSeedBinRegressor.forward s JJqM IIaL JJqMiil  H ,%,K +oo!To.R R ..4>>9=NNJ**:7IPZbfbpbp*qJ ZQ7I1crc3;!7)J:O!OPK$1 1+ +r )r,r-r)rrrr+r*)rr2 rros@@rr#r#sb:,,r r#c0V^8dQhR\R\/#)ralphagamma)floatr})rs"rrr's--U--r c\VP^WPV4,,4#)a Inverse attractor: dc = dx / (1 + alpha*dx^gamma), where dx = a - c, a = attractor point, c = bin center, dc = shift in bin center This is the default one according to the accompanying paper. Args: dx (`torch.Tensor`): The difference tensor dx = Ai - Cj, where Ai is the attractor point and Cj is the bin center. alpha (`float`, *optional*, defaults to 300): Proportional Attractor strength. Determines the absolute strength. Lower alpha = greater attraction. gamma (`int`, *optional*, defaults to 2): Exponential Attractor strength. Determines the "region of influence" and indirectly number of bin centers affected. Lower gamma = farther reach. Returns: torch.Tensor: Delta shifts - dc; New bin centers = Old bin centers + dc )divpow)dxrCrDs&&&r inv_attractorrJ&s#" 66!effUm++ ,,r c@aa]tRtRtoRV3RlltRRltRtVtV;t#)ZoeDepthAttractorLayeri:c <\S V`4VPVnVPVnVP VnW0nW n W@n WPn W`n VP;rx\P!Wx^^^4Vn\P"!RR7Vn\P!W^,^^^4Vn\P"!RR7VnR#)za Attractor layer for bin centers. Bin centers are bounded on the interval (min_depth, max_depth) Tr%N)r0r1attractor_alpharCattractor_gammagemmaattractor_kindkind n_attractorsr.r*r+memory_efficientbin_embedding_dimrrxrrr,rr- r?r@r.rSr*r+rTrr/rDs &&&&&&& rr1ZoeDepthAttractorLayer.__init__;s ++ ++ )) ( "" 0!' 8 88 YY{Q1= GGD) YYwq(8!QB GGD) r c Ve@V'd0\PPW1PR RRRR7pW,pVP V4pVP V4pVP V4pVPV4pVR,pVPwrgrVPW`P^W4pVR ,p \PPW(V 3RRR7p VP'gmR\PR \P/VP,p V !\V P!^4V P!^4, 4^R 7p M\P"!WP$R 7p \'VP4F5pV \V RVR3,P!^4V , 4, p K7 VPR8XdWP, p W,pVP(VP*, V,VP*,p \P,!V ^R 7wr\P.!WP*VP(4p W3#)a The forward pass of the attractor layer. This layer predicts the new bin centers based on the previous bin centers and the attractor points (the latter are predicted by the MLP). Args: x (`torch.Tensor` of shape `(batch_size, num_channels, height, width)`): Feature block. prev_bin (`torch.Tensor` of shape `(batch_size, prev_number_of_bins, height, width)`): Previous bin centers normed. prev_bin_embedding (`torch.Tensor`, *optional*): Optional previous bin embeddings. interpolate (`bool`, *optional*, defaults to `True`): Whether to interpolate the previous bin embeddings to the size of the input features. Returns: `tuple[`torch.Tensor`, `torch.Tensor`]: New bin centers normed and scaled. NrTrrr2rQ.meanr7rNdevice)rQrQrR.)rrrrVrr,rr-rrSrTrrZr7rRrJr\ zeros_liker\rr+r*sortclip)r?r:prev_binprev_bin_embeddingr attractorsrbrCheightwidthattractors_normedr;funcdelta_crbin_new_centerss&&&&& rrkZoeDepthAttractorLayer.forwardZs&  )%']]%>%>& :UY&?&"&A JJqM IIaL JJqMYYq\ $& '1'7'7$ v__Z1B1BAvU '|4mm//5/PZjn/o $$$EJJuyy9$))DD=):)D)DQ)G+J_J_`aJb)bcijkG&&{;M;MNG4,,-=):1a9)E)O)OPQ)RU`)`aa.yyF"!$5$55%/~~6/IDNNZ KQ7 jjnndnnM ++r ) r,r-rCrrrPrRr+rTr*rSr.)r@r2rAFNTrros@@rrLrL:s*><,<,r rLc@aa]tRtRtoRV3RlltRRltRtVtV;t#)ZoeDepthAttractorLayerUnnormedic <\S V`4W0nW nW@nWPnVP VnVP VnVPVn W`n VP;rx\P!Wx^^^4Vn\P !RR7Vn\P!W^^^4Vn\P&!4VnR#)z< Attractor layer for bin centers. Bin centers are unbounded Tr%N)r0r1rSr.r*r+rNrCrDrQrRrTrUrrxrrr,rrr-rVs &&&&&&& rr1'ZoeDepthAttractorLayerUnnormed.__init__s ( ""++ ++ )) 0 & 8 88 YY{Q1= GGD) YYwaA> KKM r c Ve@V'd0\PPW1PR RRRR7pW,pVP V4pVP V4pVP V4pVPV4pVPR Rwrg\PPW&V3RRR7pVP'gmR\PR\P/VP,p V !\VP^4VP^4, 4^R7p M\P!WP R7p \#VP$4F5p V \VR V R 3,P^4V, 4, p K7 VPR8XdWP$, p W,p T pW3#) a, The forward pass of the attractor layer. This layer predicts the new bin centers based on the previous bin centers and the attractor points (the latter are predicted by the MLP). Args: x (`torch.Tensor` of shape (batch_size, num_channels, height, width)`): Feature block. prev_bin (`torch.Tensor` of shape (batch_size, prev_num_bins, height, width)`): Previous bin centers normed. prev_bin_embedding (`torch.Tensor`, *optional*): Optional previous bin embeddings. interpolate (`bool`, *optional*, defaults to `True`): Whether to interpolate the previous bin embeddings to the size of the input features. Returns: `tuple[`torch.Tensor`, `torch.Tensor`]: New bin centers unbounded. Two outputs just to keep the API consistent with the normed version. NrTrYrZr7rNr[rQ.r])rrrrVrr,rr-rTrrZr7rRrJr\r^r\rrS) r?r:rarbrrcrdrer;rgrhrris &&&&& rrk&ZoeDepthAttractorLayerUnnormed.forwards&  )%']]%>%>& :UY&?&"&A JJqM IIaL JJqMYYq\ "((- mm//5/PZjn/o $$$EJJuyy9$))DD=)=)=a)@;CXCXYZC[)[\bcdG&&{;M;MNG4,,-=Aq#I)>)H)H)Kk)YZZ.yyF"!$5$55%/% ++r ) r,r-rCrrrDrRr+rTr*rSr.)r@r2rATrkrros@@rrmrms":3,3,r rmcHaa]tRtRtoRV3RlltV3RlRltRtVtV;t#)ZoeDepthProjectoric <\SV`4\P!W^^^4Vn\P !RR7Vn\P!W2^^^4VnR#)zProjector MLP. Args: in_features (`int`): Number of input channels. out_features (`int`): Number of output channels. mlp_dim (`int`, *optional*, defaults to 128): Hidden dimension. Tr%N)r0r1rrxrrrr)r?r out_featuresr/rDs&&&&rr1ZoeDepthProjector.__init__sL YY{Q1= 774(YYwaA> r cN<V^8dQhRS[PRS[P/#rr)rrs"rrZoeDepthProjector.__annotate__s#ELLU\\r clVPV4pVPV4pVPV4pV#r)rrrrs&&rrkZoeDepthProjector.forwards2zz,/ xx - zz,/ r )rrr)rros@@rrsrss?"r rscLaa]tRtRtoRtV3RltRV3RlRlltRtVtV;t #)ZoeDepthMultiheadAttentionizKEquivalent implementation of nn.MultiheadAttention with `batch_first=True`.c(<\SV`4W,^8wd\RV RV R24hW n\ W, 4VnVPVP ,Vn\P!WP 4Vn \P!WP 4Vn \P!WP 4Vn \P!W4Vn \P!V4VnR#)rRzThe hidden size (z6) is not a multiple of the number of attention heads ()N)r0r1rnum_attention_headsr}attention_head_size all_head_sizerr=querykeyr6out_projDropoutdropout)r?rBrrrDs&&&&rr1#ZoeDepthMultiheadAttention.__init__ s   , 1#K=1-.a1  $7 #&{'H#I !558P8PPYY{,>,>? 99[*<*<=YY{,>,>?  +; zz'* r c <V^8dQhRS[PRS[PRS[PRS[PR,RS[R,RS[S[P,/#)rquerieskeysvaluesattention_maskNoutput_attentionsrH)rrKrrr)rrs"rr'ZoeDepthMultiheadAttention.__annotate__ sk...ll. . ))D0 .  $; . u|| .r cVPwrgpVPV4PVRVPVP4P ^^4p VP V4PVRVPVP4P ^^4p VPV4PVRVPVP4P ^^4p \P!WP RR44p V \P!VP4, p Ve W,p \PPV RR7p VPV 4p \P!W4pVP!^^^^4P#4pVP%4RRVP&3,pVPV4pVP)V4pV'dW3pV#V3pV#)rTNrNrUr])rVrrrr transposerr6rmatmulmathsqrtrrr rrYrZrrr)r?rrrrrrb seq_lengthrC query_layer key_layer value_layerattention_scoresattention_probs context_layernew_context_layer_shapeoutputss&&&&&& rrk"ZoeDepthMultiheadAttention.forward s%,MM!  JJw  T*b$":":DS"S%**+BC  m4 6G=2O\M]r )rrrrrrrr6)NFrmros@@rr}r}sU+(...r r}cLaa]tRtRtoRV3RlltRV3RlRlltRtVtV;t#)ZoeDepthTransformerEncoderLayeriQc.<\SV`4VPpVPpVPp\ WFVR7Vn\P!WE4Vn \P!V4Vn \P!WT4Vn \P!V4Vn\P!V4Vn\P!V4Vn\P!V4Vn\$V,VnR#))rN)r0r1patch_transformer_hidden_size#patch_transformer_intermediate_size%patch_transformer_num_attention_headsr} self_attnrr=linear1rrlinear2 LayerNormnorm1norm2dropout1dropout2r activation)r?r@rrrBintermediate_sizerrDs&&&& rr1(ZoeDepthTransformerEncoderLayer.__init__Rs :: "FF$JJ3K^efyy@ zz'* yy!2@ \\+. \\+.  7+  7+  ,r cB<V^8dQhRS[PR,/#)rsrc_maskNr)rrs"rr,ZoeDepthTransformerEncoderLayer.__annotate__fs  ,,% r c \T;r4VPW4WR7^,pWPV4,pVPV4pVPVP VP VP V4444pWPV4,pVPV4pV#))rrrr) rrrrrrrrr)r?srcrrrsrc2s&&& rrk'ZoeDepthTransformerEncoderLayer.forwardfs ~~g~^_`aMM$''jjo||DLLc9J)KLMMM$''jjo r ) rrrrrrrrr)g?relurrros@@rrrQs-(   r rcZaa]tRtRtoV3RltR]P 3RltRtRt Vt V;t #)ZoeDepthPatchTransformerEncoderiuc &<\SV`4VPp\P!\ VP 4Uu.uFp\V4NK up4Vn\P!W!P^^^R7Vn R#uupi)zViT-like transformer block Args: config (`ZoeDepthConfig`): Model configuration class defining the model architecture. ruN) r0r1r(rr3rnum_patch_transformer_layersrtransformer_encoderrxrembedding_convPxP)r?r@rrrCrDs&& rr1(ZoeDepthPatchTransformerEncoder.__init__vs{ 00 #%==>CFDgDg>h i>h ,V 4>h i$  "$ ==1UV`a"  jsBcpuc  "\P!^W%VR7P^4p\P!^V^WTR7P^4p\P!V\P!\P !RRVR74)V, ,4pWh,p \P !\P!V 4\P!V 4.^R7p V P^R7PV^^4p V #)zGenerate positional encodings Args: sequence_length (int): Sequence length embedding_dim (int): Embedding dimension Returns: torch.Tensor: Positional encodings. )dtyper\g@r[rNr!) rrr\exprfullrWsincosrepeat) r?rbre embedding_dimr\rpositionindexdiv_term pos_encodings &&&&&& rpositional_encoding_1d6ZoeDepthPatchTransformerEncoder.positional_encoding_1ds<<?OYYZ[\ Q qMWWXYZ99UuyyBPV1W'X&X[h&hij* yy%))L"9599\;R!SYZ[ #--!-4;;J1M r c  |VPV4P^4p\PP VR4pVP ^^^4pVP wr4pW PW4WRPVPR7,p\^4FpVPV,!V4pK V#)zForward pass Args: x (torch.Tensor - NCHW): Input feature tensor Returns: torch.Tensor - Transformer output embeddings of shape (batch_size, sequence_length, embedding_dim) )r\r)rTrR) rr[rrr8rYrVrr\rrr)r?r: embeddingsrbrerrs&& rrk'ZoeDepthPatchTransformerEncoder.forwards++A.66q9 ]]&&z6: ''1a0 5?5E5E2 ]"="= ?P?PXbXhXh#>#  qA11!4Z@Jr )rr) r"r#r$r%r1rfloat32rrkr(r)rnros@@rrrus) &Y^ejerer$r rcDaa]tRtRtoV3RlV3RlltRtRtVtV;t#)ZoeDepthMLPClassifieric<V^8dQhRR/#)rrHNr!)rrs"rr"ZoeDepthMLPClassifier.__annotate__s@@T@r c<\SV`4Tp\P!W4Vn\P !4Vn\P!W24VnR#r)r0r1rr=rrrr)r?rruhidden_featuresrDs&&& rr1ZoeDepthMLPClassifier.__init__s@ %yy> '')yy? r clVPV4pVPV4pVPV4pV#r)rrr)r?rjrs&& rrkZoeDepthMLPClassifier.forwards2||L1 |4  \2 r )rrrrros@@rrrs@@r rc<aa]tRtRtoRtV3RltRtRtVtV;t #)*ZoeDepthMultipleMetricDepthEstimationHeadsizf Multiple metric depth estimation heads. A MLP classifier is used to route between 2 different heads. cH<\S V`4VPpVPpVPVnVP VnVP p\P!WD^^^R7Vn \V4Vn \^^R7Vn VP R8Xd\pMVP R8Xd\p\P !VPUu/uF7pVR,\#VVR,V^,VR,VR,R 7bK9 up4Vn\'WBV^,R 7Vn\P*!\-^4Uu.uF"p\'VP.VV^,R 7NK$ up4Vn\P !VPUU u/uFapVR,\P*!\-\3V44U u.uF#p X!VW9,VR,VR,R 7NK% up 4bKc up p4VnVP6p \P !VPUu/uF"pVR,\9VV VVR,^R 7bK$ up4VnR #uupiuupiuup iuup piuupi)rTrurrur'softplusnamer.r*r+)r.r/r*r+)rrur/r.r*r+)rN)r0r1rUnum_attractorsbin_configurationsr)r(rrxrrpatch_transformerrmlp_classifierrLrm ModuleDictr#seed_bin_regressorsrsseed_projectorr3rr projectorsrrcrr conditional_log_binomial) r?r@rUrSr( AttractorconfrC configurationrlast_inrDs && rr13ZoeDepthMultipleMetricDepthEstimationHeads.__init__s "44,, "(";"; & 7 7%88YY2UV_`jkl "A!H3RST  H ,.I  " "j 06I$&==#55 6DV 6>-2";/";/ 6  $  0+UfjkUk --q   "A " & 9 9!2-2 "    --&,%>%> &?Mf%r}}"'s<'8!9":A ""#/?&3K&@&3K&@  ": ( &?  "..(* &,%>%> &?Mf%'L%!(+&' (&?  ) %]    & s*$=J #(J63J ))J J (JJ cVPV4pVPV4R ,pVPV4p\P!VP ^RR7RR7pVP U u.uF qR,NK p p V \P!VRR7P4P4,p VP U u.uFqR,V 8XgKV NK up ^,p T R,pT R,pTPT ,pT!T4wppTPR9dTT, Y, , pMTpTPT4pTPT ,p\!TP"TT4F"wpppT!T4pT!TTTRR 7wppTpTpK$ Tp\$P&P)XTP*RR R RR 7p\$P&P)XTP*RR R RR 7pTP,T ,pT!TT4p\P !TT,^RR7pTT3#uup iuup i \d\RT R24hi;i)rQTr3rNrzbin_configurations_name z! not found in bin_configurationssr*r+rNrrY)rQrRrQrUr'hybrid2r])rrrrr r7rargmaxritem IndexErrorrrr)rrcr5rrrrrVr)r?outconv_activationrfeature_blocksrelative_depthr: embeddingr domain_voternamesbin_configurations_namer@rr*r+seed_bin_regressorrCseed_bin_centersrarbrc projector attractorr bin_embeddingbinr;lastrrhs&&&&& rrk2ZoeDepthMultipleMetricDepthEstimationHeads.forwardst JJz "**1-g6 ++I6 mmM$5$5!T$5$JPRS =A>9dnnt~~>]^H'H!003.1$//Sa-b )I'%g.M(BTbfg Cyy{H!.!4!4!6  .c "!/ 8 8 ; " 9 9 ! 2ZW[!:! yy$ 56A> 11-BCWaqu1v  ) )$ >mm// QWWRS\PZjn/o iiQ=Dyr ) rcr)rrr+r*rrrrros@@rrrRs9 v""r rcNaa]tRtRtoRtRtRtRtV3RltV3Rlt Rt Vt V;t #) ZoeDepthPreTrainedModelizoedepth pixel_valuesTc <\SV`V4\V\4'd\P !VP \P!^VP4P^R^^44\P !VP\P!VP^, .4P^R^^44R#R#)rRNrU) r0 _init_weightsrrinitcopy_rrrrrrr)r?modulerDs&&rr%ZoeDepthPreTrainedModel._init_weightss f% f0 1 1 JJv||U\\!VXX%>%C%CAr1a%P Q JJv''vxx!|n)E)J)J1bRSUV)W X 2r c&<V^8dQh/S[;R&#rr )rrs"rr$ZoeDepthPreTrainedModel.__annotate__s r r!)image) r"r#r$r%base_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointingrr'r(r)rnros@@rrrs+#$O!&*#Yr rzU ZoeDepth model with one or multiple metric depth estimation head(s) on top. cRaa]tRtRtoV3Rlt]RV3RlRll4tRtVtV;t #)ZoeDepthForDepthEstimationicT<\SV`V4\V4Vn\ VPP R4'dr\ VPP R4'dLVPP P VnVPP PVnM \R4h\V4Vn \V4Vn \VP4^8d \!V4M \#V4VnVP'4R#)rB patch_sizezXZoeDepth assumes the backbone's config to have `hidden_size` and `patch_size` attributesN)r0r1rbackbonehasattrr@rBr;r#rrneckr relative_headrrrr metric_head post_init)r?r@rDs&&rr1#ZoeDepthForDepthEstimation.__init__s  %f- 4=='' 7 7GDMMDXDXZf*>*J*JF '"mm22==DOj !( @H6,,-1 7v >26:  r c <V^8dQhRS[PRS[PR,RS[R,RS[R,RS[R,RS[S[P ,S[,/#)rrlabelsNroutput_hidden_states return_dictrH)rr LongTensorrrrKr)rrs"rr'ZoeDepthForDepthEstimation.__annotate__s{^ ^ ''^   4'^  $; ^ #Tk ^ D[ ^  u|| 3 3^ r c 6RpVe \R4hVeTMVPPpVeTMVPPpVeTMVPPpVP P WVR7pVPp VPwrp VPp W,pW,pVPWV4wp pV.V ,pVPV 4wppV.V,pVPV^,V^,VR,VR7wppVP^R7pV'g9VeVV3VR,,pMV3VR,,pVe V3V,#T#\VVVVPVP R7#) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth depth estimation maps for computing the loss. Examples: ```python >>> from transformers import AutoImageProcessor, ZoeDepthForDepthEstimation >>> import torch >>> import numpy as np >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> image_processor = AutoImageProcessor.from_pretrained("Intel/zoedepth-nyu-kitti") >>> model = ZoeDepthForDepthEstimation.from_pretrained("Intel/zoedepth-nyu-kitti") >>> # prepare image for the model >>> inputs = image_processor(images=image, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> # interpolate to original size >>> post_processed_output = image_processor.post_process_depth_estimation( ... outputs, ... source_sizes=[(image.height, image.width)], ... ) >>> # visualize the prediction >>> predicted_depth = post_processed_output[0]["predicted_depth"] >>> depth = predicted_depth * 255 / predicted_depth.max() >>> depth = depth.detach().cpu().numpy() >>> depth = Image.fromarray(depth.astype("uint8")) ```NzTraining is not implemented yet)r-rr)rrrrrNrS)rrrrr)NotImplementedErrorr@r.r-rr$forward_with_filtered_kwargs feature_mapsrVr#r&r'r(rrrr)r?rr,rr-r.kwargsrrrrCrdrer#r`rarrhr metric_depthrrs&&&&&&, rrk"ZoeDepthForDepthEstimation.forwards`  %&GH H%0%)i,r)6r&r dataclassesrrrrr activationsrbackbone_utilsrmodeling_outputsrmodeling_utilsrutilsr r r configuration_zoedepthr get_loggerr"loggerrModuler+r9rrrrrrrr r#jitscriptrJrLrmrsr}rrrrrrr!__all__r!r rrFs7 ! &!+4-992   H %   <; <  <GbiiGTbii0##2;'"));'~""J3$2993$l))"))))XR)0)0X@GBII@GF5,ryy5,p--&\,RYY\,~Q,RYYQ,h 6FFR!bii!H<bii<~BII"J"J"Z^ ^F Yo Y Y y !8y  y x ()B Cr