+ j4>^RIt^RIt^RIHtHt^RIHtHt^RIH t ^RI H t ^RI Ht.ROt!R R]4t!R R]4t ]]],,],t!RR ]4t!RR ]4t!RR ]4tR#)N)SizeTensor) functionalinit) Parameter) CrossMapLRN2d)ModuleLocalResponseNormr LayerNorm GroupNormRMSNormcraa]tRt^toRt.R OtR V3RlV3RllltV3RlRltRtV3Rlt Rt Vt V;t #) r a;Applies local response normalization over an input signal. The input signal is composed of several input planes, where channels occupy the second dimension. Applies normalization across channels. .. math:: b_{c} = a_{c}\left(k + \frac{\alpha}{n} \sum_{c'=\max(0, c-n/2)}^{\min(N-1,c+n/2)}a_{c'}^2\right)^{-\beta} Args: size: amount of neighbouring channels used for normalization alpha: multiplicative factor. Default: 0.0001 beta: exponent. Default: 0.75 k: additive factor. Default: 1 Shape: - Input: :math:`(N, C, *)` - Output: :math:`(N, C, *)` (same shape as input) Examples:: >>> lrn = nn.LocalResponseNorm(2) >>> signal_2d = torch.randn(32, 5, 24, 24) >>> signal_4d = torch.randn(16, 5, 7, 7, 7, 7) >>> output_2d = lrn(signal_2d) >>> output_4d = lrn(signal_4d) c 6<V^8dQhRS[RS[RS[RS[RR/#sizealphabetakreturnNintfloat)format __classdict__s"_/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/nn/modules/normalization.py __annotate__LocalResponseNorm.__annotate__45 %49EJ cT<\SV`4WnW nW0nW@nR#Nsuper__init__rrrrselfrrrr __class__s&&&&&rr%LocalResponseNorm.__init__4$    r c&<V^8dQhRS[RS[/#rinputrr)rrs"rrr=sVVVVVr c \P!WPVPVPVP 4#z Runs the forward pass. )Flocal_response_normrrrrr'r-s&&rforwardLocalResponseNorm.forward=s+$$UIItzz499dffUUr c :RP!R/VPB#0 Return the extra representation of the module. z){size}, alpha={alpha}, beta={beta}, k={k}r__dict__r's&r extra_reprLocalResponseNorm.extra_reprC;AARDMMRRr cJ<V^8dQh/S[;R&S[;R&S[;R&S[;R&#rrrrrr)rrs"rrrs;> I?@ LAB KCD HEr rrrr)rrrr)-C6??g?) __name__ __module__ __qualname____firstlineno____doc__ __constants__r%r4r=__annotate_func____static_attributes____classdictcell__ __classcell__r(rs@@rr r s5:3M VV Sgr craa]tRt^JtoR V3RlV3RllltV3RlRltV3RlRltV3RltRtVt V;t #) rc 6<V^8dQhRS[RS[RS[RS[RR/#rr)rrs"rrCrossMapLRN2d.__annotate__Prr cT<\SV`4WnW nW0nW@nR#r"r#r&s&&&&&rr%CrossMapLRN2d.__init__Pr*r c&<V^8dQhRS[RS[/#r,r.)rrs"rrrRYsWWVWWr c \P!WPVPVPVP 4#r0)_cross_map_lrn2dapplyrrrrr3s&&rr4CrossMapLRN2d.forwardYs+ %%eYY DIItvvVVr c <V^8dQhRS[/#rrstr)rrs"rrrR_sSSCSr c :RP!R/VPB#r7r:r<s&rr=CrossMapLRN2d.extra_repr_r?r cJ<V^8dQh/S[;R&S[;R&S[;R&S[;R&#rAr)rrs"rrrRJs1 I L K H r rB)rCrD) rErFrGrHr%r4r=rKrLrMrNrOs@@rrrJs. WW SS+r caa]tRt^itoRt.R OtR V3RlV3RllltV3RlRltV3RlRltV3RlR lt V3R lt R t Vt V;t #)r a Applies Layer Normalization over a mini-batch of inputs. This layer implements the operation as described in the paper `Layer Normalization `__ .. math:: y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta The mean and standard-deviation are calculated over the last `D` dimensions, where `D` is the dimension of :attr:`normalized_shape`. For example, if :attr:`normalized_shape` is ``(3, 5)`` (a 2-dimensional shape), the mean and standard-deviation are computed over the last 2 dimensions of the input (i.e. ``input.mean((-2, -1))``). :math:`\gamma` and :math:`\beta` are learnable affine transform parameters of :attr:`normalized_shape` if :attr:`elementwise_affine` is ``True``. The variance is calculated via the biased estimator, equivalent to `torch.var(input, correction=0)`. .. note:: Unlike Batch Normalization and Instance Normalization, which applies scalar scale and bias for each entire channel/plane with the :attr:`affine` option, Layer Normalization applies per-element scale and bias with :attr:`elementwise_affine`. This layer uses statistics computed from input data in both training and evaluation modes. Args: normalized_shape (int or list or torch.Size): input shape from an expected input of size .. math:: [* \times \text{normalized\_shape}[0] \times \text{normalized\_shape}[1] \times \ldots \times \text{normalized\_shape}[-1]] If a single integer is used, it is treated as a singleton list, and this module will normalize over the last dimension which is expected to be of that specific size. eps: a value added to the denominator for numerical stability. Default: 1e-5 elementwise_affine: a boolean value that when set to ``True``, this module has learnable per-element affine parameters initialized to ones (for weights) and zeros (for biases). Default: ``True`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`elementwise_affine` is ``True``). Default: ``True`` Attributes: weight: the learnable weights of the module of shape :math:`\text{normalized\_shape}` when :attr:`elementwise_affine` is set to ``True``. The values are initialized to 1. bias: the learnable bias of the module of shape :math:`\text{normalized\_shape}` when :attr:`elementwise_affine` is set to ``True``. The values are initialized to 0. Shape: - Input: :math:`(N, *)` - Output: :math:`(N, *)` (same shape as input) Examples:: >>> # NLP Example >>> batch, sentence_length, embedding_dim = 20, 5, 10 >>> embedding = torch.randn(batch, sentence_length, embedding_dim) >>> layer_norm = nn.LayerNorm(embedding_dim) >>> # Activate module >>> layer_norm(embedding) >>> >>> # Image Example >>> N, C, H, W = 20, 5, 10, 10 >>> input = torch.randn(N, C, H, W) >>> # Normalize over the last three dimensions (i.e. the channel and spatial dimensions) >>> # as shown in the image below >>> layer_norm = nn.LayerNorm([C, H, W]) >>> output = layer_norm(input) .. image:: ../_static/img/nn/layer_norm.jpg :scale: 50 % c 6<V^8dQhRS[RS[RS[RS[RR/#)rnormalized_shapeepselementwise_affinebiasrN_shape_trbool)rrs"rrLayerNorm.__annotate__s=  "  !      r c.<RVRV/p\SV`4\V\P4'dV3p\ V4VnW nW0nVP'd{\\P!VP 3/VB4Vn V'd1\\P!VP 3/VB4Vn M7VPRR4M$VPRR4VPRR4VP4R#)devicedtypergNweight)r$r% isinstancenumbersIntegraltuplerdrerfrtorchemptyrorgregister_parameterreset_parameters) r'rdrerfrgrmrnfactory_kwargsr(s &&&&&&& rr%LayerNorm.__init__s#FGU;  &(8(8 9 9 02  %&6 7"4  " " "# D11D^DDK%KK 5 5HH ''5  # #Hd 3  # #FD 1 r c<V^8dQhRR/#rrNr9)rrs"rrrk''$'r cVP'dS\P!VP4VPe#\P !VP4R#R#R#r")rfrones_rorgzeros_r<s&rrwLayerNorm.reset_parameterssB  " " " JJt{{ #yy$ DII&% #r c&<V^8dQhRS[RS[/#r,r.)rrs"rrrks  V  r c\P!WPVPVPVP 4#r")r1 layer_normrdrorgrer3s&&rr4LayerNorm.forwards.|| (($++tyy$((  r c <V^8dQhRS[/#r[r\)rrs"rrrk  C r cZRP!R/VPBRVPRJ/B#)zW{normalized_shape}, eps={eps}, elementwise_affine={elementwise_affine}, bias={use_bias}use_biasNr9rr;rgr<s&rr=LayerNorm.extra_repr< $f % V'+}} V?CyyPT?T V r cR<V^8dQh/S[S[R3,;R&S[;R&S[;R&#)r.rdrerfrsrrrj)rrs"rrrkis9^CHo%_` Jabcr )rgrfrerdrordrerf)h㈵>TTNNrErFrGrHrIrJr%rwr4r=rKrLrMrNrOs@@rr r isEKZFM   B''     r caa]tRt^toRt.ROtRRR/V3RlV3RlllltV3RlRltV3RlR ltV3R lR lt V3R lt R t Vt V;t #)r aApplies Group Normalization over a mini-batch of inputs. This layer implements the operation as described in the paper `Group Normalization `__ .. math:: y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta The input channels are separated into :attr:`num_groups` groups, each containing ``num_channels / num_groups`` channels. :attr:`num_channels` must be divisible by :attr:`num_groups`. The mean and standard-deviation are calculated separately over each group. :math:`\gamma` and :math:`\beta` are learnable per-channel affine transform parameter vectors of size :attr:`num_channels` if :attr:`affine` is ``True``. The variance is calculated via the biased estimator, equivalent to `torch.var(input, correction=0)`. This layer uses statistics computed from input data in both training and evaluation modes. Args: num_groups (int): number of groups to separate the channels into num_channels (int): number of channels expected in input eps: a value added to the denominator for numerical stability. Default: 1e-5 affine: a boolean value that when set to ``True``, this module has learnable per-channel affine parameters initialized to ones (for weights) and zeros (for biases). Default: ``True`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, *)` where :math:`C=\text{num\_channels}` - Output: :math:`(N, C, *)` (same shape as input) Examples:: >>> input = torch.randn(20, 6, 10, 10) >>> # Separate 6 channels into 3 groups >>> m = nn.GroupNorm(3, 6) >>> # Separate 6 channels into 6 groups (equivalent with InstanceNorm) >>> m = nn.GroupNorm(6, 6) >>> # Put all 6 channels into a single group (equivalent with LayerNorm) >>> m = nn.GroupNorm(1, 6) >>> # Activating the module >>> output = m(input) Trgc <<V^8dQhRS[RS[RS[RS[RS[RR/#)r num_groups num_channelsreaffinergrNrrrj)rrs"rrGroupNorm.__annotate__%sG                r c<RVRV/p\S V`4W!,^8wd\RV RV R24hWnW nW0nW@nVP 'dg\\P!V3/VB4Vn V'd'\\P!V3/VB4Vn M7VPRR4M$VPRR4VPRR4VP4R#)rmrnznum_channels (z#) must be divisible by num_groups ()rgNro)r$r% ValueErrorrrrerrrtrurorgrvrw) r'rrrerrmrnrgrxr(s &&&&&&&$ rr%GroupNorm.__init__%s#FGU;   $ ) .QR\Q]]^_ %( ;;;#EKK $O$OPDK%ekk,&Q.&QR ''5  # #Hd 3  # #FD 1 r c<V^8dQhRR/#r{r9)rrs"rrrGr|r cVP'dS\P!VP4VPe#\P !VP4R#R#R#r")rrr~rorgrr<s&rrwGroupNorm.reset_parametersGs? ;;; JJt{{ #yy$ DII&% r c&<V^8dQhRS[RS[/#r,r.)rrs"rrrMsVVVVVr c\P!WPVPVPVP 4#r")r1 group_normrrorgrer3s&&rr4GroupNorm.forwardMs'||E??DKKDHHUUr c <V^8dQhRS[/#r[r\)rrs"rrrPrr cZRP!R/VPBRVPRJ/B#)zI{num_groups}, {num_channels}, eps={eps}, affine={affine}, bias={use_bias}rNr9rr<s&rr=GroupNorm.extra_reprPrr cJ<V^8dQh/S[;R&S[;R&S[;R&S[;R&#)rrrrerr)rrs"rrrs>bOcdef Jgh Lir )rrgrerrro)rrrer)rTNNrrOs@@rr r sP-^DM     D'' VV  Cr caa]tRtRtoRt.R OtRV3RlV3RllltV3RlRltV3RlRltV3R lR lt V3R lt R t Vt V;t #)r iWaApplies Root Mean Square Layer Normalization over a mini-batch of inputs. This layer implements the operation as described in the paper `Root Mean Square Layer Normalization `__ .. math:: y_i = \frac{x_i}{\mathrm{RMS}(x)} * \gamma_i, \quad \text{where} \quad \text{RMS}(x) = \sqrt{\epsilon + \frac{1}{n} \sum_{i=1}^{n} x_i^2} The RMS is taken over the last ``D`` dimensions, where ``D`` is the dimension of :attr:`normalized_shape`. For example, if :attr:`normalized_shape` is ``(3, 5)`` (a 2-dimensional shape), the RMS is computed over the last 2 dimensions of the input. Args: normalized_shape (int or list or torch.Size): input shape from an expected input of size .. math:: [* \times \text{normalized\_shape}[0] \times \text{normalized\_shape}[1] \times \ldots \times \text{normalized\_shape}[-1]] If a single integer is used, it is treated as a singleton list, and this module will normalize over the last dimension which is expected to be of that specific size. eps (float, optional): a value added to the denominator for numerical stability. If not specified, uses the machine epsilon of the computation (opmath) type: fp16/bf16 and fp32 inputs use ``torch.finfo(torch.float32).eps``, while fp64 inputs use ``torch.finfo(torch.float64).eps``. Default: ``None`` elementwise_affine: a boolean value that when set to ``True``, this module has learnable per-element affine parameters initialized to ones (for weights). Default: ``True``. Shape: - Input: :math:`(N, *)` - Output: :math:`(N, *)` (same shape as input) Examples:: >>> rms_norm = nn.RMSNorm([2, 3]) >>> input = torch.randn(2, 2, 3) >>> rms_norm(input) c><V^8dQhRS[RS[R,RS[RR/#)rrdreNrfrrh)rrs"rrRMSNorm.__annotate__s7  " T\ !    r cv<RVRV/p\SV`4\V\P4'dV3p\ V4VnW nW0nVP'd1\\P!VP 3/VB4Vn MVPRR4VP4R#)rmrnroN)r$r%rprqrrrsrdrerfrrtrurorvrw)r'rdrerfrmrnrxr(s&&&&&& rr%RMSNorm.__init__s#FGU;  &(8(8 9 9 02  %&6 7"4  " " "# D11D^DDK  # #Hd 3 r c<V^8dQhRR/#r{r9)rrs"rrrs$$$$r c nVP'd#\P!VP4R#R#)zC Resets parameters based on their initialization used in __init__. N)rfrr~ror<s&rrwRMSNorm.reset_parameterss%  " " " JJt{{ # #r cN<V^8dQhRS[PRS[P/#)rxr)rtr)rrs"rrrs'KKK%,,Kr c n\P!WPVPVP4#r0)r1rms_normrdrore)r'rs&&rr4RMSNorm.forwards%zz!22DKKJJr c <V^8dQhRS[/#r[r\)rrs"rrrs  C r c :RP!R/VPB#)r8zF{normalized_shape}, eps={eps}, elementwise_affine={elementwise_affine}r9r:r<s&rr=RMSNorm.extra_reprs)  66ZCHo%[\ ]^_r )rfrerdror)NTNNrrOs@@rr r WsE)VFM   0$$KK   mr )r rr r r )rqrtrrtorch.nnrr1rtorch.nn.parameterr _functionsrrWmoduler __all__r rlistrir r r r9r rrs *(9 V7S7StSFS8 c?T !C C Le e P] f] r