+ j  ^RIt^RIt^RIHt^RIHtHt^RIHtH t ] !RR4t ]P]P]P]P]P.t]P"]P$.t]Uu/uF9q]P)V4P*]P)V4P,3bK; upt]P1]Uu/uFEq]!]P5V4P*4]!]P5V4P,43bKG up4Rt] P9R4]!] R R 4R R l4t]!] R R 4RRl4t] P9R4]!] RR 4RRl4t]!] RR 4RRl4t ] P9R4]!] RR 4RRl4t!]!] RR 4RRl4t"] P9R4]!] RR 4RR/RR ll4t#]!] RR 4RR/R!R"ll4t$] P9R#4]!] R$R 4RR/R%R&ll4t%]!] R$R 4RR/R'R(ll4t&] P9R)4]!] R*R 4RR/R+R,ll4t']!] R*R 4RR/R-R.ll4t(] P9R/4]!] R0R 4R1R2l4t)] P9R34]!] R4R 4R5R6l4t*]!] R0R 4R7R8l4t+]!] R4R 4R9R:l4t,R;t-] P9R<4]!] R=R 4R>R?l4t.]!] R=R 4R@RAl4t/] P9RB4]!] RCR 4RR/RDREll4t0]!] RCR 4RR/RFRGll4t1] P9RH4]!] RIR 4RJRKl4t2]!] RIR 4RLRMl4t3] P9RN4]!] RORP4RQRRl4t4] P9RS4]!] RTR 4RURVl4t5]!] RTR 4RWRXl4t6RYt7] P9RZ4]!] R[R 4R\R]l4t8]!] R[R 4R^R_l4t9] P9R`4]!] RaR 4]Pt3RbRcll4t;]!] RaR 4]Pt3RdRell4t<] P9Rf4]!] RgR 4RRhRill4t=]!] RgR 4RRjRkll4t>] P9Rl4]!] RmR 4^]Pt3RnRoll4t?] P9Rp4!RqRr]PP4tB]!] RsRt4RuRvl4tC]!] RsR 4RwRxl4tD] P9Ry4]!] RzR 4R{R|l4tE]!] RzR 4R}R~l4tFR#uupiuupi)N)_unsqueeze_multiple)determine_qparamsvalidate_qmin_qmax)implLibraryquantized_decomposedDEFcV\9d\RV 24h\V,wr4W8d\RV RV 24hW8d\RV RV 24hR#)zUnsupported dtype: z:quant_min out of bounds for dtype, quant_min_lower_bound: z quant_min: z:quant_max out of bounds for dtype, quant_max_upper_bound: z quant_max: N)_DTYPE_TO_QVALUE_BOUNDS ValueErrorAssertionError) quant_min quant_maxdtypequant_min_lower_boundquant_max_upper_bounds&&& e/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/ao/quantization/fx/_decomposed.py_quant_min_max_bounds_checkrs ++.ug6773J53Q0( &&;% Tensorquantize_per_tensorCompositeExplicitAutogradcV^8dQhR\PR\R\R\R\R\PR\P/#inputscale zero_pointrrrreturntorchTensorfloatintr)formats"r __annotate__r%3sZ"" <<" "" "  " ;; " \\"rcVP\P\P39d VP \P 4pVP\P 8wd\ RVP 24h\W4V4RV, p\P!\P!W,4V,W44P V4#)aAffine quantization for the Tensor using the same quantization parameters to map from floating point to quantized values Args: input (torch.Tensor): original float32 or bfloat16 Tensor scale (float): quantization parameter for affine quantization zero_point (int): quantization parameter for affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead Tensorzquantize_per_tensor.tensorcV^8dQhR\PR\PR\PR\R\R\PR\P/#rr r!r#r)r$s"rr%r%ssb << <<     ;;  \\rcVP4^8wd\RVP4 24hVP4^8wd\RVP4 24h\VVP4VP4VVV4#zAffine quantization for the Tensor using the same quantization parameters to map from floating point to quantized values Same as `quantize_per_tensor` but scale and zero_point are Scalar Tensor instead of scalar values >Expecting zero_point tensor to be one element, but received : 9Expecting scale tensor to be one element, but received : numelr ritemr5s&&&&&&rquantize_per_tensor_tensorr@ps QLZM]M]M_L` a   {{}G  W       rcV^8dQhR\PR\PR\PR\R\R\PR\P/#rr8)r$s"rr%r%sb00 <<0 <<0 0 0  0 ;; 0 \\0rcVP\P\P39d VP \P 4pVP 4^8wd\RVP 4 24hVP 4^8wd\RVP 4 24hVP\P 8wd\RVP 24h\P!WR7#)r;r<r'r3) rr r)r*r+r,r>r r4r5s&&&&&&rquantize_per_tensor_tensor_metarDs {{u}}enn55'QLZM]M]M_L` a   {{}G  W   {{emm#J5;;- X     E //rzquantize_per_tensor.tensor2(Tensor input, Tensor scale, Tensor zero_point, Tensor quant_min, Tensor quant_max, ScalarType dtype) -> Tensorzquantize_per_tensor.tensor2cV^8dQhR\PR\PR\PR\PR\PR\PR\P/#rr r!r)r$s"rr%r%sj << << ||  ||  ;;  \\rcVVP4^8wd\RVP4 24hVP4^8wd\RVP4 24h\VVP4VP4VP4VP4V4#r:r=r5s&&&&&&rquantize_per_tensor_tensor2rHs QLZM]M]M_L` a   {{}G  W       rcV^8dQhR\PR\PR\PR\PR\PR\PR\P/#rrF)r$s"rr%r%sj << << ||  ||  ;;  \\rc"\VVVVVV4#N)rDr5s&&&&&&r quantize_per_tensor_tensor2_metarLs# +    rzdequantize_per_tensor(Tensor input, float scale, int zero_point, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensordequantize_per_tensor out_dtypecV^8dQhR\PR\R\R\R\R\PR\PR,R \P/# rrrrrrrrNNrr)r$s"rr%r%sw0Q0Q <<0Q 0Q0Q 0Q  0Q ;; 0Q{{T!0Q \\0QrcVPV8wd\RV RVP 24hVf\PpV\9d VP V4V, V,#\ RV 24h)aAffine dequantization for the Tensor using the same quantization parameters to map from quantized values to floating point values Args: input (torch.Tensor): Tensor with dtype matching `dtype` argument, e.g. (`torch.uint8`), it is a per tensor quantized Tensor if combined with quantization parameters in the argument of this function (scale/zero_point) scale (float): quantization parameter for affine quantization zero_point (int): quantization parameter for affine quantization quant_min (int): minimum quantized value for input Tensor (not used in computation, reserved for pattern matching) quant_max (int): maximum quantized value for input Tensor (not used in computation, reserved for pattern matching) dtype (torch.dtype): dtype for input Tensor (not used in computation, reserved for pattern matching) out_dtype (torch.dtype?): optional dtype for output Tensor Returns: dequantized float32 Tensor Expecting input to have dtype: , but got ,Unsupported dtype in dequantize_per_tensor: )rr r r,r r+r rrrrrrrNs&&&&&&$rrMrMszJ {{e-eWJu{{m L  MM  ''#j0E99GwOPPrcV^8dQhR\PR\PR\PR\R\R\PR\PR,R \P/#rPr8)r$s"rr%r%'su 4 4 << 4 << 4  4 4  4 ;; 4{{T! 4 \\ 4rcXVf\Pp\P!WR7#)Nr3)r r,r4rUs&&&&&&$rdequantize_per_tensor_metarX&s$MM   E 33rzdequantize_per_tensor.tensor(Tensor input, Tensor scale, Tensor zero_point, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensorzdequantize_per_tensor.tensorcV^8dQhR\PR\PR\PR\R\R\PR\PR,R \P/#rPr8)r$s"rr%r%Asu << <<     ;; {{T! \\rc "VP4^8wd\RVP4 24hVP4^8wd\RVP4 24h\VVP4VP4VVVVR7#zAffine dequantization for the Tensor using the same quantization parameters to map from quantized values to floating point values Same as `dequantize_per_tensor` but scale and zero_point are Scalar Tensor instead of scalar values r;r<rNr>r rMr?rUs&&&&&&$rdequantize_per_tensor_tensorr^<s(QLZM]M]M_L` a   {{}G  W   !    rcV^8dQhR\PR\PR\PR\R\R\PR\PR,R \P/#rPr8)r$s"rr%r%dsQQ <<Q <<Q Q Q  Q ;; Q{{T!Q \\QrcVf\PpVP4^8wd\RVP4 24hVP4^8wd\RVP4 24hVPV8wd\RV RVP 24hV\ 9d\P !WR7#\RV 24h)Nr;r<rRrSr3rT)r r,r>r rr r4r rUs&&&&&&$r!dequantize_per_tensor_tensor_metaracsMM QLZM]M]M_L` a   {{}G  W   {{e-eWJu{{m L   ''77GwOPPrzdequantize_per_tensor.tensor2(Tensor input, Tensor scale, Tensor zero_point, Tensor quant_min, Tensor quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensorzdequantize_per_tensor.tensor2c&V^8dQhR\PR\PR\PR\PR\PR\PR\PR,R \P/#rPrF)r$s"rr%r%s} << << ||  ||  ;; {{T! \\rc ZVP4^8wd\RVP4 24hVP4^8wd\RVP4 24h\VVP4VP4VP4VP4VVR7#r[r]rUs&&&&&&$rdequantize_per_tensor_tensor2rds(QLZM]M]M_L` a   {{}G  W   !    rcfV^8dQhR\PR,R\P/#)rrNNr)r rr!)r$s"rr%r%s-  {{T!  \\ rc  \WW#WEVR7#)r\)rarUs&&&&&&$r"dequantize_per_tensor_tensor2_metargs - jY rzrchoose_qparams.tensor(Tensor input, int quant_min, int quant_max, float eps, ScalarType dtype) -> (Tensor, Tensor)zchoose_qparams.tensorcV^8dQhR\PR\R\R\R\PR\ \P\P3,/#rrqminqmaxepsrrr r!r#r"rtuple)r$s"rr%r%V(( <<("(*-(49(BG++( 5<< %&(rc VP\P\P\P39d\ RVP 24hV\ 9d$\ R\ P4 RV 24h\W4\P!V4wrV\VVVVV\P!V.4RR7#)3Given an input Tensor, derive the per tensor affine quantization parameter (scale and zero_point) for target quantized Tensor from the Tensor Args: input (torch.Tensor): floating point input Tensor quant_min (int): minimum quantized value for target quantized Tensor quant_max (int): maximum quantized value for target quantized Tensor dtype (torch.dtype): dtype for target quantized Tensor Returns: scale (float): quantization parameter for the target quantized Tensor zero_point (int): quantization parameter for the target quantized Tensor CExpecting input to have dtype torch.float32/16/b16, but got dtype: $Expecting target dtype to be one of , but got: F)has_customized_qrange) rr r,r)r*r r keysraminmaxrr!rrjrkrlrmin_valmax_vals&&&&& rchoose_qparams_tensorr{s" {{    QRWR]R]Q^ _   ++23J3O3O3Q2RR]^c]d e  t"}}U+G     cU# rz|choose_qparams_symmetric.tensor(Tensor input, int quant_min, int quant_max, float eps, ScalarType dtype) -> (Tensor, Tensor)zchoose_qparams_symmetric.tensorcV^8dQhR\PR\R\R\R\PR\ \P\P3,/#rirm)r$s"rr%r%rorc VP\P\P\P39d\ RVP 24hV\ 9d$\ R\ P4 RV 24h\W4\P!V4wrV\VVVVV\P!V.4R\PR7#)rqrrrsrtF)ruqscheme) rr r,r)r*r r rvrrwrr!per_tensor_symmetricrxs&&&&& rchoose_qparams_symmetric_tensorrs* {{    QRWR]R]Q^ _   ++23J3O3O3Q2RR]^c]d e  t"}}U+G     cU#**  rcV^8dQhR\PR\R\R\R\PR\ \P\P3,/#rrrrrlrrrm)r$s"rr%r%)sV <<$'47>CLQKK 5<< %&rcVP\P\P\P39d\ RVP 24hW8d\ RV RV 24h\P !^\PVPR7\P !^\PVPR73#)rrzCExpecting quant_min to be smaller than quant_max but received min: z max: rdevice) rr r,r)r*r emptydoublerint64rrrrlrs&&&&&rchoose_qparams_tensor_metar(s {{    QRWR]R]Q^ _  QR[Q\\bclbm n   ;;q U\\ BEKK U\\E rcV^8dQhR\PR\R\R\R\PR\ \P\P3,/#rrm)r$s"rr%r%>sV <<$'47>CLQKK 5<< %&rc\P!^\PVPR7\P!^\PVPR73#)rCr)r rrrrrs&&&&&r$choose_qparams_symmetric_tensor_metar=sA ;;q U\\ BEKK U\\E rc\\VP444p^W!&W^&VP\ V44pW23#)r)listrangedimpermutern)xaxis new_axis_listys&& r_permute_to_axis_zerorGsBquuw(MM! % &'A rzquantize_per_channel(Tensor input, Tensor scales, Tensor zero_points, int axis, int quant_min, int quant_max, ScalarType dtype) -> Tensorquantize_per_channelcV^8dQhR\PR\PR\PR\R\R\R\PR\P/# rrscales zero_pointsrrrrrr8)r$s"rr%r%Vsl.. <<. LL..  .  .  . ;;. \\.rcVP\P\P39d VP \P 4pVP\P 8wd\ RVP 24hW0P48d\ RVP4 24h\WEV4\W4wr^.VP4,pVP^,V^&VPV4pVPV4p\P!\P!VRV, ,4V,WE4p V P\V44p V P V4#)a<Affine per channel quantization for the Tensor using the same quantization parameters for each channel/axis to map from floating point to quantized values Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (torch.Tensor): a list of scale quantization parameter for affine quantization, one per channel zero_point (torch.Tensor): a list of zero_point quantization parameter for affine quantization, one per channel quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r'Expecting axis to be < r()rr r)r*r+r,r rrrshapeviewr-r.rrn) rrrrrrrpermute_axis_list new_shaperesouts &&&&&&& rrrUs'6 {{u}}enn55' {{emm#J5;;- X   yy{6uyy{mDEE e<4UAEeiik!I<<?IaL [[ #F""9-K ++ ES6\*+k99 C ++e-. /C 66%=rcV^8dQhR\PR\PR\PR\R\R\R\PR\P/#rr8)r$s"rr%r%sl00 <<0 LL00  0  0  0 ;;0 \\0rcVP\P\P39d VP \P 4pVP\P 8wd\ RVP 24hW0P48d\ RVP4 24h\WEV4\P!WR7#)r'rr3) rr r)r*r+r,r rrr4)rrrrrrrs&&&&&&&rquantize_per_channel_metars {{u}}enn55' {{emm#J5;;- X   yy{6uyy{mDEE e<   E //rzdequantize_per_channel(Tensor input, Tensor scales, Tensor? zero_points, int axis, int quant_min, int quant_max, ScalarType dtype, *, ScalarType? out_dtype=None) -> Tensordequantize_per_channelcV^8dQhR\PR\PR\PR,R\R\R\R\PR \PR,R \P/ # rrrrNrrrrrNrr8)r$s"rr%r%s== <<= LL=$=  =  =  = ;;={{T!= \\=rcBVPV8wd\RV RVP 24hVf\PpW0P 48d\RVP 4 24h\ WEV4\ W4wr^.VP 4,p VP^,V ^&VPV 4pVe WPV 4, V,p MW,p V PV4p V P\V44p V #)aOAffine per channel dequantization for the Tensor using the same quantization parameters for each channel/axis to map from quantized values to floating point values Args: input (torch.Tensor): Tensor with dtype matching `dtype` argument, e.g. (`torch.uint8`), it is a per channel quantized Tensor if combined with quantization parameter in the argument of this function (scales/zero_points/axis) scales (torch.Tensor): a list of scale quantization parameter for affine quantization, one per channel zero_points (torch.Tensor): a list of zero_point quantization parameter for affine quantization, one per channel quant_min (int): minimum quantized value for output Tensor (not used in computation, reserved for pattern matching) quant_max (int): maximum quantized value for output Tensor (not used in computation, reserved for pattern matching) dtype (torch.dtype): requested dtype for output Tensor (not used in computation, reserved for pattern matching) out_dtype (torch.dtype?): optional dtype for output Tensor Returns: dequantized float32 Tensor rR, but got dtype: r) rr r r,rrrrrr+rrn) rrrrrrrrNrrrrs &&&&&&&$ rrrsP {{e-eW4Eekk] S  MM  yy{6uyy{mDEE e<4UAEeiik!I<<?IaL [[ #F'' 22f<n && C ++e-. /C JrcV^8dQhR\PR\PR\PR,R\R\R\R\PR \PR,R \P/ #rr8)r$s"rr%r%s44 <<4 LL4$4  4  4  4 ;;4{{T!4 \\4rc(VPV8wd\RV RVP 24hVf\PpW0P 48d\RVP 4 24h\ WEV4\P !WR7#)zExpecting input to have dtype rrr3)rr r r,rrr4)rrrrrrrrNs&&&&&&&$rdequantize_per_channel_metars {{e,UG3DU[[M R  MM  yy{6uyy{mDEE e<   E 33rzLchoose_qparams_per_token(Tensor input, ScalarType dtype) -> (Tensor, Tensor)choose_qparams_per_tokencV^8dQhR\PR\PR\\P\P3,/#rrrrr r!rrn)r$s"rr%r% s@   <<  ;;  5<< %& rcVP4PRRR7pVP\P8XdVP 4pV\P 8Xd^p^V^, ,^, pM\RV 24hVPRR7PV4p\P!V4pW%3#)Choose quantization parameters for per token quantization. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): original float32/float16 Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor Returns: scales and zero_points, both float32 Tensors Trkeepdimz/unsupported dtype in choose_qparams_per_token: gh㈵>min) absamaxrr r)r"int8 Exceptionr-div zeros_like)rrrn_bitsrrs&& rrrs,YY[  "d  3F ||u}}$ LLN   &1*%) =eW E  \\d\ # ' ' 2F""6*K  rcV^8dQhR\PR\PR\\P\P3,/#rr)r$s"rr%r%2@ << ;; 5<< %&rc\VPRR4^.,p\P!V\PVP R7\P!V\P VP R73#Nrrrrr rrrrrrsizes&& rchoose_qparams_per_token_metar-]  CR !QC 'D ;;t5<< Eu{{ EKK H rz]_choose_qparams_per_token_asymmetric_impl(Tensor input, ScalarType dtype) -> (Tensor, Tensor))_choose_qparams_per_token_asymmetric_implCompositeImplicitAutogradcV^8dQhR\PR\PR\\P\P3,/#rr)r$s"rr%r%Fs@(?(? <<(? ;;(? 5<< %&(?rc4R^r2\P!VRRR7p\P!VRRR7p\P!V\P!V44p\P !V\P!V44p\P !\P4PpWv, \W2, 4, p V PVR7p Wi, p Wy, p W*,p W;,p \P!W,^8W*, W;, 4p\P!WV4P4pV P\P4VP\P43#)rTrrir)r aminrrrmaxfinfor,rlr"r-wherer.r+float64r)rrrjrkryrz min_val_neg max_val_posrlr descaled_min descaled_maxzero_point_from_min_errorzero_point_from_max_errorrs&& rrrAs,,s$jjB5GjjB5G))GU%5%5g%>?K))GU%5%5g%>?K ++emm $ ( (C &% *< >>rzWchoose_qparams_per_token_asymmetric(Tensor input, ScalarType dtype) -> (Tensor, Tensor)#choose_qparams_per_token_asymmetriccV^8dQhR\PR\PR\\P\P3,/#rr)r$s"rr%r%{sECC <<C ;;C 5<< %&Crc\W4#rK)rrrs&&rrrvs 5U BBrcV^8dQhR\PR\PR\\P\P3,/#rr)r$s"rr%r%rrc\VPRR4^.,p\P!V\PVP R7\P!V\P VP R73#rrrs&& r(choose_qparams_per_token_asymmetric_metarrrc2\P!\VP44RR4pW1P 48wd \ RV RVP4 24hW2P 48wd \ RV RVP4 24hR#)Nz num_tokens: z scales: z zero_points: r)mathprodrrr>r )rrr num_tokenss&&& r!_per_token_quant_qparam_dim_checkrs4 -cr23J\\^#|J Tensorquantize_per_tokenc V^8dQhR\PR\PR\PR\R\R\P/#rrrrrrrr8)r$s"rr%r%sT## <<# LL## #  # ;; #rc\W4V4\WV4VPRV, 4PV4P 4P W44P V4pV#)aPer token quantization for the Tensor using the quantization parameters to map from floating point to quantized values. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (float32 torch.Tensor): quantization parameter for per token affine quantization zero_points (int32 torch.Tensor): quantization parameter for per token affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead r()rrmuladdr.r-r+rrrrrrs&&&&&&rrrsX6 e<%e[A #, [   y $ E   Lrc V^8dQhR\PR\PR\PR\R\R\P/#rr8)r$s"rr%r%sT 0 0 << 0 LL 0 0 0  0 ;; 0rcH\W4V4\P!WR7#r3rr r4rs&&&&&&rquantize_per_token_metars e<   E //rzdequantize_per_token(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype, ScalarType output_dtype) -> Tensordequantize_per_tokencV^8dQhR\PR\PR\PR\R\R\PR\P/#rrrrrrr output_dtyper8)r$s"rr%r%sb"" <<" LL"" "  " ;; "++"rcDW, pW,pVPV4#)aPer token dequantization for the Tensor using the quantization parameters to map from floating point to quantized values. This means for a N dimension Tensor (M1, M2, ...Mn, N), we calculate scales/zero_points for each N elements and quantize every N elements with the same quantization parameter. The dimension for scales/zero_points will be (M1 * M2 ... * Mn) Args: input (torch.Tensor): quantized Tensor (uint8, int8 etc.) scales (float64 torch.Tensor): quantization parameter for per token affine quantization zero_points (int64 torch.Tensor): quantization parameter for per token affine quantization quant_min (int): minimum quantized value for input Tensor quant_max (int): maximum quantized value for input Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor output_dtype (torch.dtype): dtype (e.g. torch.float32) for output Tensor Returns: dequantized Tensor with dtype `output_dtype` )r+rrrrrrrs&&&&&&&rrrs"8  E NE 88L !!rcV^8dQhR\PR\PR\PR\R\R\PR\P/#rr8)r$s"rr%r%sb 7 7 << 7 LL 7 7 7  7 ;; 7++ 7rcH\W4V4\P!WR7#rrrs&&&&&&&rdequantize_per_token_metars e<   E 66rzquantize_per_channel_group(Tensor input, Tensor scales, Tensor zero_points, int quant_min, int quant_max, ScalarType dtype, int group_size) -> Tensorquantize_per_channel_groupc V^8dQhR\PR\PR\PR\R\R\P/#rr8)r$s"rr%r%sT%% <<% LL%% %  % ;; %rcV^8:d \R4hW`PR,8d,VPR,^8XdVPR,pVPR,V,^8wd \R4hVP4^8wd \R4hVPRV4p\P !V4P 4^8wd \R4hVPR^4pVPR^4pVPRV, 4PV4P4PW44PV4PV4pV#)rCgroup_size must be > 1/input.shape[-1] must be divisible by group_sizeinput must be 2-dimensionalzto_quant must not contain NaNsr(r) r rrreshaper isnansumrrr.clamp_r+ reshape_as) rrrrrr group_sizeto_quant input_int8s &&&&&&& rrrs$Q566KKO# R(8A(=[[_  {{2#q(NOO yy{a:;;}}R,H {{8  "a'=>> ^^B "F%%b!,K  S6\" [    % E E   rc V^8dQhR\PR\PR\PR\R\R\P/#rr8)r$s"rr%r%?sT%0%0 <<%0 LL%0%0 %0  %0 ;; %0rclV^8:d \R4hW`PR,8d,VPR,^8XdVPR,pVPR,V,^8wd \R4hVP4^8wd \R4h\P!WR7#)aGroupwise quantization within each channel for a 2-d Tensor using the quantization parameters to map from floating point to quantized values. This means for each row of a 2-d Tensor (M, N), we calculate scales/zero_points for each `group_size` elements and quantize every `group_size` elements with the same quantization parameter. The dimension for scales/zero_points will be (M * ceil(N, group_size),) Args: input (torch.Tensor): original float32 or bfloat16 Tensor scales (float32 torch.Tensor): quantization parameter for per channel group affine quantization zero_points (int32 torch.Tensor): quantization parameter for per channel group affine quantization quant_min (int): minimum quantized value for output Tensor quant_max (int): maximum quantized value for output Tensor dtype (torch.dtype): requested dtype (e.g. torch.uint8) for output Tensor Returns: Tensor with requested dtype (e.g. torch.uint8), note the quantization parameters are not stored in the Tensor, we are storing them in function arguments instead rrrr3r)r rrr r4)rrrrrrrs&&&&&&&rquantize_per_channel_group_metar >s8Q566KKO# R(8A(=[[_  {{2#q(NOO yy{a:;;   E //rzdequantize_per_channel_group(Tensor input, Tensor scales, Tensor? zero_points, int quant_min, int quant_max, ScalarType dtype, int group_size, ScalarType output_dtype) -> Tensordequantize_per_channel_groupcV^8dQhR\PR\PR\PR,R\R\R\PR\R \P/#) rw_int8rrNrrrrrr8)r$s"rr%r%rsq.. LL. LL.$. .  . ;; ..++.rcV^8:d \R4hW`PR,8d,VPR,^8XdVPR,pVPR,V,^8wd \R4hVP4^8wd \R4hVPRV4pVPR^4pVeVPR^4p M1\P !.\P VPR7p VPV 4PV4PV4PV4p V #)aGroupwise dequantization within each channel for a 2-d Tensor using the quantization parameters to map from floating point to quantized values. This means for each row of a 2-d Tensor (M, N), we calculate scales/zero_points for each `group_size` elements and quantize every `group_size` elements with the same quantization parameter. The dimension for scales/zero_points will be (M * ceil(N, group_size),) Args: input (torch.Tensor): quantized Tensor (uint8/int8 etc.) scales (float32 torch.Tensor): quantization parameter for per channel group affine quantization zero_points (int32 torch.Tensor): quantization parameter for per channel group affine quantization quant_min (int): minimum quantized value for input Tensor quant_max (int): maximum quantized value for input Tensor dtype (torch.dtype): dtype (e.g. torch.uint8) for input Tensor output_dtype (torch.dtype): dtype (e.g. torch.float32) for output Tensor Returns: dequantized Tensor with dtype `output_dtype` rz0w_int8.shape[-1] must be divisible by group_sizezw_int8 must be 2-dimensionalrr) r rrrr zerosint32rsubrrr+) rrrrrrrrw_int8_groupedzpw_dqs &&&&&&&& rr r ms DQ566LL$$b)9Q)>\\"%  ||B*$)OPP zz|q;<<^^B 3N ^^B "F  Q ' [[5;;v}} E   b ! % %f - 8 8 @ C CL QD Krzyfake_quant_per_channel(Tensor input, Tensor scales, Tensor zero_points, int axis, int quant_min, int quant_max) -> Tensorc@a]tRtRto]R4t]R4tRtVtR#)FakeQuantPerChannelicnVP\P8wd VP\P4pVP\P8wd VP\P4pVP\P8wd\ RVP 24hWAP 48d\ RVP 4 24h\\V44\\V^,VP44,p\W'4p\W74p \P!VRV, ,4V ,p \P!WV4V , V,p \P!W8W8*4p VPV 4V #)r'rr()rr r,r+rr rrrndimrr.r- logical_andsave_for_backward) ctxrrrrrrbroadcast_dimsunsqueeze_scalesunsqueeze_zero_pointstemprmasks &&&&&&& rforwardFakeQuantPerChannel.forwardsC <<5== (YYu}}-F    +%..5K ;;%-- ' Nu{{m\  99;  #:599;-!HI IeDk*T%q%**2M-NN.vF 3K P{{5C*:$:; Tensorzconvert_element_type.no_fusecxV^8dQhR\PR\PR\P/#rrF)r$s"rr%r%s2FF FU[[FU\\Frch\PPPP W4#rK)r opsprimsconvert_element_typedefaultrs&&rrBrBs# 99?? / / 7 7 EErcxV^8dQhR\PR\PR\P/#rrF)r$s"rr%r%s-00U\\0%++0%,,0rc0\P!WR7#rr<rs&&rconvert_element_type_metarFs   E //r))Grr torch._refsrtorch.ao.quantization.utilsrr torch.libraryrrquantized_decomposed_libuint8ruint16int16r_INTEGER_DTYPES float8_e5m2 float8_e4m3fn _FLOAT_DTYPESiinforrr updater#rrdefinerr6r@rDrHrLrMrXr^rardrgr{rrrrrrrrrrrrrrrrr,rrrr r autogradFunctionrr2r=rBrF)ks0rrYs +M' ##95A;; ELL%++u{{S""E$7$78 :I9HA AEKKN..//DQRMqU[[^   #ekk!n&8&8"9::MR  $@  57RS"T"J 5v>0?0"@ :  =vFG,_  79TU0Q%)0QV0Qf 7@ 4%) 4A 4_ " %)  D >GQ%)QHQ>e # %)  D ?H %) I 7  79TU(V(V7 % (  (V 7@A( A6JK@  68ST.U.b 6?0@02_  8:UV=%)=W=@ 8&A4%)4B4.R    F    c / (?  (?V] ) C  C)     @  46QR#S#L 4f= 0> 0Y  68ST!& "U"B 6?!&  7@ 7A:G0H0E'Ss =?VA V