+ "ݳjW^RIHtHt^RIt^RIHuHt^RIH t H t ^RI H t H t HtHtHtHt^RIHtHt^RIHt^RIHt^RIHtHtHt^R IHt^R IH t ^R I!H"t"!R R ]P PF4t$R,RRllt%RRlt&!RR] 4t'!RR]'4t(!RR]$4t)RRlt*RRlt+RRlt,RRlt-!R R!]'4t.!R"R#]$4t/R$R%lt0R&R'lt1R(R)lt2!R*R+]'4t3R#)-)AnyOptionalN)PerGroupPerRow)Int8DynActInt4WeightLinearWeightOnlyInt4Linear_check_linear_int4_k_replace_linear_8da4w_replace_linear_int4 groupwise_affine_quantize_tensor) TorchAODTypeZeroPointDomain)get_group_qparams_symmetric) _is_device)FakeQuantizeConfigBaseFloat8FakeQuantizeConfigIntxFakeQuantizeConfig)FakeQuantizerBase)TwoStepQuantizer)_get_qmin_qmaxcaa]tRt^*toRtR V3RlV3RllltV3RlRltV3RlRlt]R V3RlR ll4t R t Vt V;t #) FakeQuantizedLinearaY General linear layer with fake quantized weights and activations. Specific target dtypes, granularity, schemes etc. are specified through separate configs for weights and activations. Example usage:: activation_config = IntxFakeQuantizeConfig( dtype=torch.int8, granularity="per_token", is_symmetric=False, ) weight_config = IntxFakeQuantizeConfig( dtype=torch.int4, group_size=8, is_symmetric=True, ) fq_linear = FakeQuantizedLinear( 16, 32, False, activation_config, weight_config, ) fq_linear(torch.randn(16)) c \<V^8dQhRS[RS[RS[RS[S[,RS[S[,RR/#) in_features out_featuresbiasactivation_config weight_configreturnN)intboolrr)format __classdict__s"`/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torchao/quantization/qat/linear.py __annotate__ FakeQuantizedLinear.__annotate__CsS'.'.'.'. '. $$:; '.  67 '. '.c<\S V`!VVV.VO5/VB\PP R4Ve\ P !V4VnMRVnVe\V\4'dR\VP\4'd2VPpVe"W,^8wd\RV: RV: R24h\ P !V4VnR#RVnR#)z,torchao.quantization.qat.FakeQuantizedLinearNz in_features (z) % group_size (z) must be == 0)super__init__torch_C_log_api_usage_oncer from_configactivation_fake_quantizer isinstancer granularityr group_size ValueErrorweight_fake_quantizer) selfrrrrrargskwargsr3 __class__s &&&&&&*, r%r+FakeQuantizedLinear.__init__Cs          $$%ST  (->-J-J!.D *.2D *  $-)?@@Z))8FF+55 )k.F!.K$& 4*;)F)F})UD &)-D &r(cN<V^8dQhRS[PRS[P/#)rxr )r,Tensor)r#r$s"r%r&r'ls#)))%,,)r(cVPeVPV4pVPeVPVP4pM VPp\P!WVP 4#N)r0r5weightFlinearr)r6r<ws&& r%forwardFakeQuantizedLinear.forwardls\  ) ) 5..q1A  % % 1**4;;7A Axxdii((r(cH<V^8dQhRS[PP/#rr )r,nnLinear)r#r$s"r%r&r'us588??r(c\PPVPVPVP RJVP PVP PR7pVP P\P!R48wd#VP VnVP VnV#)Ndevicedtypemeta) r,rHrIrrrr@rLrM)r6 new_linears& r% to_linearFakeQuantizedLinear.to_linearusXX__       IIT !;;%%++## %  ;;  f!5 5 $ J "iiJOr(ct<V^8dQhRS[PPRS[S[,RS[S[,/#)rmodrr)r,rHrIrr)r#r$s"r%r&r's= XX__$$:; 67 r(c \\VPVPVPRJVVVPP VPP R7pVPP \P !R48wd#VPVnVPVnV#)NrrrLrMrN)rrrrr@rLrMr,)clsrSrrrOs&&&& r% from_linearFakeQuantizedLinear.from_linears) OO    HHD /'::$$**""  ::   V 4 4 # J !hhJOr()r0r5)FNN)NN) __name__ __module__ __qualname____firstlineno____doc__r+rDrP classmethodrW__static_attributes____classdictcell__ __classcell__r9r$s@@r%rr*sG0'.'.R)) r(rcXV^8dQhR\PPR\/#)rrSenabled)r,rHModuler")r#s"r%r&r&s$ 8 8  8  8r(c\V\4'dAVPeWPnVPeWPnR#R#R#)zG Helper function to enable fake quantization in `FakeQuantizedLinear`. N)r1rr0rdr5)rSrds&&r%enable_linear_fake_quantrgsL#*++  ( ( 44; ) ) 1  $ $ 007 % % - 1,r(cLV^8dQhR\PP/#rrSr,rHre)r#s"r%r&r&s11588??1r(c \VRR7R#)zH Helper function to disable fake quantization in `FakeQuantizedLinear`. F)rdN)rgrSs&r%disable_linear_fake_quantrmsS%0r(cHa]tRt^toRtV3RlRltV3RlRltRtVtR#)_LegacyQATQuantizerzE Base class for sharing common methods across legacy QAT quantizers. c0<V^8dQhRS[S[,/#rGrr)r#r$s"r%r& _LegacyQATQuantizer.__annotate__sX>T5Ur(cR#r?r6s&r%#get_activation_fake_quantize_config7_LegacyQATQuantizer.get_activation_fake_quantize_configr(c0<V^8dQhRS[S[,/#rGrq)r#r$s"r%r&rrs:P1Qr(cR#r?rtrus&r%get_weight_fake_quantize_config3_LegacyQATQuantizer.get_weight_fake_quantize_configrxr(rtN) rYrZr[r\r]rvr{r_r`r$s@r%rorosr(rocaa]tRt^toRtRR]P ]P 3V3RlV3RllltV3RlRltV3RlR lt V3R lR lt V3R lR lt V3RlRlt Rt VtV;t#) Int8DynActInt4WeightQATQuantizerz Quantizer for performing QAT on a model, where linear layers have int8 dynamic per token fake quantized activations and int4 fake quantized grouped per channel weights. Fc ^<V^8dQhRS[RS[RS[PRS[PRR/#)r groupsizepadding_allowed precisionscales_precisionr N)r!r"r,rM)r#r$s"r%r&-Int8DynActInt4WeightQATQuantizer.__annotate__sE9999;; 9  ++ 9  9r(c<\SV`4\PP R4WnW nW0nW@n\PVn R#)z9torchao.quantization.qat.Int8DynActInt4WeightQATQuantizerN) r*r+r,r-r.rrrrfloat32activation_scales_precision)r6rrrrr9s&&&&&r%r+)Int8DynActInt4WeightQATQuantizer.__init__sI  $$ G (%4&/-=+0==(r(c<V^8dQhRS[PPRS[RS[RS[PP/#rmodelr7r8r r,rHrer)r#r$s"r%r&rs=  XX__ -0 ??44;;#=%%&&JJd*$//#ll00%+%;%; $ &67-e4 5LL%%$44  UU6667TLLJJ%%+3'*+')+&::),1JJ$)*..u5U3r(c0<V^8dQhRS[S[,/#rGrq)r#r$s"r%r&r&sNNX>T5UNr(c,\VP4#r?)_get_8da4w_activation_configrrus&r%rvDInt8DynActInt4WeightQATQuantizer.get_activation_fake_quantize_config&s+D,L,LMMr(c0<V^8dQhRS[S[,/#rGrq)r#r$s"r%r&r)sOO:P1QOr(cB\VPVP4#r?)_get_8da4w_weight_configrrrus&r%r{@Int8DynActInt4WeightQATQuantizer.get_weight_fake_quantize_config)s'8M8MNNr()rrrrr)rYrZr[r\r]r,rr+rrrrvr{r_r`rarbs@@r%rrsg %!&(- 99$   .6.6`NNOOOr(rcaa]tRtRtoRtRRR]P ]P 3V3RlV3RllltR V3RlR lltR t R t Vt V;t #) ri-a, This module implements a linear layer with int8 dynamic per token fake quantized activations with int4 fake quantized grouped per channel weights. args: groupsize: the number of elements in each quantized group for weights precision: precision of weights scales_precision: precision of per group scales and zero points Note: we hardcode activation scales to use torch.fp32, but allow users to specify the weight scales (defaults to torch.fp32). Here scales_precision refers specifically to the weight scales only, not the activation scales. FNrc<V^8dQhRS[RS[RS[RS[PRS[RS[PRS[PRR /#) rrrrrLrrrr Nr!r"r,rLrM)r#r$s"r%r&*Int8DynActInt4WeightQATLinear.__annotate__;sg          ;;  ++   r(c |<\\P4p\WW4p \S V`VVVVV VVR7R#)rKN)rr,rrr*r+) r6rrrrLrrrrrr9s &&&&&&&& r%r+&Int8DynActInt4WeightQATLinear.__init__;sE9G0M         r(c <V^8dQhRS[/#rrdr")r#r$s"r%r&rS555r(cFWPnWPnR#r?r0rdr5r6rds&&r%enable_fake_quant/Int8DynActInt4WeightQATLinear.enable_fake_quantS18&&.-4""*r(c(VPR4R#FNrrus&r%disable_fake_quant0Int8DynActInt4WeightQATLinear.disable_fake_quantW u%r(rtT) rYrZr[r\r]r,rr+rrr_r`rarbs@@r%rr-sA "#!&(-   055&&r(rcLV^8dQhR\PP/#rirj)r#s"r%r&r&\s   r(cV\V\4'dVP4R#R#)zL (deprecated) Enable fake quantization for `Int8DynActInt4WeightQATLinear`. N)r1rrrls&r%enable_8da4w_fake_quantr\s##455 6r(cLV^8dQhR\PP/#rirj)r#s"r%r&r&es!!%((//!r(cV\V\4'dVP4R#R#)zM (deprecated) Disable fake quantization for `Int8DynActInt4WeightQATLinear`. N)r1rrrls&r%disable_8da4w_fake_quantres##455  6r(cDV^8dQhR\PR\/#)rqparams_precisionr )r,rMr)r#s"r%r&r&ms {{r(c V\P8XgQh\\PRRRVV\P!V4P R7#)zX Return the activation `IntxFakeQuantizeConfig` for `Int8DynActInt4WeightQATQuantizer`. per_tokenFT)rMr2 is_symmetric is_dynamicrreps)r,rrrfinfor)rs&r%rrmsL  -- - !jj). KK) * . . r(cPV^8dQhR\R\PR\/#rr3rr r!r,rMr)r#s"r%r&r&s*{{r(c @\\PVRRVVR7#)zT Return the weight `IntxFakeQuantizeConfig` for `Int8DynActInt4WeightQATQuantizer`. T)rMr3rrrr)rr INT4r3rs&&r%rrs) ").  r(caa]tRtRtoRtR^]P ]P 3V3RlV3RllltV3RlRltV3RlR lt V3R lR lt V3R lR lt Rt Vt V;t#)Int4WeightOnlyQATQuantizerizt Quantizer for performing QAT on a model, where linear layers have int4 fake quantized grouped per channel weights. rc n<V^8dQhRS[RS[S[,RS[PRS[PRR/#)rr inner_k_tilesrrr N)r!rr,rM)r#r$s"r%r&'Int4WeightOnlyQATQuantizer.__annotate__sI111 }1;; 1  ++ 1  1r(c<\SV`4\PP R4VR9gQhVR9gQhW nWnW0nW@nR#)z3torchao.quantization.qat.Int4WeightOnlyQATQuantizerN)r) @r) r*r+r,r-r.rrrr)r6rrrrr9s&&&&&r%r+#Int4WeightOnlyQATQuantizer.__init__sZ  $$ A  )))....*"" 0r(c<V^8dQhRS[PPRS[RS[RS[PP/#rr)r#r$s"r%r&rs=  XX__ -0 >1V3VV1#KMJJFG G*-iJ     "'  r(c <V^8dQhRS[/#rr)r#r$s"r%r&rrr(cFWPnWPnR#r?rrs&&r%r)Int4WeightOnlyQATLinear.enable_fake_quantrr(c(VPR4R#rrrus&r%r*Int4WeightOnlyQATLinear.disable_fake_quantrr()rr) rYrZr[r\r]r,r r+rrr_r`rarbs@@r%rrsD #!&(-  455&&r(rcLV^8dQhR\PP/#rirj)r#s"r%r&r&"s  ehhoo r(cV\V\4'dVP4R#R#)zF (deprecated) Enable fake quantization for `Int4WeightOnlyQATLinear`. N)r1rrrls&r%enable_4w_fake_quantr"s##.// 0r(cLV^8dQhR\PP/#rirj)r#s"r%r&r&+s!!uxx!r(cV\V\4'dVP4R#R#)zG (deprecated) Disable fake quantization for `Int4WeightOnlyQATLinear`. N)r1rrrls&r%disable_4w_fake_quantr+s##.//  0r(cPV^8dQhR\R\PR\/#rr)r#s"r%r&r&3s*{{r(c ^\\PVRRVV\PR7#)zN Return the weight `IntxFakeQuantizeConfig` for `Int4WeightOnlyQATQuantizer`. FT)rMr3rrrrzero_point_domain)rr,uint4rFLOATrs&&r%r r 3s0 "kk).)// r(ca]tRtRtoRt^@]P 3V3RlRlltV3RlRltV3RlRlt V3R lR lt V3R lR lt R t Vt R#)Float8ActInt4WeightQATQuantizeriJa QAT quantizer for applying dynamic rowwise float8 activation + int4 per group/channel symmetric weight fake quantization to linear layers in the model. Currently only supports rowwise granularity for float8 activations. args: group_size (Optional[int]): the number of elements in each quantized group for weights, defaults to 64. Use None for per channel. scale_precision: precision of weight scales, defaults to torch.bfloat16. cJ<V^8dQhRS[S[,RS[P/#)rr3r)rr!r,rM)r#r$s"r%r&,Float8ActInt4WeightQATQuantizer.__annotate__Ws'  SM  r(c \PPR4VeRpMRp\\P\ 4R7Vn\\PVVRRVR7Vn R#)z8torchao.quantization.qat.Float8ActInt4WeightQATQuantizerN per_group per_channel)rMr2T)rMr2r3rrr) r,r-r.r float8_e4m3fnr_activation_configrint4_weight_config)r6r3rweight_granularitys&&& r%r+(Float8ActInt4WeightQATQuantizer.__init__Wsm $$ F   !!, !. ":%%# 5***!+  r(c<V^8dQhRS[PPRS[RS[RS[PP/#rr)r#r$s"r%r&r%ps=XX__-0T5URr(c\R4h)z,Float8 FakeQuantizeConfig does not exist yetr4rus&r%rvCFloat8ActInt4WeightQATQuantizer.get_activation_fake_quantize_configs!"PQQr(c0<V^8dQhRS[S[,/#rGrq)r#r$s"r%r&r%s"":P1Q"r(cVP#r?)rrus&r%r{?Float8ActInt4WeightQATQuantizer.get_weight_fake_quantize_configs!!!r()r*r,N)rYrZr[r\r]r,r r+rrrvr{r_r`r}s@r%r#r#JsJ %'',~~  2("" RR""r(r#r)4typingrrr,torch.nn.functionalrH functionalrA torchao.quantization.granularityrr)torchao.quantization.linear_quant_modulesrrr r r r %torchao.quantization.quant_primitivesr rtorchao.quantization.utilsr torchao.utilsrfake_quantize_configrrrfake_quantizerrtwo_step_quantizerrutilsrrIrrgrmrorrrrrrrrrrr r#rtr(r%rIs! =C$ 1 q%((//qh 81 * "aO':aOH+&$7+&^ !&,ZL!4ZLz+&1+&^ !.C"&9C"r(