+ "ݳj8^RIt^RIHt^RIHt^RIHtHtHtH t ^RI t ^RI H t ^RI Ht^RIHt^RIHt^R IHtHt^R IHt^R IHtHtHtHt^R IHt^R IH t ^RI!H"t"^RI#H$t$] PLPORR4t(]"!](4t)R.t*Rt+Rt,Rt-Rt.Rt/])RRl4t0Rt1RRlt2RRlt3Rt4R t5R!t6R"t7R#t8R$t9R%t:R&t;R'tR*t?R+t@R,tAR-tBR.tCR/R0ltD]!R1R244tER3R4ltFR# ]PJt$L;i)5N) dataclass)partial)AnyCallableListOptional) out_dtype)quantized_decomposed_lib GraphModule) InternalMatch)ReplacedPatternsreplace_pattern_with_filters) WrapperModule)"_get_aten_graph_module_for_pattern,_replace_literals_with_existing_placeholders'_replace_literals_with_new_placeholders"remove_tensor_overload_for_qdq_ops) MappingType)_get_per_token_block_size)_register_custom_op)$_disable_aten_to_metadata_assertionstorchaoFRAGMENT reference_representation_rewritec\PPPWW#V\P4p\PPPVVVVV \P4p\PP P PVVV 4p\PPPVWW\P4pV#N) torchopsquantized_decomposeddequantize_per_tensorint8atenlineardefaultquantize_per_tensor)x_i8x_scale x_zero_point x_quant_min x_quant_max weight_i8 weight_scaleweight_zero_pointweight_quant_minweight_quant_max bias_fp32 out_scaleout_zero_point out_quant_min out_quant_maxx_fp32 weight_fp32out_fp32out_i8s&&&&&&&&&&&&&&& {/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torchao/quantization/pt2e/reference_representation_rewrite.py_qdq_quantized_linearr;,s"YY + + A A |+uzzF))00FF Kyy~~$$,,V[)LH YY + + ? ?)]5::F Mc\PPPWV4p\PPPWXV 4pVP \P 4pVP \P 4p\ \PPPP\PW, VV, R4pW,p\ \PPPP\PV V4pVV,p\ \PPPP\PVW,V , 4V ,p\PPPVW4P \P4pV#r)rrr#clamptoint16r r$r%int32divTensormulr")r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5x_i16 weight_i16acc_i32 bias_scalebias_i32r9s&&&&&&&&&&&&&&& r:_reference_quantized_linearrJOsJ( 99>>  ; ?D $$YBRSI GGEKK Eekk*J %%  && G'J++22EKKJWH G  IINN   % % KK   "Y .     YY^^ ! !'= H K KEJJ WF Mr<c D\PPPWW#\P4wr\PPP W WV\P4p \PPP WWV\P4p\PPP VVVVV\P4p \PPPPW V 4pV#r) rrr choose_qparamsr"r&r!r#r$r%)r6r*r+x_epsr,r-r.r/r0r1r(r)r'r7r8s&&&&&&&&&& r:_qdq_dynamic_quantized_linearrNs"II::II[G 99 ) ) = =K DYY + + A A |+uzzF))00FF Kyy~~$$,,V)LH Or<c \PPPWW#\P4wrW , p\P !V4pVP \PR7p W,p \P!WV4p V P \PR7p \PPPWGV4pV P \P4pVP \P4p\\PPPP\PW, W, R4pW,p\\PPPP\PV V4pVV,pVW,,pV#)dtypeN)rrr rLr"roundr?rAr>r#r@r r$r%rBrC)r6r*r+rMr,r-r.r/r0r1r(r)x_i32r'rErFrGrHrIr8s&&&&&&&&&& r:#_reference_dynamic_quantized_linearrTsL"II::II[G  F[[ F IIEKKI (E  E KKK 8E 88%**8 %D $$YBRSI GGEKK Eekk*J %%  & G'J++22EKKJWH G'01H Or<c \Pp\V4pRp ^p \PP P VVPV\PV V V\P\P4 wr\PP PVVV V \PV V 4p \PP PV VV V \PV V \P4pV^8gQR4hVP^,V,^8XgQR4hVP4^8XgQR4h^V3p\PP PVVVV\PR^4p\PPP P#WV4pV#)zGroup size must be positivez&Weight must be divisible by group_sizezWeight must be 2D tensor)r ASYMMETRICrrrrchoose_qparams_affinenamer"float32rAquantize_affinedequantize_affineshapedimr#r$r%)r6rM weight_i4r-r.r1 group_sizex_mapping_typeper_token_block_sizer*r+r(r)r' block_sizer7r8s&&&&&&& r:,_qdq_dynamic_quantized_linear_4bit_groupwiserfs!++N4V<KK!II--CC     G 99   , ,  DYY   0 0    F >888> ??1  *a /0 / ==?a ;!;; ZJ))##55   Kyy~~$$,,V)LH Or<cV^8dQhR\PR\R\PR\PR\PR\\P,R\\ ,R\P/#) r6rMrar-r.r1rbreturn)rrCfloatrrint)formats"r: __annotate__rms~$~$ LL~$ ~$||~$,, ~$ || ~$  % ~$S ~$ \\~$r<c 2V^,p\Pp\V4pRp ^p \PP P VVPV\PV V V\P\P4 wr\PP PVVV V \PV V 4p VPwrW,pVP\P4P\P4pV PpV PR,pV PRV4p V P^,pV PVVV4pVPVVV4pVP\P4pVP\P4p\P !VV\PVP"R7p\%V4pVVR&VP&^8XdVP)^4p\+V4EF%pVRVR3,pVRVR3,pVP-RR7pVRV3,p\/\PP0P2P4\PVVR4pVVP\P4VP^R4,,pVP\P4P^R4V PR^4,VP^R4,p VV , pEK( V PR^4p!VV!,p"Ve V"V,p"V"PV4#)av Reference implementation for dynamically quantized linear 4-bit groupwise operation. This implementation emulates actual numerics of on-device integer compute. Args: x_fp32: Input activation tensor in fp32 x_eps: Epsilon for quantization parameter computation weight_i4: 4-bit quantized weight (stored as int8 with values in [-8, 7]) weight_scale: Groupwise scales for weight dequantization weight_zero_point: Groupwise zero points for weight (unused for symmetric) bias_fp32: Optional bias tensor in fp32 group_size: Size of each group for groupwise quantization Returns: Output tensor in fp32 )rQdevice:NNN)r`NrW)rrYrrrrrZr[r"r\rAr]r_r?bfloat16viewzerosrolistndim unsqueezerangesumr r#r$r%)#r6rMrar-r.r1rbrcrdr*r+r(r)r' out_features in_features num_groups x_orig_shapek_dim batch_size x_i8_groupedweight_i4_grouped x_i32_groupedweight_i32_groupedacc_fp32 out_shape group_idxx_group weight_groupweight_group_col_sumweight_scale_group group_accweights_col_sum_adjustedx_scale_multiplierr8s#&&&&&&& r:_reference_dqlinear_int4rs6AJ ++N4V<KK!II--CC     G 99   , ,  D!*L*J ??5>>255emmDL::L JJrNE 99R DAJ99ZZ@L"|ZL!OOEKK0M*--ekk:{{L fmmH\"I IbMA#--a0 :& 9a0)!Y/: +//B/7*!Y,7 IINN ! ! ) ) KK     ill5==9A& ' %%a, - ! 66C'D!b!,,,Hi' == ##r<c d\PPPVVVVVV^V34#)z Reference implementation for dynamically quantized linear 4-bit groupwise operation. This function now delegates to the custom op implementation. )rrrreference_dqlinear_int4)r6rMrar-r.r1rbs&&&&&&&r:2_reference_dynamic_quantized_linear_4bit_groupwisers: 99   4 4  J r<c$V^8dQhR\/#)rhri)bool)rls"r:rmrms00 0r<c@RpRpVPP4EFpp\V\PP 4'dVP R8XdzVP\PPPP8Xd=VPp\V4^8d!V^,R8H;'d V^,^8Hp\V\PP 4'gKVP R8XgKVP\PPPP8XgEK7VPp\V4^8gEKVV^,\P8HpEKs T;'dT#)F call_functionrX) nodes_mapvalues isinstancerfxNodeoptargetrrr^r%argslenr]r")matchoriginal_graph pattern_graphweight_is_int4act_quant_is_int8noders&&& r:6_filter_fn_for_dynamic_quantized_linear_4bit_groupwisers N&&( tUXX]] + +?* uyy00BBJJJ99D4yA~!%aB!?!?47a< tUXX]] + +?* uyy00@@HHH99D4yA~$(Guzz$9!!)"  / ///r<c$V^8dQhR\/#)rhreplacement_pattern)r)rls"r:rmrms  ) r<c^RIHpHpRpRpVPP 4FFwrVVP \ PPPP8XgKDTpM \VP4^8dVPR,pVeJVeDVPP4Vn V!VRVP4.VPR&R#R#R#)z Port metadata for dynamically quantized linear 4-bit groupwise operation. It custom_op node's metadata with corresponding linear node's metadata. ) NodeSourceNodeSourceActionN#ReplaceInt4DynamicQuantWithCustomOp from_noderp)torch.fx.tracebackrrritemsrrrr#r$r%r replacementsmetacopyREPLACE)rrr linear_nodeint4_custom_op_node_g_ns& r::_port_metadata_for_dynamic_quantized_linear_4bit_groupwisers@K%//557 ::..66 6K 8  + +,q01>>rB#6#B#.#3#3#8#8#: 5 (( 1   -$Cr<c ^^.p^^.p^^.pRp^^.p^p\PPPWW#V\P4p\PPPVVVVV \P4p\PP P PVVV VVVVVV4 p\PPPVWW\P4pV#F) rrr r!r"r# convolutionr%r&)r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5stridepaddingdilation transposedoutput_paddinggroupsr6r7r8r9s&&&&&&&&&&&&&&& r:_qdq_quantized_conv2drs"VF!fG1vHJVN F YY + + A A |+uzzF))00FF Kyy~~))11 HYY + + ? ?)]5::F Mr<c H^^.p^^.p^^.pRp^^.p^p\PPPWV4p\PPPWXV 4pVP \P 4pVP \P 4p\ \PPPP\PVV, VV, RVVVVVV4 pW,p\ \PPPP\PV V4pVPR4pVPR4pVV,p\ \PPPP\PVW,V , 4V ,p\PPPVW4P \P4pV#)rFNrp)rrr#r>r?r@r rr%rArBrCrvrDr")r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5rrrrrrrErFrGrHrIr9s&&&&&&&&&&&&&&& r:_reference_quantized_conv2drs"VF!fG1vHJVN F 99>>  ; ?D $$YBRSI GGEKK Eekk*J ""**  &&  G'J++22EKKJWH!!"%H!!"%H G  IINN   % % KK   "Y .     YY^^ ! !'= H K KEJJ WF Mr<c \PPPWW(V \P4p \PPPW4WXV \P4p W,p \PP P V 4p \PPPWWxV \P4p V #r)rrr r!r"r#relur&r'r(r)y_i8y_scale y_zero_pointr2r3 quant_min quant_maxr6y_fp32r8r9s&&&&&&&&&& r:_qdq_quantized_add_relurksYY + + A A | 5::FYY + + A A | 5::FHyy~~""8,H YY + + ? ?^ 5::F Mr<c ~VP\P4p VP\P4p \\PP P P\PW, W, 4p \\PP P P\PW, WF, 4p W,V,p \PP PWV 4P\P4p V #)z See comments for `_reference_quantized_add` for more information on how to derive the formula for out_i8 based on x_i8 and y_i8 ) r?rrAr rr#rDrCr>r"r'r(r)rrrr2r3rrrSy_i32out_i32r9s&&&&&&&&&& r:_reference_quantized_add_relurs GGEKK E GGEKK E  !!      E  !!      E mn,G YY^^ ! !'9 E H H TF Mr<c r\PPPWW(V \P4p \PPPW4WXV \P4p W,p \PPP WWxV \P4p V #r)rrr r!r"r&rs&&&&&&&&&& r:_qdq_quantized_addrsYY + + A A | 5::FYY + + A A | 5::FH YY + + ? ?^ 5::F Mr<c NVP\P4p VP\P4p \P!W, W, ,4P\P4p \P!WF, W, ,4P\P4p W,V,p Rp^p \PP P WV 4P\P4p V #)a # How to Derive the formula for out_i8 based on x_i8 and y_i8 # (since quantized add takes x_i8, y_i8 and their quantization parameters, and produce an out_i8) # out_i8 is quantized output, we can write down the formula for it first: out_i8 = out_f32 / out_scale + out_zero_point (1) # then out_fp32 is computed from x_f32 + y_f32, and the x_fp32 and y_fp32 are the dequantized x_i8 and y_i8 out_f32 = x_f32 + y_f32 (2) x_fp32 = (x_i8 - x_zero_point) * x_scale (3) y_fp32 = (y_i8 - y_zero_point) * y_scale (4) # applying the above fomula to the out_i8 equation we can get the following: out_i8 = out_fp32 / out_scale + out_zero_point # (1) = (x_f32 + y_f32) / out_scale + out_zero_point # applying (2) to substitute out_fp32 with x_fp32 + y_fp32 = ((x_i8 - x_zero_point) * x_scale + (y_i8 - y_zero_point) * y_scale) / out_scale + out_zero_point # apply (3) and (4) rW)r?rrArRrr#r>r"rs&&&&&&&&&& r:_reference_quantized_addrs: GGEKK E GGEKK E KK,1EF G J J5;; WE KK,1EF G J J5;; WEmn,GII YY^^ ! !'i @ C CEJJ OF Mr<c r^p ^p ^p ^p Rp \PPPWW#V\P4p\PP P PWWW4wpp\PPPWWgV\P4pV#r) rrr r!r"r#max_pool2d_with_indicesr%r&)r'r(r)r*r+r2r3r4r5 kernel_sizerrr ceil_moder6r8rr9s&&&&&&&&& r:_qdq_quantized_max_pool2drsK FGHI YY + + A A |+uzzF))..88@@VhKHaYY + + ? ?^M5::F Mr<c ^p ^p ^p ^p Rp \P!WV4pVP\P4p\PP P PW, WWV 4wppWV, ,V,p\P!VWx4pVP\P4pV#r) rr>r?rArr#rr%r")r'r(r)r*r+r2r3r4r5rrrrrrSrrr8r9s&&&&&&&&& r:_reference_quantized_max_pool2drsK FGHI ;;t+ 6D GGEKK E77?? k7iJGQI-.?H{{8]BH [[ $F Mr<cz\PPPWW#V\P4pV#r)rrr r&r"r6scale zero_pointrrxs&&&&& r:_quantize_per_tensor_int8r$s0 &&::zi A Hr<cW, p\P!V4pVP\PR7pWR,p\P!WSV4pVP\P R7pV#rP)rrRr?rAr>r"rs&&&&& r:#_reference_quantize_per_tensor_int8r+s^ A  AA 5;;A A A),A 5::A Hr<cz\PPPWW#V\P4pV#r)rrr r!r")r'rrrrr6s&&&&& r:_dequantize_per_tensor_int8r;s0 YY + + A A ZIuzzF Mr<c\PPPWV4pVP \P 4V, V,P \P R7#r)rrr#r>r?r\)r'rrrrs&&&&&r:%_reference_dequantize_per_tensor_int8rBsO 99>>   ;DWWU]] #j 0E 9 = =EMM = RRr<c z\PPPWW#WE\P4pV#r)rrr quantize_per_channelr")r6scales zero_pointsch_axisrrr9s&&&&&& r:_quantize_per_channel_int8rOs2YY + + @ @ iEJJF Mr<cl\P!WR4p\PPP \P !W, 4P \P4V,WE4p\P!WcR4pVP \P4#rrp) r transposerr#r>rRr?rAr")r6rrrrrrs&&&&&& r:$_reference_quantize_per_channel_int8rXsu__Vb 1Fiinn"" FO$'' 4{BIGoog3G ::ejj !!r<c z\PPPWW#WE\P4pV#r)rrr dequantize_per_channelr")r'rrrrrr8s&&&&&& r:_dequantize_per_channel_int8rcs2yy--DD kI%**H Or<cL\PPPWV4p\P!WR4pVP \P 4pWb, P \P4V,p\P!WsR4pV#r)rrr#r>rr?rArj)r'rrrrrrSr8s&&&&&& r:&_reference_dequantize_per_channel_int8rmsq 99>>   ;D ??4" -D GGEKK E#'' 4v=Hx"5H Or<cLV^8dQhR\PP/#)rhgm)rrr )rls"r:rmrm{s0D0Dr<c ,\VR.^^R^^^/R7#)r)exclude_literalsliteral_to_ph_idxrprW)r)rs&r:'_replace_ph_qdq_per_channel_replacementr{s% 7 bTaD!S!5L r<cDa]tRtRtoRtRtRtRtRtRt V3Rlt Rt Vt R#) _RewriteInfoizData needed for rewrite, this includes example inputs, pattern and replacement functions and post transformation functions for the exported pattern and replacement GraphModule NFc<V^8dQh/S[S[R3,;R&S[;R&S[;R&S[S[S[.S[3,,;R&S[S[S[.S[3,,;R&S[S[S[RS[PPS[PP.S[ 3,,,;R&S[ ;R &S[S[R .R 3,,;R &#) rh.example_inputspattern replacementpattern_post_transreplacement_post_transr filter_fnignore_literalsrNport_metadata_fn) tuplerrrr rtrrGraphr)rl __classdict__s"r:rm_RewriteInfo.__annotate__s#s(O#!;-*D!EFM%X{m[.H%IJQ XGM NO  !!"x);(4\.\.4\V\\@4\\B44\V\\D4\\F44\V\\H4\\J4\L\L4\V \\N4\\P4\L\L4. p \SV4\U4;_uu_4V Fp V PVpV PXpV PZpV P\pV P^p\aW4p\SV4\aVV4p\SV4V'd V!V4pV'd V!V4pVPc4VPc4\eVVVV PfV PhR7pV Pj'gKVFpV PkV4K EK RRR4V# +'giT#;i)rVrP)rT)rr r )r) match_filtersr NrW)rh)rri)rrr)r rX)r)rrr)rrrp)6rrandintr"randnrjrsrktensorfinfor\epsrrrNrTrrrfrrrr;rJrrrrrrrrrrrrrrrrrrrrrrrrrr recompilerrr r )r _QUANTIZED_LINEAR_EXAMPLE_INPUTS(_DYNAMIC_QUANTIZED_LINEAR_EXAMPLE_INPUTS _QUANTIZED_CONV2d_EXAMPLE_INPUTS)_QUANTIZED_ADD_OR_ADD_RELU_EXAMPLE_INPUTS$_QUANTIZED_MAX_POOL2D_EXAMPLE_INPUTS(_QUANTIZE_PER_TENSOR_INT8_EXAMPLE_INPUTS*_DEQUANTIZE_PER_TENSOR_INT8_EXAMPLE_INPUTS)_QUANTIZE_PER_CHANNEL_INT8_EXAMPLE_INPUTS+_DEQUANTIZE_PER_CHANNEL_INT8_EXAMPLE_INPUTS9_DYNAMIC_QUANTIZED_LINEAR_4BIT_GROUPWISE_EXAMPLE_INPUTS_19_DYNAMIC_QUANTIZED_LINEAR_4BIT_GROUPWISE_EXAMPLE_INPUTS_2_REWRITE_INFO_LIST rewrite_inforrrrrmatchesms& r:rrs  dCuzz: AU[[) AUYY' dV599- cU%)), dCuzz: AU[[) AUYY' dV599- cU%)), AU[[) AU[[) AUYY' dV599- cU%)),($&  F%++.   EMM"&& dCuzz: AU[[) AUYY' dV599- cU%)), AU[[) 0,  dCUZZ@ AU[[) AUYY' dV599- cU%)), dCUZZ@ AU[[) AUYY' dV599- cU%)), AU[[) AU[[) AUYY' dV599- cU%)),($&  dCUZZ@ AU[[) AUYY' dCUZZ@ AU[[) AUYY' AU[[) AUYY' dV599- cU%)), 1-  dCUZZ@ AU[[) AUYY' dV599- cU%)), AU[[) AUYY' dV599- cU%)), ,(  Aq!Qekk2 AU[[) AUYY' dV599- cU%)), 0,  dCUZZ@ AU[[) AUYY' dV599- cU%)), 2.  Aq!Qekk2 AU[[) AUYY'   1-  dCUZZ@ AU[[) AUYY'   3/  G5;;/ EMM"&& b!WEJJ7 Aq , q    AU[[)  A=  Jekk2 EMM"&& b!WEJJ7 Aq , q    AU[[)  A=  4 7 8 = > <#'CEKK 4N4R4RTU"V  <#'CEKK 4N4R4RTU"V    E F G L M <KK .22AA#  <KK .22AA# NN W) ,  E F G L M <KK .22AA#  <KK .22AA# NN W) ,  , / 0 5 6 3 3    , / 0 5 6 ;rd S ;rd S    5 1 2 7 8  5 , - 2 3  0 3 4 9 : 3 3    4 3 4 = >  6 5 6 ? @  5 4 5 > ? 3 3    7 6 7 @ A 3 3  Uqf'u- - / /.L)88N"**G&22K!-!@!@ %1%H%H "8QG .w 7<^K /{ ;!,W5%4[A      ! ! #2*44 , < < G,,, A 11!4!5/ 0< L= 0 /< Ls"B BAFDAAFE2AFF AF, )G contextlib dataclassesr functoolsrtypingrrrrr!torch._higher_order_ops.out_dtyper $torch.ao.quantization.fx._decomposedr torch.fxr 6torch.fx.passes.utils.matcher_with_name_node_map_utilsr torch.fx.subgraph_rewriterrr&torchao.quantization.pt2e.export_utilsrtorchao.quantization.pt2e.utilsrrrr%torchao.quantization.quant_primitivesrtorchao.quantization.utilsr torchao.utilsrtorch._export.utilsr nullcontextlibraryLibrary quant_libregister_custom_op__all__r;rJrNrTrfrrrrrrrrrrrrrrrrrrrrrrrrr<r:rJsN!00 7I PU@ >@-BH MM ! !)Z 8 (3'  F2jB,^>B~$~$B006 80fOd4"J2&R88    S"   LL L&YiB+5+A+A(s &D D