+ "ݳj/G^RIt^RIt^RIHt^RIHt^RIHtHtH t ^RI t ^RI H t ^RI HtHt^RIHt^RIHtHtHtHt^R IHt^R IHt^R IHt!R R ]]4t]!RR] 44t]!]4RRl4t ]!RR] 44t!!RR]!4t"]!]!4RRl4t#]!RR] 44t$!RR]$4t%]!]$4RRl4t&!RR]4t'R R!lt(R#)"N) dataclass)Enum)AnyOptionalTuple) AOBaseConfig)_QUANTIZE_CONFIG_HANDLER register_quantize_module_handler)FakeQuantizedEmbedding)FakeQuantizeConfigFakeQuantizeConfigBaseIntxFakeQuantizeConfig_infer_fake_quantize_configs)FakeQuantizedLinear)TwoStepQuantizer_log_deprecation_warningc"]tRt^!tRtRtRtRtR#)QATStepzR Enum value for the `step` field in :class:`~torchao.quantization.qat.QATConfig`. prepareconvertN)__name__ __module__ __qualname____firstlineno____doc__PREPARECONVERT__static_attributes__r]/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torchao/quantization/qat/api.pyrr!sGGr!rcTa]tRt^*toRtR RR/V3RlRllltRtV3RltR tVt R#) QATConfiga Config for applying quantization-aware training (QAT) to a `torch.nn.Module`, to be used with :func:`~torchao.quantization.quant_api.quantize_`. This config has two steps, "prepare" and "convert". The prepare step applies "fake" quantization to the model and should be applied before training, while the convert step converts the model into an actual quantized model. Fake quantization here refers to simulating the quantization numerics (e.g. int4) using high precision arithmetic (e.g. bf16), with the goal of reducing eventual degradation from quantization. There are two ways to use this config. The first involves passing a base post-training quantization (PTQ) config, which we will use to automatically infer the corresponding fake quantization schemes to use in the prepare phase. In the convert phase, we will then apply the base PTQ config to the model. This will be the most common use case. Example usage:: from torchao.quantization import ( quantize_, Int4WeightOnlyConfig, ) from torchao.quantization.qat import QATConfig base_config = Int4WeightOnlyConfig(group_size=32) quantize_(model, QATConfig(base_config, step="prepare")) train_loop(model) quantize_(model, QATConfig(base_config, step="convert")) Currently only the following are supported as base configs: - :class:`~torchao.quantization.Int4WeightOnlyConfig` The second way to use this config involves specifying the fake quantization schemes directly. Users will pass in :class:`~torchao.quantization.qat.FakeQuantizeConfigBase` for weights and/or activations instead of the base PTQ config. This use case is mostly for experimentation, e.g. when the corresponding PTQ config does not exist yet. Example usage:: from torchao.quantization import quantize_ from torchao.quantization.qat import IntxFakeQuantizeConfig activation_config = IntxFakeQuantizeConfig( torch.int8, "per_token", is_symmetric=False, ) weight_config = IntxFakeQuantizeConfig( torch.int4, group_size=32, is_symmetric=True, ) qat_config = QATConfig( # must specify one of `base_config` or `weight_config` activation_config=act_config, weight_config=weight_config, step="prepare", ) quantize_(model, qat_config) Args: base_config (Optional[AOBaseConfig]): Base PTQ config to infer the fake quantization configs during the prepare phase, and to apply directly during the convert phase. activation_config (Optional[FakeQuantizeConfigBase]): Custom fake quantization config for input activations, always optional. Must be None if `base_config` is used. weight_config (Optional[FakeQuantizeConfigBase]): Custom fake quantization config for weights. Must be None if `base_config` is used. Keyword args: step (str): One of "prepare" or "convert", determines the QAT phase Raises: ValueError: If `base_config` and `activation_config` are both specified ValueError: If `base_config` and `weight_config` are both specified ValueError: If none of `base_config`, `activation_config`, or `weight_config` are specified ValueError: If either `activation_config` or `weight_config` is specified and `step` is "convert" ValueError: If `step` is not one of "prepare" or "convert" ValueError: If the config is applied on a module that is not a `torch.nn.Linear` or `torch.nn.Embedding`, or it is applied on `torch.nn.Embedding` with an activation config Nsteprcb<V^8dQhRS[S[,RS[S[,RS[S[,RS[/# base_configactivation_config weight_configr%rrr r)format __classdict__s"r" __annotate__QATConfig.__annotate__sD  l+ $$:;  67   r!cVWnW nW0nW@nVP 4R#N)r)r*r+r% __post_init__)selfr)r*r+r%s&&&&$r"__init__QATConfig.__init__s''!2*  r!c\PPR4VPP 4Vn\ Uu.uFqP NK ppVPV9d\RV 24hVPeVPe \R4hVPeVPe \R4hVP\ P8Xd>\VPVPVP34'g \R4hVP\ P8Xd(VPfVPe \R4h\VP\4'd5VPP P"p\V RV RV R 24hR#uupi) z"torchao.quantization.qat.QATConfigz`step` must be one of Nz9Cannot specify both `base_config` and `activation_config`z5Cannot specify both `base_config` and `weight_config`zWMust specify `base_config`, `activation_config`, or `weight_config` in the prepare stepzICannot specify `weight_config` or `activation_config` in the convert stepz~ was passed as `base_config`. Did you mean to do the following instead? qat_config = QATConfig( activation_config=z(...), weight_config=z$(...), step="prepare", ))torch_C_log_api_usage_oncer%lowerrvalue ValueErrorr)r*r+ranyr isinstancer __class__r)r4sall_step_values config_types& r"r3QATConfig.__post_init__s $$%IJIIOO% ,34Gq77G4 99O +5o5FGH H    'D,B,B,NK     'D,>,>,JTU U 99 '   t55t7I7I J1 1 i  99 '  " " .$2D2D2P[  d&&(> ? ?**44==K- --8M:))4 6  @-5sGcz<V^8dQh/S[S[,;R&S[S[,;R&S[S[,;R&S[;R&#r'r,)r-r.s"r"r/r0*sQn,''op 677qr233st Mur!)r*r)r%r+)NNN) rrrrrr5r3__annotate_func__r __classdictcell__r.s@r"r$r$*s-Sx  "  "Yr!r$cV^8dQhR\PPR\R\PP/#r(moduleconfigreturn)r8nnModuler$)r-s"r"r/r/s;ww HHOOw w XX__wr!c^RIHpHpHpHpVP pVP pV\P8XEdJVe\V4wrMVPpVPp \V\PP4'd\W4'd*Ve\W4'gQhVP!WV 4#\W4'd*Ve\W4'gQhVP!WV 4#\"P !WV 4#\V\PP$4'd&Ve \'R4h\(P*!W 4#\'R\-V4,4hV\P.8XgQRV,4hVPeQR4hVPeQR4h\V\"\(VV34'gV#/p Rp \1VR 4'd\VP2P4\64'd[VP2P4P8'd5VP2P:V R &VP2P<V R &R p \V\"W534'dVP?4pM6\V\(4'dVPA4pM\'R V R24hVekV 'dF\1VR4'd4\BPD!R4\FPH!V4pRVn%\L\-V4,!W3/V B#V#)ad During the prepare step, perform module swap to apply fake quantization. If the base PTQ config is specified, derive the fake quantization configs from it. During the convert step, first perform module swap to revert all fake quantized modules to the corresponding built-in `torch.nn.Module`s, then apply the base config directly to quantize the module. )MXFakeQuantizeConfigMXFakeQuantizedLinearNVFP4FakeQuantizeConfigNVFP4FakeQuantizedLinearN;Activation fake quantization is not supported for embedding-Module of type '%s' does not have QAT supportz!unexpected step '%s' in QATConfigzunexpected `activation_config`zunexpected `weight_config`Fweight_fake_quantizer custom_scalecustom_zero_pointTzEncountered unexpected module z, should never happenintx_choose_qparams_algorithmz'Disabling intx_choose_qparams_algorithm)'torchao.prototype.qatrQrRrSrTr)r%rrrr*r+r?r8rNLinear from_linearr Embeddingr=r from_embeddingtyperhasattrrWrLrrange_learningscale zero_point to_linear to_embeddingloggingdebugcopydeepcopyrZr ) rKrLrQrRrSrTr)r% act_configr+kwargshas_custom_scale_and_zero_points && r"_qat_config_transformrns2$$K ;;D w  "*F{*S 'Z11J"00M fehhoo . .-AA!)Z..0;; M@@!)Z..-88 +66  2 2 3 3% Q*88O O?$v,N  w&R(Kd(RR&''/Q1QQ/##+I-II+ #&%(    M*/' F3 4 4677>>@VWW,,33BBB%+%A%A%G%GF> "*0*F*F*Q*QF& '.2 +   "7 R  %%'F  6 7 7((*F08MN   ".7<44 GH"mmK8 <@ 9+D,=>'- Mr!c>a]tRtRtoRtRtRtRtV3RltRt Vt R#)#IntXQuantizationAwareTrainingConfigi6ay (Deprecated) Please use :class:`~torchao.quantization.qat.QATConfig` instead. Config for applying fake quantization to a `torch.nn.Module`. to be used with :func:`~torchao.quantization.quant_api.quantize_`. Example usage:: from torchao.quantization import quantize_ from torchao.quantization.qat import IntxFakeQuantizeConfig activation_config = IntxFakeQuantizeConfig( torch.int8, "per_token", is_symmetric=False, ) weight_config = IntxFakeQuantizeConfig( torch.int4, group_size=32, is_symmetric=True, ) quantize_( model, IntXQuantizationAwareTrainingConfig(activation_config, weight_config), ) Note: If the config is applied on a module that is not `torch.nn.Linear` or `torch.nn.Embedding`, or it is applied on `torch.nn.Embedding` with an activation config, then we will raise ValueError as these are not supported. Nc\V4R#r2rr4s&r"r31IntXQuantizationAwareTrainingConfig.__post_init__V  &r!cR<V^8dQh/S[S[,;R&S[S[,;R&#)r(r*r+)rr )r-r.s"r"r/0IntXQuantizationAwareTrainingConfig.__annotate__6s+: 67>;<23:=r!r) rrrrrr*r+r3rFr rGrHs@r"rprp6s$6;?6:M'Ar!rpc]tRtRtRtR#) intx_quantization_aware_trainingi[rNrrrrr rr!r"rxrx[r!rxcV^8dQhR\PPR\R\PP/#rJ)r8rNrOrp)r-s"r"r/r/`s;VV HHOOV /V XX__Vr!cTpVPpVPp\V\PP 4'd\ P!VVV4#\V\PP4'd&Ve \R4h\P!W$4#\R\V4,4h)NrUrV) r*r+r?r8rNr\rr]r^r=r r_r`)rKrLmodr*r+s&& r"+_intx_quantization_aware_training_transformr~_s C00((M#uxx''"..     C++ , ,  (M &44SHHH4PS9TUUr!c*a]tRtRtoRtRtRtVtR#)'FromIntXQuantizationAwareTrainingConfigixag (Deprecated) Please use :class:`~torchao.quantization.qat.QATConfig` instead. Config for converting a model with fake quantized modules, such as :func:`~torchao.quantization.qat.linear.FakeQuantizedLinear` and :func:`~torchao.quantization.qat.linear.FakeQuantizedEmbedding`, back to model with the original, corresponding modules without fake quantization. This should be used with :func:`~torchao.quantization.quant_api.quantize_`. Example usage:: from torchao.quantization import quantize_ quantize_( model_with_fake_quantized_linears, FromIntXQuantizationAwareTrainingConfig(), ) c\V4R#r2rrrs&r"r35FromIntXQuantizationAwareTrainingConfig.__post_init__rtr!rN)rrrrrr3r rGrHs@r"rrxs&''r!rc]tRtRtRtR#)%from_intx_quantization_aware_trainingirNryrr!r"rrrzr!rcV^8dQhR\PPR\R\PP/#)r(r}rLrM)r8rNrOr)r-s"r"r/r/s6     3  XX__ r!c\V\4'dVP4#\V\4'dVP 4#V#)z If the given module is a fake quantized module, return the original corresponding version of the module without fake quantization. )r?rrer rf)r}rLs&&r"0_from_intx_quantization_aware_training_transformrs@#*++}} C/ 0 0!! r!cZa]tRtRtoRtV3RlRltV3RlRltV3RlRltR tVt R #) ComposableQATQuantizeria* Composable quantizer that users can use to apply multiple QAT quantizers easily. Quantizers will be applied in the order they are specified in the constructor. Note: the quantizers provided must apply to different modules in the model, e.g. nn.Linear and nn.Embedding, otherwise the behavior will be undefined. Example usage:: my_quantizer = ComposableQATQuantizer([ QATQuantizer1(), QATQuantizer2(), QATQuantizer3(), ]) model = my_quantizer.prepare(model) train(model) model = my_quantizer.convert(model) c0<V^8dQhRS[S[,/#)r( quantizers)listr)r-r.s"r"r/#ComposableQATQuantizer.__annotate__s%%4(8#9%r!cP\PPR4WnR#)z/torchao.quantization.qat.ComposableQATQuantizerN)r8r9r:r)r4rs&&r"r5ComposableQATQuantizer.__init__s $$%VW$r!c<V^8dQhRS[PPRS[RS[RS[PP/#r(modelargsrlrMr8rNrOr)r-r.s"r"r/r=XX__-0.__annotate__s""EHHOO"r!c><\VS4'd RVnR#R#)TN)r? _initialized)rrs&r"_set_initialized4initialize_fake_quantizers.._set_initializeds a* + +!AN ,r!N)r8r9r:'torchao.quantization.qat.fake_quantizerrapply)rrrrs&& @r"initialize_fake_quantizersrs? HH  !VWJ"" KK ! >r!))rirg dataclassesrenumrtypingrrrr8torchao.core.configr%torchao.quantization.transform_moduler r embeddingr fake_quantize_configr r rrlinearrtwo_step_quantizerrutilsrstrrr$rnrprxr~rrrrrrr!r"rs# !'' , . (0+c4 M M M`"),w-wt  ', '  'H 'J ""EFVGV0 'l' '2 ,S ""IJ K  $-$Nr!