+ jC?^RIt^RIHt^RIHt^RIHtHtHtH t H t H t H t H t HtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtH t ^RI!H"t"^RI#H$t$^RI%H&t&^RI'H(t(^R I)H*t*^R I+H,t,^R I-H.t.^R I/H0t0^R I1H2t2^RI3H4t4^RI5H6t6^RI7H8t8^RI9H:t:^RI;Ht>^RI?H@t@^RIAHBtB^RICHDtD^RIEHFtF^RIGHHtH^RIIHJtJ^RIKHLtL^RIMHNtN^RIOHPtP^RIQHRtR^RISHTtT^RIUHVtV/R ](bR!],bR"].bR#]@bR$]$bR%]LbR&]NbR']8bR(]:bR)]2bR*]BbR+]DbR,]0bR-]4bR.]TbR/]*bR0]VbR1]RR2]6R3]>R4]JR5]6R6]&R7]HR8]FR9]PR:]4t^R?R@lt_RARBlt`RDRCltaR#)EN) AutoConfig)logging) AqlmConfigAutoRoundConfig AwqConfigBitNetQuantConfigBitsAndBytesConfigCompressedTensorsConfig EetqConfigFbgemmFp8ConfigFineGrainedFP8ConfigFourOverSixConfig FPQuantConfigGemmaQuantizationConfig GgufConfig GPTQConfig HiggsConfig HqqConfig MetalConfig Mxfp4Config NVFP4ConfigQuantizationConfigMixinQuantizationMethod QuantoConfig QuarkConfig SinqConfig SpQRConfig TorchAoConfig VptqConfig) HfQuantizer)AqlmHfQuantizer)AutoRoundQuantizer) AwqQuantizer)BitNetHfQuantizer)Bnb4BitHfQuantizer)Bnb8BitHfQuantizer)CompressedTensorsHfQuantizer)EetqHfQuantizer)FbgemmFp8HfQuantizer)FineGrainedFP8HfQuantizer)FourOverSixHfQuantizer)FPQuantHfQuantizer)GemmaQuantizer)GgufHfQuantizer)GptqHfQuantizer)HiggsHfQuantizer)HqqHfQuantizer)MetalHfQuantizer)Mxfp4HfQuantizer)NVFP4HfQuantizer)QuantoHfQuantizer)QuarkHfQuantizer)SinqHfQuantizer)SpQRHfQuantizer)TorchAoHfQuantizer)VptqHfQuantizerawqbitsandbytes_4bitbitsandbytes_8bitgptqaqlmquantoquark fouroversixfp_quanteetqhiggshqqzcompressed-tensors fbgemm_fp8torchaobitnetvptqspqrfp8ggufnvfp4mxfp8z auto-roundmxfp4metalsinqgemmacPa]tRt^toRt]V3RlRl4t]R4tRtVt R#)AutoQuantizationConfigz The Auto-HF quantization config class that takes care of automatically dispatching to the correct quantization config given a quantization config stored in a dictionary. c <V^8dQhRS[/#)quantization_config_dict)dict)format __classdict__s"]/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/quantizers/auto.py __annotate__#AutoQuantizationConfig.__annotate__s>>>c VPR4pVPRR4'gVPRR4'd5VPRR4'dRMRp\PV,pMVf \R4hV\9d-\RV R\ \ P44 24h\V,pVPV4#) quant_method load_in_8bitF load_in_4bit_4bit_8bitThe model's quantization config from the arguments has no `quant_method` attribute. Make sure that the model has been correctly quantizedUnknown quantization type, got  - supported types are: ) getrBITS_AND_BYTES ValueError AUTO_QUANTIZATION_CONFIG_MAPPINGlistAUTO_QUANTIZER_MAPPINGkeys from_dict)clsrXrasuffix target_clss&& r\rp AutoQuantizationConfig.from_dicts/33NC # ' ' > >BZB^B^_motBuBu 8 < <^U S SWY`F-<>( # #r_rUc|a]tRt^toRt]V3RlRl4t]R4t]V3RlRl4t] R4t Rt Vt R #) AutoHfQuantizerz The Auto-HF quantizer class that takes care of automatically instantiating to the correct `HfQuantizer` given the `QuantizationConfig`. c0<V^8dQhRS[S[,/#)rWrv)rrY)rZr[s"r\r]AutoHfQuantizer.__annotate__s99.E.L9r_c \V\4'd\PV4pVPpV\ P 8XdG\V\4'g \R4hVP'd VR, pM VR, pV\9d-\RV R\\P44 24h\V,pV!V3/VB#)zZFound `quant_method=bitsandbytes` but `quantization_config` is not a `BitsAndBytesConfig`.rerdrgrh) isinstancerYrUrprarrjr TypeErrorrbrnrkrmro)rqrvr|rarss&&, r\ from_configAutoHfQuantizer.from_configs )4 0 0"8"B"BCV"W *77  -<< <13EFFp#///' ' 5 51,@/44678:  ,L9 -888r_c R\P!V3/VBpVPV4#N)rUrxr)rqr{r|rvs&&, r\rxAutoHfQuantizer.from_pretraineds*4DDEbmflm233r_cD<V^8dQhRS[S[,RS[R,/#)rWrvquantization_config_from_argsN)rYr)rZr[s"r\r]rs*/#/#!$;;/#(?'E/#r_c fVeRpMRp\V\4'dC\V\4'd\P!V4pM\PV4pVejVP P VP P 8wd;\RVP P RVP P R24h\V\4'dx\V\4'dbVP4pVP4FwrV\WV4K V'd%VR\VP44 R2, pVR8wd:\V\\\ 34'g\"P$!V4V#\&P)V4V#)zt handles situations where both quantization_config from args and quantization_config from model config are present. zYou passed `quantization_config` or equivalent parameters to `from_pretrained` but the model you're loading already has a `quantization_config` attribute. The `quantization_config` from the model will be used.zThe model is quantized with z but you are passing a z| config. Please make sure to pass the same quantization config class to `from_pretrained` with different loading attributes.z"However, loading attributes (e.g. z]) will be overwritten with the one you passed to `from_pretrained`. The rest will be ignored.)rrYrrprU __class__rrkLOADING_ATTRIBUTES_CONFIG_TYPESget_loading_attributesitemssetattrrmrorrr warningswarnloggerinfo)rqrvr warning_msgloading_attr_dictattrvals&&& r\merge_quantization_configs*AutoHfQuantizer.merge_quantization_configss ) 4y  K )4 0 07II&5&?&?@S&T#&<&F&FGZ&[# * 5#--66:W:a:a:j:jj./B/L/L/U/U.VVmoLoVoVo_o_n``FF  )+J K KPZ )+JQ Q !> T T V .446 +377!!CDIZI_I_IaDbCcdA BB " Z0CkS^`tEu%v%v MM+ &#" KK $""r_c VPRR4pVPRR4'gVPRR4'd5VPRR4'dRMRp\PV,pMVf \R4hV\9d:\ P RV R \\P44 R 24R#R #) raNrbFrcrdrerfrgrhz~. Hence, we will skip the quantization. To remove the warning, you can delete the quantization_config attribute in config.jsonT) rirrjrkrlrwarningrmrnro)rXrarrs& r\supports_quant_method%AutoHfQuantizer.supports_quant_methods/33NDI # ' ' > >BZB^B^_motBuBu 8 < <^U S SWY`F-<.register_config_fn5sK 5 5xx/CDE E#677HI I36(0 r_rw)rrsf r\register_quantization_configr2s r_c$V^8dQhR\/#)rWnamer)rZs"r\r]r]Bs ! !S !r_caV3RlpV#)zRegister a custom quantizer.c<S\9d\RS R24h\V\4'g \ R4hV\S&V#)z Quantizer 'rz!Quantizer must extend HfQuantizer)rnrkrr r)rqrs&r\register_quantizer_fn1register_quantizer..register_quantizer_fnEsJ ) ){4&0DEF F#{++?@ @'*t$ r_rw)rrsf r\register_quantizerrBs ! r_cVe/Vf\VR7pM>\V\4'dWQnM!\V\4'd \R4h\ VRR4;'g\ VP RR7RR4pVRJpV'd\ PV4'gRpV'gVeLV'd\ PWa4Vn MWn \ PVPVR7pMRpVeVe5VPP\P8wd \R4hVPVVR 7VPV4pVP!V4pVP#V4p\ VPR R4'g'VPPp \ V R V 4VR &WV3#) N) gguf_filezLoading a GGUF checkpoint needs the file named as `from_pretrained(..., gguf_file=...)`. `GgufConfig` carries the loading options and does not have to repeat it.rvT)decoderF) pre_quantizedzYou cannot combine Quantization and loading a model from a GGUF file, try again by making sure you did not passed a `quantization_config` or that you did not load a quantized model from the Hub.) device_map weights_only dequantizevaluequant)rrrrkryget_text_configrrrrvrrarGGUFvalidate_environmentupdate_device_mapupdate_tp_planupdate_ep_plan) configrvrr user_agentrquantization_params_from_configr hf_quantizerras &&&&&& r\get_hf_quantizerrRs  &",y"A  +Z 8 8,5 ) ' 4 4 W  '.f6KT&R''V]t,.CTW#44?M_BBCbcc +7 )8)S)S/*F &*= &&22  & &'3    \%E%E%R%RVhVmVm%mv  ))!% * "33J? ,,V4,,V4|77uMM';;HHL"),"NJw   ++r_r)brmodels.auto.configuration_autorutilsrutils.quantization_configrrrrr r r r r rrrrrrrrrrrrrrrrrrbaser quantizer_aqlmr!quantizer_auto_roundr" quantizer_awqr#quantizer_bitnetr$quantizer_bnb_4bitr%quantizer_bnb_8bitr&quantizer_compressed_tensorsr'quantizer_eetqr(quantizer_fbgemm_fp8r)quantizer_finegrained_fp8r*quantizer_fouroversixr+quantizer_fp_quantr,quantizer_gemmar-quantizer_ggufr.quantizer_gptqr/quantizer_higgsr0 quantizer_hqqr1quantizer_metalr2quantizer_mxfp4r3quantizer_nvfp4r4quantizer_quantor5quantizer_quarkr6quantizer_sinqr7quantizer_spqrr8quantizer_torchaor9quantizer_vptqr:rnrlr get_loggerrrrUrrrrrwr_r\rs7:+4'/22F+6@92+++-)---/-++1+ <++ O   O     )" O  >6&!  !" O#$ O $ O  &$   O ^=B$ J$ 9$+$+ $  J $  J $ J$ l$ [$$$ $ 9$1$/$ [$ }!$" #$$ J J  [ !/ [ [ J $7$ >  #   H %&#&#Rll^  ! 4,r_