+ j*^RIHt^RIHtHtHt^RIHt^RIH t ]!4'd^RI t ^RI H t ^RI HtHt]P !]4tRR lt!R R ] 4tR#) )deepcopy)is_compressed_tensors_availableis_torch_availableloggingCompressedTensorsConfig) HfQuantizerN)WeightConverter)DecompressExpertsget_experts_schemec$V^8dQhR\/#returnbool)formats"u/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/quantizers/quantizer_compressed_tensors.py __annotate__r sUUdUc~VPpVRJ;'d'VPR8H;'dVP^8H#)zJWhether a compressed-tensors quantization scheme quantizes weights to FP8.Nfloat)weightstypenum_bits)schemers& r_is_fp8_schemer s;nnG $  T T7<<7#: T Tw?O?OST?TTrcaa]tRt^&toRtRtV3RlV3RlltRtRtRt Rt ] R 4t ] V3R lR l4t V3R lR ltV3RlRltRtRtV3RltRtVtV;t#)CompressedTensorsHfQuantizera Quantizer for the compressed_tensors package. Loads and restores models to quantized state with compressed_tensors. With `use_optimized_inference=True`, FP8 checkpoints are kept in FP8 and their matmuls run through row-wise FP8 kernels (`torch.nn.functional.scaled_mm`) via `CompressedTensorsFP8Linear`, when FP8 matmul hardware is available (CUDA SM89+ or XPU). This is opt-in and inference only. Otherwise the model goes through the regular compressed-tensors route: `dequantize=True` dequantizes the weights at load time, while `dequantize=False` leaves them compressed and lets compressed-tensors decompress them on the first forward pass. Tc <V^8dQhRS[/#rquantization_configr)r __classdict__s"rr)CompressedTensorsHfQuantizer.__annotate__7s 7 7,C 7rc <\SV`!V3/VBVP4^RIHpVP V4VnWnR#)r)ModelCompressorN)super__init__ post_initcompressed_tensors.compressorsr&from_compression_config compressorr")selfr"kwargsr& __class__s&&, rr(%CompressedTensorsHfQuantizer.__init__7s@ ,77 %%'B)AABUV#6 rc\4'g \R4hVPPVnVP'd\ P P4'gh\ PP4'd$\ PP4R8g\PR4RVnR#R#R#R#)zyUsing `compressed_tensors` quantized models requires compressed-tensors>=0.15.0: `pip install compressed-tensors>=0.15.0`zIgnoring `use_optimized_inference=True`: FP8 matmul kernels need a CUDA GPU with compute capability >= 8.9 (e.g. 4090/H100) or an Intel XPU, and none was found.FN) ) r ImportErrorr"use_optimized_inferenceuse_fp8_kerneltorchxpu is_availablecudaget_device_capabilitylogger warning_once)r-argsr.s&*,rvalidate_environment1CompressedTensorsHfQuantizer.validate_environmentDs.00; #66NN     II " " $ $)@)@)B)BuzzGgGgGimsGs   O #(D  Ht $ rc VPPp\VP4pVP'd^RIHpVP4UUu/uFwrg\V4'gKWgbK pppVP4UUu/uFwrgWh9gK WgbK pppVPVRVP4Vn VP4UU u.uFqwPFqNK K p pp V!VV VPVPR7^RIHp \#V4p WLnV !WRR7VPP$'dVPP'VR7R#R#uuppiuuppiuup pi)r)*replace_with_compressed_tensors_fp8_linearN)targetsignoremodules_to_not_convert)apply_quantization_configF)run_compressedmodel)r,r"dict config_groupsr6integrations.compressed_tensorsrBitemsrget_modules_to_not_convert_keep_in_fp32_modulesrEvaluesrCrDcompressed_tensors.quantizationrFris_quantization_compressedcompress_model) r-rIr. ct_configremaining_groupsrBnamegroup fp8_groupstargetrCrFremaining_configs &&, r$_process_model_before_weight_loadingACompressedTensorsHfQuantizer._process_model_before_weight_loadingXsTOO77  7 78     d:J9O9O9Qk9Q+$UcdiUj+$+9QJk?O?U?U?Wr?W [_[q  ?W r*.*I*I%QUW\WrWr*sD '+5+<+<+>[+>%]]6v]v+>G[ 6 '''+'B'B   N#I.)9&!%%P  # # > > > OO * * * 7 ?)lr\sE61E6 E<E<Fc VPP'd=VPP'dVPP VR7R#R#R#)aDequantize the layers loaded through the compressed-tensors wrappers, but only when asked for: with `dequantize=False` the weights are left compressed, and the hook `compress_model` registered decompresses them on the first forward pass instead. FP8-kernel modules were never wrapped (their weights loaded directly in FP8) and hold no compressed-tensors scheme, so neither the call below nor that hook touches them.rHN)r" dequantizerRr,decompress_model)r-rIr.s&&,r#_process_model_after_weight_loading@CompressedTensorsHfQuantizer._process_model_after_weight_loadingxsE  # # . . .43K3K3f3f3f OO , ,5 , 94g .rc RRRRRRRRRR/pVP4eFVP4Pe*VP4PPV4V#)z0layers.*.feed_forward.experts.*.gate_proj.weightcolwisez6layers.*.feed_forward.experts.*.gate_proj.weight_scalez.layers.*.feed_forward.experts.*.up_proj.weightz4layers.*.feed_forward.experts.*.up_proj.weight_scalez0layers.*.feed_forward.experts.*.down_proj.weightrowwise)get_text_configbase_model_tp_planupdate)r-configadditional_plans&& rupdate_tp_plan+CompressedTensorsHfQuantizer.update_tp_plansi > Di    ! ! # /F4J4J4L4_4_4k  " " $ 7 7 > > O rc$VP'*#)N)r6r-s&r is_trainable)CompressedTensorsHfQuantizer.is_trainables&&&&rc <V^8dQhRS[/#rr)rr#s"rrr$srcR#)Trms&ris_compileable+CompressedTensorsHfQuantizer.is_compileablerc <V^8dQhRS[/#rr)rr#s"rrr$snn$nrc VP'dR#VPP;'gVPP'*#)z7Loaded Models can carry out quantization aware trainingF)r6r"r^rRrms&ris_qat_trainable-CompressedTensorsHfQuantizer.is_qat_trainables:    ''22mm$:R:R:m:m6mmrc <V^8dQhRS[/#rr)rr#s"rrr$src R#)z>Models quantized using compressed tensors can be saved to diskTrrrms&ris_serializable,CompressedTensorsHfQuantizer.is_serializablerurc dVP'g.#^RIHp\R.R.V!4.R7.#)uOn the FP8 kernel path, a generic converter reshapes the checkpoint ``weight_scale`` tensors into the row-wise kernel layout once at load time — see :class:`ConvertFP8LinearScale`.)ConvertFP8LinearScale weight_scalesource_patternstarget_patterns operations)r6rLrr )r-rs& rget_weight_conversions3CompressedTensorsHfQuantizer.get_weight_conversionssA"""IK !/ 0!/ 0134   rc j.pVEFp\V\4'dO\;QJd&RVP4F 'gK RM RM!RVP44'dVP V4K{VPUu.uFqDP R4'gKVNK ppV'Ed%\ VPP4pVUu.uF qDR,NK ppVPUu.uFqDP R4'dKVNK pp\V4'd(VUu.uF qDR,NK upV,V,p MDVUu.uF qDR,NK p pVUu.uF qDR,NK p pW,V ,V,p \WR 7.\VP4,p \V VPV R 7pVP V4EK VPVP44V#uupiuupiuupiuupiuupiuupi) aAttach the quantization sources (scales, packed weights) of MoE expert converters to their bucket and prepend a :class:`DecompressExperts` op, so the per-expert (weight, scale) pairs are dequantized *before* the merge / concat ops collapse the per-expert structure. FP8 checkpoints keep the plain ``weight`` name; packed formats use ``weight_packed`` / ``weight_shape``. The generic converters from :meth:`get_weight_conversions` are appended at the end: converters are matched in order, so the expert converters keep their scale keys. c3,"TF pRV9xK R#5i)expertsNrr).0ps& r ICompressedTensorsHfQuantizer.update_weight_conversions..s;mXlSTIQrsPPP?4W   H %U x;xr