+ ܳj(^RIt^RIHt^t^ t]],t.R Ot.ROt.ROtRRRllt RRRllt Rt R t R]3R R llt R tR#)N)ListcolumncDV^8dQhR\PR\/#)imatrix directiontorchTensorstr)formats"V/home/luca/screen-checking/.venv/lib/python3.14/site-packages/awq/utils/quant_utils.py __annotate__rs%,,3c\P!^\\VPR7pVP \P 4p\P!V^4pVR8XdbVPRVP^,\,\4p\P!WR,4PRR7pMgVR8XdaVPVP^,\,\R4p\P!WR,4P^R7pXP \P4pV#)z Packs a 4-bit integer matrix into a packed 32-bit integer matrix. Args: imatrix (torch.Tensor): matrix of integers direction (str): direction of packing, either "column" or "row" Returns: qmatrix (torch.Tensor): packed matrix of integers devicerdimrowNNNNNNrN)r arange STORAGE_BITSQ_BITSrtoint8 bitwise_andviewshapePACK_NUMbitwise_left_shiftsumint32)rrshiftsqmatrixs&& rpackr)s\\!\6'.. IFjj$G.GH,,r7==#3x#?J**7=4IJNNSUNV e ,,w}}Q/8;XrJ**7=4IJNNSTNUjj%G NrcDV^8dQhR\PR\/#)rr(rr )r s"rrr*sELLSrc\P!^\\VPR7pVR8XdH\P !VR,VR,4P VP^,R4pMMVR8XdG\P !VR,VR,4P RVPR,4pXP\P4^,pV#) a Unpacks a 32-bit packed integer matrix into a 4-bit integer matrix. Args: qmatrix (torch.Tensor): matrix of packed integers direction (str): direction of unpacking, either "column" or "row" Returns: imatrix (torch.Tensor): matrix of integers rrr)rrNrr)rNrr) r rrrrbitwise_right_shiftr!r"rr)r(rr'rs&& runpackr-*s\\!\6'.. IFH++ J  !6 $w}}Q $  e ++ J  !6 $r7==$ % jj$t+G Nrc(VP\P4^,p\P!WP V^R7, VP V^R7,4pVP\P4^,pV#)aG Quantizes a matrix of 16-bit floats into a matrix of 4-bit integers. Args: fmatrix (torch.Tensor): matrix of 16-bit floats scales (torch.Tensor): matrix of 16-bit floats zeros (torch.Tensor): matrix of 4-bit integers group_size (int): group size Returns: imatrix (torch.Tensor): matrix of 4-bit integers r)rr rroundrepeat_interleave)fmatrixscaleszeros group_sizers&&&& rquantizer5Fsu HHUZZ 4 'Ekk ..zq.A A%%ja%8 9Gjj$t+G Nrc>VP\P4^,pVP\P4^,pWPV^R7, VPV^R7,pVP\P4pV#)a: Dequantizes a 4-bit integer matrix into a float matrix. Args: imatrix (torch.Tensor): matrix of 4-bit integers scales (torch.Tensor): matrix of 16-bit floats zeros (torch.Tensor): matrix of 4-bit integers group_size (int): group size Returns: fmatrix (torch.Tensor): matrix of 16-bit floats r)rr rr0float16)rr2r3r4r1s&&&& r dequantizer8as} HHUZZ 4 'Ejj$t+G ))*!)<<   34Gjj'G NrcfV^8dQhR\PR\R\\,/#)rrrorder)r r r rint)r s"rrrzs. \\ 9rcVR8Xd;VPR\4RV3,PVP4pV#VR8Xd9VP\R4VR3,PVP4pV#)a0 Applies the order to a 4-bit integer matrix. Args: imatrix (torch.Tensor): matrix of integers direction (str): direction of applying order, either "column" or "row" order (List[int]): order to apply, default is ordinal packing order Returns: imatrix (torch.Tensor): matrix of integers rrrr)r!r#r")rrr:s&&&r apply_orderr=zsr H,,r8,QX6;;GMMJ N e ,,x,UAX6;;GMMJ Nrc\VRR7p\VRR7p\VR\R7p\VR\R7pV^, p\VRR7p\VRR7pW3#)r)r)rr:r)r-r=REVERSE_AWQ_PACK_ORDERr))qweightqzerosizerosiweightss&& rawq_to_exllamarDsc Fh /Fg2H8;Q RF8x?UVH aZF &H -F8u-G ?r)rr)rrrGrIrErFrHrJ)rrGrErHrrIrFrJ)r)r typingrrrr#ORDINAL_PACK_ORDERAWQ_PACK_ORDERr?r)r-r5r8r=rDrrrOsX   6 !-)18866)0r