+ "ݳj^RIHt^RIHtHtHtHtHt^RIt^RI H t ^RI H t ] !RR44tRRltRR ltRR R lltRR R lltRRRlltRRltRtR#)) defaultdict)DictListOptionalTupleUnionN) convert_bytesc#r"TF-p\\V4'gK\\V4xK/ R#5i)N)hasattrtorchgetattr).0names& U/home/luca/screen-checking/.venv/lib/python3.14/site-packages/gptqmodel/utils/vram.py rs3 $udGE4s77c8V^8dQhR\P/#)dtype)r r)formats"r __annotate__rsDD5;;DcNV\P8XdR#VR8XdR#VR8XdR#VR8Xd^#V\9d^#V\P8XgV\P8Xd^#V\P 8XgV\P 8Xd^#\RV R24h) z Returns the size (in bytes) occupied by one parameter of type `dtype`. Example: ```py >>> dtype_byte_size(torch.float32) 4 ``` int2int4fp8z`dtype` is not a valid dtype: .g?g?g?)r bool_ONE_BYTE_FLOATX_DTYPESfloat16bfloat16float32int32 ValueErrorrs&rdtype_byte_sizer%s   & & % ) ) %-- 5ENN#: %-- 5EKK#79%BCCrczV^8dQhR\\\P3,R\P/#)rrreturn)rstrr devicer)rs"rrr8s,U3 #45%++rcv\V\4'd#VPRR4p\\V4pV#)z, Just does torch.dtype(dtype) if necessary. ztorch.) isinstancer(replacer r r$s&r_get_proper_dtyper.8s2% h+u% Lrc\V^8dQhR\PR\R\R\/#)rmoduleinclude_buffersrecurseremove_non_persistentnnModuler)rs"rrrBs0## II#(,#>B#cg#rc#"VPVR7RjxL V'dJ\4pV'd \WR7pVPVR7FpVwrgWd9gKVxK R#R#LW5i)a  A helper function that gathers all the tensors (parameters + buffers) of a given module. If `include_buffers=True` it's the same as doing `module.named_parameters(recurse=recurse) + module.named_buffers(recurse=recurse)`. Args: module (`torch.nn.Module`): The module we want the tensors on. include_buffer (`bool`, *optional*, defaults to `True`): Whether or not to include the buffers in the result. recurse (`bool`, *optional`, defaults to `False`): Whether or not to go look in every submodule or just return the direct parameters and buffers. remove_non_persistent (`bool`, *optional*, defaults to `False`): Whether or not to remove the non persistent buffer from the buffers. Useful only when include_buffers = True r2N)named_parameterssetget_non_persistent_buffers named_buffers)r0r1r2r3non_persistent_buffers named_bufferr_s&&&& rnamed_module_tensorsr@Bsm$&&w&777!$ %?%X ""000AL"GD1""B 8sA2A0 A2:A2$ A2cDV^8dQhR\PR\/#)rr0r2r4)rs"rrr_s&&ryy&4&rcVPpV'd,VP4Fwr4W$P,pK V#)a: Gather all non persistent buffers of a given modules into a set Args: module (`nn.Module`): The module we want the non persistent buffers on. recurse (`bool`, *optional*, defaults to `False`): Whether or not to go look in every submodule or just return the direct non persistent buffers. )_non_persistent_buffers_set named_modules)r0r2non_persistent_buffers_setr?ms&& rr;r;_s?"(!C!C((*DA &*G*G G &+ &%rc V^8dQhR\PR\\\\ P 3,,R\\\\\\ P 3,3,,R\/#)rmodelrspecial_dtypes buffers_only) r5r6rrr(r r)rr)rs"rrrrsd%% 99% E#u||+, -%T#uS%,,->'?"?@A% %rc Ve\V4p\V4pVe[VP4UUu/uFwrVV\V4bK pppVP4UUu/uFwrVV\V4bK ppp\\4pV'g\ VRR7p MVP RR7p V EFAwrVe&W9d V P4XV ,,p MVf,V P4\V P4,p M\V P4PR4'd,V P4\V P4,p M4V P4\X\V P44,p V PR4p \\V 4^,4F*pVRPV RV4;;,V , uu&K, EKD V#uuppiuuppi)z6 Compute the size of each submodule of a given model. NTr8r)z torch.uintz torch.intz torch.bool)r.r%itemsrintr@r<numelrr( startswithminsplitrangelenjoin)rHrrIrJ dtype_sizekeydtypspecial_dtypes_size module_sizes module_listrtensorsize name_partsidxs&&&& rcompute_module_sizesr_rs !%($U+ !HVH\H\H^_H^93#066H^_KYK_K_KabKaicsOD$99Kabs#L *5$? ))$)7 #   %$*@<<>$7$==D ]<<>OFLL$AAD   ) )*S T T<<>OFLL$AAD<<>C OFLL4Q$RRDZZ_ Z1,-C *Tc"23 4 < 4.$ 1`bs G:HcV^8dQhR\\\\PP 3,,R\ \\3,R\\,/#)rmodulesrYno_split_module_classes)rrr(r r5r6rrM)rs"rrrsI!! %UXX__,- .!>B38n!gklogp!rc.pVP4p\V4^8dVP^4wrV\V\P P 4'd\VP44M.p\V4^8XgVPPV9d'W,pVPV\V434KVU U u.uFwrV RV 2V 3NK up p V,pKV#uup p i)a+ from accelerate.utils get_max_layer_size Utility function that will scan a list of named modules and return the maximum size used by one full layer. The definition of a layer being: - a module with no direct children (just parameters and buffers) - a module whose class name is in the list `no_split_module_classes` Args: modules (`List[Tuple[str, torch.nn.Module]]`): The list of named modules where we want to determine the maximum layer size. module_sizes (`Dict[str, int]`): A dictionary mapping each layer name to its size (as generated by `compute_module_sizes`). no_split_module_classes (`List[str]`): A list of class names for layers we don't want to be split. Returns: `List[Tuple[str, str]]`: The size of all layer with the list of layer names and size str. r) copyrSpopr,r r5r6listnamed_children __class____name__appendr ) rarYrb layer_sizesmodules_to_treat module_namer0modules_childrenr\nvs &&& rget_all_layer_sizerqs,K||~  ! #.2215 rs[ $55 *$ $ $ D8#:&&%N!F"r