+ j YL^RIt^RIt^RIt^RIt^RIHt^RIHt^RIHtH t ^RI H t ^RI H t ^RIt^RIt^RIHt^RIHt^R IHt^R IHt^R IHt^R IHt^R IHt^RI H!t!H"t"H#t#H$t$H%t%^RI&H't'^RI(H)t)^RI*H+t+]!],4t-]$!4'd ^RI.H/uH0t1RRlt2RRlt3RRlt4RRlt5RRlt6RRlt7RRlt8R R!lt9R"R#lt:R:R$R%llt;R&t]"!R+4'd ]P~M ]P^t@]@PP]P]]P.tE]"!R,4'd&]EP]PP4RN)encode) OrderedDict)partialreduce) MethodType)Optional)Version) save_file)write_basic_config) get_logger) PartialState)FSDP_PYTORCH_VERSION)DistributedType)is_deepspeed_availableis_numpy_availableis_torch_distributed_availableis_torch_xla_availableis_weights_only_available)id_tensor_storage) convert_model)is_torch_versioncXV^8dQhR\PPR\/#modulereturntorchnnModulebool)formats"W/home/luca/screen-checking/.venv/lib/python3.14/site-packages/accelerate/utils/other.py __annotate__r$6s&HHuxxH4Hc\\R4'gR#\V\PPP 4#)z< Check whether the module was compiled with torch.compile() _dynamoF)hasattrr isinstancer' eval_frameOptimizedModulers&r#is_compiled_moduler-6s1 5) $ $ femm66FF GGr%cXV^8dQhR\PPR\/#rr)r"s"r#r$r$@s"   T r%c\\R4'gR#VP'dOVP4F:p\ V\P P P4'gK9R# R#)zT Check whether the module has submodules that were compiled with `torch.compile()`. r'FT)r(r_modulesmodulesr)r'r*r+r submodules& r#has_compiled_regionsr4@sS 5) $ $ )I)U]]%=%=%M%MNN* r%cXV^8dQhR\PPR\/#rr)r"s"r#r$r$Os"  uxx 4 r%ca\S\PP4;'dQ\ S4^8;'d;\ ;QJdV3RlS4F 'dK R# R#!V3RlS44#)z Check whether the module is a repeated block, i.e. `torch.nn.ModuleList` with all children of the same class. This is useful to determine whether we should apply regional compilation to the module. c3^<"TF"p\VS^,P4xK$ R#5i)rN)r) __class__).0mrs& r# %is_repeated_blocks..Xs%CFq 1fQi1122Fs*-FT)r)rr ModuleListlenallr,sfr#is_repeated_blocksr@Osq 6588../ D D K!O D D CCFCCC  CFC Cr%cXV^8dQhR\PPR\/#rr)r"s"r#r$r$\s"   D r%cVP'd,VP4Fp\V4'gKR# R#)z Check whether the module has repeated blocks, i.e. `torch.nn.ModuleList` with all children of the same class, at any level of the module hierarchy. This is useful to determine whether we should apply regional compilation to the module. TF)r0r1r@r2s& r#has_repeated_blocksrC\s1 )I!),,* r%cV^8dQhR\PPR\PP/#rrrr )r"s"r#r$r$js.HHEHHOOH%((//Hr%c jaRV3RlloS!V3/VBpRVP9dWPR&V#)a Performs regional compilation where we target repeated blocks of the same class and compile them sequentially to hit the compiler's cache. For example, in `GPT2LMHeadModel`, the repeated block/class is `GPT2Block`, and can be accessed as `model.transformer.h[0]`. The rest of the model (e.g. model.lm_head) is compiled separately. This allows us to speed up the compilation overhead / cold start of models like LLMs and Transformers in general. See https://pytorch.org/tutorials/recipes/regional_compilation.html for more details. Args: module (`torch.nn.Module`): The model to compile. **compile_kwargs: Additional keyword arguments to pass to `torch.compile()`. Returns: `torch.nn.Module`: A new instance of the model with some compiled regions. Example: ```python >>> from accelerate.utils import compile_regions >>> from transformers import AutoModelForCausalLM >>> model = AutoModelForCausalLM.from_pretrained("gpt2") >>> compiled_model = compile_regions(model, mode="reduce-overhead") >>> compiled_model.transformer.h[0] OptimizedModule( (_orig_mod): GPT2Block( (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): GPT2Attention( (c_attn): Conv1D(nf=2304, nx=768) (c_proj): Conv1D(nf=768, nx=768) (attn_dropout): Dropout(p=0.1, inplace=False) (resid_dropout): Dropout(p=0.1, inplace=False) ) (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): GPT2MLP( (c_fc): Conv1D(nf=3072, nx=768) (c_proj): Conv1D(nf=768, nx=3072) (act): NewGELUActivation() (dropout): Dropout(p=0.1, inplace=False) ) ) ) ``` cV^8dQhR\PPR\PP/#rrE)r"s"r#r$%compile_regions..__annotate__s*uxxr%c <\V4'dP\PP4pVF)pVP \P !V3/VB4K+ V#\ V4'dVPPVP4pVPPVP4\VPP44FOwrE\VR4'gK\VRR4VJgK,\VP V4VPV&KQ /VnVP%4FwrCVP'VS!V3/VB4K V#\P !V3/VBpV#)__func____self__N)r@rrr=appendcompilerCr8__new____dict__updatelistitemsr(getattrrrJr0named_children add_module)rcompile_kwargs new_moduler3namevalue_compile_regionss&, r#rZ)compile_regions.._compile_regionssF f % %,,.J# !!%-- "L^"LM$! ( ())11&2B2BCJ    & &v 7#J$7$7$=$=$?@ 5*--'%T2RV\2\0:5>>:0VJ''- A#%J #)#8#8#:%%d,~ >' ((r%cV^8dQhR\PPR\PP/#rrE)r"s"r#r$r$s*%((//r%c \V4'dVFpVP!R/VBK V#\V4'd&VP4Fp\ V3/VBK V#VP!R/VBV#)a Like `compile_regions`, but uses the in-place `module.compile()` instead of `torch.compile(module)`. Needed for the FSDP2 prepare path: `torch.compile(module)` returns an `OptimizedModule` whose `__call__` bypasses `nn.Module._call_impl`, so forward/pre hooks added later by `fully_shard` never fire and per-layer all-gather/reshard is lost. The in-place `module.compile()` keeps `_call_impl` (and its runtime hook check) on the call path, so FSDP hooks installed afterwards still run. Args: module (`torch.nn.Module`): The model to compile. **compile_kwargs: Additional keyword arguments to pass to `module.compile()`. r`)r@rMrCracompile_regions_fsdp2rcs&, r#rgrgsy&!!I    / /  M V $ $__&E !% :> :' M (( Mr%cXV^8dQhR\PPR\/#rmodelrr)r"s"r#r$r$s&CCUXX__CCr%ca\RR4'd^RIHoM^RIHo\;QJd-V3RlVP 44F 'gK R# R#!V3RlVP 444#)z Check if the model has DTensor parameters. Args: model (`torch.nn.Module`): The model to check. Returns: `bool`: Whether the model has DTensor parameters. >=z2.5.0)DTensorc3<<"TFp\VS4xK R#5iN)r))r9prms& r#r;$model_has_dtensor..sB/A!z!W%%/AsTF)rtorch.distributed.tensorrmtorch.distributed._tensorany parameters)rjrms&@r#model_has_dtensorrvsUg&&4 6 3Bu/?/?/AB33B3B3Bu/?/?/AB BBr%cnV^8dQhR\PPR\\,/#ri)rrr rint)r"s"r#r$r$s&88UXX__8#8r%c\VRR4pVeV#\\VRR4RR4p\V\4'dVPR4#\VRR4#)z Get the tensor parallel degree a `transformers` model was sharded with, or `None` if it was not sharded. Args: model (`torch.nn.Module`): The model to inspect. Returns: `Optional[int]`: The model's tensor parallel size. tp_sizeNconfigdistributed_config)rSr)dictget)rjrzr|s& r#get_model_tp_sizersfeY-G $!?AUW[\$d++!%%i00 %y$ 77r%c<V^8dQhR\R\R\/#)rkeep_fp32_wrapperkeep_torch_compile recursiver!)r"s"r#r$r$s)UU"U?CUW[Ur%ca \PPP\PP3p\ V4p\ V4pRpV'dTpVPpM.V'd'RVP9dTpVPR,p\4'd^RI H pWH3, p\R\4'd \4'd^RIHp WI3, p\#W4'dVP$pKV'dV 3Rlo S !V4pV'gVP&p VPP)RR4p V e8\+V R4'dV P,p W8XgK&\/W4Vn\1VR R 4'd\3VR R 7V'd/Ve+V'd WnTpV#V'dWPR&TpV#) a_ Extract a model from its distributed containers. Args: model (`torch.nn.Module`): The model to extract. keep_fp32_wrapper (`bool`, *optional*): Whether to remove mixed precision hooks from the model. keep_torch_compile (`bool`, *optional*): Whether to unwrap compiled model. recursive (`bool`, *optional*, defaults to `False`): Whether to recursively extract all cases of `module.module` from `model` as well as unwrap child sublayers recursively, not just the top-level distributed containers. Returns: `torch.nn.Module`: The extracted model. Nr\)DeepSpeedEnginerl)FullyShardedDataParallelc<\VR4'dS!VP4pMTpVP4Fwr#\WS!V44K V#r,)r(rrTsetattr)runwrapped_modulerXrd_recursive_unwraps& r#r6extract_model_from_parallel.._recursive_unwrapDsTvx((#4V]]#C #) />>@ (0A%0HI A# #r%_original_forward __wrapped__ _converted_to_transformer_engineF)to_transformer_engine)rrparallelDistributedDataParallel DataParallelr-r4r\rOr deepspeedrrrr2torch.distributed.fsdp.fully_sharded_data_parallelrr)rforwardpopr(rrrSr) rjrrroptions is_compiled has_compiledcompiled_modelrFSDProriginal_forwardrs &&&& @r#extract_model_from_parallelrs(xx  88%((:O:OPG$U+K'.LN  %.. ("NNN;/E-%%2338V8X8Xg7 U $ $  $"%( -- >>--.A4H  ''=11!--.&w6EM 5 Make sure all processes will reach this instruction otherwise one of your processes will hang forever. N)r wait_for_everyoner`r%r#rrjsN$$&r%c$V^8dQhR\/#)r state_dict)r})r"s"r#r$r$ws   r%c ,\P!\4pVP4F>wr#\ V\ 4'dKV\ V4,PV4K@ VP4UUu/uFwrE\V4^8gKWEbK ppp\4pVP4FFpVUu.uF q"V9gK VNK ppVPVR,4VR,FpWK KH \V4^8d\PRV R24VP4U U u/uF8wrT \ V \P4'dV P!4MT bK: pp p V#uuppiuupiuup p i)z Cleans the state dictionary from a model and removes tensor aliasing if present. Args: state_dict (`dict`): The state dictionary from a model :NNzRemoved shared tensor zk while saving. This should be OK, but check by verifying that you don't receive any warning while reloading) collections defaultdictrQrRr)strrrLr>setvaluesrPloggerwarningrTensor contiguous) rptrsrXtensorptrnames shared_ptrs warn_names found_nameskvs & r# clean_state_dict_for_safetensorsrwsa  " "4 (D"((* &#&& "6* + 2 24 8+ 15 O *#E Q:3: KOJ##% ).D1Ctt D+b/*OOD $& :$ZL1\ ] WaVfVfVhiVhda!z!U\\'B'BQ\\^IVhJi #PEjsFFF F >Fc0V^8dQhR\R\/#)rsave_on_each_nodesafe_serializationr)r"s"r#r$r$sDdr%c\4P\P8Xd\P !V4pV'd6\ \RR/R7p\V\4'd \V4pM\Pp\4P'dV'g V!W4R#\4P'dV'd V!W4R#R#R#)a Save the data to disk. Use in place of `torch.save()`. Args: obj: The data to save f: The file (or file-like object) to use to save the data save_on_each_node (`bool`, *optional*, defaults to `False`): Whether to only save on the global main process safe_serialization (`bool`, *optional*, defaults to `False`): Whether to save `obj` using `safetensors` or the traditional PyTorch way (that uses `pickle`). r"pt)metadataN)r distributed_typerXLAxm_maybe_convert_to_cpursafe_save_filer)rrrsaveis_main_processis_local_main_process)objfrr save_funcs&&&& r#rrs$~&&/*=*==&&s+Nh5EF c; ' '237CJJ ~%%%.?#  - - -2C#3D -r%z2.0.0z1.25.0c |\4'dO\PP4pRV9dRVR&\PP \ 4MVP RR4\P!V3RV/VBp\4'dF\PP4X'd \PP V4V# \4'dH\PP4X'd!\PP T4iii;i)a Compatible drop-in replacement of `torch.load()` which allows for `weights_only` to be used if `torch` version is 2.4.0 or higher. Otherwise will ignore the kwarg. Will also add (and then remove) an exception for numpy arrays Args: f: The file (or file-like object) to use to load the data map_location: a function, `torch.device`, string or a dict specifying how to remap storage locations **kwargs: Additional keyword arguments to pass to `torch.load()`. weights_onlyTN map_location) rr serializationget_safe_globalsadd_safe_globalsTORCH_SAFE_GLOBALSrloadclear_safe_globals)rrkwargsold_safe_globals loaded_objs&&, r#rrs G $ & &$22CCE V+)-~&    0 01C D JJ~t ,ZZG GG $ & &    2 2 4##445EF  % & &    2 2 4##445EF  'sB C""6D;"D;c\VR4'g \VR4'g\VRV4p\VR4'd VP#\VR4'd VP#\ V4#)z Gets a pretty name from `obj`. __qualname____name__r8)r(rSrrr)rs&r#get_pretty_namersd 3 ' 'Z0H0Hc;,sN##sJ|| s8Or%cVP4F>wr#\V\4'd VPV/4p\ W44K:W1V&K@ V#)z Recursively merges two dictionaries. Args: source (`dict`): The dictionary to merge into `destination`. destination (`dict`): The dictionary to merge `source` into. )rRr)r} setdefault merge_dicts)source destinationkeyrYnodes&& r#rrsMlln  eT " "))#r2D  $$  % r%cFV^8dQhR\\,R\/#)rportr)rrxr!)r"s"r#r$r$ s66#6$6r%cVfRp\P!\P\P4;_uu_4pVPRV34^8HuuRRR4# +'giR#;i)z Checks if a port is in use on `localhost`. Useful for checking if multiple `accelerate launch` commands have been run and need to see if the port is already in use. Ni> D [[F XX0$,,?NAKw'+..&wi/[\g[his s  sd3 /r%c$V^8dQhR\/#)rattr)r)r"s"r#r$r$?s555r%cLRp\W .VPR4,4#)z Recursive `getattr`. Args: obj: A class instance holding the attribute. attr (`str`): The attribute that is to be retrieved, e.g. 'attribute1.attribute2'. c\W4#ro)rS)rr s&&r#_getattr#recursive_getattr.._getattrJs s!!r%.)rr)rr rs&& r#recursive_getattrr?s#" (EDJJsO3 44r%cV^8dQhR\PPR\R\\PP,/#)rrj return_fqnsr)rrr r!rQ)r"s"r#r$r$Ps<!!!t!X\]b]e]e]l]lXm!r%cV'gTMRV3pV.p.pV'dVP4pV'dVwreVP4Fxwrx\V\PP 4'gK1V'd/X'dVR,V,MTp VP W34KgVP V4Kz V'dVP XV34KVP V4KVRRR1,#)a)Traverse the model in bottom-up order and return the children modules in that order. Args: model (`torch.nn.Module`): the model to get the children of Returns: `list[torch.nn.Module]`: a list of children modules of `model` in bottom-up order. The last element is the `model` itself. rrN)rrTr)rrr rL) rjrtopstackordered_modulescurrent_modulecurrent_module_namerXr  child_names && r#get_module_children_bottom_uprPs#%U C EEO  2@ / (779JD$00EX!4s!:T!A^bJLL*!34LL& :   " "$7#H I  " "> 2 4R4  r%)TTF)FFro)F)Rrrrrcodecsrr functoolsrrtypesrtypingrnumpynprpackaging.versionr safetensors.torchr rcommands.config.defaultr loggingr stater constantsr dataclassesrimportsrrrrrmodelingrtransformer_enginerversionsrrrtorch_xla.core.xla_modelcore xla_modelrr-r4r@rCr]rbrgrvrrrrr_corenp_core multiarray _reconstructndarraydtyperrLdtypes UInt32DTyperrrrrrr rrr`r%r#r9sU #% %98 +((-& H ))H   HV)04C(8.Up ' FH)11"((rww ##JJ HH hbii334@ $6 ""4$5"!!r%