+ jRt^RIt^RIt^RIHt^RIHtHt^RIH t H t ^RI H t ^RI t ^RIHtHt^RIHt^RIHtHt^R IHtHtHt^R IHt]!4'd ^RIHuHt]!4'd^R I H!t!R t"R t#Rt$Rt%Rt&R]"RR/Rlt'RARlt(Rt)Rt*Rt+Rt,Rt-Rt.Rt/Rt0!RR]14t2R t3R!t4]3R"4t5R#R$lt6R%R&lt7R'R(lt8RBR)lt9RCR*lt:] Pv^] Px^] Pz^] P|^] P~^] P^] P^] P^] P^ ] P^ / tE]EP4UUu/uFwrWbK upptGR+tHRDR,R-lltI]3RBR.R/ll4tJRBR0R1lltKRBR2R3lltLRER4ltMRBR5ltN!R6R7]O4tP]4RFR8l4tQRBR9ltR]3RGR:l4tSR;tT!R<R=4tUR>tVR?tW]RHR@l4tXR#uuppi)IzB A set of basic tensor ops compatible with tpu, gpu, and multigpu N)Mapping)contextmanager nullcontext)update_wrapperwrapsAny)AcceleratorState PartialState)!TORCH_DISTRIBUTED_OPERATION_TYPES)DistributedTypeTensorInformation)is_npu_availableis_torch_distributed_availableis_torch_xla_available)is_torch_version)ReduceOpc6\V\P4#N) isinstancetorchTensortensors&\/home/luca/screen-checking/.venv/lib/python3.14/site-packages/accelerate/utils/operations.pyis_torch_tensorr-s fell ++c v\V\PP\PP\PP \PP \PP\PP\PP4#r) rrxpu FloatTensor ByteTensor IntTensor LongTensor HalfTensor DoubleTensorBFloat16Tensorrs&ris_torch_xpu_tensorr&1sm            rc"\V\4#r)rr  tensor_infos&ris_tensor_informationr*>s k#4 55rcr\V\4;'d!\VR4;'d \VR4#)z Checks if `data` is a `namedtuple` or not. Can have false positives, but only if a user is trying to mimic a `namedtuple` perfectly. _asdict_fields)rtuplehasattrdatas&r is_namedtupler2Bs0 dE " \ \wtY'? \ \GDR[D\\rcv\V4'd\V4!\V4!#\V4!V4#)zG Cast a generator to the same type as obj (list, tuple, or namedtuple) )r2typelist)obj generators&&r honor_typer8Js1 SCy$y/**Cy##r test_typeerror_on_other_typeFcaaaaa\V\\34'd\VVVVVV3RlV44#\V\4'dG\ V4!VP 4UUu/uFwrgV\SV.SO5RSRS/SBbK upp4#S!V4'd S!V.SO5/SB#S'd3\R\ V4 RSP RSP R24hV#uuppi)aX Recursively apply a function on a data structure that is a nested list/tuple/dictionary of a given base type. Args: func (`callable`): The function to recursively apply. data (nested list/tuple/dictionary of objects accepted by `test_type`): The data on which to apply `func` *args: Positional arguments that will be passed to `func` when applied on the unpacked data. test_type (`Callable[[Any], bool]`, *optional*, defaults to `is_torch_tensor`): The predicate deciding whether `func` should be applied to a given leaf object. error_on_other_type (`bool`, *optional*, defaults to `False`): Whether to return an error or not if after unpacking `data`, we get on an object for which `test_type` returns `False`. If `False`, the function will leave such objects unchanged. **kwargs (additional keyword arguments, *optional*): Keyword arguments that will be passed to `func` when applied on the unpacked data. Returns: The same data structure as `data` with `func` applied to every object for which `test_type` returns `True`. c3L<"TFp\SV.SO5RSRS/SBxK R#5i)r9r:Nrecursively_apply).0oargsr:funckwargsr9s& r $recursively_apply..nsF A"!".7M`dj!$r9r:zUnsupported types (z ) passed to `z?`. Only nested list/tuple/dicts of objects that are valid for `z` should be passed.) rr.r5r8rr4itemsr> TypeError__name__)rBr1r9r:rArCkvsf&ddjl rr>r>Us!,$ &&        D' " "Dz !JJL  )DA$!".7M`dj)     4D*4*6** !$t*]4==/J++4+=+=*>>Q S   K s0 C. c.aaa\V4'g\VR4'dSR8XdRoVPSSR7#\ V\\34'd\VVVV3RlV44#\ V\4'dj\ S\4'dS.oMSf.o\V4!VP4UUu/uFwrVYUS9dTM\VSSSR7bK upp4#V# \dTPS4u#\d8p\ 4'd!\ S\4'dRS 2oRp?M ThRp?ii;iTPSSR7# \dTPS4u#i;iuuppi) aO Recursively sends the elements in a nested list/tuple/dictionary of tensors to a given device. Args: tensor (nested list/tuple/dictionary of `torch.Tensor`): The data to send to a given device. device (`torch.device`): The device to send the data to. non_blocking (`bool`, *optional*, defaults to `False`): If `True`, the transfer to the device is performed asynchronously, which can overlap data movement with computation. Only effective when the device supports it (e.g. CUDA). skip_keys (`str` or `List[str]`, *optional*): A key or list of keys in a dictionary `tensor` whose values should not be sent to the given `device`. Entries with these keys are left on their original device. Returns: The same data structure as `tensor` with all tensors sent to the proper device. tonpuznpu:0) non_blockingznpu:Nc3B<"TFp\VSSSR7xK R#5i)rO skip_keysN)send_to_device)r?tdevicerOrRs& rrD!send_to_device..s#ohncd^AvLT]^^hnsrQ)rr/rMrHAssertionErrorrrintr.r5r8rstrr4rGrS)rrUrOrRerrorrJrTs&fff rrSrSs{&v'&$"7"7 U?F 99V,9? ? FUDM * * ohno   FG $ $ i % %" I  IF|#LLN *DAY1N1fS_kt,uu*    = %99V$ $  !!fc**#F8_F   %99V,9? ? %99V$ $ % sAC9 #F 9EE E!+EEEE//F FcRp\W4#)a/ Recursively gathers the information needed to rebuild a nested list/tuple/dictionary of tensors. Args: data (nested list/tuple/dictionary of `torch.Tensor`): The data to send to analyze. Returns: The same data structure as `data` with [`~utils.TensorInformation`] instead of tensors. cD\VPVPR7#))shapedtype)r r]r^rs&r_get_data_structure/get_data_structure.._get_data_structures v||6<<HHrr=)r1r_s& rget_data_structurerasI 0 77rcRp\W4#)a Recursively gathers the shape of a nested list/tuple/dictionary of tensors as a list. Args: data (nested list/tuple/dictionary of `torch.Tensor`): The data to send to analyze. Returns: The same data structure as `data` with lists of tensor shapes instead of tensors. c,\VP4#r)r5r]rs&r _get_shapeget_shape.._get_shapesFLL!!rr=)r1rds& r get_shaperfs" Z ..rc*Rp\W\R7#)z Recursively initializes tensors from a nested list/tuple/dictionary of [`~utils.TensorInformation`]. Returns: The same data structure as `data` with tensors instead of [`~utils.TensorInformation`]. cV\P!VPRVP/#r^)remptyr]r^r(s&r_initialize_tensor.initialize_tensors.._initialize_tensors"{{K--G[5F5FGGrr9)r>r*)data_structurerks& rinitialize_tensorsrosH /K` aarc\V\\\34'd)\ V4^8Xd\ R\ V4 R24h\V\\34'd\V^,4#\V\4'd*VP4Fp\W,4u# M8\V\P4'g\R\ V4 R24hVP^,#)z Recursively finds the batch size in a nested list/tuple/dictionary of lists of tensors. Args: data (nested list/tuple/dictionary of `torch.Tensor`): The data from which to find the batch size. Returns: `int`: The batch size. z&Cannot find the batch size from empty .z0Can only find the batch size of tensors but got ) rr.r5rlen ValueErrorr4find_batch_sizekeysrrrHr])r1rJs& rrtrts$g.//SY!^A$t*QOPP$ &&tAw'' D' " "A"47+ + ell + +J4PT:,VWXYY ::a=rcJ\V4# \\3dR#i;i)a Same as [`utils.operations.find_batch_size`] except will ignore if `ValueError` and `TypeErrors` are raised Args: data (nested list/tuple/dictionary of `torch.Tensor`): The data from which to find the batch size. Returns: `int`: The batch size. N)rtrsrHr0s&rignorant_find_batch_sizerw s, t$$  "   s ""cRp\W4#)a; Recursively finds tensors in a nested list/tuple/dictionary and converts them to a list of numbers. Args: data (nested list/tuple/dictionary of `torch.Tensor`): The data from which to convert to regular numbers. Returns: The same data structure as `data` with lists of numbers instead of `torch.Tensor`. cVP4P4pVP\P8Xd VP \P 4pVP4#r)detachcpur^rbfloat16rMfloat32tolistrs&r_convert_to_list!listify.._convert_to_list'sF$$& <<5>> )YYu}}-F}}rr=)r1rs& rlistifyrs - 44rcPRp\WRR7p\P!4V#)cVP^8XdVP4R,pVP4'gVP4p\P !V4#N)ndimclone is_contiguous contiguousxm all_gatherrs&r_tpu_gather_one$_tpu_gather.._tpu_gather_one4sL ;;! \\^D)F##%%&&(F}}V$$rTr:)r>r mark_step)rrress& r _tpu_gatherr3s#% O NCLLN Jrc aa\4o\PPoSPP R8Xd1\ RR4'd\PP4VV3Rlp\WRR7#)rz<=z2.8c<VP^8XdVP4R,pVP4'gVP4pSPeSPR8wd~\ P !SPVP4,VPSPR7pS!W4VP!R.VP4R,O5!#\SP4Uu.uFp\ P!V4NK pp\ PP!W4\ P"!V^R7#uupi)rNgloor^rU:NNdim)rrrrbackendrrj num_processesnumelr^rUviewsizerange empty_like distributedrcat)routput_tensors_ gather_opstates& r_gpu_gather_one$_gpu_gather.._gpu_gather_oneJs  ;;! \\^D)F##%%&&(F == $&)@ #[[##flln4ll||N n -!&&r>FKKM",=> > AFeFYFY@Z[@Z1e..v6@ZN[    ( ( @99^3 3\sETr) r rrall_gather_into_tensorrUr4rr synchronizer>)rrrrs& @@r _gpu_gatherrBs` NE!!88I ||E!&6tU&C&C 48 _$ OOrc]tRtRtRtRtR#)DistributedOperationExceptioniiz} An exception class for distributed operations. Raised if the operation cannot be performed due to the shape of the tensors. N)rI __module__ __qualname____firstlineno____doc____static_attributes__rrrrris   rrc0a\S4V3Rl4pV#)zn Verifies that `tensor` is the same shape across all processes. Only ran if `PartialState().debug` is `True`. c |<\4P\P8Xg\4P'g S !V/VB#S P RS P 2pRV9d VR,pM V^,p\4PP\V4P8wdh\RV RVPP R\4PP R\4PP RV R2 4h\V4p\V.4pV^,erVPV^,4\V48HpV'gFRP\!V4UUu.uF wrxRV R V 2NK upp4p \R V R V 24hS !V/VB#uuppi) rqrz%One or more of the tensors passed to z were not on the z+ while the `Accelerator` is configured for z. Please move it to the z before calling z - zProcess z: znCannot apply desired operation due to shape mismatches. All shapes across devices must be valid. Operation: `z` Input shapes: - )r distributed_typer NOdebugrrIrUr4 find_devicerrf gather_objectcountrrjoin enumerate) rArC operationrshapesoutputare_sameir]process_shape_strfunctions *, rwrapper!verify_operation..wrapperws > * *o.@.@ @ H\H\H\T,V, ,**+1X->->,?@ v H%F!WF > % %V)<)A)A A/7 {BSTZTaTaTfTfSghST`TbTiTiTnTnSoo))5)>)>)C)C(DDTU^T__`b 6"x( !9 ||F1I.#f+=H$,MM[dek[l2m[lxqXaS5'3J[l2m$n!3''0k1GHYGZ\ ((( 3nsF8 rrrsf rverify_operationrrs"  8_))4 Nrc0a\S4V3Rl4pV#)z Checks that `verify_operation` failed and if so reports a more helpful error chaining the existing `DistributedOperationException`. c<S!V/VB# \d1pSP RSP 2p\RT R24ThRp?ii;i)rqzError found while calling `z1`. Please see the earlier error for more details.N)rrrI)rArCerrs*, rr"chained_operation..wrappersc T,V, ,, #../q1B1B0CDI/-i[8ij  s A+AArrsf rchained_operationrs"  8_ Nrc\4P\P8Xd \ V4#\4P\ 9d \ V4#V#)a Recursively gather tensor in a nested list/tuple/dictionary of tensors from all devices. Args: tensor (nested list/tuple/dictionary of `torch.Tensor`): The data to gather. Returns: The same data structure as `tensor` with all tensors sent to the proper device. )r rr XLArr rrs&rgatherrsF~&&/*=*==6""  ( (,M M6"" rc$V^8dQhR\/#objectr)formats"r __annotate__rs33#3rc b^RIp\4pVPpVPpVP 4p\ P !W4VP4p\V4p\P!V.\PVR7p\V4U u.uF*p \P!^\PVR7NK, p p \PPW4\!\#RV 444p Rp W8d V ^,p K\P$P'V4p \P(!V 4P+V4pVP-V 4\P.!W,\P0VR7p\V4Uu.uFpWV,V V^,,NK pp\PPVV4.p\3V4Fwpp\!V V,P544pVRVP74P94P;4p\ P<!V4p\?V\@4'dVPCV4KVPEV4K V#uup iuupi)aGather picklable objects from all ranks with padded allgather sizes. On Neuron devices every unique tensor shape triggers a new NEFF compilation. The standard ``all_gather_object`` sizes its byte tensor to the exact max pickle size, which varies per call and causes unbounded compilation cache growth. This variant rounds the allgather size up to the next power of 2 so the number of distinct compiled shapes stays bounded to O(log(max_size)). Nrc3@"TFqP4xK R#5ir)item)r?ss& rrD(_neuron_gather_object..s3Avvxxs)#ior rUrBytesIOpickledumpgetvaluerrrrlongrzerosrrrXmax ByteStorage _from_bufferr rMresize_rjuint8rrr{numpytobytesloadsrr5extendappend)rrrrU group_sizebuf raw_bytes local_sizelocal_size_tensorr size_listmax_size padded_size byte_storage input_tensor coalescedrrresultrobj_size obj_bytesr6s& r_neuron_gather_objectrs NE \\F$$J **,C KK IYJ j\FSJOPZJ[\J[QQejj@J[I\   >33334HK   $$11)F~. 6y|((*+9H%))+113;;= ll9% c4 MM#  MM# / M;] es 0J'+"J,c$V^8dQhR\/#rr)rs"rrrs22s2rc\\4P4Uu.uFpRNK pp\PP W 4VUUu.uF q3FqDNK K upp#uupiuuppir)rr rrrall_gather_object)rroutput_objectsyxs& r_gpu_gather_objectrsa$),.*F*F$GH$Gqd$GNH ''?% 1~!q!AqA~ 11I 2s A'A,c$V^8dQhR\/#rr)rs"rrrs#rc\4P\P8Xd \ R4h\4P\ 9d:\4P PR8Xd \V4#\V4#V#)a Recursively gather object in a nested list/tuple/dictionary of objects from all devices. Args: object (nested list/tuple/dictionary of picklable object): The data to gather. Returns: The same data structure as `object` with all the objects sent to every device. z&gather objects in TPU is not supportedneuron) r rr rNotImplementedErrorr rUr4rr)rs&rrrsh~&&/*=*==!"JKK  ( (,M M > % % 1(0 0!&)) rc(RRlp\W RVR7#)rcF\PPWR7V#)src)rr broadcast)rr s&&r_gpu_broadcast_one*_gpu_broadcast.._gpu_broadcast_ones ##F#4 rT)r:r rr=)r1r rs&& r_gpu_broadcastrs /4UX YYrc taa\V\\34'd \VV3Rl\ V444#\V\ 4'dE\ V4!VP4UUu/uFwr4V\VS RV 2R7bK upp4#\P!SVV3Rl4#uuppi)c3L<"TFwr\VS RV 2R7xK R#5i)rnameN)_tpu_broadcast)r?rrTrs& rrD!_tpu_broadcast..s)"gUfTQ>!TF!A3-#H#HUfrFrrc<VS,#rr)rr s&r _tpu_broadcast..s !C&r) rr5r.r8rrr4rGrr mesh_reduce)rr rrJrKs&ff rrrs&4-((&"gU^_eUf"ghh FG $ $F|RXR^R^R`aR`$!Qq$q} EER`abb >>$(8 99bs2B4 cRp\4p\P!V\PVPR7pVedVP p\ VP,p\P!\V4V.,\R7VR\V4^,%\VRR7pW3P4,p\VRR^,4pVRRpW63#)z] Grabs the shape of `tensor` only available on one process and returns a tensor of its shape rNrisum reductionrr) r rrjrXrUr]TENSOR_TYPE_TO_INTr^rr5rrreducenonzero)rmax_tensor_dimensionr base_tensorr] tensor_dtyper^s& rgather_tensor_shaper&.s ! NE++2%))ELLYK  )&,,7 (- T%[L>5QY\(] $c%j1n%6K1134K  BC # $Ecr"K  rc8V^8dQhR\P/#)rreturn)rr)rs"rrrGs++5<<+rc\4p\V4wr#Vf=\P!V\V,R7P VP 4p\VRR7#)at Copies a tensor that only exists on a single device and broadcasts it to other devices. Differs from `broadcast` as each worker doesn't need to know its shape when used (and tensor can be `None`) Args: tensor (`torch.tensor`): The tensor that should be sent to all devices. Must only have it be defined on a single device, the rest should be `None`. rirr)r r&rrTENSOR_INT_TO_DTYPErMrUr!)rrr]r^s& rcopy_tensor_to_devicesr+GsN NE&v.LE ~U*=e*DEHHV &E **rc$V^8dQhR\/#r from_processrX)rs"rrrYsCrc\4P\P8Xd\ WRR7#\4P\ 9d \ WR7#V#)a Recursively broadcast tensor in a nested list/tuple/dictionary of tensors to all devices. Args: tensor (nested list/tuple/dictionary of `torch.Tensor`): The data to gather. from_process (`int`, *optional*, defaults to 0): The process from which to send the data Returns: The same data structure as `tensor` with all tensors broadcasted to the proper device. zaccelerate.utils.broadcast)r rr )r rr rrr r)rr.s&&rr r XsI~&&/*=*==f=YZZ  ( (,M Mf77 rc$V^8dQhR\/#r-r/)rs"rrrns22S2rc^RIp\4pVPpVPV8XdCVP 4p\ P !W4VP4p\V4pMRp^p\P!V.\PVR7p\PPV4\VP44p Rp W8d V ^,p KVPV8XdW\P P#V4p \P$!V 4P'V4p V P)V 4M'\P*!V \P,VR7p \PP/WR7\ P0!V RV P34P54P744p \9\V44F pW,W&K V#)aBroadcast a list of picklable objects with padded tensor sizes. On Neuron devices ``ProcessGroupNeuron.broadcast()`` is implemented as an allreduce, so every unique tensor shape triggers a new NEFF compilation. The standard ``broadcast_object_list`` sizes its byte tensor to the exact serialized length, which varies per call and causes unbounded compilation cache growth. This variant rounds the broadcast tensor size up to the next power of 2, bounding the number of distinct compiled shapes to ~O(log(max_size)). Nrrr r)rr rU process_indexrrrrrrrrrr all_reducerXrrrr rMrrrr rr{rrr) object_listr.rrrUrrr size_tensor real_sizerr data_tensorrrs&& r_neuron_broadcast_object_listr9ns NE \\F l*jjl K%LLN ^   ,, |5::fMK   -K$$&'IK  !  l*((55i@ &&|477? K(kk+U[[P   >\\+jy1557==?GGI JF 3{# $ % rc$V^8dQhR\/#r-r/)rs"rrrsSrca\4P\P8Xd5\ V4F#wr#\ P !RVV3Rl4W&K% V#\4P\9dT\4PPR8Xd\VSR7V#\PPVSR7V#)ar Broadcast a list of picklable objects from one process to the others. Args: object_list (list of picklable objects): The list of objects to broadcast. This list will be modified inplace. from_process (`int`, *optional*, defaults to 0): The process from which to send the data. Returns: The same list containing the objects from process 0. z&accelerate.utils.broadcast_object_listc<VS,#rr)rr.s&rr'broadcast_object_list..s efgsetrr)r.r )r rr rrrrr rUr4r9rrbroadcast_object_list)r5r.rr6s&f rr>r>s~&&/*=*== ,FA^^,TVY[tuKN-   ( (,M M > % % 1 )+L Q     3 3K\ 3 R rc Rp\W@V4#)a* Recursively takes a slice in a nested list/tuple/dictionary of tensors. Args: data (nested list/tuple/dictionary of `torch.Tensor`): The data to slice. tensor_slice (`slice`): The slice to take. Returns: The same data structure as `data` with all the tensors slices. cW,#rr)r tensor_slices&&r _slice_tensor$slice_tensors.._slice_tensors ##rr=)r1rAr3rrBs&&&& r slice_tensorsrDs$ ], ??rcaa\S^,\\34'd8\S^,VV3Rl\ \ S^,4444#\S^,\ 4'da\S^,4!S^,P4UUu/uF&q"\SUu.uF q3V,NK upSR7bK( upp4#\S^,\P4'd\P!SSR7#\S\\34'd\ S4^8Xd S^,#\R\S^,4 24huupiuuppi)a Recursively concatenate the tensors in a nested list/tuple/dictionary of lists of tensors with the same shape. If there is only a single batch of data, it is returned as-is. Args: data (nested list/tuple/dictionary of lists of tensors `torch.Tensor`): The data to concatenate. dim (`int`, *optional*, defaults to 0): The dimension on which to concatenate. Returns: The same data structure as `data` with all the tensors concatenated. c3t<"TF(p\SUu.uF q"V,NK upSR7xK* R#uupi5i)rN) concatenate)r?rdr1rs& rrDconcatenate..s4#lXkSTKt0Dt!1t0D#$N$NXk0Ds 83 8rz%Can only concatenate tensors but got )rr.r5r8rrrrr4rurGrrrrH)r1rrJrHsff rrGrGs$q'E4=))$q'#lX]^abfghbi^jXk#lmm DGW % %DG}UYZ[U\UaUaUcdUcPQD-ADqddD-As!KKUcdee DGU\\ * *yy3'' D5$- ( (SY!^Aw?T!W OPP .Bds&E1 4E, E1 ,E1 c]tRtRtRtR#)CannotPadNestedTensorWarningirN)rIrrrrrrrrKrKsrrKc *RRlp\W@RWVR7#)a Recursively pad the tensors in a nested list/tuple/dictionary of tensors from all devices to the same size so they can safely be gathered. Args: tensor (nested list/tuple/dictionary of `torch.Tensor`): The data to gather. dim (`int`, *optional*, defaults to 0): The dimension on which to pad. pad_index (`int`, *optional*, defaults to 0): The value with which to pad. pad_first (`bool`, *optional*, defaults to `False`): Whether to pad at the beginning or the end. caa a \VRR4'd\P!R\4V#S\ VP 48gS\ VP 4)8dV#S^8dS\ VP 4, o\ P!VP VPR7R,p\V4P4p\V3RlV44o S VP S,8XdV#VP o \S 4pS VS&VP\V44V,pV'd\\;QJd+.VV V 3Rl\\ V444FNK 5M$!VV V 3Rl\\ V4444pMX\;QJd*.VV 3Rl\\ V444FNK 5M#!VV 3Rl\\ V4444pWV&V#) is_nestedFzHCannot pad nested tensors without more information. Leaving unprocessed.)rUNc34<"TF qS,xK R#5irr)r?rrs& rrDFpad_across_processes.._pad_across_processes..s-u!vvusc3z<"TF0qS8Xd\SSS,, S4M \R4xK2 R#5irslice)r?rrrold_sizes& rrDrPs4`t[\#Xh#.95QU;V`ts8;c3l<"TF)qS8Xd\^SS,4M \R4xK+ R#5irrRr?rrrTs& rrDrPs,oZnUVCxE!Xc]3U4[PZn14)getattrwarningswarnrKrrr]rrrUrr{rr5 new_zerosr.r) rr pad_index pad_firstrsizesnew_size new_tensorindicesrrTs &f&& @@r_pad_across_processes3pad_across_processes.._pad_across_processess 6; . . MMZ, M #fll# #sc&,,.?-?'?M 7 3v||$ $C||FLL?Et   "-u-- v||C( (M<<>  %%eHo6B e`efijrfs`tee`efijrfs`tGeoZ_`cdl`mZnoeeoZ_`cdl`mZnooG$7rT)r:rr\r]rrFr=)rrr\r]rbs&&&& rpad_across_processesres!" D 4Sir rc .RRlp\VVRVVVR7#)z Takes a `tensor` of arbitrary size and pads it so that it can work given `num_processes` needed dimensions. New tensors are just the last input repeated. E.g.: Tensor: ([3,4,4]) Num processes: 4 Expected result shape: ([4,4,4]) caa W,pWV,, pW,^8Xd W!, pMW!V,, pWVV,u;8d^8d MMWV, pVPo \S 4pW,V^&VP\V44p\;QJd*.VV 3Rl\ \ V444FNK 5M#!VV 3Rl\ \ V4444p WV &V#)rc3l<"TF)qS8Xd\^SS,4M \R4xK+ R#5irrRrVs& rrD@pad_input_tensors.._pad_input_tensors..?s,kVjQR8a#/tLVjrW)r]r5r[r.rrr) r batch_sizerr remainder last_inputsto_padr_r`rarTs &&&f @r_pad_input_tensors-pad_input_tensors.._pad_input_tensors0s/  $=>  &! +"/F"M&ABF & ,1 , )F<<> ) %%eHo6 %kV[\_`h\iVjk%%kV[\_`h\iVjkk$7rT)r:rjrrrr=)rrjrrrns&&&& rpad_input_tensorsrp%s(&  #   rc(RRlp\W0RWR7#)a. Recursively reduce the tensors in a nested list/tuple/dictionary of lists of tensors across all processes by the mean of a given operation. Args: tensor (nested list/tuple/dictionary of `torch.Tensor`): The data to reduce. reduction (`str`, *optional*, defaults to `"mean"`): A reduction method. Can be of "mean", "sum", "max", or "none" scale (`float`, *optional*): A default scaling value to be applied after the reduce, only valid on XLA. Returns: The same data structure as `data` with all the tensors reduced. c\4pVP4pVP\P8XdV#VP\P 8Xdk\ P!4VR8Xd\ PM\ Pp\ P!WT.V4\ P!4MeVPP\9dGVR8Xd\PM\Pp\ P"PWF4VR8XdWCP$,pV#)rmean)r rrr rrrr REDUCE_MAX REDUCE_SUMr4valuer rMAXSUMrrr)rrscaler cloned_tensorxla_optorch_ops&&& r_reduce_across_processes(reduce.._reduce_across_processes_s   ! !_%7%7 7  ! !_%8%8 8 LLN&/5&8R]]bmmF MM&/5 9 LLN  # # ) )-N N'0E'9x||x||H    ( ( A   00 0MrT)r:rryrsg?r=)rrryr}s&&& rr!r!Ms$*  di rc(RpRp\WVR7#)aZ Recursively converts the elements nested list/tuple/dictionary of tensors in FP16/BF16 precision to FP32. Args: tensor (nested list/tuple/dictionary of `torch.Tensor`): The data to convert from FP16/BF16 to FP32. Returns: The same data structure as `tensor` with all tensors that were in FP16/BF16 precision converted to FP32. c"VP4#r)floatrs&r_convert_to_fp32)convert_to_fp32.._convert_to_fp32s||~rc\V4;'g\VR4;'d.VP\P\P 39#ri)rr/r^rfloat16r|rs&r_is_fp16_bf16_tensor-convert_to_fp32.._is_fp16_bf16_tensorsH'CC767+C   MM NNZ J  rrmr=)rrrs& rconvert_to_fp32rys -AU VVrc6a]tRtRtoRtRtRtRtRtVt R#)ConvertOutputsToFp32iaE Decorator to apply to a function outputting tensors (like a model forward pass) that ensures the outputs in FP16 precision will be convert back to FP32. Args: model_forward (`Callable`): The function which outputs we want to treat. Returns: The same function as `model_forward` but with converted outputs. c(Wn\W4R#r) model_forwardr)selfrs&&r__init__ConvertOutputsToFp32.__init__s*t+rc8\VP!V/VB4#r)rr)rrArCs&*,r__call__ConvertOutputsToFp32.__call__st114B6BCCrc.\P!R4h)zCannot pickle a prepared model with automatic mixed precision, please unwrap the model with `Accelerator.unwrap_model(model)` before pickling it.)r PicklingError)rs&r __getstate__!ConvertOutputsToFp32.__getstate__s"" `  r)rN) rIrrrrrrrr__classdictcell__) __classdict__s@rrrs ,D  rrc8a\S4oV3RlpSVnV#)c<S!V/VB#rr)rArCrs*,rforward(convert_outputs_to_fp32..forwardsd-f--r)r __wrapped__)rrsf rconvert_outputs_to_fp32rs!(7M.(G NrcV\V\4'd-VP4Fp\V4pVfKVu# R#\V\\ 34'dVFp\V4pVfKVu# R#\V\ P4'd VP#R#)z Finds the device on which a nested dict/list/tuple of tensors lies (assuming they are all on the same device). Args: (nested list/tuple/dictionary of `torch.Tensor`): The data we want to know the device of. N) rrvaluesrr.r5rrrU)r1r6rUs& rrrs$  ;;=C %F! ! D5$- ( (C %F!  D%,, ' '{{ (rc#"\4P\P8wg?\4Pe4\4PP 4'g \ 4pM!^RIpVPPWW#R7pV;_uu_4RxRRR4R# +'giR#;i5i)z| Wrapper around `deepspeed.runtime.zero.GatheredParameters`, but if Zero-3 is not enabled, will be a no-op context manager. N) modifier_rank fwd_moduleenabled) r rr DEEPSPEEDdeepspeed_pluginis_zero3_init_enabledr deepspeedzeroGatheredParameters)paramsrrrgather_param_contextrs&&&& rrrs**o.G.GG++7 "33IIKK*}(~~@@ J A        sBCB-# C- B> 8 C)FNr)rzbroadcast tensorr)NNrdr)NNT)YrrrYcollections.abcr contextlibrr functoolsrrtypingrrrr r constantsr dataclassesr r importsrrrversionsrtorch_xla.core.xla_modelcore xla_modelrtorch.distributedrrr&r*r2r8r>rSrarfrortrwrrr Exceptionrrrrrrrrrrdoublehalfr|rint8int16int32int64boolr rGr*r&r+r r9r>rDrG UserWarningrKrerpr!rrrrr)rJrKs00rrs,#2+ 28; '))!##*, 6]$0?0X]0f7t8$/$ b."5. $PN I  F&&3l2*Z: KK LL! JJ NNA KK JJ KK KK KK JJ );(@(@(BC(Bqt(BC2+"*2j0@(Q4 ; 44n%P((VW0  4 *E Ds1 G7