+ j($^RIt^RIt^RIt^RIt^RIHtHt^RIHtH t H t ^RI t ^RI t ^RI Ht^RIHuHt^RIHt^RIHt^RIHt^RIHt^RIHt^R I H!t"]%]&,]%]%]&,,,]PN,],](R ]&3,,]PR,t*^RIH+t+Rt,RRRllt-RRRllt.RRRllt/RRRllt0RRRllt1RRRllt2RRRllt3RRRllt4RR R!llt5RR"R#llt6RR$R%llt7RR&R'llt8RR(R)llt9RR*R+llt:RR,R-llt;R.tR3R4lt?R5t@] PPR6]?]@R77R8R9ltCR:tD] PPR;]C]DR77R<R=ltER>tF] PPR?]E]FR77R@RAltGRBtH] PPRC]G]HR77RDREltIRFtJ] PPRG]I]JR77] PPRH]E]FR77] PPRI]G]HR77] PPRJ]I]JR77RKRLltKRMtL] PPRN]K]LR77RORPltMRQtN] PPRR]M]NR77RSRTltORUtP] PPRV]O]PR77RRWRXlltQ!RYRZ] P4tSRR[R\lltTRR]R^lltURR_R`lltV] PPRaRRbRc7RdRel4tX]XPRfRgl4tZRhRilt[Rjt\]XP][]\R77RkRllt]RmRnlt^]PRRoRpll4t`RqtaRrtbRstcRttdRuteRvtfRwtgRxthRytiRztjR{tkR|tlR}tmR~tnRtoRtpRtqRtrRtsRttRtuRtv] PPRR4tx]xPR]dR4]xPR]lR4]xPR]kR4]xPR]nR4]xPR]eR4]xPR]fR4]xPR]gR4]xPR]hR4]xPR]qR4]xPR]rR4]xPR]sR4]xPR]tR4]xPR]uR4]xPR]vR4]xPR]pR4]xPR]cR4]xPR]mR4] PPRR4tz]zPR]rR4]zPR]tR4]zPR]pR4] PPP] PPPXEP4] PPP] PPPX4] PPP] PPEPEP4] PPP] PPEP4] PPP] PPEPEP4] PPP] PPEP4] PPP] PPEPEP4] PPP] PPEP4] PPRR4t] PPRR4t.ROt]EP],t]Fat]^]EPR4t]!]R] 24t]EP]] EP EP"R7]P]]R4Kc RRRlltRRRllt]EP(EP*R]EP(EP,R]EP(EP.R]EP(EP0R]EP(EP2R]EP(EP4R]EP(EP6R]EP(EP8R/tRRRlltRRRlltRRRlltRRRlltRRRlltRRRlltRRltRRlt^RIHtHtHtHtH/tH.tH=tHtHtHtHtH1tH5tRtRtRtRtRtRtRtRtRt]]]]]]]]]]]]]]]]]]]]]]]]]]/ tRRlt!RR] EPEP4tR# ]d ^RIHtE Li;i ]#d]PH!R ^R 7R t"E Li;i)N)CallableSequence)Anycast TYPE_CHECKING)_maybe_view_chunk_cat) DeviceMesh)get_proxy_mode)_functional_collectives_impl) tree_map_only)is_dynamo_compilingzdUnable to import torchdynamo util `is_torchdynamo_compiling`, so won't support torchdynamo correctly stacklevelcR#)Frj/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/distributed/_functional_collectives.pyis_torchdynamo_compilingr"srzdist.tensor.DeviceMesh)_chunk_or_narrow_catcT\PPPV4#)z Wait on a tensor returned by the collectives ops. Waiting follows device semantics, which means blocking on CPU and synchronizing streams on CUDA. )torchops_c10d_functional wait_tensor)tensors&rrrs 99 % % 1 1& 99rc\V^8dQhR\PR\R\R\/#)selfsrcgrouptagrTensorint RANK_TYPESstr)formats"r __annotate__r(s, & &ELL &s &: &C &rc\W#4p\PPP W\ V44p\ V4#)a Broadcasts the tensor to all processes in the given process group. Args: src (int): Source rank group (ProcessGroup or List[int]): The process group to work on. tag (str, optional): A unique identifier for the collective. Default: empty string )_resolve_grouprrr broadcast_group_or_group_name_maybe_wrap_tensor)rrr r!rs&&&& rr+r+s@ 5 &E YY ' ' 1 1 '.F f %%rc\V^8dQhR\PR\R\R\/#)rrreduceOpr r!)rr#r&r%)r's"rr(r(s,&&U\\&S&&#&rc\W#4p\PPP WP 4\ V44p\V4#)a Reduces the tensor data across all machines in such a way that all get the final result. The input tensor is left unmodified. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. )r*rrr all_reducelowerr,r-)rr/r r!rs&&&& rr1r1sG" 5 &E YY ' ' 2 2 nn 4U ;F f %%rc |V^8dQhR\PR\R\R\R\P/#rr gather_dimr r!returnr")r's"rr(r(sB'' ,,'' ' '  \\ 'rc\W#4p\P!V4p\PP P W\V44p\V4pV^8wd\V\4'di\VP4pV^8d\P!V^V4M^pV^,V8H;'dV^8Hp V 'gVP4p\!WdV4pV#)a Gather tensor data across from all machines and concatenate over ``gather_dim``. Note that it currently only supports gather_dim = 0. The input tensor is left unmodified. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. )r*c10d_get_group_size_by_namerrrall_gather_into_tensorr,r- isinstanceAsyncCollectiveTensorlistshapemathprodwaitr) rr5r r! group_sizerresr> numel_between can_use_views &&&& rall_gather_singlerFs, 5 &E--e4J YY ' ' > > .u5F V $CQ c0 1 1OE>H1nDIIeAj&9:RSM 8z1HHmq6HLhhj#CZ@ Jrc\V^8dQhR\PR\R\R\/#rrr5r r!r")r's"rr(r(s4;; ,,;; ; ;rc\WW#4#)a$ Gather tensor data across from all machines and concatenate over ``gather_dim``. Note that it currently only supports gather_dim = 0. This function is the same as all_gather_single but will propagate the backwards gradient across workers. See all_gather_single for more details on usage. )rFrr5r r!s&&&&rall_gather_single_autogradrKs" Tu ::rc hV^8dQhR\PR\R\R\R\/#rrr/ scatter_dimr r!rr#r&r$r%)r's"rr(r(s>'' ,,'''  ' 'rc\W44p\P!V4pVPV4V,^8wd"\ RVP^4 RV R24hV^8wd\ WV^R7p\ PPPVVP4V\V44p\V4pV#)a Reduces the tensor data across all machines in such a way that all get the final result, then scatter the results to corresponding ranks. The input tensor is left unmodified. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. zinput dimension 0 (" must be a multiple of group_size )) narrow_dimcat_dim) r*r8r9sizeAssertionErrorrrrrreduce_scatter_tensorr2r,r-)rr/rNr r!rBrrCs&&&&& rreduce_scatter_singlerXs, 5 &E--e4J yy *a/!$))A,/QR\Q]]^ _  a#DVWX YY ' ' = = U# F V $C Jrc hV^8dQhR\PR\R\R\R\/#rMrO)r's"rr(r(!sEJJ ,,JJJ  J Jrc\WW#V4#)a` Reduces the tensor data across all machines in such a way that all get the final result, then scatter the results to corresponding ranks. This function is the same as reduce_scatter_single but will propagate the backwards gradient across workers. Currently only the "sum" reduceOp is supported. See reduce_scatter_single for more details on usage. )rXrr/rNr r!s&&&&&rreduce_scatter_single_autogradr\!s& !S IIrc |V^8dQhR\PR\R\R\R\P/#r4r")r's"rr(r(=sB ; ; ,, ; ;  ; ;  \\ ;rc\PP4'g\P!R\ ^R7\ WW#4#)z`torch.distributed._functional_collectives.all_gather_tensor` is deprecated. Please use `torch.distributed._functional_collectives.all_gather_single` instead.r)rcompiler is_compilingwarningswarn FutureWarningrFrJs&&&&rall_gather_tensorrd=s> >> & & ( (  `   Tu ::rc\V^8dQhR\PR\R\R\/#rHr")r's"rr(r(Ms: D D ,, D D  D Drc\PP4'g\P!R\ ^R7\ WW#4#)z`torch.distributed._functional_collectives.all_gather_tensor_autograd` is deprecated. Please use `torch.distributed._functional_collectives.all_gather_single_autograd` instead.r)rr_r`rarbrcrKrJs&&&&rall_gather_tensor_autogradrgMs> >> & & ( (  i   &d CCrc hV^8dQhR\PR\R\R\R\/#rMrO)r's"rr(r(]sEJJ ,,JJJ  J Jrc\PP4'g\P!R\ ^R7\ WW#V4#)z`torch.distributed._functional_collectives.reduce_scatter_tensor` is deprecated. Please use `torch.distributed._functional_collectives.reduce_scatter_single` instead.r)rr_r`rarbrcrXr[s&&&&&rrWrW]s@ >> & & ( (  d   !S IIrc hV^8dQhR\PR\R\R\R\/#rMrO)r's"rr(r(nsESS ,,SSS  S Src\PP4'g\P!R\ ^R7\ WW#V4#)z`torch.distributed._functional_collectives.reduce_scatter_tensor_autograd` is deprecated. Please use `torch.distributed._functional_collectives.reduce_scatter_single_autograd` instead.r)rr_r`rarbrcr\r[s&&&&&rreduce_scatter_tensor_autogradrlns@ >> & & ( (  m   *$+c RRrc V^8dQhR\\P,R\R\R\R\\P,/#)rrr/r r!r6)r=rr#r&r%)r's"rr(r(sF66 u|| 6(+64>6EH6 %,,6rc\W#4p\PPP VVP 4\ V44p\\\V44#)a Reduces a list of tensors across all machines in such a way that all get the final result. The all tensors in the input list are left unmodified. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. ) r*rrrall_reduce_coalescedr2r,r=mapr-)rr/r r! tensor_lists&&&& rrorosR& 5 &E)),,AA U#K & 4 55rc V^8dQhR\\P,R\R\R\\P,/#rrr r!r6r=rr#r%r&)r's"rr(r(s>66 u|| 6%/6696 %,,6rc\W4p\P!V4p\PP P VV\V44p\\\V44#)a Gather a list of tensors across from all machines. Note that it currently only supports gather_dim = 0. The input tensor is left unmodified. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. ) r*r8r9rrr all_gather_into_tensor_coalescedr,r=rpr-)rr r!rBrqs&&& rall_gather_single_coalescedrws[& 5 &E--e4J)),,MM U#K & 4 55rc V^8dQhR\\P,R\R\\,R\ R\R\\P,/#rinputsr/rNr r!r6r=rr#r&r$r%)r's"rr(r(sZ-6-6  -6-6c-6  -6 -6  %,, -6rc \W44p\P!V4p\V4\V48wd%\ R\V4 R\V4 R24h\ \ W 44FwpwrxVPV4V,^8wd'\ RV RVPV4 RV RV 24hV^8wgKT\P!WVR7p \P!V 4W&K \PPPVVP4V\V44p \!\#\$V 44#) a Reduces a list of tensors across all machines in such a way that all get the final result, then scatter the results to corresponding ranks. The input tensors are left unmodified. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. zLength of scatter_dim (z) must equal length of inputs (rRzinput dimension z (rQz for tensor at index )dim)r*r8r9lenrV enumerateziprUrchunkcatrrreduce_scatter_tensor_coalescedr2r,r=rpr-) rzr/rNr r!rBidxr}rrqs &&&&& rreduce_scatter_single_coalescedrsH, 5 &E--e4J ;3v;&%c+&6%77VWZ[aWbVccd e  (K(@A]c ;;s j (A - "3%r&++c*:);;]^h]ii~@CDE  !8++fcBK))K0FKB)),,LLU# K & 4 55rc V^8dQhR\\P,R\R\R\\P,/#rsrt)r's"rr(r(s> 9 9 u||  9%/ 969 9 %,, 9rc\PP4'g\P!R\ ^R7\ WV4#)z`torch.distributed._functional_collectives.all_gather_into_tensor_coalesced` is deprecated. Please use `torch.distributed._functional_collectives.all_gather_single_coalesced` instead.r)rr_r`rarbrcrw)rr r!s&&&rrvrvs> >> & & ( (  j   'tC 88rc V^8dQhR\\P,R\R\\,R\ R\R\\P,/#ryr{)r's"rr(r(sbVV  VVcV  V V  %,, Vrc\PP4'g\P!R\ ^R7\ WW#V4#)z`torch.distributed._functional_collectives.reduce_scatter_tensor_coalesced` is deprecated. Please use `torch.distributed._functional_collectives.reduce_scatter_single_coalesced` instead.r)rr_r`rarbrcr)rzr/rNr r!s&&&&&rrrsA >> & & ( (  n   +6[QT UUrc\V\PP4'g\ R\ V4 24h\P PVP4\PP4'dR#VPp\VP4^8dEVP^,pVPRJ;'dVPP'*#R#)z$Expected torch._ops.OpOverload, got FN)r;r_ops OpOverloadrVtype_C%_dispatch_has_kernel_for_dispatch_keyname DispatchKeyCompositeImplicitAutograd_schemar~ arguments alias_infois_write)tgtschema first_args& r _is_view_opr s c5::00 1 1CDI;OPP xx55  E%%?? [[F 6  q $$Q' ##4/UU 8L8L8U8U4UU!rc V^8dQhR\PR\\,R,R\\,R,R\R\ R\P/#rroutput_split_sizesNinput_split_sizesr r!r6rr#r=r$r%r&)r's"rr(r(s^2&2& ,,2&S D(2&Cy4'2&  2& 2&  \\ 2&rcrVeI\;QJdRV4F 'dK RM RM !RV44'g\RV 24hVeI\;QJdRV4F 'dK RM RM !RV44'g\RV 24h\W44p\P!V4pVeVf8VfVe \R4hVP ^,V,.V,pTp\ PPPVVV\V44p\V4#)a Each process splits input tensor and then scatters the split list to all processes in a group. Then concatenate the received tensors from all the processes in the group and return single output tensor. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one dimension of the DeviceMesh :: N.B. If you pass a PG or a 1D list to perform a MPMD collective, the compiler won't be able to recover that information and perform collective algebraic optimization. Use other forms of input for that. c3b"TF%p\V\\P34xK' R#5iNr;r$rSymInt.0rUs& r $all_to_all_single..2s' >PdJtc5<<0 1 1>P-/FTz2All output_split_sizes must be int or SymInt, got c3b"TF%p\V\\P34xK' R#5irrrs& rrr9s%WEVT:dS%,,$788EVrz1All input_split_sizes must be int or SymInt, got z^output_split_sizes and input_split_sizes must either be specified together or both set to None) allrVr*r8r9r>rrrall_to_all_singler,r-)rrrr r!rBrs&&&&& rrrs9,%s >P sss >P   !DEWDXY $sWEVWsssWEVWWW CDUCVW  5 &E--e4J!%6%>"*/@/H 9 #jjmz9:ZG. YY ' ' 9 9 U# F f %%rc V^8dQhR\PR\\,R,R\\,R,R\R\ R\P/#rr)r's"rr(r(Psf V V ,, VS D( VCy4' V  V V  \\ Vrc\WW#V4#)z2 Same as all_to_all_single but supports autograd. )r)rrrr r!s&&&&&rall_to_all_single_autogradrPs T7HQT UUrc8V^8dQhR\P/#r grad_outputrr#)r's"rr(r(cs  5<< rcV#)z Backward for wait_tensor: identity (no-op). Wait is just a synchronization primitive, so gradient flows through unchanged. Args: ctx: Context object grad_output: Gradient from downstream operations Returns: Gradient unchanged (identity) rctxrs&&rwait_tensor_backwardrcs  rcR#)z Setup context for wait_tensor backward. Args: ctx: Context object to save state for backward inputs: Tuple of (tensor,) output: Output from forward pass Nrrrzoutputs&&&rwait_tensor_setup_contextrrs rz_c10d_functional::wait_tensor) setup_contextc8V^8dQhR\P/#rr)r's"rr(r(s++%,,+rcVPpVPpVR9d\RV R24h\PP P VP4W24p\V4RR3#)aS Backward for all_reduce: all_reduce with same reduce_op. Forward aggregates tensors, backward aggregates gradients. Args: ctx: Context object grad_output: Gradient from downstream operations Returns: Tuple of (grad_input, grad_group_name, grad_reduce_op) grad_group_name and grad_reduce_op are None (not differentiable) zCall_reduce backward only supports 'sum' and 'avg' reductions, got ''N)sumavg) group_name reduce_op RuntimeErrorrrrr1 contiguousr)rrrrrs&& rall_reduce_backwardrszJ I &QR[Q\\] ^  YY ' ' 2 2 )F v d **rcFVwr4pWPnVP4VnR#)z Setup context for all_reduce backward. Args: ctx: Context object to save state for backward inputs: Tuple of (input, reduce_op, group_name) output: Output from forward pass Nrr2r)rrzrinputrrs&&& rall_reduce_setup_contextrs!$* EjNOO%CMrz_c10d_functional::all_reducec8V^8dQhR\P/#rr)r's"rr(r(s++ell+rcVPpVPp\PPP VP 4RVV4p\V4RR3#)a Backward for all_gather_into_tensor: reduce_scatter with sum. Forward gathers tensors from all ranks, backward scatters gradients back with sum reduction. Args: ctx: Context object with group_name and group_size grad_output: Gradient from downstream operations Returns: Tuple of (grad_input, grad_group_size, grad_group_name) grad_group_size and grad_group_name are None (not differentiable) rN)rrBrrrrWrr)rrrrBrs&& rall_gather_into_tensor_backwardrs[JJYY ' ' = =   F v d **rc(Vwr4pWPnW@nR#)z Setup context for all_gather_into_tensor backward. Args: ctx: Context object to save state for backward inputs: Tuple of (input, group_size, group_name) output: Output from forward pass NrrB)rrzrrrBrs&&& r$all_gather_into_tensor_setup_contextrs%+!EzNNrz(_c10d_functional::all_gather_into_tensorc8V^8dQhR\P/#rr)r's"rr(r(s11U\\1rcVPpVPpVPpVR8wd\RV R24h\P P PVP4VV4p\V4RRR3#)a Backward for reduce_scatter_tensor: all_gather. Forward reduces and scatters tensors to ranks, backward gathers gradients from all ranks. Args: ctx: Context object with group_name, group_size, and reduce_op grad_output: Gradient from downstream operations Returns: Tuple of (grad_input, grad_reduce_op, grad_group_size, grad_group_name) grad_reduce_op, grad_group_size, grad_group_name are None (not differentiable) rzCreduce_scatter_tensor backward only supports 'sum' reduction, got 'rN) rrBrrrrrr:rr)rrrrBrrs&& rreduce_scatter_tensor_backwardrsJJ IEQR[Q\\] ^  YY ' ' > > F v dD 00rcRVwr4rVW`nWPnVP4VnR#)z Setup context for reduce_scatter_tensor backward. Args: ctx: Context object to save state for backward inputs: Tuple of (input, reduce_op, group_size, group_name) output: Output from forward pass NrrBr2r)rrzrrrrBrs&&& r#reduce_scatter_tensor_setup_contextr s&06,EjNNOO%CMrz'_c10d_functional::reduce_scatter_tensorc8V^8dQhR\P/#rr)r's"rr(r(!s111rcVPpVPpVPp\PP P VP4VVV4p\V4RRR3#)a Backward for all_to_all_single: all_to_all with reversed split sizes. Forward does all-to-all with specified split sizes, backward reverses them. Args: ctx: Context object with group_name, output_split_sizes, and input_split_sizes grad_output: Gradient from downstream operations Returns: Tuple of (grad_input, grad_output_split_sizes, grad_input_split_sizes, grad_group_name) All except grad_input are None (not differentiable) N) rrrrrrrrr)rrrrrrs&& rall_to_all_single_backwardr!slJ//--YY ' ' 9 9  F v dD 00rc4Vwr4rVW`nW@nWPnR#)z Setup context for all_to_all_single backward. Args: ctx: Context object to save state for backward inputs: Tuple of (input, output_split_sizes, input_split_sizes, group_name) output: Output from forward pass N)rrr)rrzrrrrrs&&& rall_to_all_single_setup_contextr=s"@Fc(Vwr4pWPnW@nR#)z Setup context for all_gather_into_tensor_coalesced backward. Args: ctx: Context object to save state for backward inputs: Tuple of (tensor_list, group_size, group_name) output: Output from forward pass Nr)rrzrrqrBrs&&& r.all_gather_into_tensor_coalesced_setup_contextrs+1'KZNNrz2_c10d_functional::all_gather_into_tensor_coalescedcNV^8dQhR\\P,/#rr)r's"rr(r(sCCU\\@RCrcJVPpVPpVPpVR8wd\RV R24h\P P PVUu.uFqUP4NK upVV4p\\\V44RRR3#uupi)a Backward for reduce_scatter_tensor_coalesced: all_gather each gradient. Forward reduces and scatters tensors to ranks, backward gathers gradients from all ranks. Args: ctx: Context object with group_name, group_size, and reduce_op grad_outputs: Gradients from downstream operations (one per input tensor) Returns: Tuple of (grad_inputs..., grad_reduce_op, grad_group_size, grad_group_name) grad_reduce_op, grad_group_size, grad_group_name are None (not differentiable) rzMreduce_scatter_tensor_coalesced backward only supports 'sum' reduction, got 'rN) rrBrrrrrrvrr=rpr)rrrrBrrrs&& r(reduce_scatter_tensor_coalesced_backwardrsJJ IE[\e[ffg h  )),,MM5AB\k   !\BK [+. /tT BB Cs"B cRVwr4rVW`nWPnVP4VnR#)z Setup context for reduce_scatter_tensor_coalesced backward. Args: ctx: Context object to save state for backward inputs: Tuple of (tensor_list, reduce_op, group_size, group_name) output: Output from forward pass Nr)rrzrrqrrBrs&&& r-reduce_scatter_tensor_coalesced_setup_contextrs&6<2KJNNOO%CMrz1_c10d_functional::reduce_scatter_tensor_coalescedc V^8dQhR\PR\\,R\R\ R\P/#)rrsrc_dstr r!r6r)r's"rr(r(sMVV ,,V #YV V V  \\ Vrcn\W#4wrEp\P!WEV4p^.V,p^.V,p \V4FawrV \P !V48XdVP 4W&V \P !V48XgKOVP 4W&Kc \WWV4#)a Permutes the elements of the tensor according to the given source/destination pairs. `src_dst` should be defined such that src_dst[m] == n means m sends to n. Group can be one of: List[int]: ranks participating in the collective. List[List[int]]: 2D mesh of ranks taking part of this collective in MPMD. ProcessGroup: Will perform a collective using the ranks and tag of the PG. DeviceMesh: Do a SPMD collective over all ranks of the mesh (DeviceMesh, int): Do a MPMD collective over one ) _expand_groupr8#_find_or_create_pg_by_ranks_and_tagrdistget_ranknumelr) rrr r!tranksetrBlocal_pgrrrdsts &&&& rpermute_tensorrs"+56A 77JOHz)j(g& $--) )%)ZZ\  " $--) )&*jjl  # ' T7HQT UUrca]tRtRtoRtRR.t]V3RlRl4tRtRt ]R 4t RV3R lR llt V3R lRlt Rt V3RlRltRt]RRl4tRtV3RltRtVtR #)r<i#a A Tensor wrapper subclass that is used to trigger a call to wait prior to first use of the underlying tensor. Use it inside functional collective pytorch wrappers like the following: def functional_collective(self, group, tag): tag, rankset, group_size = _expand_group(group, tag) tensor = torch.ops.c10d_functional.{collective}(self, tag, rankset, group_size) return _maybe_wrap_tensor(tensor) elem completedc4<V^8dQhRS[P/#)rrr)r' __classdict__s"rr("AsyncCollectiveTensor.__annotate__4s  5<< rc \PPVVP4VP 4VP 4VP VPVPVPR7pWn RVn V#))stridesstorage_offsetdtypelayoutdevice requires_gradF) rr#_make_wrapper_subclassrUstriderrrrrrr)clsrrs&& r__new__AsyncCollectiveTensor.__new__3sm LL / /  IIKKKM..0**;;;;,, 0  rc R.R3#)rNrrs&r__tensor_flatten__(AsyncCollectiveTensor.__tensor_flatten__Csx~rc>VP4P4#r) trigger_waittolistrs&rrAsyncCollectiveTensor.tolistFs  "))++rcHVe \R4hVR,p\V4#)Nz5meta must be None for AsyncCollectiveTensor unflattenr)rVr<) inner_tensorsmeta outer_size outer_striders&&&& r__tensor_unflatten__*AsyncCollectiveTensor.__tensor_unflatten__Is/   G V$$T**rNc4<V^8dQhRS[RS[R,/#)rexpected_metadata expected_typeN)rr)r'rs"rr(rRs!##!$#59D[#rcNV\PJdR#VP4#r)rr#r)rrr s&&&r#__coerce_same_metadata_as_tangent__9AsyncCollectiveTensor.__coerce_same_metadata_as_tangent__Rs"  ,  ""rc <V^8dQhRS[/#rr6)r&)r'rs"rr(rZs??#?rc*RVP4 R2#)zAsyncCollectiveTensor(rR)rrs&r__repr__AsyncCollectiveTensor.__repr__Zs'(9(9(;'>rczVP'g\VP4pRVnV#VP#T)rrr)routs& rr"AsyncCollectiveTensor.trigger_wait]s/~~~dii(C!DNJ99 rc4<V^8dQhRS[P/#r%r)r'rs"rr(res&&ell&rc,\VP4#r)rrrs&rrAAsyncCollectiveTensor.waites499%%rc VP#)zOThis method enables _functional_collectives_impl to test if a tensor is an ACS)rrs&r_get_acs_underlying_tensor0AsyncCollectiveTensor._get_acs_underlying_tensorhs yyrca V\PPPPJd/V!V^,P V^,4p\ V4pV#\V4o RV 3RllpRRlp\\ Ws4p \\ Wt4p V!V /V Bp S 'd\\PW4p V #)rc$V^8dQhR\/#rer<)r's"rr(>AsyncCollectiveTensor.__torch_dispatch__..__annotate__xs  + rcL<S'gVP4#VP#r)rr)r6 is_view_ops&runwrap8AsyncCollectiveTensor.__torch_dispatch__..unwrapxs~~''66Mrc8V^8dQhR\P/#r5r)r's"rr(r8~s  ELL rc^\V\4'd \R4h\V4pV#)zICannot wrap an AsyncCollectiveTensor inside another AsyncCollectiveTensor)r;r<rV)r6rCs& rwrap6AsyncCollectiveTensor.__torch_dispatch__..wrap~s1!233$_(*CJr) rratenviewdefaultrr<rr r#) r functypesargskwargsrC wrapper_resr;r?unwrapped_argsunwrapped_kwargsr+r:s &&&&& @r__torch_dispatch__(AsyncCollectiveTensor.__torch_dispatch__ls 599>>&&.. .tAw||T!W-C/4K  &     ''ON7&67  d8C rc>VP4P4#r)rAnumpyrs&rrNAsyncCollectiveTensor.numpysyy{  ""rcF<V^8dQh/S[P;R&S[;R&#)rrr)rr#bool)r'rs"rr(r#s" ,,Or)rrrN)__name__ __module__ __qualname____firstlineno____doc__ __slots__ staticmethodr rrrr"r'rrAr1 classmethodrKrN__annotate_func____static_attributes____classdictcell__rs@rr<r<#s%I  ,++##??&&$$L#arr<c ~V^8dQhR\R\R\\\\,\3,/#rr r!r6)r%r&tupler=r$)r's"rr(r(s6L&L&L&#L&uS$s)S=P7QL&rc \'dRpRpMRpRp\V\4'd\V^,\4'deV!V4p.pR pVFPpVPV4VR 8wd+V\ V48wd\ RV R\ V4 24h\ V4pKR EMV!V4p\ V4pEM\V\ P4'dD\ P!V4p\ V4pT;'g\P!V4pEM_\V\4'doVP^8wd \R4hVP4p\ P!V4p\ V4pT;'g\P!V4pM\V\4'd\ V4^8Xd\V^,\4'd\V^,\ 4'dfV^,p V^,p V PV 4p\ P!V4p\ V4pT;'g\P!V4pM\ R4h\ R 4hWV3#) a% _expand_group desugars the different RANK_TYPES types into a canonical format that is traceable. By having this be part of the explicit eager codepath, we avoid having to specialize behavior inside torchdynamo and can still interoperate with processgroup objects or other untraceable forms. cN\\\\,,V4#rrr=r$xs&rcast_listlistint'_expand_group..cast_listlistintsT#Y+ +rc8\\\,V4#rrdres&r cast_listint#_expand_group..cast_listintsS 1% %rcV#rrres&rrgrhHrcV#rrres&rrjrkrmrz$group sizes must be identical found z and JOnly 1D mesh is supported, pass in (DeviceMesh, int) together if mesh > 1Dz1Invalid tuple for group must be (DeviceMesh, int)z[Invalid type for group, must be one of List, Processgroup, DeviceMesh or (DeviceMesh, int).)rr;r=extendr~ ValueErrorr ProcessGroupget_process_group_ranksr8_get_group_tagr ndimrV get_grouprar$) r r!rgrj nested_listrrBrspgdmeshr}s && rrrs} , &  % eAh % %*51KGJ!r"# c"g(=$>zl%PSTVPWyY!W "#5)GWJ E4,, - -..u5\ //T((/ E: & & ::? \ __ ..r2\ ,,T((, E5 ! ! J!O58Z0058S))!HE(C%B2226GWJ00,,R0CPQ Q i   * %%rczV^8dQhR\R\R\P\P ,/#r`)r%r&rrsr8 GroupName)r's"rr(r(s88L8L 8L8L '8Lrc\V\P4'dV#\V\4'd\ \ P V4pV#\V\4'dVP^8wd \R4h\PPP'd+\PPP!V^4#VP"^,#\V\$4'd\'V4^8Xd\V^,\4'd\V^,\(4'dyV^,pV^,p\PPP'd*\PPP!W44#VP"V,#\+R\-V^,4\-V^,43 24h\V\.4'd^\14'g\2P4!R\6^R7\ P8!\ \.\(,V4V4#\+R\-V4 RV 24h)zA Given group in RANK_TYPES, return a ProcessGroup or group name. roz?Invalid tuple for group must be (DeviceMesh, int). Instead got zThe combination of ranks + tag as process group identifier has been deprecated. Please switch to using ProcessGroup, DeviceMesh, or group name instead.rzUnsupported group type: z, )r;rrsr&rr8r}r rvrVrr_configcompile_on_one_rankr_dtensormesh_get_process_group_dim_group_namesrar~r$rrrr=rrarbrc$_resolve_group_name_by_ranks_and_tag)r r!rr{r}s&& rr*r*s%**++ E3   $..%0  E: & & ::? \  >> 4 4 499%%< Tensor) mutates_argsrcXV^8dQhR\PR\P/#rrr6r)r's"rr(r(6s" ( ( (%,, (rc\V4#)aO Custom op that allows autograd to propagate from a normal Tensor to an AsyncCollectiveTensor. This is the low-level implementation. Users should call _maybe_wrap_tensor directly. Args: input: Input tensor to wrap in AsyncCollectiveTensor Returns: AsyncCollectiveTensor wrapping the input (or wait_tensor result if tracing) r7rs&r_wrap_tensor_autogradr1s$ ! ''rcXV^8dQhR\PR\P/#rr)r's"rr(r(Gs"##U\\#ell#rc.\P!V4#)z( Meta kernel for _wrap_tensor_autograd. r empty_likers&r_rFs   E ""rc8V^8dQhR\P/#rr)r's"rr(r(Ns  U\\ rcV#)a Backward for _wrap_tensor_autograd: identity (no-op). The wrapping is just for async optimization, gradients flow through unchanged. Args: ctx: Context object (unused) grad_output: Gradient from downstream operations Returns: Gradient unchanged (identity) rrs&&r_wrap_tensor_autograd_backwardrNs  rcR#)z Setup context for _wrap_tensor_autograd backward. Args: ctx: Context object to save state for backward (nothing to save) inputs: Tuple of (input,) output: Output from forward pass Nrrs&&&r#_wrap_tensor_autograd_setup_contextr^s rc$V^8dQhR\/#r%rQ)r's"rr(r(ps ( ( (rcZ\4'dR#\PP\PPP 4eR#\PP \PPP4'dR#\4RJ#)TN) rrr_get_dispatch_mode_TorchDispatchModeKeyFAKE&_dispatch_tls_is_dispatch_key_includedrPythonDispatcherr rrr_are_we_tracingrpss!! xx""588#A#A#F#FGS xx66 --  4 ''rc8V^8dQhR\P/#r%r)r's"rr(r(~s'' 'rcN\4'd \V4#\V4#r)rrrrs&rr-r-~s!4   &&rc$V^8dQhR\/#)rvaluer)r's"rr(r(s# # # rc#h"\PPP4p\PPP V4Rx\PPP V4R# \PPP T4i;i5i)a Context manager to temporarily set whether inflight collectives are allowed as torch.compile graph inputs. Common use case is when the collective is issued in eager (with `async_op=True`) but waited in compiled region: ``` def all_reduce_eager(x): y = x * x req = dist.all_reduce(y, op=dist.ReduceOp.SUM, async_op=True) return y @torch.compile(fullgraph=True) def all_reduce_wait_compiled(y): torch.ops.c10d_functional.wait_tensor(y) return y * y x = torch.ones(1280, 1280, device="cuda") + self.rank # the context manager ensures that `wait_tensor(y)` will wait on the correct work object with allow_inflight_collective_as_graph_input_ctx(): y = all_reduce_eager(x) z = all_reduce_wait_compiled(y) ``` With this context manager, when a collective is called, under the hood the work object of the collective will be registered in the work registry, and the wait_tensor() in compiled region called on the output tensor of the collective will wait on the correct work object. N)rr_distributed_c10d)_allow_inflight_collective_as_graph_input-_set_allow_inflight_collective_as_graph_input)rpreviouss& r,allow_inflight_collective_as_graph_input_ctxrsv8xx))SSUH  ""PPQVW  ""PP  ""PP  s)B2-B+B2+B//B2c\VP44p\V4^8XdVPV4MV^;;,V,uu&VP V4pV#r)r=rUr~append new_empty)rrBout_size out_tensors&& r_make_all_gather_out_tensorrsLEJJL!H 8} # z! *J rcBVUu.uFp\WC4NK up#uupirr)rr!rrBrs&&&& r&_all_gather_into_tensor_coalesced_metars @D E1 ' 6 EE Esc.\P!V4#rrrrFs&*r_broadcast_metar   D !!rcN\P!V\PR7#)) memory_format)rrcontiguous_formatrs&*r_all_reduce_metars   D0G0G HHrc.\P!V4#rrrs&*r_wait_tensor_metarrrc\\P!^VPVPR7#)rrr)remptyrrrs&*r _isend_metars ;;q 4;; ??rc.\P!V4#rrrs&*r _irecv_metarrrc \W4UUu.uF;wrVVR8XdTM,\P!^VPVPR7NK= upp#uuppi)irecvr)rrrrr)op_list peer_listtag_listtensorsroprs&&&&& r_batch_p2p_ops_metarsP* *EB7] AQWWQXX NN*  sAAc\W4#rr)shardr!rrBs&&&&r_all_gather_into_tensor_metar &u 99rc\VP44pV^;;,V,uu&VPV4#rr=rUr)rrr!rrBrs&&&&& r_reduce_scatter_tensor_metars/EJJL!H QKJK ??8 $$rcXVUu.uFp\P!V4NK up#uupirr)rrFrs&* r_all_reduce_coalesced_metars%)- .AE  Q  .. .s'cV#rrinprFs&*r_all_reduce__metar JrcV#rrrs&*r_broadcast__metarrrcV#rr)rzrFs&*r_all_reduce_coalesced__metars MrcHaV3RlpVUu.uF qe!V4NK up#uupi)c<\VP44pV^;;,S,uu&VPV4pV#rr)rrrrBs& r mk_out_tensor<_reduce_scatter_tensor_coalesced_meta..mk_out_tensors5 % " __X. rr)rzr/r!rrBrrs&&&&f r%_reduce_scatter_tensor_coalesced_metars' '- -fM! f -- -scVf VPVP44#VFp\P!V^84K \ VP44p\ V4V^&VPV4#r)rrUr_checkr=r)rrrrFrGsrs&&&*, r_all_to_all_single_metarsg!uzz|,,#A LLa $ %,- x((rc\W4#rr)rrBrr+s&&&$r'_all_gather_into_tensor_out_native_metar rrc\W4#rr)rrBrs&&&r#_all_gather_into_tensor_native_metarrrcDVUu.uFp\W1V4NK up#uupir)r)rzrBrrs&&& r-_all_gather_into_tensor_coalesced_native_metars0 E ,EzJ  c\VP44pV^;;,V,uu&VPV4#rr)rrrBrr>s&&&& r"_reduce_scatter_tensor_native_metars/  E !HH == rc\VP44pV^;;,V,uu&VPV4#rr)rrrBrr+r>s&&&&$ r&_reduce_scatter_tensor_out_native_metar s1  E !HH == rc DVUu.uFp\WAW#4NK up#uupir)r)rzrrBrrs&&&& r,_reduce_scatter_tensor_coalesced_native_metar(s0  C +3:R  rrIMPLr1Meta all_reduce_roall_reduce_coalesced_risendr batch_p2p_opsall_gather_into_tensor_outr:rvrWreduce_scatter_tensor_outrrr+ broadcast__c10d_functional_autogradc10d_functionalDEF(r)tagsrc |V^8dQhR\PR\PR\R\R\/#)r output_tensor input_tensorasync_opr!r5)rr#rQr&r$)r's"rr(r(sIXX<<X,,X X X  XrcV'd \R4hT;'g\PPpVf \R4hVP \ WW$44#@Can't remap async version of inplace op to functional collectivegroup cannot be None)rVrr WORLDcopy_rF)rrr rr!r5s&&&&&&rall_gather_tensor_inplacer sZ N    % %TZZ%%E }344   05V WWrrc V^8dQhR\PR\PR\R\R\R\/#)rrrrrrNr!)rr#r&rQr$)r's"rr(r(sTSS LLS <<S S  S  S Src V'd \R4hT;'g\PPpVf \R4hVP \ WWSV44#r)rVrr r r rX)rrrr rrNr!s&&&&&&&rreduce_scatter_tensor_inplacersZ N    % %TZZ%%E }344 <<-eSQ RRrrproductminmaxbandborbxorc\V^8dQhR\PR\R\R\/#)rrrrr!)rr#r&rQ)r's"rr(r(s4<< LL< < < .s$$V+QVVAY&++a.%@+s/2FTz7Remapping variable size all_gather is not yet supportedr ) rVr}rrr r rFrUrrr )rqrr rr!r output_splitsoffsetr is_scalart_offsetr+rrs&f&&& rall_gather_inplacer&s  N  zz|q$V+$V$V+$V!V!VVWW  % %TZZ%%E }344 vq% 5FM F EEGqL !1qvvay )fnvv?P/QS! 3 #4 rc V^8dQhR\PR\\P,R\R\P R,R\ R\/#)rr input_listrr Nrr!)rr#r=r&rrsrQ)r's"rr(r(sdYY LLYU\\"Y Y   t # Y  Y Yrc JaV'd \R4h\;QJdV3RlV4F 'dK RM RM!V3RlV44'g \R4hT;'g\PPpVf \R4hSP 4^8XdM\ \P!V4V^W54pSPVPSP44#SP\ \P!V4V^W544#)r c3d<"TF%qP4SP48HxK' R#5irr )rrrs& rr)reduce_scatter_inplace..&s =*Qvvx6;;=(*s-0FTzPreduce_scatter requires every input_list element to have the same size as outputr ) rVrrr r r}rXrstackr reshaper>r)rr(rr rr!resultsf&&&&& rreduce_scatter_inplacer/s N   3=*=333=*= = = ^    % %TZZ%%E }344 zz|q&u{{:'>AuR||FNN6<<899 <<-eii .CREW XXrc V^8dQhR\PR\R,R\R\PR,R\/#)rrrNr!r group_dstrr#r$rrs)r's"rr(r(6sM&& LL& t& &   t # &  &rc~Vf\PPpVf \R4hVR8wdVe \ R4hTpM&Ve\ P !W14pM \ R4h\V4p\PPPWW&4p\4'dV#\V4#)Nr z@Cannot specify both 'dst' and 'group_dst' args as per eager implz(Must specify either 'dst' or 'group_dst'rp)rr r rVrrr8get_group_rankrrrrrrr-)rrr!r r1 local_dstrs&&&&& r isend_inplacer66s }    }344B ?R  ''3 CDD$U+J YY ' ' - -f QF f %%rc V^8dQhR\PR\R,R\R\PR,R\/#)rrrNr!r group_srcr2)r's"rr(r(SsM&& LL& t& &   t # &  &rcZVf\PPpVf \R4hVR8wdVe \ R4hTpM&Ve\ P !W14pM \ R4h\V4p\PPPWW&4p\V4#)Nr z@Cannot specify both 'src' and 'group_src' args as per eager implz(Must specify either 'src' or 'group_src'rp) rr r rVrrr8r4rrrrrr-)rrr!r r8 local_srcrs&&&&& r irecv_inplacer;Ss }    }344B ?R  ''3 CDD$U+J YY ' ' - -f QF f %%rc V^8dQhR\\,R\\,R\\,R\\P,R\ /#)rrrrrr)r=r&r$rr#r%)r's"rr(r(msO22 #Y2Cy23i2%,,  2  2rc\P!4'g \R4hVeVR8Xd\P!4p\ V4p\ V\4'dTM VPp\PPPWW#V4p\4'd2\W4UUu.uFwrgVR8Xd \V4MTNK upp#\!\#\V44#uuppi)z%torch.distributed must be initializedr)ris_initializedrVr8_get_default_groupr*r;r&rrrrrrrr-r=rp)rrrrrresolvedrrs&&&&& rbatch_p2p_ops_inplacerBms    DEEZ2-,,. j)H'#66H?$F "rcV\9dR#^RIp\V,p\VPV4P!T/T;'g/BP 4pV\ PP\ PP\ PP\ PP\ PP39dRV9d\VR,,VR&VRV3#)aNRedirect a legacy ``torch.distributed`` collective to its functional equivalent, following Dynamo's CollectiveFunctionRewriteVariable. Returns ``(mapped_func, (), kwargs)`` if ``func`` is a remappable collective, else None. Shared by the make_fx compile_on_one_rank mode below and non-strict export's _NonStrictTorchFunctionHandler. Nrr)traceable_collective_remapsinspectr_ signaturebindrr distributedr1rIrXrWrFREDUCE_OP_TO_STR)rDrFrGrb mapped_funcbounds&&& r_remap_traceable_collectiveris ..-d3K ""4(--tF "FPP QE  $$ (( // // ..  5=*5;7E$K E !!rc:a]tRtRtoRtRV3RlRlltRtVtR#)!_LegacyToFunctionalCollectiveModeiaRedirect legacy ``torch.distributed`` collectives to functional collectives while tracing, mirroring Dynamo and non-strict export. Used by ``make_fx`` under ``compile_on_one_rank``: the in-place ``c10d.*`` ops bind the ProcessGroup as a torchbind constant that cannot be serialized, whereas the functional collectives take the group as an op argument, so the group (e.g. a ``DeviceMesh.get_group()`` -> ``mesh_get_process_group`` output) flows into the graph instead of being baked in. Nc <V^8dQhRS[RS[3,RS[S[R3,RS[S[R3,RS[S[S[3,R,RS[/#)rrD.rErFrGNr6)rrrarr_r&)r'rs"rr(._LegacyToFunctionalCollectiveMode.__annotate__%sd - -sCx  -T3Y -CHo - S#X% - -rc`\WV4pVeVwrgpV!V/VB#T!T/T;'g/B#r)ri) rrDrErFrGremappedrg mapped_args mapped_kwargss &&&&& r__torch_function__4_LegacyToFunctionalCollectiveMode.__torch_function__%sF/t6B  6> 3Km =}= =T,fll,,rrrR)rSrTrUrVrWrrr\r]r^s@rrkrks - - -rrk)r>rr*) zObroadcast(Tensor self, int src, str tag, int[] ranks, int group_size) -> TensorzUall_reduce(Tensor self, str reduceOp, str tag, int[] ranks, int group_size) -> Tensorzcall_reduce_coalesced(Tensor[] self, str reduceOp, str tag, int[] ranks, int group_size) -> Tensor[]z"wait_tensor(Tensor self) -> Tensorz>isend(Tensor self, int dst, int tag, str group_name) -> Tensorz>irecv(Tensor self, int src, int tag, str group_name) -> Tensorzkbatch_p2p_ops(str[] op_list, int[] peer_list, int[] tag_list, Tensor[] tensors, str group_name) -> Tensor[]zTall_gather_into_tensor(Tensor shard, str tag, int[] ranks, int group_size) -> Tensorzball_gather_into_tensor_coalesced(Tensor[] input, str tag, int[] ranks, int group_size) -> Tensor[]zareduce_scatter_tensor(Tensor input, str reduceOp, str tag, int[] ranks, int group_size) -> Tensorzpreduce_scatter_tensor_coalesced(Tensor[] inputs, str reduceOp, str tag, int[] ranks, int group_size) -> Tensor[]zall_to_all_single(Tensor input, SymInt[]? output_split_sizes, SymInt[]? input_split_sizes, str tag, int[] ranks, int group_size) -> Tensor)NFr>r)rNFrr>)rNFr>)NNNFr>)NFr>)NrNrp) contextlibr?sysracollections.abcrrtypingrrrrtorch.compiler.configtorch.distributedrer"torch.distributed.distributed_c10ddistributed_c10dr8 torch._utilsrtorch.distributed.device_meshr "torch.fx.experimental.proxy_tensorr r>r fun_col_impltorch.utils._cxx_pytreer ImportErrortorch.utils._pytreetorch.compilerr r Exceptionrbr=r$rsrar}r%rrr+r1rFrKrXr\rdrgrWrlrorwrrvrrrrrrlibraryregister_autogradrrrrrrrrrrrrrrrr#r<rr*r custom_opr register_fakerrrrr-contextmanagerrrrrrrrrrrrrrrrrrrrrrrrLibrarylib_implimpllib_impl_autogradfxnodehas_side_effectrrrCrrr legacy_liblegacy_lib_implops_defsmodulesrS my_moduleop_defindexop_namegetattr backend_impldefineTagpt2_compliant_tagr rReduceOpSUMAVGPRODUCTMINMAXBANDBORBXORrfrrr&r/r6r;rBr,rElegacy_all_gather_baserFlegacy_reduce_scatter_baserGlegacy_all_gatherr:legacy_allgatherlegacy_allgather_singlelegacy_allreducelegacy_all_to_all_singlerHlegacy_batch_p2p_ops legacy_irecv legacy_isendrIlegacy_reduce_scatterlegacy_reducescatter_singlelegacy_reducescatterrLrNrPrRrTrVrXr[r]rari overridesTorchFunctionModerkrrrrst  .++  11.4=:25 N #J  I 49o  $c )* +  nn  .:: & &0'T;('TJ8 ; D J"S"686:-6f 9V& V 2&j V&   #+ +@ & "* +8   .#6 1D & -"5 18 . )1  7#6   6"5   21 => & ,!4 =8   8-@ CD & 7,? V>q#ELLq#h L&^8Lv - % (  ( $$#%#   ''"5( ('  # # LF "I"@":% /. )::   == !3V < l,f5 m.7 $&@&I %'BFK m.7 g{F+ g{F+ o2F;  "I6 &(KVT &1   %'I6R !G %0   !#:FC k?F3 l,f5 MM))*EvNA6?*,CVL eii88DDLLM eii88DDE eii88>>FFG eii88>>? eii88>>FFG eii88>>? eii88FFNNO eii88FFG ]] " "#4e < --''(96B  KK ! FQc*+G<1WI7Lf599#>#>?,0KL X(S, MMuMMuMM9MMuMMuMMMMuMM <&:"JY8&:&420 2/ 3( ((,* * 2)01!8)9+3 7///1""<-(I(I-I@2112  MMn  s$hh* h'&h'*i  i