+ j$a0t$^RIHt^RIt^RIt^RIt^RIt^RIt^RIH t ^RI H t ^RIH t H t HtHtHt^RIHtHtHt^RIt^RIHt^RIHt^R IHtHt] 'd$^R IH t H!t!^R I"H#t#^R I$H%t%] ]!,t&R ]'R&M]PPt&R ]'R&^RI)H*t*.RXOt+]!R4t,]!R4t-RRlt.RRlt/]0!]PbR4'g^]*!R4]PbPdR&]*!R4]PbPdR&]*!R4]PbPdR&^RI3H4t4H5t5H6t6RRlt7RR lt8R!R"lt9!R#R$4t:] !4t;R%]'R&&] !4tR%]'R)&] !4t?R%]'R*&R+R,lt@R-R.ltAR/R0ltBR1R2ltCR3R4ltDR5R6ltER7R8ltF] !4tGR9]'R:&R;R<ltHR=R>ltIR?R@ltJ!RAR]54tK!RBRC]P4tMRDREltNRFRG/RHRIlltORFRG/RJRKlltPRLRMltQ!RNR4tR]RO] RP]S3,3,tTR ]'RQ&] RYRRRSll4tU] RYRTRUll4tURYRVRWlltUR# ]d RtRtELi;i)Z) annotationsN) OrderedDict)Callable)overload TYPE_CHECKING TypeAlias TypeGuardUnion) ParamSpecSelfTypeVar)Tensor)_check_cuda_bindings)driverruntime) _POOL_HANDLEMemPool_CUDAGraphInputLivenessTrackerRemovableHandler _GraphPool) _dummy_type CUDAGraphgraph_R_Pc V^8dQhRRRR/#)poolobjectreturnzTypeGuard[MemPool])formats"R/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/cuda/graphs.py __annotate__r&?sEEvE"4Ecf\\PRR4pVRJ;'d \W4#)rN)getattrtorchcuda isinstance)r mempool_clss& r% _is_mem_poolr.?s,%**i6K d " D Dz$'DDr'c V^8dQhRRRR/#)rr _GraphPool | Noner"z_POOL_HANDLE | Noner#)r$s"r%r&r&Ds--(--@-r'cVfR#\V4'd"\P!RVP4#\P!RV4#)Nr)r.typingcastid)r s&r% _get_pool_idr5Ds; |D{{>47733 ;;~t ,,r'_CudaStreamBase _CUDAGraph_graph_pool_handle_cuda_isCurrentStreamCapturing)r9r7r8cV^8dQhRR/#rr"boolr#)r$s"r%r&r&Ws,,T,r'c\4#)zReturn True if CUDA graph capture is underway on the current CUDA stream, False otherwise. If a CUDA context does not exist on the current device, returns False without initializing the context. )r9r#r'r%is_current_stream_capturingr>Ws * ++r'cV^8dQhRR/#rr"rr#)r$s"r%r&r&`s99<9r'cP\PP\44#)zReturn an opaque token representing the id of a graph memory pool. See :ref:`Graph memory management`. .. warning:: This API is in beta and may change in future releases. )r*r+rr8r#r'r%graph_pool_handlerB`s :: " "#5#7 88r'cV^8dQhRR/#rr"Noner#)r$s"r%r&r&ks     r'c,\f \R4hR#)zRaise a uniform error if the cuda-bindings package is unavailable. ``_cuda_runtime`` and ``_cuda_driver`` are imported together, so checking one suffices. NzcThis CUDAGraph API requires the cuda-bindings package; install it with `pip install cuda-bindings`.) _cuda_runtime RuntimeErrorr#r'r%_require_cuda_bindingsrIks!  ;  r'c2]tRt^xtRtRRltRRltRtR#)_RetainedCallbacksaHolds destroy callbacks and retained objects for a single capture cycle of a :class:`CUDAGraph`. A :class:`CUDAGraph` owns one of these and arms a :func:`weakref.finalize` bound to :meth:`fire`. The holder deliberately does NOT reference the graph, so the finalizer cannot keep the graph alive; :meth:`fire` can still sync the replay streams because they are mirrored onto the holder, not read off the graph. Callbacks and objects live in ``OrderedDict``s keyed by ``RemovableHandle`` id so registrations can be individually removed. cV^8dQhRR/#rDr#)r$s"r%r&_RetainedCallbacks.__annotate__s " "$ "r'c |\4Vn\4Vn\4VnRVnRVnR#FN)r callbacksobjectssetreplay_streamssync_before_fire_firedselfs&r%__init___RetainedCallbacks.__init__s/?J}1< 7:e&+! r'cV^8dQhRR/#rDr#)r$s"r%r&rMsdr'c VP'dR#RVnVP'd%VPFpVP4K \ VP P44F pV!4K VP P4VPP4R# \dKi;i \dKei;i)NT) rUrTrS synchronize ExceptionlistrPvaluesclearrQ)rWstreamcbs& r%fire_RetainedCallbacks.fires ;;;   --&&(. t~~,,./B 0  !   s#B:9C : C C  CC)rUrPrQrSrTN)__name__ __module__ __qualname____firstlineno____doc__rXrc__static_attributes__r#r'r%rKrKxs  "r'rK-OrderedDict[int, Callable[[CUDAGraph], None]]_global_capture_start_hooks_global_capture_end_hooks_global_instantiate_hooks_global_replay_start_hooks_global_replay_end_hooksc$V^8dQhRRRRRR/#)rregistryrkfnCallable[[CUDAGraph], None]r"rr#)r$s"r%r&r&s&;#r'c>^RIHpV!V4pWVP&V#)rr)torch.utils.hooksrr4)rrrsrhandles&& r%_register_global_hookrxs"2 X &FVYY Mr'c$V^8dQhRRRRRR/#)rrrrktorch_cuda_graphrr"rEr#)r$s"r%r&r&s&; r'c|\VP44F pV!V4K R# \dK i;iN)r^r_r])rrrzrss&& r%_run_global_hooksr}s<8??$%   &   s , ;;c V^8dQhRRRR/#rrsrtr"rr#)r$s"r%r&r&s B B# B Br'c"\\V4#)aRegister a hook run with each CUDA graph as its capture begins. Returns a RemovableHandle; call ``.remove()`` to unregister. .. warning:: The hook runs with capture already live on the current stream, so it must not issue CUDA work: anything it launches is captured into the graph, and under the default ``"global"`` capture error mode an unsafe call (e.g. an allocation) raises. Querying capture state is fine. Do preparation that needs CUDA before the capture instead. )rxrlrss&r%!register_graph_capture_start_hookrs !!>r'z,OrderedDict[int, Callable[[set[int]], None]]_global_destroy_hooksc V^8dQhRRRR/#)rrszCallable[[set[int]], None]r"rr#)r$s"r%r&r& s$>?r'cP^RIHpV!\4pV\VP&V#)ztRegister ``fn(exec_ids)`` to run when a CUDA graph is destroyed. Returns a handle whose ``remove()`` unregisters it.r)rvrrr4)rsrrws& r%register_graph_destroy_hookr s&2 2 3F')&))$ Mr'cV^8dQhRR/#r;r#)r$s"r%r&r&s''T'r'c \\4#)zqTrue when any graph-destroy hook is registered -- the gate a CUDAGraph checks before arming its destroy callback.)r<rr#r'r%_graph_destroy_hooks_activers % &&r'c V^8dQhRRRR/#)rexec_graph_idsset[int]r"rEr#)r$s"r%r&r&sX$r'c\\P44F pV!V4K R# \dK i;i)zInvoke every registered hook with the destroyed exec graph ids, swallowing per-hook errors so one failure does not abort the rest (matching the destroy-callback fire semantics). The single entry point a graph's destroy callback calls.N)r^rr_r])rrss& r%_run_graph_destroy_hooksrs>(//12  ~ 3   s 0 ??cja]tRtRt$RtR]R&R]R&R]R&R]R &R ]R &R]R &R ]R&R ]R&R ]R&R]R&R]R&R ]R&R ]R&R]R&R]R&RURV3RllltRRltRR ltR!R"lt R#R$lt R%R&lt R'R(lt R)R/R*R+llt R)R/R,R-lltR.R/ltR0R1ltR2R3ltRVR4V3R5llltR6V3R7lltR8V3R9lltR:R;ltR<V3R=lltR>V3R?lltR@V3RAlltRBV3RClltRDV3RElltRFV3RGlltRHV3RIlltRJRK/RLRMlltRNV3ROlltRPV3RQlltRRRSlt RTt!V;t"#)Wri(aWrapper around a CUDA graph. Arguments: keep_graph (bool, optional): If ``keep_graph=False``, the cudaGraphExec_t will be instantiated on GPU at the end of ``capture_end`` and the underlying cudaGraph_t will be destroyed. Users who want to query or otherwise modify the underlying cudaGraph_t before instantiation can set ``keep_graph=True`` and access it via ``raw_cuda_graph`` after ``capture_end``. Note that the cudaGraphExec_t will not be instantiated at the end of ``capture_end`` in this case. Instead, it will be instantiated via an explicit call to ``instantiate`` or automatically on the first call to ``replay`` if ``instantiate`` was not already called. Calling ``instantiate`` manually before ``replay`` is recommended to prevent increased latency on the first call to ``replay``. It is allowed to modify the raw cudaGraph_t after first calling ``instantiate``, but the user must call ``instantiate`` again manually to make sure the instantiated graph has these changes. Pytorch has no means of tracking these changes. .. warning:: This API is in beta and may change in future releases. z%_CUDAGraphInputLivenessTracker | None_trackerr<_has_graph_execz int | None_capture_graph_id_remapped_exec_idr_recorded_exec_ids _keep_graphz&dict[int, Callable[[CUDAGraph], None]]_capture_start_hooks_capture_end_hooks_post_instantiate_hooks_caching_graph_dataz dict | None_instantiate_graph_data_replay_start_hooks_replay_end_hooksrK _retainedzweakref.finalize | None_retained_finalizerFc V^8dQhRRRR/#)r keep_graphr<r"r r#)r$s"r%r&CUDAGraph.__annotate__hs$r'c Z<\SV`W4pRVnRVnRVn\ 4VnWn\4Vn \4Vn \4Vn RVn RVn \4Vn\4VnRVnVP#4V#)NF)super__new__rrrrRrrrrrrrrrrr _arm_retained)clsrinstance __class__s&& r%rCUDAGraph.__new__hs7?33 %)"%)"&)e#)(3 %&1m#+6=(',$+/('2}$%0]"'+$ r'cV^8dQhRR/#rDr#)r$s"r%r&r{sWWtWr'c a\4Vn\P!WPP4Vn\ 4'd$VPoVPV3Rl4R#R#)c<\S4#r|)r)exec_idssr%)CUDAGraph._arm_retained..s 3KH3Ur'N) rKrweakreffinalizercrrrregister_destroy_callback)rWrs&@r%rCUDAGraph._arm_retained{sV ,-#*#3#3D..:M:M#N  ' ( (..H  * *+U V )r'c V^8dQhRRRR/#rhookrtr"rr#)r$s"r%r&rs/ r'c f^RIHpV!VP4pWPVP&V#)aRegister ``hook(graph)`` to run when capture begins on this graph, right after capture is under way on the current stream. Hooks fire in registration order. Returns a handle whose ``remove()`` deregisters the hook. .. warning:: The hook runs inside the capture: any CUDA work it issues is captured into the graph, and under the default ``"global"`` capture error mode an unsafe call raises. See :func:`torch.cuda.graphs.register_graph_capture_start_hook`. r)rvrrr4rWrrrws&& r%register_capture_start_hook%CUDAGraph.register_capture_start_hooks. 6 !:!:;/3!!&)), r'c V^8dQhRRRR/#rr#)r$s"r%r&r  /   r'c f^RIHpV!VP4pWPVP&V#)a+Register ``hook(graph)`` to run when capture ends, after capture completes but before the graph is finalized. The captured ``cudaGraph_t`` is live (via :meth:`raw_cuda_graph`) for both ``keep_graph`` modes. Hooks fire in registration order. Returns a handle whose ``remove()`` deregisters the hook. r)rvrrr4rs&& r%register_capture_end_hook#CUDAGraph.register_capture_end_hooks. 6 !8!89-1 * r'c V^8dQhRRRR/#rr#)r$s"r%r&rrr'c f^RIHpV!VP4pWPVP&V#)aRegister ``hook(graph)`` to run after each instantiation (including re-instantiation, which produces a fresh exec graph). The instantiated graph is available via :meth:`raw_cuda_graph_exec`. Hooks fire in registration order. Returns a handle whose ``remove()`` deregisters the hook. r)rvrrr4rs&& r%register_post_instantiate_hook(CUDAGraph.register_post_instantiate_hooks. 6 !=!=>26$$VYY/ r'c V^8dQhRRRR/#rr#)r$s"r%r&rs/ r'c f^RIHpV!VP4pWPVP&V#)aRegister ``hook(graph)`` to run at the start of every :meth:`replay`, just before the graph is launched (after any on-demand instantiation, so :meth:`raw_cuda_graph_exec` is valid). Hooks fire in registration order. Returns a handle whose ``remove()`` deregisters the hook. .. note:: Replay is the hot path and a registered hook runs on every replay -- keep it cheap. With no hook registered the cost is a single dict emptiness check. r)rvrrr4rs&& r%register_replay_start_hook$CUDAGraph.register_replay_start_hooks. 6 !9!9:.2  + r'c V^8dQhRRRR/#rr#)r$s"r%r&rs/ r'c f^RIHpV!VP4pWPVP&V#)a?Register ``hook(graph)`` to run at the end of every :meth:`replay`, just after the graph is launched. The launch is asynchronous, so the hook runs once the replay is *enqueued*, not once the GPU work completes. Hooks fire in registration order. Returns a handle whose ``remove()`` deregisters the hook. See the hot-path note on :meth:`register_replay_start_hook`. End hooks fire even if the launch raises -- so a start hook is always balanced by an end -- and the launch error then propagates. (Start hooks that raise abort the replay before launch, and no end hook fires.) r)rvrrr4rs&& r%register_replay_end_hook"CUDAGraph.register_replay_end_hooks. 6 !7!78,0vyy) r'synchronize_before_releasec$V^8dQhRRRRRR/#)rrbzCallable[[], None]rr<r"rr#)r$s"r%r&rs( %)   r'c ^RIHpV'dRVPnV!VPP4pWPPVP &V#)aRegister ``cb()`` to run when this graph is destroyed (finalized) or explicitly :meth:`reset`, just before its CUDA resources are freed. Callbacks fire once per capture cycle, in registration order; exceptions are swallowed so one failure does not abort the rest. ``cb`` must NOT reference this graph: the finalizer that fires it is held by a global registry, so a callback reachable to the graph keeps the graph alive until interpreter exit (it is never collected, hence never fired). Returns a handle whose ``remove()`` deregisters the callback. Teardown does not synchronize CUDA, and ``cudaGraphExecDestroy`` frees an in-flight graph only asynchronously, so a callback that frees device memory the graph reads/writes is a use-after-free if a replay is still in flight. Pass ``synchronize_before_release=True`` to synchronize every stream this graph was replayed on before firing. Otherwise callbacks must not free anything the graph references. rT)rvrrrTrPr4)rWrbrrrws&&$ r%r#CUDAGraph.register_destroy_callbacksE, 6 %.2DNN + !9!9:.0  + r'c$V^8dQhRRRRRR/#)robjr!rr<r"rr#)r$s"r%r&rs( %)   r'c ^RIHpV'dRVPnV!VPP4pWPPVP &V#)aKeep ``obj`` alive for this graph's current capture cycle and release it when the graph is destroyed (finalized) or explicitly :meth:`reset`. No callback runs; normal refcounting drops ``obj`` when the retained reference is released. Returns a handle whose ``remove()`` drops the retained reference early. As with :meth:`register_destroy_callback`, ``obj`` must NOT reference this graph, or the graph is kept alive until interpreter exit and ``obj`` is never released. ``synchronize_before_release`` has the same meaning and caveats as in :meth:`register_destroy_callback`: set it if releasing ``obj`` frees device memory the graph reads/writes (e.g. ``obj`` is the last reference to a tensor the graph uses) and replays may still be in flight. rT)rvrrrTrQr4)rWrrrrws&&$ r% retain_objectCUDAGraph.retain_objectsE& 6 %.2DNN + !7!78,/vyy) r'cV^8dQhRR/#rDr#)r$s"r%r&rs@@$@r'c VPfR#^RIHpV!V4VPe(VPP VP4R#R#)N)remap_to_exec_graph)rtorch.cuda._graph_annotationsrrradd)rWrs& r%_maybe_remap_annotations"CUDAGraph._maybe_remap_annotationssK  ! ! ) ED!  ! ! -  # # ' '(>(> ? .r'cV^8dQhRR/#rDr#)r$s"r%r&r0s4r'c XVPRuqnVeVP4R#R#r|)rstop)rWtrackers& r%_release_python_resources#CUDAGraph._release_python_resources0s( "&   LLN r'cV^8dQhRR/#rDr#)r$s"r%r&r:sr'c LVP4R# \dR#i;ir|)rr]rVs&r%__del__CUDAGraph.__del__:s%   * * ,   s  ##c(V^8dQhRRRRRRRR/#) rr r0capture_error_modestrcheck_input_livenessr<r"rEr#)r$s"r%r&r@s2000 0# 0  0r'c <VPe"VPP4RVn\SV` \ V4VR7V'd-^RIHpV!4VnVPP4\'d\\V4\VPP44F pV!V4K R#)aBegin capturing CUDA work on the current stream. Typically, you shouldn't call ``capture_begin`` yourself. Use :class:`~torch.cuda.graph` or :func:`~torch.cuda.make_graphed_callables`, which call ``capture_begin`` internally. Arguments: pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) or :class:`~torch.cuda.MemPool` that hints this graph may share memory with the indicated pool. See :ref:`Graph memory management`. capture_error_mode (str, optional): specifies the cudaStreamCaptureMode for the graph capture stream. Can be "global", "thread_local" or "relaxed". During cuda graph capture, some actions, such as cudaMalloc, may be unsafe. "global" will error on actions in other threads, "thread_local" will only error for actions in the current thread, and "relaxed" will not error on these actions. Do NOT change this setting unless you're familiar with `cudaStreamCaptureMode `_ check_input_liveness (bool, optional): If ``True``, tracks external tensor inputs during graph capture and raises an error if any are deallocated before replay. This helps debug "use after free" errors where input tensors are garbage collected between capture and replay. Default: ``False``. .. note:: Custom CUDA kernels added outside PyTorch (e.g., via cuLaunchKernel or DLPack) are not tracked by this mechanism. N)r rr) rrr capture_beginr5torch.utils._cuda_debugrstartrlr}r^rr_)rWr rrrrrs&&&& r%rCUDAGraph.capture_begin@s> == $ MM    DM d#8J   N:00779:D J;     r'cV^8dQhRR/#rDr#)r$s"r%r&rs00T0r'c 2<\SV`4VP4RVn\'d\ \V4\ VPP44F pV!V4K RVnRVn R# RTnRTn i;i)zInstantiate the CUDA graph. Will be called by ``capture_end`` if ``keep_graph=False``, or by ``replay`` if ``keep_graph=True`` and ``instantiate`` has not already been explicitly called. Does not destroy the cudaGraph_t returned by ``raw_cuda_graph``. TFN) rrrrrnr}r^rr_rrWrrs& r%rCUDAGraph.instantiates  %%' $(  0((!";TBT99@@BCT D(-D $+/D ((-D $+/D (s BABBcV^8dQhRR/#rDr#)r$s"r%r&rsr'c <VPe*VPPVP44VPP'dBVPP P \PP44VP'gVP4\'d\\V4VP'd3\VPP!44F pV!V4K \"SV`I4\&'d\\&V4VP('d5\VP(P!44F pV!V4K R#R# \&'d\\&T4TP('d4\TP(P!44F pT!T4K ii;i)z,Replay the CUDA work captured by this graph.N)r check_alivepoolsrrTrSrr*r+current_streamrrror}rr^r_rreplayrprr s& r%rCUDAGraph.replays\ == $ MM % %djjl 3 >> * * * NN ) ) - -ejj.G.G.I J###     % % 8$ ?  # # #T55<<>?T @  GN ''!":DA%%% !7!7!>!>!@ADJB&('!":DA%%% !7!7!>!>!@ADJB&sF .G.:4G.cV^8dQhRR/#rDr#)r$s"r%r&rstr'c "<VP4VPP4VPeVPP 4\ 4VnVP4RVnRVn \SV`-4R#)z1Delete the graph currently held by this instance.N) rrrcrdetachrRrrrrrresetrs&r%rCUDAGraph.resetsr &&(   # # /  $ $ + + -#&% "&!%  r'cV^8dQhRR/#r@r#)r$s"r%r&rs77l7r'c Z<\PP\SV`44#)zReturn an opaque token representing the id of this graph's memory pool. This id can optionally be passed to another graph's ``capture_begin``, which hints the other graph may share the same memory pool. )r*r+rrr rs&r%r CUDAGraph.pools zz&&uw|~66r'cV^8dQhRR/#)rr"zlist[_POOL_HANDLE]r#)r$s"r%r&rsKK)Kr'c <\SV`4Uu.uF"p\PP V4NK$ up#uupi)zAReturn opaque tokens for all memory pools retained by this graph.)rrr*r+r)rWr rs& r%rCUDAGraph.poolss0:?'-/J/$ ''-/JJJs(=c V^8dQhRRRR/#)rr rr"rEr#)r$s"r%r&rs----r'c V<\V4pVf \R4h\SV` V4#)Nz-CUDAGraph._retain_pool expected a memory pool)r5rHr _retain_pool)rWr pool_idrs&& r%r"CUDAGraph._retain_pools.t$ ?NO Ow#G,,r'cV^8dQhRR/#rDr#)r$s"r%r&rs  4 r'c 2<\SV`4RVnR#)zRetain the captured graph (equivalent to ``keep_graph=True``) so it can be inspected, e.g. via :meth:`debug_dump`. Kept for backward compatibility.TN)renable_debug_moderrs&r%r'CUDAGraph.enable_debug_modes !#r'verboseTc$V^8dQhRRRRRR/#)r debug_pathrr)r<r"rEr#)r$s"r%r&rs! ; ;S ;d ;d ;r'c \WVR7R#)alDump the captured graph to ``debug_path`` in Graphviz DOT format. The graph's template must be live: ``keep_graph=True`` (or :meth:`enable_debug_mode`), or called from a capture-end hook. Requires the ``cuda.bindings`` package. Arguments: debug_path (required): Path to dump the graph to. verbose: If ``True`` (default), use the most verbose DOT output. r)N_dump_graph_dot)rWr+r)s&&$r% debug_dumpCUDAGraph.debug_dumps ':r'cV^8dQhRR/#rr"intr#)r$s"r%r&r s(((r'c <\SV`4#)aReturns the underlying cudaGraph_t. The template must be live: this requires ``keep_graph=True`` (it persists after ``capture_end``), or access from within a capture-end hook (before the template is destroyed for ``keep_graph=False``). See the following for APIs for how to manipulate this object: `Graph Management `_ and `cuda-python Graph Management bindings `_ )rraw_cuda_graphrs&r%r6CUDAGraph.raw_cuda_graph sw%''r'cV^8dQhRR/#r3r#)r$s"r%r&rs--S-r'c <\SV`4#)aReturns the underlying cudaGraphExec_t. ``instantiate`` must have been called if ``keep_graph`` is True, or ``capture_end`` must have been called if ``keep_graph`` is False. If you call ``instantiate()`` after ``raw_cuda_graph_exec()``, the previously returned cudaGraphExec_t will be destroyed. It is your responsibility not to use this object after destruction. See the following for APIs for how to manipulate this object: `Graph Execution `_ and `cuda-python Graph Execution bindings `_ )rraw_cuda_graph_execrs&r%r:CUDAGraph.raw_cuda_graph_execs w*,,r'cV^8dQhRR/#)rr"dictr#)r$s"r%r&rsXXXr'c VPe VP#^RIHp\4\e \ f \ R4hV!4'd \R4h\PPR\PPR\PPR\PPR\PPR \PPR \PPR \PP R \PP"R \PP$R\PP&R/ p\PP\PP&0p\)4pVP+4p\-\P.!V^R74wrg\-\P.!WWR74wr/p .p \1V4EFp W,p W\3V 4&\-\P4!V 44p W9d!VP7VP9W44\-\P:!V 44pV^ , pVR,pRpV \PP8Xd\ P<!\3V 4R7p\ P>!V4wppV\ P@PB8Xd\3VPD4'd\ PF!\3VPD4R7p\ PH!V4wppV\ P@PB8Xd)\KV\L4'dVPO4MTp^pV \PP 8Xd*\3\-\PP!V 444pMGV \PP8Xd)\3\-\PR!V 444p^pRpV \PP8XdY\PT!V 4wppV\PVPX8Xd!\3VPZ4p\]V4pV P_RV RVP9V \aV 44RVRVRVRVRVRVRVR.R./ 4EK \-\Pb!V^R74wppV^8d\-\Pb!VVR74wpppp\1V4Fp V P9\3VV ,44pV P9\3VV ,44pVfKKVfKQV V,R,P_V4V V,R,P_V4K \Pd!VPg4R7p \-\Ph!V 44p!V F"p"V!^ ,V"R,,V"R&V!V"R&K$ V'd&\jPl!R\oV4 R 2^R!7R"V!R#V /p#VPp'dV#VnV##)$aReturn a dictionary describing the graph's topology and node metadata. ``keep_graph`` must be True. The graph must have been instantiated (via :meth:`instantiate`) before calling this method. Requires the ``cuda.bindings`` package. Returns a dictionary with structure:: { "exec_graph_id": int, "nodes": [ { "index": int, "node_type": str, "tools_id": int, "graph_id": int, "node_id": int, "kernel_name": str or None, "event_ptr": int, "host_fn_addr": int, "host_fn_name": str or None, "dependencies": [int, ...], "dependents": [int, ...], }, ..., ], } ``event_ptr`` is the ``cudaEvent_t`` handle (as an int) an event-record or event-wait node records / waits on -- these nodes produce no timed CUPTI record, so matching a wait to the record that signals it is the only way to reason about the cross-stream sync it encodes. It is ``0`` for other node types. ``host_fn_addr`` / ``host_fn_name`` are populated for host nodes (a CPU callback run as a graph node): the callback address and a best-effort demangled symbol name for it (``None`` when it resolves to no exported symbol). They are ``0`` / ``None`` for other node types. Each node's ``graph_id`` is remapped to the exec graph id so that ``tools_id`` values match those reported by CUPTI-based profilers. ``dependencies`` and ``dependents`` are lists of node indices within the ``nodes`` list. This structure is useful for inspecting a profiler trace and establishing whether a particular dependency observed in the profile is a true dependency (encoded in the graph) or a fake dependency caused by mapping of independent streams to the same hardware channel. Child-graph and conditional nodes are reported as nodes in their own right, but this walk does not descend into their bodies (those are separate ``cudaGraph_t`` objects), so the work inside them is absent from ``nodes`` and a warning is issued. The ids that *are* reported stay valid: the exec graph preserves top-level node ids. N)_is_tools_id_unavailablez6expected _cuda_runtime and _cuda_driver to be not Nonezget_graph_data requires cudaGraphNodeGetToolsId which needs cuda.bindings >= 13.1 and CUDA driver >= 13.1 (or cuda-compat >= 13.1 in LD_LIBRARY_PATH)kernelmemcpymemsethost child_graphempty wait_event event_record mem_allocmem_free conditional)numNodesl) init_valueindex node_typetools_idgraph_idnode_id kernel_name event_ptr host_fn_addr host_fn_name dependencies dependents)numEdgesz$get_graph_data: this graph contains z nodes; the work inside them is in a separate cudaGraph_t that this walk does not descend into, so it is absent from the returned nodes) stacklevel exec_graph_idnodes)9rrr?rIrG _cuda_driverAssertionErrorrHcudaGraphNodeTypecudaGraphNodeTypeKernelcudaGraphNodeTypeMemcpycudaGraphNodeTypeMemsetcudaGraphNodeTypeHostcudaGraphNodeTypeGraphcudaGraphNodeTypeEmptycudaGraphNodeTypeWaitEventcudaGraphNodeTypeEventRecordcudaGraphNodeTypeMemAlloccudaGraphNodeTypeMemFreecudaGraphNodeTypeConditionalrRr6rcudaGraphGetNodesranger4cudaGraphNodeGetTypergetcudaGraphNodeGetToolsId CUgraphNodecuGraphKernelNodeGetParamsCUresult CUDA_SUCCESSfunc CUfunction cuFuncGetNamer,bytesdecode cudaGraphEventRecordNodeGetEventcudaGraphEventWaitNodeGetEventcudaGraphHostNodeGetParams cudaError_t cudaSuccessrs_resolve_host_fn_nameappendrcudaGraphGetEdgescudaGraphExec_tr:cudaGraphExecGetIdwarningswarnsortedr)$rWr?node_type_namesnested_graph_typesnested_node_typesraw_numr[ handle_to_idx node_infosinodentyperOrPrQrRcu_nodeerrparamscu_funcnamerSrTrU num_edges from_nodesto_nodes _edge_datasrcdst exec_handlerZinfodatas$& r%get_graph_dataCUDAGraph.get_graph_datast  ' ' 3// /J  L$8 H  $ % %>   + + C CX  + + C CX  + + C CX  + + A A6  + + B BM  + + B BG  + + F F  + + H H.  + + E E{  + + D Dj  + + H H-    + + B B  + + H H '*e!!#%m&E&EcTU&VW)  + +C > )+ !# sA8D'(#d) $()K)KD)QRE*!%%o&9&9%&GH+M,Q,QRV,WXH2~H+GK 77OOO&22c$iH*EEgN V,//<<<V[[AQAQ*55V[[AQRG , : :7 CICl33@@@7A$7N7NdkkmTX I 77TTT(%FFtL -99TTT(%DDTJ LL 77MMM+FFtL V-33???#&vyy>L#8#FL   Q!4!4UCJ!Gw!;"L"L"B "  iH2  + +C! < 1a q=:N//iH; 7J*i9%#''JqM(:;#''HQK(89?ssOL188=sON3::3? &$33//1 -  , ,[ 9 D - 3tIFD ,D   MM6v>O7P6QRVV   ] Z   # # #+/D ( r') rrrrrrrrr)F)NglobalF)#rerfrgrhri__annotations__rrrrrrrrrrrrrrrrrrrr rr"r'r0r6r:rrj __classcell__)rs@r%rr(sb443"!"!! @@>>CC ((@?=="!00&W&$  &(,1 <,1 6@$ 00d"" !! 800.8*77KK--    ;T ;((--XXr'c]tRtRtR]P 3R]P 3R]P 3R]P 3.tRtR#)_DlInfoi dli_fname dli_fbase dli_sname dli_saddrr#N) rerfrgrhctypesc_char_pc_void_p_fields_rjr#r'r%rrs9 foo& foo& foo& foo& Hr'rc V^8dQhRRRR/#)raddrr4r"z str | Noner#)r$s"r%r&r&s 7 7 7 7r'clV'gR#\4p\P!R4P\P!V4\P !V44'gR#VP 'gR#\PPVP P44#)zBest-effort symbol name for a host-node callback address via dladdr + demangle. Returns None when the address resolves to no exported symbol (e.g. an anonymous trampoline). Host nodes carry no name in the CUPTI record, so this is the only source.N) rrCDLLdladdrrbyrefrr*_C _demanglerw)rrs& r%r}r}sp  9D ;;t  # #FOOD$96<<;M N N >>> 88  dnn335 66r'r)Tc(V^8dQhRRRRRRRR/#) r cuda_graphrpathrr)r<r"rEr#)r$s"r%r&r&s)    $ RV r'c\4V'd\PPM^p\ \P !VP 4VP4V44R#)zCWrite ``cuda_graph``'s template to ``path`` in Graphviz DOT format.N)rIrGcudaGraphDebugDotFlagscudaGraphDebugDotFlagsVerbosercudaGraphDebugDotPrintr6encode)rrr)flagss&&$ r%r/r/sT  ,,JJ  ,,  % % ' r'c$V^8dQhRRRRRR/#)rrrr)r<r"rtr#)r$s"r%r&r&s"  S d 6Q r'caaRVV3RllpV#)zReturn a capture-end hook that dumps the captured graph to ``path`` in Graphviz DOT format. Register it with :meth:`CUDAGraph.register_capture_end_hook`; works for both ``keep_graph`` modes since it runs while the template is still live.c V^8dQhRRRR/#rrrr"rEr#)r$s"r%r& export_dot..__annotate__#s;;);;r'c$<\VSSR7R#)r-Nr.)rrr)s&r%_hookexport_dot.._hook#s D':r'r#)rr)rsfd r% export_dotrs ;; Lr'c V^8dQhRRRR/#)rrrr"rtr#)r$s"r%r&r&)s  C $? r'caRV3RllpV#)a]Return a post-instantiate hook that pickles :meth:`CUDAGraph.get_graph_data` to ``path``. Register it with :meth:`CUDAGraph.register_post_instantiate_hook`: ``get_graph_data`` needs the graph instantiated (it remaps node ids to the exec graph id), and at post-instantiate time the template is still live, so this works for both ``keep_graph`` modes.c V^8dQhRRRR/#rr#)r$s"r%r&'export_graph_data..__annotate__0s88)88r'c<^RIp\SR4;_uu_4pVPVP4V4RRR4R# +'giR#;i)rNwb)pickleopendumpr)rrfrs& r%r export_graph_data.._hook0s; $   KK 113Q 7   s !A A r#)rrsf r%export_graph_datar)s88 Lr'cR]tRtRt$RtRtR]R&R RRlltRR ltR R lt R t R#)ri9a. Context-manager that captures CUDA work into a :class:`torch.cuda.CUDAGraph` object for later replay. See :ref:`CUDA Graphs ` for a general introduction, detailed use, and constraints. Arguments: cuda_graph (torch.cuda.CUDAGraph): Graph object used for capture. pool (optional): Opaque token (returned by a call to :func:`~torch.cuda.graph_pool_handle()` or :meth:`other_Graph_instance.pool()`) or :class:`~torch.cuda.MemPool` hinting this graph's capture may share memory from the specified pool. See :ref:`Graph memory management`. stream (torch.cuda.Stream, optional): If supplied, will be set as the current stream in the context. If not supplied, ``graph`` sets its own internal side stream as the current stream in the context. capture_error_mode (str, optional): specifies the cudaStreamCaptureMode for the graph capture stream. Can be "global", "thread_local" or "relaxed". During cuda graph capture, some actions, such as cudaMalloc, may be unsafe. "global" will error on actions in other threads, "thread_local" will only error for actions in the current thread, and "relaxed" will not error on actions. Do NOT change this setting unless you're familiar with `cudaStreamCaptureMode `_ enable_annotations (bool, optional): If ``True``, enables kernel annotation recording on entry and automatically calls :func:`~torch.cuda._graph_annotations.resolve_pending_annotations` before the capture ends. Annotations are **not** cleared on exit so that multiple graphs in the same workload can accumulate annotations. Requires ``cuda.bindings`` package and cuda-compat >= 13.1 or CUDA driver >= 13.1. check_input_liveness (bool, optional): If ``True``, tracks external tensor inputs during graph capture and raises an error if any are deallocated before replay. This helps debug "use after free" errors where input tensors are garbage collected between capture and replay. Default: ``False``. .. note:: Custom CUDA kernels added outside PyTorch (e.g., via cuLaunchKernel or DLPack) are not tracked by this mechanism. .. note:: For effective memory sharing, if you pass a ``pool`` used by a previous capture and the previous capture used an explicit ``stream`` argument, you should pass the same ``stream`` argument to this capture. .. warning:: This API is in beta and may change in future releases. .. _cudaStreamCaptureMode: https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__STREAM.html#group__CUDART__STREAM_1g9d0535d93a214cbf126835257b16ba85 Ntorch.cuda.Stream | Nonedefault_capture_streamc 0V^8dQhRRRRRRRRR R R R /#) rrrr r0rarrrenable_annotationsr<rr#)r$s"r%r&graph.__annotate__gsF999 9) 9  9 ! 9#9r'c VfFVPPf.\PP 4VPn\ V4pVfRMV3VnVeTMVPPVnVPf \R4h\PPVP4Vn Wn W@n WPn W`nR#)Nzcapture_stream must not be Noner#)rrr*r+Streamr5r capture_streamr]ra stream_ctxrr_enable_annotationsr)rWrr rarrrr#s&&&&&&& r%rXgraph.__init__gs >dnnCCK49JJ4E4E4GDNN 1t$/Bz (Fdnn.S.S     & !BC C**++D,?,?@$"4#5 $8!r'cV^8dQhRR/#rDr#)r$s"r%r&rs&>&>4&>r'c h\PP4\PPP 'd\ P!4\PP4\PP4^RI H pV!VP4VPP4VP P"!VP$RVP&RVP(/^RI HpV!\PP-44R#)r)_set_annotations_enabledrr)maybe_stamp_capture_rootN)r*r+r\compilerconfigforce_cudagraph_gcgccollect empty_cacher_host_emptyCacherrrr __enter__rrr rrrr)rWrrs& r%rgraph.__enter__s  >> 3 3 3 JJL   !!# K !9!9: !!# %% YY  $66  "&!:!:   K !:!:!<=r'c V^8dQhRRRR/#)rargsr!r"rEr#)r$s"r%r&rs,,f,,r'c ^RIHpHpVP'dV!4VPP 4VP P!V!V!R4R# T!R4i;i)r)rresolve_pending_annotationsFN)rrrrrrr__exit__)rWrrrs&* r%rgraph.__exit__sV ,'''+- OO ' ' ) OO $ $d + %U + $U +sA!;A!! A+)rrrrrr r)NNrFF) rerfrgrhrirrrXrrrjr#r'r%rr9s+)V8<4;9<&>P,,r'torch.nn.Module._ModuleOrCallablec8V^8dQhRRRRRRRRR R R R R RRR/#)r callablesr sample_argstuple[Tensor, ...]num_warmup_itersr4allow_unused_inputr<r r0rrrr"r#)r$s"r%r&r&sX #     r'cR#r|r#rrrrr rrs&&&&&&&r%make_graphed_callablesrsr'c8V^8dQhRRRRRRRRR R R R R RRR/#)rrztuple[_ModuleOrCallable, ...]rztuple[tuple[Tensor, ...], ...]rr4rr<r r0rrrr"r#)r$s"r%r&r&sX((,(/(( (  (  ((#(r'cR#r|r#rs&&&&&&&r%rrs%(r'c8V^8dQhRRRRRRRRR R R R R RRR/#)rrz1_ModuleOrCallable | tuple[_ModuleOrCallable, ...]rz3tuple[Tensor, ...] | tuple[tuple[Tensor, ...], ...]rr4rr<r r0rrrr"r#)r$s"r%r&r&sbaa@aDaa a  a  aa7ar'c 6\P!4'd'\P!4'd \R4hRp\ V\ 4'g0RpV3p\ P!\ \R3,V43pM5\ P!\ \ \R3,R3,V4p.p \W4EFqwr\ V \PP4'd\V P4^8Xd5\V P4^8Xd\V P4^8Xg \!R4h\";QJd*RV P%44F 'dK RM RM!RV P%444'g \!R4h\P&P(P*!V !p V P-\ V 44\";QJdRV 4F 'dK RM RM !RV 44'dEKi\!R4h V U u.uFp \V 4NK p p VU u.uFHp \ V \PP4'd\ V P/44MRNKJ pp \1\V44Uu.uFpW,W,,NK pp\1\V44Uu.uF!p\P2P54NK# pp\1\V44Uu.uF!p\P2P54NK# ppVf \74M \9V4p\P2P;4\P2P=4p\P2P?V4;_uu_4\WV4EFwpp pRRRppp\1V4Fp\P&P(PAV!V !4p\ ;QJd.RV4FNK 5M !RV44p\V4^8gKn\PBPET\ ;QJd.R V4FNK 5M !R V44\ ;QJd.R V4FNK 5M !R V44RVR 7pK VVV3Fp?K EK RRR4\P2P;4.p.p\WV4Fwpp p\P2PGVVVVVR 7;_uu_4V!V !pRRR4\P&P(PIX4wp p!VP-\ V 44VP-V!4K .p".p#\\KV4\KV4\KV44EFwpp$p%\ ;QJd.R V$4FNK 5M !R V$44p&\ ;QJd.RV$4FNK 5M !RV$44pRp\V4^8d\P2PGV%VVVVR 7;_uu_4\PBPET\ ;QJd.RV4FNK 5M !RV44\ ;QJd.RV&4FNK 5M !RV&44RVR 7pRRR4.p'^p(VFMp)V)PL'd(Ve$V'P-VV(,4V(^, p(K<V'P-R4KO \ V'4p'V"P-V&4V#P-V'4EK V"PO4V#PO4RRlp*.p+\QV4FwppV*!VV,VV,W,W,VV,VV,VV,V"V,V#V,4 p,\ V\PP4'd>RRlp-V-!VVPRV,VPT4Vn*V+P-V4KV+P-V,4K V'd V+^,#\ V+4#uup iuup iuupiuupiuupi +'giEL;i +'giEL;i +'giEL;i)aAccept callables (functions or :class:`nn.Module`\ s) and returns graphed versions. Each graphed callable's forward pass runs its source callable's forward CUDA work as a CUDA graph inside a single autograd node. The graphed callable's forward pass also appends a backward node to the autograd graph. During backward, this node runs the callable's backward work as a CUDA graph. Therefore, each graphed callable should be a drop-in replacement for its source callable in an autograd-enabled training loop. See :ref:`Partial-network capture` for detailed use and constraints. If you pass a tuple of several callables, their captures will use the same memory pool. See :ref:`Graph memory management` for when this is appropriate. Arguments: callables (torch.nn.Module or Python function, or tuple of these): Callable or callables to graph. See :ref:`Graph memory management` for when passing a tuple of callables is appropriate. If you pass a tuple of callables, their order in the tuple must be the same order they'll run in the live workload. sample_args (tuple of Tensors, or tuple of tuples of Tensors): Samples args for each callable. If a single callable was passed, ``sample_args`` must be a single tuple of argument Tensors. If a tuple of callables was passed, ``sample_args`` must be tuple of tuples of argument Tensors. num_warmup_iters (int): The number of warmup iterations. Currently, ``DataDistributedParallel`` needs 11 iterations for warm up. Default: ``3``. allow_unused_input (bool): If False, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to False. pool (optional): Token (returned by :func:`~torch.cuda.graph_pool_handle` or :meth:`other_Graph_instance.pool()`) or :class:`~torch.cuda.MemPool` that hints this graph may share memory with the indicated pool. See :ref:`Graph memory management`. enable_annotations (bool, optional): If ``True``, the forward and backward captures record kernel annotations from :func:`torch.cuda.graph_annotations.mark_kernels` scopes inside the callables (backward kernels are tagged via the scopes' autograd node hooks). See :mod:`torch.cuda.graph_annotations`. Default: ``False``. .. note:: The ``requires_grad`` state of each Tensor in ``sample_args`` must match the state that's expected for the corresponding real input in the training loop. .. warning:: This API is in beta and may change in future releases. .. warning:: ``sample_args`` for each callable must contain only Tensors. Other types are not allowed. .. warning:: Returned callables do not support higher order differentiation (e.g., double backward). .. warning:: In any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. Buffers must have ``requires_grad=False``. .. warning:: After you pass a :class:`torch.nn.Module` through :func:`~make_graphed_callables`, you may not add or remove any of that Module's parameters or buffers. .. warning:: :class:`torch.nn.Module`\s passed to :func:`~torch.cuda.make_graphed_callables` must not have module hooks registered on them at the time they are passed. However, registering hooks on modules *after* passing them through :func:`~torch.cuda.make_graphed_callables` is allowed. .. warning:: When running a graphed callable, you must pass its arguments in the same order and format they appeared in that callable's ``sample_args``. .. warning:: The automatic mixed precision is supported in :func:`~torch.cuda.make_graphed_callables` only with disabled caching. The context manager `torch.cuda.amp.autocast()` must have `cache_enabled=False`. z_make_graphed_callables does not support the autocast caching. Please set `cache_enabled=False`.FT.c3<"TFqPRJxK R#5irO requires_grad.0bs& r% )make_graphed_callables..KsEA%/sc3V"TFp\V\P4xK! R#5ir|)r,r*r)rargs& r%rrSsHKS:c5<<00Ks')Nc3L"TFqP'gKVxK R#5ir|rros& r%rrxs$K1??QQ$ $c3L"TFqP'gKVxK R#5ir|rrrs& r%rr|s%';!AA';rc3t"TF.qP'gK\P!V4xK0 R#5ir|rr* empty_liker s& r%rrs(+9@AOO/E,,Q//s88)outputsinputs grad_outputs only_inputs allow_unused)rar rrc3t"TF.qP'd\P!V4MRxK0 R#5ir|rr s& r%rrs)$ FT???E  Q  .__annotate__sl====6= = < = 1 =+=7=/= =r'c aaaaaaaaaa !VVVVVVV3RlR\PP4o RV VV3Rllp V #)c<]tRtRt]RVVVV3Rll4t]]PPPRVVV3Rll44t Rt R#)Omake_graphed_callables..make_graphed_autograd_function..Graphedic$V^8dQhRRRRRR/#)rctxr!rrr"rr#)r$s"r%r&\make_graphed_callables..make_graphed_autograd_function..Graphed.__annotate__s' A AV Af A9K Ar'c <\S4FSpSV,P4W,P48wgK5SV,PW,4KU SP4\ S\ 4'g\ R\S4 24h\ ;QJd.RS4FNK 5#!RS44#)z"static_outputs must be tuple, got c3@"TFqP4xK R#5ir|rr s& r%rjmake_graphed_callables..make_graphed_autograd_function..Graphed.forward..s@AXXZZs)rkdata_ptrcopy_rr,tupler]type)r,rrrr!r#r$s&* r%forwardWmake_graphed_callables..make_graphed_autograd_function..Graphed.forwards}-A+A.779VY=O=O=QQ,Q/55fi@.  "!.%88(.make_graphed_autograd_function..Graphed.backward..s(0#$-AHHJQ6/s#%) lenr]zipr2r3rr,r4r5)r,r9ggradrr&r%s&* r%backwardXmake_graphed_callables..make_graphed_autograd_function..Graphed.backwardsu:%8!99(%c%j\1NsSfOgNhi ##6>GA}::<4==?:GGDM ?   """4e<<(@FXAY@Z[u0uu0r'r#N) rerfrgrh staticmethodr6r*autogradfunctiononce_differentiablerArj)rrr!r&r%r#r$sr%Graphedr*sN  A A A ^^ $ $ 8 8  9 r'rGc V^8dQhRRRR/#)r user_argsr!r"r#)r$s"r%r&Tmake_graphed_callables..make_graphed_autograd_function..__annotate__ s R Rv R& Rr'c<\PPP!V!pSP!\ V4S,!p\PPP VS4#r|)r*utils_pytreearg_tree_leavesapplyr4tree_unflatten)rIflatten_user_argsoutrGr r"s* r%functionalizedVmake_graphed_callables..make_graphed_autograd_function..functionalized sW!& 3 3 C CY O --%(9":]"JLC;;&&55c;PQ Qr')r*rDFunction) rrr r!r"r#r$r%r&rSrGs fffffffff @r%make_graphed_autograd_function>make_graphed_callables..make_graphed_autograd_functions4( ( enn--( T R Rr'c ,V^8dQhRRRRRRRRRR/#) rrsrgraph_training_stater<graphedzCallable[_P, _R]orig_fwdr"r#)r$s"r%r&r'$s:  % &* * +   "  r'c&aaaaRVVVV3RllpV#)c$V^8dQhRRRRRR/#)rrIz_P.args user_kwargsz _P.kwargsr"rr#)r$s"r%r&Jmake_graphed_callables..make_graphed_forward..__annotate__*s&CCC CbCr'cF<SPS8Xd S!V/VB#S!V/VB#r|)training)rIr^rsrYrZr[s*,r%new_fwdEmake_graphed_callables..make_graphed_forward..new_fwd*s2}}(<<& A[AA'BkBBr'r#)rsrYrZr[rbsffff r%make_graphed_forward4make_graphed_callables..make_graphed_forward$s CCr'zModules must not have hooks registered at the time they are passed. However, registering hooks on modules after passing them through make_graphed_callables is allowed.zIn any :class:`~torch.nn.Module` passed to :func:`~make_graphed_callables`, only parameters may be trainable. All buffers must have ``requires_grad=False``.zfIn the beta API, sample_args for each callable must contain only Tensors. Other types are not allowed.r#)+r*is_autocast_enabledis_autocast_cache_enabledrHr,r4r2r3rr>nnModuler=_backward_hooks_forward_hooks_forward_pre_hooksr]allbuffersrLrMrNr~ parametersrkr+rrBr5r\rra tree_leavesrDr@r tree_flattenreversedrreverse enumeraterar6).rrrrr rrjust_one_callable _sample_argsflatten_sample_argscr flatten_argper_callable_len_user_argsper_callable_module_paramsr"per_callable_static_input_surfacesr fwd_graphs bwd_graphsmempoolrarsr# grad_inputsr outputs_gradvper_callable_static_outputs"per_callable_output_unflatten_specr func_outputsflatten_outputsspec per_callable_static_grad_outputsper_callable_static_grad_inputsr$rr%r&grad_idxr rVretrZrds.&&&&&&& r%rrsd   ""u'F'F'H'H m   i ' ' L  E&#+$6 DF {{5vs{);S)@#A;O y/ a ) )A%%&!+(()Q.,,-2$a3EE333EEEE$1 kk))994@ ""5#56sHKHsssHKHHH ^ )069L!L8K#d)8K!L"A",Auxx!?!?allnRG" s9~&*&A !;!>>>&'* 38I2GH2GQ%**&&(2GJH27I2GH2GQ%**&&(2GJH%)\!|D7IG  JJZZ   F   6 " "03 %G1 ,D$,26tT,K+,++--99$+F$u$K$Kuu$K$KK |$q("'.."5"5 ,$u%';%uu%';% &+U+9@+UU+9@+&%)%7#6 #K -|[9:'1  #. JJ#%)+&!$Yj!IdI ZZ   11    ;L !& 3 3 @ @ N#**5+AB*11$7"J(*$&(#;>34,-<7ni$e$ FT$ ee$ FT$  uJJuuJJJ  | q !!#5#5 "$nn11( 5 T,@ T55 T,@ TT!&&W2E&W&W2E&W!W $!3 2 $ 'C   [%<"))+h*?@A "))$/ ( ##56(//0CD'../ABW<\%,,.#++-=@$&CY'40 qM qM & ) & ) .q 1 .q 1 ' * ,Q / +A .   dEHHOO , ,  0dmmWdllDL JJt  JJw I(L1v :G"M"* IH # " "B   <s]cAc c;'c:'cBc3$c*c>c*c3Ad/(d c0 3 d  d ) r>rBrrrrrrrrrrr)FNrF)V__conditional_annotations__ __future__rrrr2rr collectionsrcollections.abcrrrrr r typing_extensionsr r r r*rtorch.cuda._utilsr cuda.bindingsrr\rrG ImportError torch.cudarrrrrvrrrAny_utilsr__all__rrr.r5hasattrr__dict__torch._Cr9r7r8r>rBrIrKrlrmrnrorprxr}rrrrrrrrrr Structurerr}r/rrrr!rr)rs@r%rs}"" #$GG66 20F1(72J 2"JJJ &  " T]t_E -uxx*++&1,&?EHHl#.9:N.OEHH*+:E(;EHH67TS,9  ++lMJLW=HXKV=HXMIKV-GW B@@ A?GRmCS' L  L ^f 7   T   E,E,R %%6f8M%MN9N   ( (aaK&LMsI%% I43I4