+ jsa0t$Rt^RIHt^RIt^RIt^RIt^RIH t ^RI H t ^RI H t ^RIHtHtHtHt]'d ^RIHtHt^RIt^RIHtHtHt^R IHtHt] !]!4t"]#]]$],3,t%R ]&R &]'](])](,3,t*R ]&R &Rs+R ]&R&Rs,R]&R&Rs-R]&R&RRlt.RRlt/RRlt0RRlt1RRlt2RRlt3R R!lt4R"R#lt5R$R%lt6R&R'lt7R(RR)R/R*R+llt8] !]$4t9R,]&R-&Rs:R.]&R/&R0R1lt;RsR5]&R6&R7t?R8R9lt@R:R;ltAR<R=ltB]C!4tDR>]&R?&^sER@]&RA&!RBRC]4tFRDREltGRFRGltH!RHRI]P4tJ]C!4tKR>]&RJ&RKRLltLRMRNltM] RORP/RQRRll4tNRSRTltORURVltPRWRXltQRYRZltRR[R\ltSR]R^ltTR_R`ltURaRbltV^= 13.1 or appropriate cuda-compat). When unavailable, ``mark_kernels`` silently becomes a no-op. Annotation recording is enabled per capture via ``torch.cuda.graph``'s ``enable_annotations`` argument, which also resolves and remaps the recorded annotations to the exec graph automatically on context exit. Usage during capture:: from torch.cuda._graph_annotations import mark_kernels, get_kernel_annotations with torch.cuda.graph(graph, enable_annotations=True): with mark_kernels("phase_A"): y = workload_a(x) with mark_kernels("phase_B"): z = workload_b(y) annotations = get_kernel_annotations() When you need to drive this outside the context manager's automatic path, ``resolve_and_remap(graph)`` is shorthand for ``resolve_pending_annotations()`` followed by ``remap_to_exec_graph(graph)``; call those directly for finer control (e.g. resolving once before remapping several graphs). ) annotationsN) defaultdict)contextmanager) getLogger)Any NamedTuple TYPE_CHECKING TypeAlias)IterableMapping)_check_cuda_bindings_check_cuda_bindings_driver_HAS_CUDA_BINDINGS)driverruntimer _CaptureState_ExistingDirectDependentsz bool | None_tools_id_availableFbool_annotations_enabledz int | None_capture_root_graph_idc V^8dQhRRRR/#)enabledrreturnNone)formats"^/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/cuda/_graph_annotations.py __annotate__rfs&&d&t&c"VsV'gRsR#R#)z}Set whether annotation recording is active. Used by ``torch.cuda.graph`` to scope annotations to a capture; not a public API.N)rr)rs&r_set_annotations_enabledr"fs# !% r c V^8dQhRRRR/#)rgraphrrintr)rs"rrrosSSr c@\\P!V44#)zThe driver's unique id for a ``cudaGraph_t``. Identifies a graph across its whole lifetime, unlike the handle itself: that is a pointer, so a graph created after another is destroyed can reuse the value and compare equal to it.)r _cuda_runtimecudaGraphGetId)r$s&r _graph_idr)os $$U+ r c V^8dQhRRRR/#)rstreamrrrr)rs"rrrzs55S5T5r cRs\'d\4'dR#\P!VP R7p\ V4pVe\V^,4sR#R#)zRecord the top-level capture graph of the capture just begun on ``stream``. Called by ``torch.cuda.graph`` right after ``capture_begin``, while ``stream`` is still capturing into the top-level graph. Not a public API.N init_value)rr_is_tools_id_unavailabler' cudaStream_t cuda_stream_get_capture_stater))r+ stream_handlestates& rmaybe_stamp_capture_rootr5zs[ "  #;#=#=!..%%M } -E !*58!4r cV^8dQhRR/#rrrr)rs"rrrs!!!r c:\\R4'g8\PP R4p\ P RV4R#\P!^4vrV\PP8Xd\ PR4R#R#)zProbe whether cudaGraphNodeGetToolsId is supported by the driver. Calls with a null node: cudaErrorInvalidValue means the API exists in the driver (good), cudaErrorCallRequiresNewerDriver means it does not (bad). cudaGraphNodeGetToolsIdz cuda-bindingszcudaGraphNodeGetToolsId API not found in cuda-bindings. Current version: %s, required: >= 13.1.0. CUDA graph kernel annotations will be disabled. To enable annotations, upgrade cuda-bindings: pip install --upgrade cuda-bindingsFzvcudaGraphNodeGetToolsId requires a newer driver (missing cuda-compat?); CUDA graph kernel annotations will be disabledT) hasattrr' importlibmetadataversionloggerwarningr9 cudaError_t cudaErrorCallRequiresNewerDriverinfo)cuda_bindings_versionerr_s r_probe_tools_idrFs ="; < < ) 2 2 : :? K 2 "  33 GC  $ $ E E F   =  r cV^8dQhRR/#r7r)rs"rrrs##$#r cn\'gR#\e \'*#\4s\'*#)z5Return True if cudaGraphNodeGetToolsId is not usable.T)rrrFrr rr/r/s.  &&&&)+" ""r cV^8dQhRR/#r7r)rs"rrrs H Hd Hr cl\PP4;'d\4'*#)ais_available() -> bool Return whether CUDA graph annotation recording is supported. Requires a CUDA device, the ``cuda-bindings`` package, and a driver that supports ``cudaGraphNodeGetToolsId`` (CUDA >= 13.1 or an equivalent cuda-compat package). When this returns ``False``, :func:`mark_kernels` is a silent no-op and no annotations are recorded. The first call may probe the CUDA driver; the result is cached. )torchcuda is_availabler/rr rrMrMs& :: " " $ G G-E-G)GGr c V^8dQhRRRR/#)rr+rrz_CaptureState | Noner)rs"rrrs * *s *'; *r c\\P!V44wrr4rVV\PP8wdR#V\ VRV43#)zBReturn ``(graph, frontier)`` for an active capture, else ``None``.N)r r'cudaStreamGetCaptureInfocudaStreamCaptureStatuscudaStreamCaptureStatusActivelist)r+status_idr$_deps _edge_data _num_depss& rr2r2sZ7K..  84Fz   0 0 N N O $uZi() ))r c V^8dQhRRRR/#)rr$rr list[Any]r)rs"rrrs##3#9#r c\\P!V44wrV^8Xd.#\\P!WR74wr2\VRV4#)z,Return the current root nodes for the graph.) pNumRootNodesN)r r'cudaGraphGetRootNodesrS)r$rE num_rootsrootss& r_get_root_nodesr`s_'++  LA A~ +++  E jy! ""r c V^8dQhRRRR/#)rnoderrrZr)rs"rrrs--s-y-r c\\P!V44wrV^8Xd.#\\P!WR74wr4p\VRV4#)z-Return the direct dependents of a graph node.)pNumDependentNodesN)r r'cudaGraphNodeGetDependentNodesrS)rbrEnum_dependents dependentsrWs& r_get_dependent_nodesrhse/44  Aq  -A44  .*JN  ?N+ ,,r c V^8dQhRRRR/#)rrbrrr)rs"rrrs    r c@\\P!V44#)aReturn graph node type without tripping runtime bugs on newer node kinds. The runtime ``cudaGraphNodeGetType`` can return ``cudaErrorUnknown`` (999) for valid nodes whose driver type is ``CU_GRAPH_NODE_TYPE_BATCH_MEM_OP``. Query via the driver API instead. )r _cuda_drivercuGraphNodeGetType)rbs&r_get_node_typerms '''- r existing_direct_dependentsinclude_start_nodesc(V^8dQhRRRRRRRR/#) r start_nodesrZrnz _ExistingDirectDependents | Nonerorrzdict[int, Any]r)rs"rrrs3!!!!A! !  !r cT;'g/pVUu0uFp\V4kK pp/p\V4pV'dVFpW5\V4&K V'dVP4pVP\V4\ 44p\ V4FBp\V4p W9gW9dKVP V 4WV &VPV4KD KV#uupi)aWalk dependent edges starting at ``start_nodes``. ``existing_direct_dependents`` maps each node in ``start_nodes`` to the direct dependent node keys that were already present at scope entry. Those edges are skipped so the traversal only follows nodes added after scope entry. )r%rSpopgetsetrhaddappend) rqrnrorbseen descendantsstackold_dependents dependent dependent_keys &$$ r_collect_descendantsr~s" [] run 2: prehook push [e2]; node throws, no pop -> [e2] run 3: prehook push [e2, e3]; posthook pops e3 -> [e2] Pops always take the top, so the leaked e2 sits below all later entries and every subsequent run still pops its own. The only loss is e2 itself: that run's kernels go unrecorded, which is moot since its backward failed. (The region/creation-hook TLS the prehook set is not leaked -- the engine task's ThreadLocalStateGuard restores those on unwind.) cV^8dQhRR/#rrrr)rs"rr_BracketState.__annotate__s##$#r c .VnR#rrz)selfs&r__init___BracketState.__init__s " r rN)rrrrrrrrr rrrs.##r r_SKIPPEDc$V^8dQhRRRRRR/#)rrbr generationr%rrr)rs"rrrs!55c5s5t5r c\V\PPP4'dR#\ VP 9dR#\4pVfR#RVP \ &\WV4R#)aUNode creation hook: freeze the current region into ``node``'s bracket. Reads the region TLS slot and, if a region is open, installs the node's single backward hook pair closing over that snapshot. Every backward- annotating ``mark_kernels`` scope pushes one of these (and so does an executing hooked node, to cover nodes created during its backward), so a node created under nested scopes sees several identical firings; the ``node.metadata`` marker makes only the first attach. AccumulateGrad is always excluded: a leaf's node is created once and cached (possibly during warmup), so which scope would own it is an accident of first use, and its work (the ``.grad`` accumulation) belongs to the leaf rather than to any forward region. If accumulation ever needs annotation it should get a dedicated leaf-accumulation region, not first-use assignment. NT) isinstancerKr _functionsAccumulateGradrr<r_attach_backward_hooks)rbrfrozens&& r_freeze_region_hookrs["$++::;;dmm#  F ~!%DMM+44r c(V^8dQhRRRRRRRR/#) rrbrrrrr%rrr)rs"rrr+s/D!D!D!nD!#D!RVD!r caaaa\4oRV3RlloRVVVV3RllpRV3RllpVPV4VPV4R#)aRegister one pre/post hook pair bracketing ``node``'s backward execution. ``frozen`` is the region that was current when the node was created: the annotations of every scope then open, already collapsed inner-wins. The pair brackets the node's execution the same way ``mark_kernels`` brackets forward work: snapshot the capture frontier before, collect the new nodes after. It runs on the autograd engine thread executing the node, which during a whole-graph capture participates in the capture; when backward runs outside a capture the prehook sees no active capture and the pair is a no-op. Executing the node is semantically re-entering its frozen region, so the prehook merges ``frozen`` over the live region (a ``mark_kernels`` enclosing the whole ``backward()`` call, if any) and installs the result while the node runs. That gives nodes created during its backward (``create_graph=True``, checkpoint recomputation) the forward region's ownership -- a creation hook is pushed alongside so they get hooked even when this backward runs eagerly and no scope's hook is live -- and it puts a ``mark_kernels`` opened inside the execution dynamically inner, so it outranks ``frozen``, which outranks the enclosing scope. The posthook records the merged region once; scope completion order then yields exactly that precedence at resolve. The pops are exception-safe without a finally: each engine task runs under an ``at::ThreadLocalStateGuard`` that restores both the creation-hook TLS and the region slot even when the node throws. ``generation`` is the annotation generation the owning scope was opened under; the bracket only records (and only propagates to created nodes) while it is current, so ``clear_kernel_annotations`` makes stale hooks inert. c V^8dQhRRRR/#)rchildrrrr)rs"rr,_attach_backward_hooks..__annotate__Ns//S/T/r c <\VS4R#rr)rrs&r creation_hook-_attach_backward_hooks..creation_hookNs E:.r c V^8dQhRRRR/#)r _grad_outputsrrrr)rs"rrrQs 2 2s 2t 2r c<S\8wg\4'd"SPP\4R#/\ 4;'g/CSCp\ V4p\PPPS4\'d \4MRpSPPW2V34R#r) rr/rzrwrrrrKr _autograd_push_node_creation_hookrr)rmergedrrrrrr4s& rprehook'_attach_backward_hooks..prehookQs / /3K3M3M KK  x ( 9_&,,"88V$ 33MB)=)=#%4 E01r c$V^8dQhRRRRRR/#)r _grad_inputsrrrrr)rs"rrr^s!88s83848r c^<SP'gR#SPP4pV\JdR#Vwr4p\PP P 4\V4VfR#\V4pV'd!/VCRR/Cp\PWv34R#R#)Nautograd_phasebackward) rzrsrrKrr_pop_node_creation_hookrrrrw) rrentryrrrrtaggedr4s && rposthook(_attach_backward_hooks..posthook^s{{{  ! H  #V 224T = %e, == 0*=F  " "F#6 7 r N)rregister_prehook register_hook)rbrrrrrr4s&ff @@rrr+sIB OE// 2 288  '"x r rTc V^8dQhRRRR/#)r annotationstr | dict[str, Any]rrr)rs"rrrss88188r c#a"\'d\4'dRxR#\V\4'dRV/p\ 4pVfRxR#\ e=\ VP4\ 8wd\P!R^R7RxR#\oRV3Rllp\/\4;'g/CVC4pV'd@\PPPV4;_uu_4RxRRR4MRx\!V4\#V4pV'd\$P'W34R#R# +'giLH;i \!T4i;i5i)amark_kernels(annotation, *, backward=True) Context manager that annotates GPU work captured within its scope. Must be used inside an active :class:`torch.cuda.graph` capture with ``enable_annotations=True``. Every kernel, memcpy, and memset node the capture adds within the scope is tagged with :attr:`annotation`. Outside a capture, with annotations disabled, or when :func:`is_available` is ``False``, the context manager is a no-op. When scopes overlap on the same node (e.g. nested scopes), their annotation dicts are merged key-by-key with the inner scope winning common keys. By default backward work is annotated too: autograd nodes created by forward operations inside the scope get hooks (via :class:`torch.autograd.graph.node_creation_hook`) that bracket their backward execution, so when the backward pass is itself captured -- in the same capture as the forward or in a later one -- its kernels are tagged with the same annotation, plus an ``"autograd_phase": "backward"`` key marking them as backward work (``"autograd_phase"`` is therefore reserved: backward annotation overwrites it). When backward runs outside a capture the hooks are a no-op. Ownership extends to higher-order gradients: nodes created while a hooked node executes (``create_graph=True``, checkpoint recomputation) inherit its annotations, so a later grad-of-grad capture is attributed too. ``AccumulateGrad`` nodes are never annotated: a leaf's node is created once and cached, so scope ownership would be an accident of first use. Pass ``backward=False`` to annotate only the forward work, e.g. when a wrapper implements its own backward attribution. The keyword's presence also serves as the feature probe for that native backward support: ``"backward" in inspect.signature(mark_kernels).parameters``. Implementation: on entry, records the current stream's capture frontier and its existing direct dependents; on scope exit, walks only the dependent nodes added since entry (falling back to newly created graph roots when the scope is the first captured work). Args: annotation (str or dict): Metadata to attach to each captured node. A string ``s`` is recorded as ``{"name": s}``. Dict values must be picklable. The key ``"name"`` names the region in trace tooling; ``"stream"`` is reserved for stream-lane assignment. backward (bool): Whether to also annotate the backward kernels of autograd nodes created inside the scope. Default: ``True``. .. note:: The nodes to annotate must be reachable from the capture frontier of the stream that is current on scope entry. Work on a different already-capturing stream must be synchronized with the current stream first. .. note:: Child-graph and conditional nodes have bodies in a separate ``cudaGraph_t`` that this walk does not descend into, so their work is left unannotated and a warning is issued. Descending is possible (``cudaGraphNodeGetParams`` exposes the body graphs), but would not be enough on its own: a body's nodes are numbered in that graph's id space and are renumbered again when the exec graph inlines them, and nothing exposes that renumbering, so :func:`remap_to_exec_graph` could not key the annotations to what a profiler reports. For the same reason a scope *inside* a conditional body (``torch.cond`` / ``torch.while_loop``) records nothing at all. .. warning:: This API is in prototype and may change in future releases. Example:: >>> # xdoctest: +SKIP("requires cuda-bindings and driver >= 13.1") >>> g = torch.cuda.CUDAGraph() >>> x = torch.randn(8, device="cuda") >>> with torch.cuda.graph(g, enable_annotations=True): ... with torch.cuda.graph_annotations.mark_kernels("phase_A"): ... y = x + 1 Nrzmark_kernels: this scope is inside a CUDA graph conditional-node body (torch.cond / torch.while_loop), which is captured into a separate cudaGraph_t whose node ids are never remapped to the exec graph; nothing is annotated for itrc V^8dQhRRRR/#)rrbrrrr)rs"rr"mark_kernels..__annotate__s..C.D.r c <\VS4R#rr)rbrs&rr#mark_kernels..creation_hooks D*-r )rr/rstrrrr)r$rrrrrrKautogradnode_creation_hookrrrrw)rrrrrrrs&$ @r mark_kernelsr rs&\ #;#=#= *c""j)  !E }  * ekk "&< <   *   'J.. D_.44"DD ED %%88GGHG T!%(I 67HG TsBB:E/> E/ E1EE  E8E/ E E E,,E/c V^8dQhRRRR/#rtorch_cuda_graphztorch.cuda.CUDAGraphrrr)rs"rrrs..3G.D.r c\'d\4'dR#\\P!VP 444VnRVnR#)aVRecord the captured graph's id on the graph object for later remap. Called from ``capture_end`` in the window after capture ends and before the graph is finalized, where the template ``cudaGraph_t`` is live for both ``keep_graph`` modes (and reachable via ``raw_cuda_graph``). ``remap_to_exec_graph`` later matches this graph's annotations to its exec id via this stamp, without relying on call ordering. No-op if annotations are disabled or ``cudaGraphNodeGetToolsId`` is unavailable. Harmless when no ``mark_kernels`` regions run: the annotation map stays empty so resolve and remap are no-ops. N)rr/r r'r(raw_cuda_graph_capture_graph_id_remapped_exec_idrs&rmaybe_stamp_capture_graph_idrsK #;#=#= *>$$  + + - *& *.&r cV^8dQhRR/#rr)rs"rrrs  T r c\'gR#\\4p\F%wrVFpW,PV4K K' VP 4Fwr4\ V4^8Xd&\ V,PV^,4K:/pVFWp\V\4'd-VP 4FwrgVPWg4K KEVPRV4KY \ V,PV4K \P4R# \d\PR4L8i;i \P4i;i)z7Resolve pending scope toolsIds into kernel annotations.Nrz"resolve_pending_annotations failed)rrrSrwitemslenrrdict setdefault Exceptionr> exceptionclear) per_tools_idrrtools_idann_listrkeyvalues rresolve_pending_annotationsr%s" ? 4?4E %4 !J%&--j9&&5#/"4"4"6 H8}!#H-44Xa[A%'F& j$//&0&6&6&8 ))#5'9%%fj9 '  ) 0 0 8#7  ?=>? s$DD-- E EEEE)c V^8dQhRRRR/#rr)rs"rrr-s,7,7*>,74,7r cbVPp\'dVfR#\\P!VP 444pVP fTM VP pW28XdR#\\W24p\P4\PV4W nR#)aRemap annotation keys from capture graph ID to exec graph ID. During capture, toolsId encodes the capture graph's ID in the upper 32 bits. After instantiation, the profiler uses the exec graph's ID. This function rewrites the keys so annotations match the trace. The graph's capture id is read from the ``_capture_graph_id`` stamped on it by ``maybe_stamp_capture_graph_id`` in the capture_end window, so only the annotations belonging to this graph are rekeyed. This is order-independent and correct when several graphs are captured in sequence: call once per graph. Graphs captured with annotations disabled have no capture id and are skipped. The exec graph id is only defined once the graph is instantiated. With ``keep_graph=True`` instantiation is deferred past the ``torch.cuda.graph()`` context, so the remap is driven from the graph's ``instantiate()``/ ``replay()`` instead of context exit. Each ``instantiate()`` (even on an unmodified template) produces a fresh exec graph id, so this rekeys from wherever the annotations are currently keyed -- the capture id before the first remap, the previous exec id after a re-instantiate -- to the current exec id. ``_remapped_exec_id`` tracks that current key; when it already matches the live exec id (e.g. replay after instantiate) this is a no-op. N) rrr r'cudaGraphExecGetIdraw_cuda_graph_execr_rekey_annotationsrupdate)rcapture_graph_id exec_graph_idcurrent_key_idremappeds& rremap_to_exec_graphr0-s.(99  "2":(((  0 0 2 M  - - 5   / / &!"5~UHx()6&r c(V^8dQhRRRRRRRR/#)rrzdict[int, list[Any]]r,r%r-rr)rs"rrr\s0% r c/pVP!4F^wrEV^ , V8wdWSV&KVR,pV^ ,V,pWs9dW7,PV4KQ\V4W7&K` V#)aRekey one graph's annotations from its capture id to its exec id. A toolsId packs the graph id in the upper 32 bits and the node id in the lower 32. Only entries whose upper bits match ``capture_graph_id`` are rewritten to ``exec_graph_id``; entries from other graphs (already remapped to their own exec ids, or pending their own remap) are kept as-is. When two capture-side ids collide on the same rekeyed id their lists are merged. l)rextendrS)rr,r-r/r!r"node_id new_tools_ids&&& rr*r*\sz&(H)//1 r>- -!)X  Z'%+w6  #  " ) )( 3%)(^H "2 Or c V^8dQhRRRR/#rr)rs"rrrws**(<**r c0\4\V4R#)zResolve any pending scopes and remap one graph in a single call. Shorthand for ``resolve_pending_annotations()`` followed by ``remap_to_exec_graph(graph)``; the pair normally run after a capture. N)r%r0rs&rresolve_and_remapr8ws  !()r cV^8dQhRR/#)rrzMapping[int, list[Any]]r)rs"rrrs 7r c\#)a9get_kernel_annotations() -> Mapping[int, list] Return the live registry of recorded kernel annotations. Keys are opaque integers matching the ``graph node id`` field that CUPTI-based profilers attach to kernel events; values are the lists of annotation dicts recorded for that node. The registry accumulates across captures and is global to the process. The returned mapping is a **live view**: it is updated in place when a graph is instantiated (annotation keys are rekeyed to the executable graph's ids), so a reference obtained early stays current. Keys are valid for joining against a profiler trace once the corresponding graphs have been instantiated. Treat the mapping as read-only; snapshot it with ``dict(...)`` if isolation is needed. .. warning:: This API is in prototype and may change in future releases. Example:: >>> # xdoctest: +SKIP("requires cuda-bindings and driver >= 13.1") >>> annotations = torch.cuda.graph_annotations.get_kernel_annotations() >>> with open("annotations.pkl", "wb") as f: ... pickle.dump(dict(annotations), f) )rrr rget_kernel_annotationsr;s 6 r cV^8dQhRR/#rr)rs"rrrs  $ r cp\P4\P4\^, sR#)aclear_kernel_annotations() -> None Clear all recorded kernel annotations. The annotation registry is process-global and accumulates across captures; long-running workloads that capture many graphs should clear it once recorded annotations have been consumed (e.g. after saving them alongside a profiler trace). Clearing forgets everything recorded so far and revokes recording from every scope opened before the clear: backward hooks that :func:`mark_kernels` attached to existing autograd nodes stay on the graph but become inert. Scopes opened after the clear record normally. .. warning:: This API is in prototype and may change in future releases. N)rrrrrr rclear_kernel_annotationsr>s'&ar c V^8dQhRRRR/#)rexec_graph_idsz Iterable[int]rrr)rs"rrrs%%m%%r c\V4pV'gR#\Uu.uFq"^ , V9gKVNK upF p\VK R#uupi)zDrop kernel-annotation entries whose exec graph id (tools_id >> 32) is in exec_graph_ids, so the map does not grow across the run. Run by the annotation resolver's graph-destroy handler.N)rur)r@idskr#s& rremove_kernel_annotationsrDsE n C .A.ar'S..A  $BAs A A _stream_id_counterzdict[int, int]_stream_id_mapc V^8dQhRRRR/#)rr+torch.cuda.Streamrr%r)rs"rrrs,r c~VPpV\9d\\V&\^, s\V,#)z;Return a small, stable stream ID for the given CUDA stream.)r1rFrE)r+r#s& r_get_stream_idrJs7   C . 0sa # r c V^8dQhRRRR/#)rpg_keyr rr%r)rs"rrrs""c"c"r cfV\9d\\V&\^, s\V,#)z:Return a unique stream ID for the given process group key.)rFrE)rLs&rget_stream_for_pgrNs,^#!3va & !!r c V^8dQhRRRR/#)rr+rHrrr)rs"rrrs)7Kr c #"\'g7\PPV4;_uu_4RxRRR4R#VP\PP 4P8Xd#\ V4;_uu_4RxRRR4R#\V\4'dRV/p\V\4'd\V4VR&\ V4;_uu_4\PPV4;_uu_4RxRRR4RRR4R# +'giR#;i +'giR#;i +'giLB;i +'giR#;i5i)uZSwitch to stream, inject its ID into annotation, and mark kernels. If *stream* is already the current stream, no stream switch or stream ID injection happens — the kernels stay on whatever stream is active (which keeps the trace faithful when e.g. FSDP uses the current stream for copy-in instead of a separate one). When switching to a different stream, this snapshots the current capturing branch before the target stream runs marked work. If the target stream is already capturing, the marked work must still be synchronized with the current stream so it is reachable from that snapped frontier. Nrr+) rrKrLr+r1rr rr rrJ)r+rs&&r mark_streamrQs  ZZ  v & & ' UZZ668DDD * % % & % j# & & *-J j$ ' '#1 Jx * % %""6**+& %' & % % %+*& % %se4E=D.AE=EAE=/(E)E E)$ E=. D? 9 E= E E= E& !E)) E: 4 E=)\__conditional_annotations__r __future__rimportlib.metadatar; threadingr collectionsr contextlibrloggingrtypingrrrr collections.abcr r rKtorch.cuda._utilsr r r cuda.bindingsrrkrr' ImportErrorrr>tuplerSrrrr%rurrrrr"r)r5rFr/rMr2r`rhrmr~rrrrrrrrrrobjectrrrrrlocalrrrrr rr%r0r*r8r;r>rDrErFrJrNrQ)rRs@rrasN)V##%<<1  8 !d3i0 y0'+CSM':9: $(['#d"&* )&5"!H# H *#"-" !DH!!& !J4?t3D0D'+O* ,(,_+H02,11 :8 S  7:7S&*Z#IOO#@#58D!N8t88D.4 <,7^6*< 0%C!##"mLMsG00 G?>G?