Mj:UdZddlmZddlmZddlmZddlmZddl Z ddl m Z m Z  ddl mZeeZdaedzed <d aeed <d*d Zd*dZd efdZded edzfdZded efdZded efdZeeZeeeefed<da e!edzed<d e!efdZ"gZ#ee$eeefed<da%eed<da&edzed<ede'e(e'efzfdZ)d*dZ*de jVjXd dfd Z-d e(eeeffd!Z.d*d"Z/d#a0eed$<iZ1e(eefed%<de jVjdd efd&Z3d'e'd efd(Z4ede jVjdde'e(e'efzfd)Z5y#e$rdZY>wxYw)+aAnnotate CUDA graph kernel nodes during capture. During CUDA graph capture, ``mark_kernels`` uses ``cudaGraphGetNodes`` to count nodes before and after the wrapped region. Nodes at indices ``[before, after)`` are the ones added within the scope. Each kernel or memcpy node found is annotated by its ``toolsId`` so it can later be matched to profiler trace events. The annotations can be pickled and later merged into a Chrome profiler trace using ``torch.cuda._annotate_cuda_graph_trace``. Requires ``cuda.bindings`` package and a CUDA driver that supports ``cudaGraphNodeGetToolsId`` (CUDA >= 13.1 or appropriate cuda-compat). When unavailable, ``mark_kernels`` silently becomes a no-op. Usage during capture:: from torch.cuda._graph_annotations import ( enable_annotations, mark_kernels, resolve_pending_annotations, remap_to_exec_graph, ) enable_annotations() with torch.cuda.graph(graph): with mark_kernels("phase_A"): y = workload_a(x) with mark_kernels("phase_B"): z = workload_b(y) resolve_pending_annotations() remap_to_exec_graph(graph) ) defaultdict)contextmanager) getLogger)AnyN)_check_cuda_bindings_HAS_CUDA_BINDINGS)runtime_tools_id_availableF_annotations_enabledreturncday)z#Enable kernel annotation recording.TNr \/var/www/html/ia-enhanced/venv/lib/python3.12/site-packages/torch/cuda/_graph_annotations.pyenable_annotationsrAs  rcday)z$Disable kernel annotation recording.FNrrrrdisable_annotationsrGs !rcFtsytduryttdsyy)zBReturn True if we already know cudaGraphNodeGetToolsId is missing.TFcudaGraphNodeGetToolsId)rr hasattr _cuda_runtimerrr_is_tools_id_unavailablerMs$ e# ="; < rnodect# ttj|}da|Sttj|S#t$rdat j dYywxYw)z$rtjAdY7wxYw#tjdawxYw)a Resolve pending scope index ranges into kernel annotations. Enumerates all graph nodes and annotates kernel/memcpy nodes whose indices fall within recorded scope ranges. Must be called while still inside the ``torch.cuda.graph()`` capture context. NrBz6resolve_pending_annotations: no graph handle availablerr. c36K|]\}\}}}||||fywr7r).0iannstartends r z.resolve_pending_annotations..s, (A(UCeS!$ sc"|d|d |d fS)Nr)ss rz-resolve_pending_annotations.. s1Q4!A$1.r)keyrXz:resolve_pending_annotations: toolsId unavailable, abortingnamez"resolve_pending_annotations failed)!r<r=rrErFrGrHrIr,rwarningclearr3rr0r r>r;sorted enumeraterangepoplenrJcudaGraphNodeGetTyper4reversedrDdictitems setdefaultr exception)r!r(r2nodes first_tid annotatable sorted_scopes scope_ptr active_stackrRrS _start_idxend_idx_idxr node_typermergedr1akavs rresolve_pending_annotationsrzs[  ' '::,,.::F v &E } }OPZe$ !8  ! ! #!N h g*  + +  s 7%eAh/I:Ci2o ",.  ,5o,F /   .0 s0 =A<#3A#6!#;  "<#3A#6!#;c-00]95Ma5PTU5U1>y1I.Z$Q; ''#7Q c-00]95Ma5PTU5U  8D,22I  +$T*HP %%'!%* )< A%#H-44\!_Q5GH*,&|47FAs!#t,&)iik6FB"--b"56))&#6 7 $H-44V? sF&LB(L-ALA$LB+L-L)L&#L)%L&&L))Mtorch_cuda_graphctsytj|j}t tj |}t }i}tjD]K\}}|dz }| ||k7r|||<|dz}|dz|z} | |vr|| j|>t||| <Mtjtj|y)a\Remap annotation keys from capture graph ID to exec graph ID. During capture, toolsId encodes the capture graph's ID in the upper 32 bits. After instantiation, the profiler uses the exec graph's ID. This function rewrites the keys so annotations match the trace. Must be called after the ``torch.cuda.graph()`` context exits. NrBrOl) r4rcudaGraphExec_traw_cuda_graph_execrcudaGraphExecGetIdr>rjextendlistraupdate) r{ exec_handle exec_graph_idcapture_graph_idremappedrann_listgraph_idnode_id new_tools_ids rremap_to_exec_graphrIs //#779K)((  M.%'H1779 4(r>  'H8H,H!)HX  Z'%+w6 8 # \ " ) )( 3%)(^H\ " 4x(rctS)zBReturn the current kernel annotation map (toolsId -> annotations).)r4rrrget_kernel_annotationsrts rcXtjtjday)z9Clear all recorded kernel annotations and pending scopes.N)r4rar<r=rrrclear_kernel_annotationsrys!Nr<_stream_id_counter_stream_id_mapch|j}|tvrtt|<tdz at|S)z;Return a small, stable stream ID for the given CUDA stream.rX)rIrr)r!r]s r_get_stream_idrs7   C . 0sa # rpg_keycP|tvrtt|<tdz at|S)z:Return a unique stream ID for the given process group key.rX)rr)rs rget_stream_for_pgrs,^#!3va & !!rc#RKts-tjj|5ddddy|jtjj jk(rt |5ddddyt|trd|i}t|trt||d<tjj|5t |5dddddddy#1swYyxYw#1swYyxYw#1swY*xYw#1swYyxYww)uTSwitch to stream, inject its ID into annotation, and mark kernels. If *stream* is already the current stream, no stream switch or stream ID injection happens — the kernels stay on whatever stream is active (which keeps the trace faithful when e.g. FSDP uses the current stream for copy-in instead of a separate one). NrAr!) r rFrGr!rIrHrMrDrArir)r!r?s r mark_streamrs ZZ  v &    UZZ668DDD * %     j# &,J j$ '#1 Jx ZZ  v & j)           se&D'C7A D'6D;AD' D!D&D. D'7D<D'D D'D DD$ D')r N)6__doc__ collectionsr contextlibrloggingrtypingrrFtorch.cuda._utilsrr cuda.bindingsr r ImportError__name__rr bool__annotations__r rrrintr r,r3rr4r5setr;r<tupler=r>rArirMrzrG CUDAGraphrrrrrStreamrrrrrrrs1"H$% F 8  $(TD['#d" ! $d 4 s s 334?t3D[d3i0D'+CHtO*C/1eCcM*+0&*d ))ES4S>1)E)EXrj()%***>*>()4()VS$s)^ 4 C!#S#X#5::,,"c"c" ))sT#s(^7KU MsE##E.-E.