+ j`\^RIt^RIt^RIt^RIt^RIHt^RIHt^RIt^RI t ^RI H uH t ^RI Ht^RIHt^RIHtHtHt^RIHt^RIHtHtHtHtHtHtHt^R IH t ^R I!H"t"^R I#H$t$^R I%H&t&]PN!](4t)R Rlt*!RR]PV4t,!RR]4t-R#)N)CallableAny)counters)ir)SubgraphCPUBenchmarkRequestSubgraphGPUBenchmarkRequest TensorMeta)KernelTemplate)Buffer FixedLayoutget_free_symbolsget_symbolic_inputsgm_original_output_stridesir_node_to_tensorLayout) benchmarker)do_bench_using_profilingV) OrderedSetcV^8dQhR\PPR\\,R\ R\/#)gminputsnamereturn)torchfx GraphModulelistrstr)formats"a/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/_inductor/codegen/subgraph.py __annotate__r%&s:)) )&*3i)7:))c^RIHp\PPpV\PnV!W4V\Pn# T\Pni;i)aInline a subgraph by converting its FX operations to individual IR nodes. This converts a subgraph to multiple ComputedBuffer nodes (fusable), enabling epilogue fusion with subsequent operations. Returns: TensorBox containing the final operation result as individual IR nodes )process_subgraph_nodes)torch._inductor.loweringr(rgraphmodule)rrrr(original_modules&&& r$inline_subgraph_to_ir_nodesr-&sF@ggnnO)%b1(s AA+c:aa]tRt^>?$FB? B?r&c <\S V`WW44.p.Vn\P;_uu_4\ VP 4Fwr\\V P4RR74^8wd \R4h\\V P4RR74^8wd \R4hV PP4VP\V 44Ve0W9d*VPPWh,!V 44KVPP\V RR74K RRR4V!V!Vn\#VP 4Wpn\'VP 4VnVP+4VnRVn/Vn/VnRVnRVn\8P:'dN\P;_uu_4VP=4VnVP?4VnRRR4R#R# +'giL;i +'giR#;i)T) unbacked_onlyz*expected no unbacked symbols in input sizez,expected no unbacked symbols in input strideN)replace_symbols_with_hints) super__init__benchmark_inputsr fake_mode enumerater1lenrget_sizeAssertionError get_stridedata freeze_layoutappendrrrexample_inputsr sym_inputs_compute_sym_input_valuessym_input_values decompositiondecomposition_kwargsconfig_patches_compiled_module_bmreqconfigpipeline_max_autotune_gemm_compile_for_benchmarking_create_benchmark_request) selfrr1r2r3r4r5 trace_inputsiinp __class__s &&&&&&& r$r?SubgraphChoiceCaller.__init__Bs F@ " [[[#D$4$45' dKLPQQ()UVV'(8MNRSS()WXX&&(##$5c$:;!,1C))001A#1FG))00)#$O!6, ."477+*-d.>.>? $ > > @9=46!.0%)   , , ,(,(F(F(H%"<<>  -Y[ZsDH,5+H?, H< ? I c0<V^8dQhRS[S[,/#rr)r!r7)r#r9s"r$r%r:s49r&c  \VPUu.uF#p\VR4'gKVPNK% up4p/p\ VP VP 4FwrE\V\P4'gK'\ VP4VP4Fgwrg\V\P4'd\V4W6P&K>\V4V9gKP\V4V\V4&Ki K .pVPFp \V \P4'd5V PV9d$VP!W9P,4KW\"P$P&P(P+V ^R7p VP!\V 44K V#uupi)a Extract concrete dimension values for sym_inputs from benchmark_inputs. The compiled module expects symbolic dimension values as runtime arguments. This maps each symbolic variable to its concrete value from the benchmark tensors. Used for range based autotuning. r)fallback)rrKhasattrrzipr1r@ isinstancerr8rDshapesympySymbolr7r"rIrr*sizevars shape_envoptimization_hint) rWssym_input_namessym_name_to_valueinp_node benchmark_inpsym_dim actual_dimresultsym_varhints & r$rL.SubgraphChoiceCaller._compute_sym_input_valuess_%!__ C_60BVQVV_ C  -/'*4+;+;T=R=R'S #H-66+.%%')<)<,'G"'5<<88:=j/),,7W8:=j/)#g,7 ,(TG'5<<00W\\EV5V / =>ww''11CCGVWCX c$i( '  - Ds GGcT<V^8dQhRS[RS[3,RS[S[S[3,RR/#)rrN.kwargsrNrrr6r")r#r9s"r$r%r:s6++%c3h/+9=c3h+ +r&c WnW nR#)zHCache decomposition function and kwargs for range-based dispatch lookup.N)rNrO)rWrNrvs&&&r$cache_decomposition(SubgraphChoiceCaller.cache_decompositions+$*!r&c <V^8dQhRS[/#r^r")r#r9s"r$r%r:s...r&c"RVP R2#)zSubgraphCaller()rrWs&r$__str__SubgraphChoiceCaller.__str__s  1--r&c <V^8dQhRS[/#r^r)r#r9s"r$r%r:s-=-=3-=r&c  ^RIHpVPPRR4PRR4pVP'd VP M VP p\PRVPVP4VPf \R4hV!VPV\PP\PP\PP\PP \PP"\PP$RV 2R 7 pVPF@pWTP&VP&VP(P+VP4KB \P,!V4;_uu_4R R R R R RRR RR /VP.Cp\0P2!V4;_uu_4VP4!V!VP74uuRRR4uuRRR4# +'giM;iRRR4R# +'giR#;i)zCCompile the subgraph for benchmarking, returns the compiled module.) GraphLoweringz::_.z#Benchmark compile %s: sym_inputs=%sNexpected self.gm to be set benchmark_) rrJrh cpp_wrapperaot_modeextern_node_serializer is_inference is_backwardr max_autotuneFmax_autotune_gemmmax_autotune_gemm_backendsATENbenchmark_fusionrT)torch._inductor.graphrrreplacerKrJr@logdebugrrErr* _shape_envrrrrr graph_inputsgraph_input_namesrIset_graph_handlerrPrSpatchruncompile_to_module)rWr safe_namecompile_inputsbm_graph_loweringsym_inpbenchmark_configs& r$rU.SubgraphChoiceCaller._compile_for_benchmarkings7II%%dC088cB $(???D  8M8M  7DOOT 77? !=> >)ww)gg((++WW%%#$77#A#A--++i[)  G;B * *7<< 8  / / 6 6w|| D' !2 3 3#U,f"E,e 0 %% 0 .//!%%~6(::<0/4 30//4 3 3 3s$=7I4 H)  I) H9 4I I c0<V^8dQhRS[S[,/#r^r r)r#r9s"r$r%r:s  $'B B r&c  VPf \R4h\P!VP4p\P!VP 4pVP P PR8Xd\pM\pV!VPVV\4VPPVPPVPR7#)z0Create a benchmark request for async autotuning.z9Module must be compiled before creating benchmark requestcpu) kernel_nameinput_tensor_metaoutput_tensor_meta extra_args module_pathmodule_cache_keyrM)rQrEr from_irnodesr1r2devicetyperr rtuple__file__keyrM)rWrr bmreq_clss& r$rV.SubgraphChoiceCaller._create_benchmark_requests  ( K '33D4D4DE'44T[[A ;;   " "e +3I3I /1w--66!2266!22  r&c0<V^8dQhRS[S[,/#r^r)r#r9s"r$r%r:s $'B Br&c LVPf \R4hVP#)z`Benchmark request for async autotuning. Pre-compiled when pipeline_max_autotune_gemm is enabled.zMbmreq accessed but pipeline_max_autotune_gemm was not enabled during __init__)rRrErs&r$bmreqSubgraphChoiceCaller.bmreqs* ;;  _ {{r&c<V^8dQhRR/#)rrN)r#r9s"r$r%r:sEE$Er&c PVPfVP4VnR#R#)zKEnsure the module is compiled. Used for lazy compilation in non-async path.N)rQrUrs&r$_ensure_compiled%SubgraphChoiceCaller._ensure_compileds$  ($($B$B$DD ! )r&cP<V^8dQhRS[S[,RS[PRS[/#)rargsoutr)r!rrr8float)r#r9s"r$r%r: s*  tCy u||  r&c raaaVP4VPPoVPoRVVV3RllpVP'd\ P !V4#\P'd \V4#\ P!V\ P!.SOSO5!R7#)z>Regular benchmarking: compile if needed, then use benchmarker.c$V^8dQhR\/#r^r)r#s"r$r%4SubgraphChoiceCaller.benchmark..__annotate__s 1 1C 1r&c<S!.SOSO4#Nr)rbm_funcrKsr$fn*SubgraphChoiceCaller.benchmark..fns/Z/$/0 0r&)r) rrQcallrM_benchmark_with_cudagraphsrbenchmark_gpu_with_cuda_graphrS/profile_bandwidth_with_do_bench_using_profilingr benchmark infer_device)rWrrrrrKs&$j @@r$rSubgraphChoiceCaller.benchmark s '',,**  1 1  * * *< >xx   a(+ 151A1AB1A##clln%1AB 483C3CD3CC#cnn&'3CD DGGMM"    CDs "C "C c4<V^8dQhRS[P/#r^)r TensorBox)r#r9s"r$r%r:-s   R\\  r&c XVPf \R4h\PP \P !VP VPVPVPVPVP'dVPR74#RR74#)Nr)r2r1rrJ subgraph_namerP) rrErrcreateSubgraphBufferr2r1rJrrPrs&r$ output_node SubgraphChoiceCaller.output_node-s 77? !=> >||""   {{ ,,77#22"ii6:6I6I6It22    PT    r&c6<V^8dQhRS[S[S[3,/#r^r6r"r)r#r9s"r$r%r:;s  4S> r&c "RRRVP/#)zRInformation returned here is logged to the autotune log file when that is enabled.backendsubgraphrrrs&r$ info_dictSubgraphChoiceCaller.info_dict;s z 499  r&c <V^8dQhRS[/#r^r|)r#r9s"r$r%r:Bs''#'r&c RVP 2#) subgraph_rrs&r$autoheuristic_id%SubgraphChoiceCaller.autoheuristic_idBs499+&&r&) rRrQr@rPrNrOrJrrMrKr)__name__ __module__ __qualname____firstlineno____doc__r?rLryrrUrVpropertyrrrrrrrr__static_attributes____classdictcell__ __classcell__r[r9s@@r$r/r/<s B?B?H@++..-=-=^  4EE   &DD        '''r&r/caa]tRtRtoRt]P !4tV3RlV3RlltRV3RlRllt RV3RlRllt V3R lR lt V3R lR lt V3R lRlt RV3RlRlltRtVtV;t#)SubgraphTemplateiFz A template for subgraph evaluation to be used in autotuning. This class allows creating customized subgraphs that can be appended as choices during the autotuning process, enabling the selection of optimal implementations for complex operations. c <V^8dQhRS[/#)rrr|)r#r9s"r$r%SubgraphTemplate.__annotate__Qs $ $ $r&c (<\SV`VR7R#)zd Initialize a subgraph template. Args: name: The name of this template graph: The FX graph rN)r>r?)rWrr[s&&r$r?SubgraphTemplate.__init__Qs d#r&c<V^8dQhRS[RS[S[,RS[RS[RS[3,RS[RS[S[S[S[.S[P3,3,R,R S[R S[ /#) rrr1r2r4.r3r5Nrvr) r"r!r rrrr6r7rr8r/)r#r9s"r$r%r^s! ! ! &\!  !  S) !  ! C3%*=!>>?$F! !  ! r&c \\V R\\P4 2VVVVVR7#)a! Generate a SubgraphChoiceCaller instance for autotuning. Args: name: The name for this subgraph choice input_nodes: List of input nodes to the subgraph layout: Memory layout information for the output make_fx_graph: Callable that creates the FX graph for this subgraph description: Optional description of this choice input_gen_fns: Optional dict mapping input indices to tensor generators **kwargs: Additional keyword arguments Returns: SubgraphChoiceCaller: A callable object that can be used for autotuning r)rr1r2r3r4r5)r/nextr index_counter)rWrr1r2r4r3r5rvs&&&&&&&,r$generateSubgraphTemplate.generate^s;4$64 0 > >?@A##''   r&cZ<V^8dQhRS[RS[S[RS[3,,RS[S[,RS[S[S[S[3,,RS[RS[3,R,RS[S[S[S[.S[P3,3,R,R S[S[S[S[3,,R,R S[S[ ,/#) rrdecompositions.r1non_tensor_args default_implNr5config_patches_listr) r"r!rrr r6r7rr8r/)r#r9s"r$r%rssssXc3h/0s&\ s d38n- s sCx(4/ sC3%*=!>>?$Fs"$sCx.1D8s " #sr&c  V'g.#\V4\V48wd%\R\V4 R\V4 R24hVfVUu.uFp/NK pp\W$4U U u.uFwrVPW9WV4NK p p p VP WV 4V ^,p .p \W$V4F|wrp^RIHpRV RV/RRllpVPW4pVPV R V 2VV VR V P 2VR 7pTPY4TTnT PT4K~ V #uupiuup p i TdE\PR T P4\R ,R;;,^, uu&Ki;i)a Generate multiple SubgraphChoiceCaller instances for custom op autotuning. This method extends SubgraphTemplate to support custom op decompositions, allowing multiple implementations to compete in autotuning. Args: name: Base name for the choices decompositions: List of decomposition functions to compete in autotuning input_nodes: List of tensor inputs. All tensor arguments must be passed here. non_tensor_args: List of non-tensor kwargs only, one dict per corresponding decomposition. default_impl: Default implementation for layout inference input_gen_fns: Optional dict mapping input indices to tensor generators config_patches_list: Optional list of config patches per decomposition Returns: List of SubgraphChoiceCaller instances for autotuning z>decompositions and non_tensor_args must have same length, got z decompositions and z kwargs)_ShapeEnvGuardErrordecomp decomp_kwargscV^8dQhR\R\R\3,R\\\3,R\/#)rrr.rr)rrr6r")r#s"r$r%ASubgraphTemplate.generate_custom_op_choices..__annotate__sA    c*  $CH~   r&c H^RIHp^RIHpV!4p\P P pVeVP4M\P!4pV;_uu_4V!\P!V3/VBVRR7!V!uuRRR4# +'giR#;i)r)make_fx)select_decomp_tableNsymbolic)decomposition_table tracing_mode) "torch.fx.experimental.proxy_tensorrrNrrrArherror_on_new_guards contextlib nullcontext functoolspartial)rrrrrrrh guard_ctxs$$* r$r4BSubgraphTemplate.generate_custom_op_choices..make_fx_graphsG?&9&;#KK11 !,113#//1 Y"!))&BMB,?%/ YYYs !$B B! rz CustomOp )rr1r2r4r3r5z5Skipping decomposition %s: adds guards during tracinginductorcustom_op_decomp_guard_skips)rCrErb_infer_custom_op_layout_validate_layout_equivalence%torch.fx.experimental.symbolic_shapesr_generate_variant_namerrrinforryrPrI)rWrr r1r r r5r rrrvlayoutsr2choicesrrPrr4 variant_namechoices&&&&&&&& r$generate_custom_op_choices+SubgraphTemplate.generate_custom_op_choicess8I ~ #o"6 6 >*++?O@T?UU\^   &/=">~!2~ ">#&n"F  #G  ( (V= #G   ))$H.058 -@6 1F> R -3 1> 8 66vML  6<.1 +!"/"+FOO+< ="/ '"  & &v =$2F ! NN6 "w6 zY#? v' KOO$%CDID  s D7+D<)EAF  F cV<V^8dQhRS[RS[3,RS[S[S[3,RS[/#)rr.rvrrw)r#r9s"r$r%rs6--sCx(-26sCx.- -r&c aa^RIoVPpV'gV#RV3RlloRPV3Rl\VP 4444pV RV 2#)zLGenerate a descriptive name for a decomposition variant with its parameters.Nc0V^8dQhR\R\/#)rvr)rr")r#s"r$r%=SubgraphTemplate._generate_variant_name..__annotate__s  c c r&c<\V4pSPRRV4pV'd'V^,P4'd RV,pV#)z7Convert a value to a valid Python identifier component.z [^a-zA-Z0-9_]r)r"subisdigit)r1rjres& r$sanitize_value?SubgraphTemplate._generate_variant_name..sanitize_values>AA'a0AQqT\\^^!GHr&rc3B<"TFwrV RS!V4 2xK R#5i)rNr).0kr1r7s& r$ :SubgraphTemplate._generate_variant_name.. s( 3I41qc>!$% &3Is)r6rrsorteditems)rWrrv base_name param_suffixr6r7s&&& @@r$r&'SubgraphTemplate._generate_variant_names_ OO    xx 39&,,.3I  Al^,,r&c:<V^8dQhRS[S[S[3,RR/#)rrvrNr)r#r9s"r$r%rs#$sCx.Tr&c  VP4FFwr#\V\P\34'gK-\ RV R\ V4 R24h R#)z8Validate that kwargs contains only non-tensor arguments.zkwargs['z'] contains tensor zo. Tensor arguments should be in input_nodes, not kwargs. Only scalar/non-tensor parameters should be in kwargs.N)r?rcrr8r rEr)rWrvrvalues&& r$_validate_non_tensor_kwargs,SubgraphTemplate._validate_non_tensor_kwargssV ,,.JC%%,,!788$se#6tE{mDMN)r&cd<V^8dQhRS[RS[S[RS[3,,RS[S[,RR/#)rop_namer .r(rN)r"r!rrr)r#r9s"r$r%rsEXc3h/0f   r&c bV'gR#V^,p\VR,^R7EFwrVVPVPVPVP3VPVPVPVP38wgKg\ RV RW%,P RVP RVP RVP RVP RV^,P RVP RVP RVP RVP R 24h R#) zXEnsure all layouts have consistent stride, device, dtype, and sizes for fair autotuning.N:NN)startzLayout mismatch in custom op 'z': decomposition 'z ' produces (z, z) but 'r~)rBrdtypesizestriderEr)rWrIr r( referencerYr2s&&&& r$r$-SubgraphTemplate._validate_layout_equivalences  AJ "72;a88IA v||V[[&--H    M %4WI>&&4&7&@&@%AB bb R W*1-6678!(()IOO+NbQZQaQaPbbc e9r&c<V^8dQhRS[S[,RS[RS[3,RS[S[S[3,RS[RS[3,R,RS[S[S[S[.S[P3,3,R,RS[ /#) rr1function_decomposition.rvr Nr5r) r!r rrr6r"r7rr8r)r#r9s"r$r%r3s44&\4!)c 24S#X 4 sCx(4/ 4 C3%*=!>>?$F 4 4r&c  ~^RIp^RIHpVPV4VP;_uu_4.p\ V4FwrV'dW9dWY,!V 4p MV P 4p VPPPV 4p V P4pVPPPV4p\P!V VV P4V P4R7p VPV 4K VP !V3/VBpV!V!p\#V\P$4'g\'R\)V4 R24h\+VP,VP.VP0VP34R7uuRRR4# +'giR#;i)zInfer output layout for custom ops using the default implementation when available. Note that the Subgraph assumes custom ops return exactly one tensor output. TODO: Add support for multiple output custom ops. Nr)rMrz#Expected single tensor output, got z:. Multi-output custom ops not yet supported in autotuning.)rrMrNrO)rtorch._inductor.virtualizedrrFrArBrDr*rgoptimization_hintsrFr empty_strided get_dtype get_devicerIrrcr8rErr rrMrdrO)rWr1rSrvr r5rrrJrYrZ fake_tensor raw_shapeconcrete_shape raw_strideconcrete_strideroutputs&&&&&& r$r#(SubgraphTemplate._infer_custom_op_layout3s` 1 ((0 [[[N#K0 Q%7"/"23"7K # I%&WW%5%5%H%H%SN!$!1J&'gg&6&6&I&I*&UO"'"5"5&'!mmo"~~/ #K %%k21 ""#9DVDB(Ffell33$9$v,HOP }}ll\\}} 9[[[s E0F++ F< r)N)NNN)NN)rrrrr itertoolscountrr?rr,r&rFr$r#rrrrs@@r$rrFsgOO%M $ $! ! Fssj--24444r&r).rrrbloggingcollections.abcrtypingrrertorch._inductor.config _inductorrStorch._dynamo.utilsrtorch._inductorr torch._inductor.autotune_processrr r torch._inductor.codegen.commonr torch._inductor.irr r rrrrr$torch._inductor.runtime.benchmarkingrtorch._inductor.utilsrrUrtorch.utils._ordered_setr getLoggerrrr- ChoiceCallerr/rrr&r$rss$ ''( :=:)/!),G'2??G'Ta~ar&