+ jG <Rt^RIt^RIt^RIt^RIt^RIt^RIt^RIHt^RI H t H t ^RI H t Ht^RIt^RIHt^RIHtHtHt^RIHtHtHtHtHtHtHt^RIH t H!t!H"t"^R I#H$t$^R I%H&t&H't'H(t(H)t)H*t*H+t+H,t,^R I-H.t.H/t/H0t0H1t1H2t2H3t3H4t4H5t5H6t6H7t7H8t8H9t9H:t:H;t;Ht>H?t?H@t@HAtA^R IBHCtC^RIDHEtE^RIFHGtG]'d ^RIHHItI^RIJHKtKRRltL] RRl4tMRRltNRRltORRltP!RR]A4tQ!RR]G4tR!R R!]R4tS!R"R#]R4tTR#)$aQ This module implements variable tracking for PyTorch nn.Module instances during Dynamo tracing. It provides specialized handling for different types of nn.Module instances through several key classes: - NNModuleVariable: Handles instance-specific module tracing, specializing on module id() and placing parameters directly on the torch.fx.GraphModule. This creates one graph per module instance. - UnspecializedNNModuleVariable: Provides class-level module tracing, treating nn.Modules like other user-defined objects and passing parameters as inputs to the FX graph. This creates one graph per module class. - UnspecializedBuiltinNNModuleVariable: Specifically handles built-in PyTorch modules (e.g. nn.Linear) with appropriate optimizations. - FSDPManagedNNModuleVariable: Special handling for FSDP-wrapped modules with modified guarding behavior and parameter handling. The module integrates with Dynamo's broader tracing functionality to handle module method calls, parameter access, hooks, and other nn.Module behaviors while maintaining proper scoping and guarding of module state. N)Iterable)contextmanager nullcontext)Any TYPE_CHECKINGSource)graph_break_hints trace_rules variables)handle_observed_exceptionObservedAttributeErrorraise_observed_exceptionraise_type_error unimplementedUnspecializeRestartAnalysis Unsupported) GuardBuilder install_guardmake_dupe_guard)GenerationTracker) AttrSourceConstDictKeySourceDictGetItemSourceFSDPNNModuleSource GetItemSourceNNModuleSourceUnspecializedNNModuleSource)"enumerate_items_with_dict_positionget_custom_getattrget_fake_valueis_lazy_module is_namedtupleis_safe_constantistensoristypennmodule_has_hooksobject_has_getattributeproxy_args_kwargsraise_args_mismatchset_example_valueunpatched_nn_module_callunpatched_nn_module_call_impl)MethodtypestrValueMutationNewVariableTracker)invoke_and_store_as_constant)LazyVariableTracker)UserDefinedObjectVariable)InstructionTranslatorBaseConstantVariablec V^8dQhRRR\PPR\\,R\ \ \3,RR/#)txr4modargskwargsreturnN)torchnnModulelistr0dictstr)formats"b/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/_dynamo/variables/nn_module.py __annotate__rFWsQ77#7 7  7 o% & 7  7cxaa \P!VRR4enRV V3Rllo \W#4wrEVUu.uF pS !V4NK ppVP4UU u/uFwrVS !V 4bK p pp VP WV 4R#R#uupiuup pi \ d\ \ SR.R7R#i;i)aj Fairly coupled helper used by NNModuleVariable and UnspecializedNNModuleVariable. Used to cause lazy module to be initialized (and delete its init hook) before tracing. Especially useful now that 'allowed' modules graph-break on hooks, calling this first ensures there is no hook by the time we trace __call__ and thus no graph-break for lazy allowed modules. _initialize_hookNc0V^8dQhR\R\/#)r8xr=r)rDs"rErF,initialize_lazy_module..__annotate__fs  s s rGcZ<\V4'd\V4!V3RlV4!#\V\4'd,VP 4UUu/uFwrVS!V4bK upp#\V\ \ \34'd\V4!V3RlV44#\V\PP4'EdM\VPS4p\V\P4'Ed\;QJd&RVP4F 'gK RM RM!RVP44'dVPUu.uF;p\V\P 4'dVPP"MTNK= pp\$;QJdRV4F 'dK RM RM !RV44'g\'RV 24h\P(!WSP*VP,R7#V#V#uuppiuupi)c34<"TF pS!V4xK R#5iN.0elemconvert_to_fakes& rE Binitialize_lazy_module..convert_to_fake..hs E14!6!61c34<"TF pS!V4xK R#5irPrQrRs& rErVrWlsCt44rXc3V"TFp\V\P4xK! R#5irP) isinstancer>SymIntrSss& rErVrWos!:9CAJq%,,//s')TFc3B"TFp\V\4xK R#5irP)r[intr]s& rErVrWxsA5az!S115sz*expected all shape entries to be int, got )dtypedevice)r"typer[rBitemsrAtuplesetr>fxProxyr nodeTensoranyshaper\hintallAssertionErroremptyrarb)rKkvfaker^rlrUr9s& rErU/initialize_lazy_module..convert_to_fakefsQAw E1 EFFAt$$:;'')D)$!?1--)DDAeS122AwCCCCAuxx~~..%affb1dELL11cc:9=:ccc:9=:77"&!+A(2!U\\'B'B I!+3A5A333A5AAA,HP!;;ZZ  /Es H"'AH(z0AttributeError during lazy module initializationr;)inspectgetattr_staticr(rd_infer_parametersAttributeErrorr) r9r:r;r< proxy_args proxy_kwargsarg fake_argsrqrr fake_kwargsrUs f&&& @rEinitialize_lazy_modulerWsc#5t<H  :$5T#B 5?@Zc_S)Z @9E9K9K9MN9Mq/!,,9M N   ! !#+ >GI@AN  $HI   sB B6BB98B9c tV^8dQhR\R\RRR\PPR\ /#)r8 module_keysourcer9r4r:r=)rCrr>r?r@r)rDs"rErFrFsB+++ + $+  +  +rGc#V"VPp\P!RRV4pVPP V^4pV^8dV RV 2MTpWCP 3VP V&V^,VPV&RxVP VR# TP Ti;i5i)z9\._(?:modules|parameters|buffers)\[(['\"])([^'\"\]]+)\1\]z.\2@N)nameresub num_callsget __class__nn_module_stack)rrr9r:fully_qualified_namers&&&& rErecord_nn_module_stackrs";;66D    !5q9I09A J )N:&-6] )*   z *B  z *sAB)4BB)B&&B)cjV^8dQhR\R,R\PPRR/#)r8rNr:r=)rr>r?r@)rDs"rErFrFs0&& TM& %& &rGc V'Ed>RVP9d\VPR,4'dVPR,p\VR4p\V4\P Jd7\VR4p\ VP\P44R#\V\P4'd\WW24R#\ VP\P44R#\ VP\P!\PRR744R#R#)forward__func__attrN)__dict__callablerrctypes MethodTyper make_guardr CLOSURE_MATCHr[ functoolspartial.guard_to_detect_forward_partial_monkeypatchingNOT_PRESENT_IN_GENERIC_DICT)rr:fwdforward_sources&& rE&guard_to_detect_forward_monkeypatchingrs v  $#,,y2I)J)J,,y)C' :NCyE,,,!+NJ!Gn77 8R8RSTC!2!233>n77 8R8RST !!%%$@@y !rGc V^8dQhR\R\PPR\R\P \ ,RR/#)r8 module_sourcer:partial_source partial_objr=N)rr>r?r@rrr)rDs"rErFrFsM         ""3'    rGc Z\VP\P44\ VR4p\ VP \P4'd\WWCP 4M(\VP\P44\ VR4p\VP\P44\VP4Fwrg\W\WV4V4K \ VR4pVP f+\VP\P"44R#\VP\P$44VP P'4Fwr\W\)W4V 4K R#)funcr;keywordsN)rrr TYPE_MATCHrr[rrrrrSEQUENCE_LENGTH enumerater;%guard_to_detect_forward_partial_valuerr NONE_MATCHDICT_KEYS_MATCHrdr) rr:rr func_source args_sourceir|keywords_sourcekeyvalues &&&& rErrs? .++L,C,CDE^V4K+""I$5$5666  -=-=  k,,\-G-GHI^V4K+(()E)EFGK,,--  k =s . !r?r@r)rDs"rErFrFsIFFF FF  F  FrGc>W1Jd@\W 4p\TPT;'g\P44R#\ V\ P4'd\WW#4R#\V4\PJd+\VP\P44R#\V4'd+\VP\P44R#\VP\P44R#rP)rrrrID_MATCHr[rrrrcr FunctionTyperr#CONSTANT_MATCH)rr:rr dupe_guards&&&& rErrs  |$\A l--j.Q.QLlR?lt%V3R@lRAlt&V3RBlRClt'V3RDlRElt(V3RFlRGlt)V3RHlRIlt*V3RJlRKlt+V3RLlRMlt,RN]-!]4RO]-!]4RP]-!] 4RQ]-!]!4RR]-!]"4RS]-!]#4RT]-!]$4RU]-!]%4RV]-!]&4RW]-!]'4RX]-!](4RY]-!])4RZ]-!]*4R[]-!]+4R\]-!],4/t.RcV3R]lV3R^lllt/V3R_lR`lt0Rat1Vt2V;t3#)dNNModuleVariablei c ^<V^8dQhRS[RS[RS[PPRS[RR/#)r8 module_typerrr<r=N)rcrCr>r?r@r)rD __classdict__s"rErFNNModuleVariable.__annotate__s= 2 2 2-0 29> 2TW 2  2rGc <\SV`!R/VBWnW nW0nVP VnVP VnR#)NrQ)super__init__rrrrnn_module_stack_source)selfrrrr<rs&&&&,rErNNModuleVariable.__init__s> "6"&$ #kk &*kk#rGc <V^8dQhRS[/#r8r=r)rDrs"rErFr FrGclVP;'g VPpV'g \R4hV#z'nn_module_stack_source must not be Nonerrrorress& rEget_nn_module_stack_source+NNModuleVariable.get_nn_module_stack_source -))88T[[ !JK K rGc$<V^8dQhRS[RR/#r8rr=Nr)rDrs"rErFr&---D-rGcWnR#rPrrrs&&rEset_nn_module_stack_source+NNModuleVariable.set_nn_module_stack_source&&,#rGc <V^8dQhRS[/#r)rc)rDrs"rErFr)s  T rGcVP#rP)rrs&rE python_typeNNModuleVariable.python_type)srGc2<V^8dQhRRRS[R,/#r8r9r4r=N)r`)rDrs"rErFr,s <<4<tr? ModuleDictrdr0rregister_attr_or_modulerrrappend ModuleList ParameterList Sequentialror.r)rr9baseresultrrname_varidxs&& rEunpack_var_sequence$NNModuleVariable.unpack_var_sequenceMsWyy&&t7(* dEHH// 0 0 $ *00: 11OO)- T*JK 2  h'!-M 588&&(>(>@S@ST  !I'RV-Y %T?KC MM 11OO)- S*IJ 2 + rGc(<V^8dQhRRRS[RR/#)r8r9r4rr=r6)rC)rDrs"rErFrps%11-1581 1rGc VPPVP4p\W24p\ VP P \P!\PVR744\P!W4#)r) rrrhasattrrrrrrrHASATTRr0r)rr9rr:rs&&& rEcall_obj_hasattr!NNModuleVariable.call_obj_hasattrpsgii%%doo6# KK " "9#4#4\5I5IPT#U V $$R00rGc$<V^8dQhRRRS[/#r)r)rDrs"rErFrzs//9/d/rGcfVPPVP4p\VRR4#)trainingF)rrrgetattrrs&& rE is_trainingNNModuleVariable.is_trainingzs)ii%%doo6sJ..rGc"<V^8dQhRRRR/#rrQ)rDrs"rErFr~s**+F*4*rGc VPPVP4p\P!V4VP P R8wd \P!\V44\h)zIRestart analysis treating this module as an UnspecializedNNModuleVariabler) rrrrtagf_codeco_namemark_class_dynamicrcrrs&& rEconvert_to_unspecialized)NNModuleVariable.convert_to_unspecialized~sTii%%doo6c" 99   *  0 0c ;))rGc l<V^8dQhRS[PPRRRS[RS[RS[R,/#)r8rr9r4r obj_sourcer=N)r>r?r@rCrr0)rDrs"rErFrsJ= = hhoo=  (=  =  = 4  = rGc \V4'd\P!\V4R4pV'd \ VR4MRp\ P !VVVR7PV\ PPV4./4#\!VR R 7pVfR#\#V\$P&4'g'\R RV RV 2R R.\POR 7R\ VR4/p\ P !Wp3/VBPV\(P*!W#4./4# \d\T4L\d*\RRT RT 2RR.\POR 7Li;i)zDCheck for a __getattr__ and handle it specially if it is implemented__getattribute__Nrz5Custom __getattribute__ in nn.Module attribute accesstp_getattro_impl  zXDynamo could not trace through the custom `__getattribute__` method on this `nn.Module`.z[Simplify your `__getattribute__` implementation, or replace it with a targeted `@property`.gb_typecontext explanationhintsT)ignore_nn_module_getattrz6torch.nn.Module with a non-function custom __getattr__zDynamo detected a nn.Module object with a custom `__getattr__` method, but this method is not a standard Python function (e.g., it might be implemented in C/C++). Dynamo cannot currently trace into such non-standard `__getattr__` methods.zAvoid using objects with non-standard __getattr__ methods within the compiled region. If possible, implement __getattr__ as a standard Python function.r __getattr__)r'rvrwrcrr UserMethodVariable call_functionr6rr r rrr SUPPORTABLErr[rrr0r) rrr9rr*getattribute_fn new_source getattr_fnrs &&&&& rE_custom_getattr_fallback)NNModuleVariable._custom_getattr_fallbacks #4 ( (%44T$ZASTO>H :'9:d   33#% -Y%?%?%F%Ft%L$MrR S((tL  *e&8&899 P+D64&9-A'22  $Z MBC++JHHVV &&r012  Q* .)"- S/vQtf=!EE+66  s AD55F F'F?Fc*<V^8dQhRRRS[RS[/#r8r9r4rr=rCr0)rDrs"rErFrs*u2u2-u258u2 u2rGc  <VP;'d\VPV4pVPPVP4p\ P VR4pRp\4p\P!VP4F,pVPVPP44K. VP'g&\RRV RV 2R.\P OR7VR8XdVP#V4#Rp W%9d WR,p MRV9d%W%R,9dW'9dVR,V,p McR V9dW%R ,9dVR ,V,p M>R V9dW%R ,9dVR ,V,p M\P$!WB4p R pVR8Xd\1VPV4VR8Xd+V'g#\2P4!WPVR7#V'dh\2P4!W\7V44p \9V \:\<34'd*V P?\VPA4V44V #\CV \D4'drVP'd-\\VPR4V4p\VR4p\FPH!V PJVR7PMW./4#\CV \N4'dA\FPP!V PR\FPT!\-V44VR7#\CV \V4'd(\FPH!V PYV4VR7#\CV \ZP\4'd\FPP!WVR7#\_V 4'g\aV 4'd!\2P4!W\7V44#\RR\cV4 RV R\cV 4 2R\cV 4 R2RV R\cV 4 R\cV4 R2R.\PdOR7\fS V`W4# \&dWTP)YAY PR 7p T eT u#\+\&TR \-T4P. RT R 2.R7EL%i;i)rTzgetattr with no sourcer-r.zDynamo does not know how to access an attribute on an `nn.Module` instance that lacks a source. This is usually an internal error in Dynamo.r/N_modules _parameters_buffersF)rr9rr*'' object has no attribute 'rurrrfgetz$Unsupported nn.Module attribute typeznn.Module subclass: z, name: z, attribute type: z>Dynamo does not support tracing nn.Module attributes of type ``zRefactor your code so that `z ` (type `z`) is not an attribute of `zqCurrently supported attribute types are methods, classmethods, staticmethods, properties, constants, and tensors.)5rrrrrrr,rfrvgetmrorupdaterkeysrr DYNAMO_BUG get_dict_vtrwryr<rrc__name__rr0rrr[rUnspecializedNNModuleVariablerrr%propertyr UserFunctionVariablerGr7 classmethodr6rr3 staticmethod__get__rrr#r$r.r8rtp_getattro_impl) rr9rrr base_dict object_memberall_class_attribute_namesrKsubobjroutrs &&& rErU!NNModuleVariable.tp_getattro_implsG>>DKK!>yy&&t7++D*=  $'E!/A % , ,QZZ__-> ?0{{{ 0+D64&976)445   : ##B' '  _F ) #*--5z*40F i 'Dm4L,L}-d3F 9 $:1F)Fz*40F  //; %  9  24;; E ; }"((^^FK K !''N64JKC# 02OPQQ..t>>@$GJfh'';;;' 4;; (LdSF'7F 55KK! -Hb12 ,, 33OO77T C!  -- 55NN4(! 2 233 33FPP!&))Xf-=-=&,,R9OPPB274=/$Oabijpbqars"`ahioap`qqr s6tfIgfoEVVqryz~rrAABC>+66  w'11Q" 664KK7%!M("d4j1122MdVSTUV sQ66.S&-SScV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r8r9r4r;r<r=rAr0rBrC)rDrs"rErFr>sHnn 'n?#nS/)* n  nrGcVPPVP4p\VPVP 4W4;_uu_4\ V4p\ V\PP4'Ed/VPP\PPPJd\V4'dVPV4V'd \R4hV'd\VRR\!V4 R24VwpVP"P%4FmwrxVP'VPP)VVPV\+\-VP.V44R7V./4VP14pKo VuuRRR4#V'd,VP2eVP2Vn\7WW#4VPP94'dVP:P=R4'dVP:R8wdp\VRRR7'dVPV4^R IH p V !VVPPB!R VP.\EW#4O5!R 7uuRRR4#\ V\PFPH4'd$VPp \-VP.R 4p M"VPJp \-VP.R 4p \MV \NPP4'd-V PRp \-V R4p V.\UV4,pM7\MV \NPV4'g\R\YV 4 24hVP[\\P^!WR7VVRR7uuRRR4# +'giR#;i)z3Expected lazy sequential isn't a valid combination?ztorch.nn.Module.Sequentialz0 kwargs kwargsrNztorch.nn.utils.parametrizeT)check_forward_hookscheck_backward_hooks wrap_fx_proxy call_moduler9proxyr _call_implrExpected FunctionType, got allow_nested_graph_breaks)z torch.nn.z torch.ao.)0rrrrrr!r[r>r?rrrr&r'ror)lenrBrdr7r rrrpop cls_to_becomerris_root_tracer __module__ startswithbuilderrd create_proxyr(rg GraphModulerhr%rrrrArrcinline_user_function_returnr rQ) rr9r;r<r:is_lazyr| child_namerrdfn fn_sources &&&& rEr7NNModuleVariable.call_function>s" ii%%doo6 # OOT<<>  %S)G3 3 344MM))UXX-@-@-H-HH%c**11"5(M'4"v;-w/  *-,,*<*<*>&J$$ 99" OO*T[[*2U#V :  &&(C+?Q  T$$0'*'8'8D$ 'r= ((**NN--.HIINN&BB%T11"52$))00%+48Q  bc588#7#788B *4;; BIB *4;; EI"e..//B *9j AI 6DJ.D!"e&8&899,/J4PR8*-UVV55222H.2 6C    s5BO0OB*O8OA O BO,DO O# c&<V^8dQhRRRRRR/#)r8r9r4rr0r=rQ)rDrs"rErFrs.` ` '` `   ` rGc ^RIHp^RIHpVPP VP 4p\PPP\PPP\PPP\PPP\PPP3p\V4PV9dVP!4'd+\#VP%4\&\(34'g\+RRV RV 2RV R2R.R 7VPP,p\#V\.P04'g\3R \V4 24h\5\5VP6R 4R 4pVP9\:P<!WxR 7W./RR7#\#W#4'EdCVPP>'Ed.p \A\C\EV444VP%4,p \GWRP%4,4FYwrW,p \I\KVP6V 44pV PMVPPOV V VR 74K[ WRP%4,pVPPOVV R2\I\KVP6VP%444R 7pV#VPQV4^p\#W$4'dVPSVP4pMr?r rr  ParameterDictr rrcis_python_constantr[as_python_constantrCr`rrrrrorrrur rQexportrArangerlrrrr r r' evaluate_expr)rr9rr~rmodulebuiltin_supportedrxsrcrrKrrrq new_modulenew_module_variable key_values&&& rEmp_subscript_impl"NNModuleVariable.mp_subscript_implsL )+((9 HH   + + HH   + + HH " " . . HH " " . . HH   + +   < # #+< <&&((s557#sDDT1$q>!==CHDo!qVW ##,,Bb%"4"455$'B48*%MNNZ ]CZPC11..r> *. 2  c ) )yyE#f+./0F0F0HI#,V4J4J4L-M#NKC A(t{{A)FGCMM 99"#&:$O$$:$:$<= &(ii&G&Gf/0)%dkk33I3I3KL'H'#+*--b1 c + +))"))4I  # # % %..0I H-dV1SE:5   "yy00  OO !- Y"GH 1  rGc$<V^8dQhRRRS[/#rr)rDrs"rErFrs  :  rGcP^RIHpV!VPV4\4R7#)r})ListIteratorVariable mutation_type)rrr/)rr9rs&& rE tp_iter_implNNModuleVariable.tp_iter_impls&*#  $ $R (8H8J  rGc,<V^8dQhRS[RS[RS[/#)r8rrr=)rrC)rDrs"rErFrs"F#&rGcVPR4pV^,R8XdV#\V4^8dVP^4p\W4pK.V#).r)splitrlrmr)rr name_splitrKs&& rE _gen_sourceNNModuleVariable._gen_sourcesIZZ_ a=B M*o!q!A*F rGc 6<V^8dQhRRRS[RS[RS[RS[/#r8r9r4rr;r<r=)rCrr0)rDrs"rErFr!s5  - 58 @C MP  rGc BVPPVP4pVPPRVPR/4p\ VP V4\ W44wrx^RIHp V !VVPPRW&.VO5VR7R7#)get_attrrc call_method)r;r<rfrQ) rrrrsr*rir(rrrd) rr9rr;r<r mod_proxyrzr{rds &&&&& rE_generic_call_method_helper,NNModuleVariable._generic_call_method_helper!s ((9II**:tBO )..&1#4T#B *))((t*Bz*B<)  rGc 4<V^8dQhRRRS[RS[RS[RR/#)r8r9r4rr;r<r=NrCr)rDrs"rErFr3s5  - 58 @C MP   rGc \;QJd>R\P!W4P444F 'dK RM3 RM/!R\P!W4P4444'gJVPP VP 4p\RRV RV RV RV 2RV RV R2.R 7R #R #) c3@"TFqP4xK R#5irP)rrSrKs& rErVANNModuleVariable._assert_all_args_kwargs_const..6s ,Rq " ",RFTz&non-const argument in nn.Module method call_method: r.z(Dynamo does not support calling method `z` of ``nn.Module`` z with non-constant arguments.r/N)rn itertoolschainvaluesrrrr)rr9rr;r<rs&&&&& rE_assert_all_args_kwargs_const.NNModuleVariable._assert_all_args_kwargs_const3ss ,5OOD--/,R sss ,5OOD--/,R   YY,,T__=F @'vQtfAdV1VHE& 3F8;XZ    rGcR<V^8dQhRRRS[RS[RS[RS[RS[S[S[3,/#)r8r9r4rr;r<namesr=)rCrrB)rDrs"rErFrBsO11 '11 1  1  1 c3h1rGc VPWW44VPPVP4p\ Wb4p\ P !V4P!VUu.uFqP4NK up/VP4U U u/uFwrWP4bK up p Bp V P4V Pp VU u/uF qW,bK up #uupiuup p iuup irP) rrrrrrv signaturebindrrdapply_defaults arguments) rr9rr;r<rrrxrKrqrr bound_argss &&&&&* rE _get_kwargsNNModuleVariable._get_kwargsBs **2TB((9 V "&&r*///34t!##%t4 5;\\^D^TQq&&((^D  !!#)) */0%Q:= %00 5D1s'C*C/C5cN<V^8dQhRRRS[S[S[S[3,,RR/#)r8r9r4rdr=zvariables.ListIteratorVariable)rrer)rDrs"rErFrUs7 X X- X6>uS#X6O X ) XrGc.pVF`wrEVPVPPVVPV\ VP VP V44R74Kb \P!V\4R7#)rr) r rr rrrrr rr/)rr9rdrrrs&&& rE _wrap_valuesNNModuleVariable._wrap_valuesUsw)+!LD MM 11OO)$*:*:4;;*MN 2 "--fDTDVWWrGc.<V^8dQhRRRS[RS[RR/#)r8r9r4robjr=zvariables.TupleVariabler)rDrs"rErFrds-   -  58  ?B  "  rGc\P!\P!W4VPP VVP V\VPVPV44R7.4#)r) r TupleVariabler0rrr rrrr)rr9rrs&&&&rE _named_embedNNModuleVariable._named_embeddsg&&%%b/ 11OO)$*:*:4;;*MN 2   rGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrssB44 '4?#4S/)* 4  4rGc&VPWV4#rP)r7)rr9r;r<s&&&&rErhNNModuleVariable._call_implss!!"F33rGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFr}sH Q Q ' Q?# QS/)* Q  QrGc VPPVP4p\VPVP 4W4;_uu_4VP VRW#4uuRRR4# +'giR#;i)rN)rrrrrrrr9r;r<rs&&&& rErNNModuleVariable.forward}sf((9 # OOT<<>  33B 4P    s A11 B cV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrsB  ' ?# S/)*    rGc fVPPVP4pV^,P4'g\ VRV R24\ V^,\ P4'g\ VRV R24V^,PV^,P4,p\ V\4'dV#VPpVPPV4pVPPWvV\\VPV44R7#)r}z``nn.Module`` zB's call method _get_item_by_idx requires a constant index argumentzB's call method _get_item_by_idx requires a tuple as first argumentr)rrrrrr[r rrdrrOr rrr)rr9r;r<rmod_varrrs&&&& rE_get_item_by_idx!NNModuleVariable._get_item_by_idxs ((9Aw))++  (jk $q'9#:#:;;  (jk q'--Q : : <= g< = =N  ((-yy00 ^M$++s4S%T1  rGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrHXX 'X?#XS/)* X  XrGc VPPVP4pVPPP \ VP R44V'g V'd'\VRR\V4 R\V4 R24.pVP4F&wrgVPVPWV44K( \P!V\4R7#)rBnamed_children0 args and 0 kwargs args and r`r)rrrguard_on_key_orderaddrrr)rlrr rr rr/rr9r;r<rrnrs&&&& rErNNModuleVariable.named_childrens ((9 $$((DKK)LM 6  %t9+ZF }G<   )+..0IA MM$++B6: ;1--fDTDVWWrGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrsH X X ' X?# XS/)* X  XrGcVPPVP4pVPPP \ VP R44VPVRW#RR4p.pVP!R/VBF&wrxVPVPWV44K( \P!V\4R7#)rCnamed_parametersprefixrecurserrQ)rrrrrrrrrr rr rr/) rr9r;r<rkwrrparams &&&& rEr!NNModuleVariable.named_parameterss ((9 $$((DKK)OP   b"4dHi X(*//5"5HA MM$++B59 :6--fDTDVWWrGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrHXX 'X?#XS/)* X  XrGc VPPVP4pVPPP \ VP R44VPVRW#RRR4p.pVP!R/VBF&wrxVPVPWV44K( \P!V\4R7#)rD named_buffersrrremove_duplicaterrQ)rrrrrrrrrr rr rr/) rr9r;r<rrrrbuffers &&&& rErNNModuleVariable.named_bufferss ((9 $$((DKK)LM    xDV )+--33IA MM$++B6: ;4--fDTDVWWrGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrrrGc VPPVP4pVPPP \ VP R44VPVRW#RRR4p.pVP!R/VBF&wrxVPVPWV44K( \P!V\4R7#)rB named_modulesmemorrrrQ)rrrrrrrrrr rr rr/) rr9r;r<rrrrrs &&&& rErNNModuleVariable.named_moduless ((9 $$((DKK)LM    vxAS )+--33IA MM$++B6: ;4--fDTDVWWrGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrsB>> '>?#>S/)* >  >rGc hVPPVP4pVPPP \ VP R44V'g V'd'\VRR\V4 R\V4 R24VPWP44#)rBchildrenrrr`) rrrrrrrr)rlrrrs&&&& rErNNModuleVariable.childrens ((9 $$((DKK)LM 6 %t9+ZF }G<     %:%:%<==rGcV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFrsB== '=?#=S/)* =  =rGcVPPVP4pVPPP \ VP R44VPWP44#)rB) rrrrrrrrrrs&&&& rEmodulesNNModuleVariable.modulessY ((9 $$((DKK)LM  %9%9%;<R \RPT!W4PW444F 'dK RM3 RM/!R \RPT!W4PW4444'dVPYWW44#\ZS V`W\IV4V4#)r}r5_check_input_dimT__result__iter__ __contains___get_abs_string_index _conv_forward_output_paddingrrc3@"TFqP4xK R#5irP) is_tensorrs& rErV/NNModuleVariable.call_method..sR+QaKKMM+QrF)/rr6rrrr is_torch_inline_allowedrvgetfilerrrrrNr1rr[r>r?r rrr0rrrBrconv_ConvNd_ConvTransposeNdrrrrur rQrA tp_methodsrrrnrrrrrr) rr9rr;r<rr6rrx const_nameryrs &&&&&& rErNNModuleVariable.call_methodas /((9 % %+*M*M OOF,,== >+ + $**40 0 &B",,556avWEJ/ &Q Q : $$R( ( N "6EHH$7$79O9O#PQQQ**,,"((G..0FOOC  + +6588#3#3#8#8#@#@AAO+6588#3#3#8#8#I#IJJ--&//B":dkk4#@*MI11..rDd#   '((111))224899R9??4+QRR9??4+QRRR33BdK K7&rdVD DrGc"<V^8dQhRRRR/#)r8r9r4r=r0rQ)rDrs"rErFrs66!<6AR6rGc VPPVP4p\P!V\ V44#)zrrrr)F)4rNrp __qualname____firstlineno__r0_nonvar_fieldsrrrrrrrrrrrrr'r<rUr7rrrSrrrrrrrhrrrrrrrrrrrKrrdr-r"rr'__static_attributes____classdictcell__ __classcell__rrs@@rErr sx  ' ' N 2 2 --  << 2 244!!F11//**= = ~u2u2nnn`` ` D    $  11& X X   44 Q Q  4XX( X XXX XX >>"== D D A AXX(55 XX( fZ(fZ06'?F#34&0F#34 . .F8$6'?fZ(6'?t &.J$?E?EB666rGrcNaa]tRtRto0Rm]P mtV3RlV3RlltV3RlRltV3RlRltV3RlR lt ] ] PV3R lR l44t V3R lV3R lltV3RlRltV3RlV3RlltV3RlRltV3RlV3RlltV3RlRltRtVtV;t#)rOicR<V^8dQhRS[PPRS[RR/#r8rr<r=Nr>r?r@r)rDrs"rErF*UnspecializedNNModuleVariable.__annotate__s*22ehhoo222rGc <RV9d2\VR,RR4p\V4VJd\V4VR&\SV` !RRV/VBRVnVP VnR#) value_typernNrFrQ)rrcrris_state_mutatedrr)rrr<lazy_value_to_becomers&&, rEr&UnspecializedNNModuleVariable.__init__sh 6 !#*6,+?RV#W E{22 (,E{|$ /u// %'+kk#rGc&<V^8dQhRS[RS[/#r8 attr_sourcer=r)rDrs"rErFr46rGcV#rPrQrr<s&&rE _wrap_source*UnspecializedNNModuleVariable._wrap_sourcerGc <V^8dQhRS[/#rr)rDrs"rErFr4rrGclVP;'g VPpV'g \R4hV#rrrs& rEr8UnspecializedNNModuleVariable.get_nn_module_stack_sourcerrGc$<V^8dQhRS[RR/#rr)rDrs"rErFr4rrGcWnR#rPrrs&&rEr8UnspecializedNNModuleVariable.set_nn_module_stack_sourcerrGc0<V^8dQhRS[S[,/#r)rfr`)rDrs"rErFr4s   3s8  rGc\PPP\PPP\PPP 0p\PPP P4Uu0uF4p\VR4'gKW9gK\VP4kK6 up#uupi)__code__) r>r?r@ __setattr__r __delattr__rrrrrK) supportedrKs rE_nn_module_method_ids3UnspecializedNNModuleVariable._nn_module_method_idss HHOO ' ' HHOO $ $ HHOO ' ' XX__--446 6q*% *+*< BqzzN6   s C9CCc4<V^8dQhRRRS[S[,/#rr)rDrs"rErFr4s"//-/ o /rGc<\P!VPR4pT\ P PP\ P PP\ P PP39d8TP\P!Y4T./4PT4#\ST`9T4# \d p\ThRp?ii;i)rN)rvrwr6ryNotImplementedErrorr>r?r rr rrur0rrr)rr9rxers&& rEr1UnspecializedNNModuleVariable.unpack_var_sequences -''DB  HH   ( ( HH " " + + HH   ( (  11%%b-  "!"%  &w*2..' -%1 , -s!C C1%C,,C1cV<V^8dQhRRRS[S[,RS[S[S[3,RS[/#r]r^)rDrs"rErFr4sH\\ '\?#\S/)* \  \rGc Vaaa a SPp\V4'd-VPeVPSn\ SWBV4\ V\ PP4'gRp\SPV4pMRp\SPV4p\VP\P4'Edn\VP\P4'EdCVPP\ JEd$VPP\"JEdSP%SR4'g\&P(!VR4p\ V\P*4'dSP,o RV3Rllo \.;QJd V V3RlR 4F 'gK RM RM!V V3RlR 44;'g=\.;QJd V V 3R lR 4F 'gK RM RM!V V 3R lR 44pV'gRpSPP0pSP2'dSP5SV4p MRp \7SP2V4SP2'd/\9\;\=V44SP?4SV4M \A4p T ;_uu_4\ V\P*\ PBPD34'gH\FPH!SWiRR 7p V PKSS.\MV4,V4uuRRR4#\NPP!WiR 7PKSS.\MV4,V4uuRRR4# +'giR#;i)N__call__rhrc<V^8dQhR\R\R\/#)r8rrr=)r0rCr`)rDs"rErFAUnspecializedNNModuleVariable.call_function..__annotate__%s!$$$$$rGc<VPSV4p\VR4'dVP4MTpVP4#)realize)rUrr\rl)rrvtr9s&& rE_hooks_dict_lenDUnspecializedNNModuleVariable.call_function.._hooks_dict_len%s9--b$7B)0Y)?)?RB668OrGc36<"TFpS!SV4xK R#5irPrQ)rSrr^rs& rErV>UnspecializedNNModuleVariable.call_function..*s& !$D$//!TFc36<"TFpS!SV4xK R#5irPrQ)rSrr^ globals_vts& rErVra4s&!$J55!rbrr\r)_backward_hooks_backward_pre_hooks_forward_hooks_forward_hooks_with_kwargs_forward_pre_hooks_forward_pre_hooks_with_kwargs)_global_backward_pre_hooks_global_backward_hooks_global_forward_hooks_global_forward_pre_hooks))rr!rnr6rr[r>rgrtrr%rXrrrhrr+r,has_key_in_generic_dictrvrwrnn_modules_globals_vtrkrrget_source_by_walking_mrorrrCrrrjitScriptFunctionr0rr7rAr rQ)rr9r;r<r:rrxforward_method has_hooksrctxfn_vtr^rdsff&& @@rEr7+UnspecializedNNModuleVariable.call_functions jj #    ,"%"3"3 "2s& 9#uxx3344D$/BD$/B 3<<!1!1 2 2s~~u'7'788 %%)AA''+HH00Y??$33CCN.%*<*<==55 $$  C ! CCC !  S!SSS!(!$D00B ;;;33B=FF.t{{C@{{{ #BsG //1   Sb5#5#5uyy7O7O"PQQ'--b"TR**2vT /BFKS!55bHVVd+VSSSsA7N8N N( c \<V^8dQhRRRS[RS[S[,RS[S[S[3,RS[/#r)rCrAr0rB)rDrs"rErFr4`sZ\A\A '\A\A?# \A S/)* \A  \ArGc <VR9dy\VPV4pVP'dVPW4pMRp\P !WVRR7pVP W.\V4,V4#VPW4'Eg\P!\VP4V4p\V\4'dQ\!VPW4R4p\P !WP"VRR7pVP WV4#\%VR4'de\'VP(4VP+49d=\-RRV RV RV RV 2R V R VP 2R .\.P0OR 7VR 8XEd)RVPP29Ed V^,P54p V^,p V P74'd-V P94\:P<P>Jg"WPP2R,9d RVn MWPP2R,9d RVn MX\V \BPD\BPF34'g"WPP2R,9dRVn V\:P<PHPJJd(\V^,\BPL4'g)V\:P<PHPNJdU\P !V\:P<PHPN4pVP WV^,.V4#\PS V`W\V4V4# \dRpELi;i)rhNTrerrKz,UnspecializedNNModuleVariable missing methodrr.z'Dynamo does not support tracing method z of nn.Module z@Dynamo does not really define unspecialized nn.Module very well.r/rLrCrDrB)rhr)*rr6rrrr0rr7rArprvrwrcrryr[rSrrrrrKrOrr DIFFICULTrrrrr>r? Parameterr7r rrOr@rLDeletedVariablerMrr) rr9rr;r<rxrrxmethod attr_namerrs &&&&& rEr)UnspecializedNNModuleVariable.call_method`s  7 7$/B{{{77A#))"NE&&r6DJ+>G G++B55  //TZZ0@$G&,//#222<V^8dQhRRRS[RS[RS[R,/#)r8r9r4fieldname_vtr=Nr@)rDrs"rErFr4s2-69DS 4 rGcVPW4p\V\P4'd VPp\V\P 4'dVP V4#R#rP)rUr[r UserDefinedDictVariable_base_vtConstDictVariablemaybe_getitem_const)rr9rrdict_vts&&&& rEgetattr_helper,UnspecializedNNModuleVariable.getattr_helpersY''2 gy@@ A A&&G gy:: ; ;..w7 7rGc*<V^8dQhRRRS[RS[/#r?r@)rDrs"rErFr4s*_2_2-_258_2 _2rGcT<aaaSPPPV4'd[VR 9dTVP'dBSPPP V4'd\ RRV RV R2R.R7VR 9Ed SPPP W4'g\VPV4p\V\4'd\V4^8XdVP'd\VPV4MRpV'd)\VP\ P"44\V\$P&4'd\(P*M\(P,pV!/VR7#VP'dVR9dSPPP W4'g\VPV4p\VPV4o\SP\ P.44SPP0P3S4R VV3R llo\V3R l\5V444p\(P6!VSR7#\8S V`uSV4#)rz,getattr() on nn.Module with pending mutationzgetattr(z, )zPIntentionally graph breaking on getattr() on a nn.Module with a pending mutationr/Nrc jV^8dQhR\R\R\R\\\3,/#)r8rrqrrr=)r`rrer0)rDs"rErFDUnspecializedNNModuleVariable.tp_getattro_impl..__annotate__s6 " " " "#& "78 "rGc<\P!SV4p\SV4p\SV4p\P !W%SR7pW63#))r9)r0rrrr2r) rrqrrr source_key source_valuerhooks_dict_sourcer9s &&& rEbuild_key_valueGUnspecializedNNModuleVariable.tp_getattro_impl..build_key_valuesM&++B200A1E 01BJO +221rJz!rGc3<<"TFwrpS!WV4xK R#5irPrQ)rSrrqrrrs& rErVAUnspecializedNNModuleVariable.tp_getattro_impl..!s&MGA! a((Ms)rrrrrr)rfrgrirhrkrj)rkrjrirh)r side_effectsis_attribute_mutationr7has_pending_mutationrhas_pending_mutation_of_attrrrr[rBrlrrrrrEMPTY_NN_MODULE_HOOKS_DICT collections OrderedDictr OrderedDictVariablerrrrrNNModuleHooksDictVariablerrU) rr9r hooks_dict hooks_source hooks_vt_clsrrrrs &f& @@rErU.UnspecializedNNModuleVariable.tp_getattro_impls& II " " 8 8 > >%%% &&;;DAA F"4&4&2*     99))FFtRR$TZZ6 j$//C Oq4H9= 4;;5$!$%(33 , G G&j+2I2IJJ"55&88! (<@@ KKKII**GGSS T2J *4;; =  +66|7S7ST U II ( ( , ,-> ? " "A*MF 66vFWX Xw'D11rGc*<V^8dQhRRRS[RS[/#r?r@)rDrs"rErFr4)s%-58 rGc  R^RIHpVPW4pVPVRV4pVfVPVRV4pVfVPVRV4pVf8\ \ VR\ VP4P RV R2.R7Vf\RV R24hV#) z Dynamo tracing of nn.Module __getattr__ can be expensive if the model has deep submodule hierarchy. Since the __getattr__ is stable, we can directly look into the underlying datastructures. This saves a lot of compilation time. )SourcelessBuilderrCrBrDrErFruz;manually_trace_nn_module_getattr failed to find attribute ') rrrrrrryrcrrNro)rr9rrrrZs&&& rE manually_trace_nn_module_getattr>UnspecializedNNModuleVariable.manually_trace_nn_module_getattr)s /#**24!!"mW= ;%%b*g>C ;%%b*g>C ; $TZZ(1122MdVSTU  ; MdVSTU  rG)r7rr6>r6r7r)rNrpr)r*r3r+rr@rrrSrcacherOrr7rrrUrr,r-r.r/s@@rErOrOs # 1 1 N22( --__    //4\\|\A\A|_2_2BrGrOc6a]tRtRtoRtV3RlRltRtVtR#)$UnspecializedBuiltinNNModuleVariableiIz_ Differentiates between builtin nn modules (e.g. torch.nn.Linear) and user defined nn modules. c&<V^8dQhRS[RS[/#r;r)rDrs"rErF1UnspecializedBuiltinNNModuleVariable.__annotate__Nr=rGcV#rPrQr?s&&rEr@1UnspecializedBuiltinNNModuleVariable._wrap_sourceNrBrGrQN)rNrpr)r*__doc__r@r,r-)rs@rErrIsrGrcTaa]tRtRtoRtV3RlV3RlltV3RlRltRtVtV;t #)FSDPManagedNNModuleVariableiSa Tracing behavior: trace into submodules and treat them as Unspecialized, do not register parameters to the top-level, treat them as function inputs. Guards behavior: if 'skip_fsdp_guards', many guards that would be installed by a vanilla UnspecializedNNModuleVariable are simply dropped, on the basis that a user wrapping their model in FSDP(model) is already opting into a requirement to not modify internal model state, which would already break FSDP without compilation. cR<V^8dQhRS[PPRS[RR/#r2r3)rDrs"rErF(FSDPManagedNNModuleVariable.__annotate___s*ehhoorGc x<VPR4pVf \R4h\SV` !RRV/VBW0nR#)rNzMFSDPManagedNNModule depends on having an accurate source to control guarding.rrQ)rrorrr)rrr<rrs&&, rEr$FSDPManagedNNModuleVariable.__init___sBH% > _  /u// rGc&<V^8dQhRS[RS[/#r;rL)rDrs"rErFrisrGc\V\\34'gA\PP P 'd \V4#\V4#V#rP)r[rrr>_dynamoconfigskip_fsdp_guardsr?s&&rEr@(FSDPManagedNNModuleVariable._wrap_sourceisO ,.IJ  }}##444)+662;??rGr) rNrpr)r*rrr@r,r-r.r/s@@rErrSs# rGr)Urrrrvrrrcollections.abcr contextlibrrtypingrrtorch.nnr> torch._guardsrrr r r excr r rrrrrguardsrrrmutation_guardrrrrrrrrrutilsrrr r!r"r#r$r%r&r'r(r)r*r+r,rr-r.r/r0 functionsr1lazyr2 user_definedr3torch._dynamo.symbolic_convertr4rr6rrrrrrrOrrrQrGrErs. $2% 88BA."ED3%3H*7t++.&R  FF*[6[6|^$=^B +H"?rG