+ j mZ^RIt^RIt^RIt^RIt^RIt^RIHt^RIHt^RIH t ^RI H t H t H t Ht^RIt^RIt^RIHtHt^RIHtHtHtRR .t!R R ]P04t!R R ]4tRtRRltRRltRRltRRRllt!RR]P@4t!RRRllt"R#)N) OrderedDict)deepcopy)chain)AnyCallableOptionalUnion)fxnn)_CodeOnlyModule _copy_attr_USER_PRESERVED_ATTRIBUTES_KEYcreate_feature_extractorget_graph_node_namescNaa]tRt^toRtV3RltV3RlV3RlltRtVtV;t #)LeafModuleAwareTracerz An fx.Tracer that allows the user to specify a set of leaf modules, i.e. modules that are not to be traced through. The resulting graph ends up having single nodes referencing calls to the leaf modules' forward methods. cr</VnRV9dVPR4pW0n\SV` !V/VBR#) leaf_modulesN)rpopsuper__init__)selfargskwargsr __class__s&*, f/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torchvision/models/feature_extraction.pyrLeafModuleAwareTracer.__init__s9 V #!::n5L ,  $)&)c@<V^8dQhRS[PRS[RS[/#)mmodule_qualnamereturn)r Modulestrbool)format __classdict__s"r __annotate__"LeafModuleAwareTracer.__annotate__"s&:: :C:D:rcp<\V\VP44'dR#\SV`W4#T) isinstancetuplerris_leaf_module)rr!r"rs&&&rr/$LeafModuleAwareTracer.is_leaf_module"s. at001 2 2w%a99r)r) __name__ __module__ __qualname____firstlineno____doc__rr/__static_attributes____classdictcell__ __classcell__rr(s@@rrrs *:::rrcvaa]tRt^(toRtV3RltV3RlRltR V3RlV3RllltV3RlRltR t Vt V;t #) NodePathTracera, NodePathTracer is an FX tracer that, for each operation, also records the name of the Node from which the operation originated. A node name here is a `.` separated path walking the hierarchy from top level module down to leaf operation or leaf module. The name of the top level module is not included as part of the node name. For example, if we trace a module whose forward method applies a ReLU module, the name for that node will simply be 'relu'. Some notes on the specifics: - Nodes are recorded to `self.node_to_qualname` which is a dictionary mapping a given Node object to its node name. - Nodes are recorded in the order which they are executed during tracing. - When a duplicate node name is encountered, a suffix of the form _{int} is added. The counter starts from 1. cT<\SV`!V/VBRVn\4VnR#)N)rrcurrent_module_qualnamernode_to_qualname)rrrrs&*,rrNodePathTracer.__init__;s) $)&)')$ !, rcN<V^8dQhRS[PPRS[/#)r r!forward)torchr r$r)r'r(s"rr)NodePathTracer.__annotate__Es#88UXX__8x8rc VPpVPV4pW`nVPW4'gV!V/VBpVWPn#VPRWcV4WPn# YPni;i)a_ Override of `fx.Tracer.call_module` This override: 1) Stores away the qualified name of the caller for restoration later 2) Adds the qualified name of the caller to `current_module_qualname` for retrieval by `create_proxy` 3) Once a leaf module is reached, calls `create_proxy` 4) Restores the caller's qualified name into current_module_qualname call_module)r>path_of_moduler/ create_proxy)rr!rBrr old_qualnamer"outs&&&&& rrFNodePathTracer.call_moduleEsv33  8"11!4O+: (&&q::t.v.,8 ($$]O6R+7 (< (s7A% A%%A-c|<V^8dQhRS[RS[PPRS[PP /#)r kindtargetr#)r%r nodeTargetproxyProxy)r'r(s"rr)rDZs5   !#   rc <\S V`WW4WV4pVPVPVP4VP VP&V#)z Override of `Tracer.create_proxy`. This override intercepts the recording of every operation and stores away the current traced module's qualified name in `node_to_qualname` )rrH_get_node_qualnamer>rOr?) rrMrNrrname type_expr_rQrs &&&&&&&* rrHNodePathTracer.create_proxyZsK$T4Q,0,C,CDD`D`bgblbl,mejj) rcT<V^8dQhRS[RS[PPRS[/#)r r"rOr#)r%r rONode)r'r(s"rr)rDfs*""#"RWW\\"c"rcTpVPR8wd,\V4^8d VR, pV\V4, p\P!RV4eVP R^4^,p\ VPP44Fmp\P!V R2V4fK!VPVR4p\V4'd\VR,4^,pM^pVRV 2, pV# V#)rF.z .+_[0-9]+$rWz (_[0-9]+)?$r=:NN) oplenr%rematchrsplitreversedr?valuesreplaceint)rr"rO node_qualnameexisting_qualnamepostfix next_indexs&&& rrT!NodePathTracer._get_node_qualnamefs' 77m #=!A%$ SY &M 88M= 1 =)00a8;M"*$*?*?*F*F*H!I xxM?+68IJV+33M2Fw<r?)NN) r1r2r3r4r5rrFrHrTr6r7r8r9s@@rr;r;(s2$.88*  """rr;ca\V4o\;QJdV3RlV4F 'dK R# R#!V3RlV44#)zNCheck if y is a subsequence of x https://stackoverflow.com/a/24017747/4391249 c3<a"TF?o\;QJdV3RlS4F 'gK RM RM!V3RlS44xKA R#5i)c3,<"TF qS8HxK R#5iN).0x_itemy_items& r '_is_subseq...s9&V#&sTFN)any)rqrsiter_xs&@rrt_is_subseq..s/Jfss9&9sss9&999sA A  A FT)iterall)xyrws&&@r _is_subseqr}s7!WF 3JJ33J3J3JJ JJrc0V^8dQhR\R\/#)r train_tracer eval_tracer)r;)r's"rr)r)s((.(~(rc\VPP44p\VPP44p\V4\V48XdO\;QJd%R\ W#44F 'dK RM RM!R\ W#444'dR#Rp\ W#4'dRpM\ W24'dRpMRp\P!WT,4R#) zx Utility function for warning the user if there are differences between the train graph nodes and the eval graph nodes. c3."TF wrW8HxK R#5irorp)rqtes& rrt*_warn_graph_differences..s2cFbda16FbsFTNztWhen choosing nodes for feature extraction, you may need to specify output nodes for train and eval mode separately.znNOTE: The nodes obtained by tracing the model in eval mode are a subsequence of those obtained in train mode. znNOTE: The nodes obtained by tracing the model in train mode are a subsequence of those obtained in eval mode. zdThe nodes obtained by tracing the model in train mode are different to those obtained in eval mode. ) listr?rdr_rzzipr}warningswarn)rr train_nodes eval_nodessuggestion_msgmsgs&& r_warn_graph_differencesrs |44;;=>Kk2299;#># 99>#DcN+># >#DcN+ >#  49d3i  >#rc\V4pVPp\R/VBpVPVP 4VR7\R/VBpVPVP 4VR7\ VPP44p\ VPP44pV'g \WV4VP V4Wx3#)a Dev utility to return node names in order of execution. See note on node names under :func:`create_feature_extractor`. Useful for seeing which node names are available for feature extraction. There are two reasons that node names can't easily be read directly from the code for a model: 1. Not all submodules are traced through. Modules from ``torch.nn`` all fall within this category. 2. Nodes representing the repeated application of the same operation or leaf module get a ``_{counter}`` postfix. The model is traced twice: once in train mode, and once in eval mode. Both sets of node names are returned. For more details on the node naming conventions used here, please see the :ref:`relevant subheading ` in the `documentation `_. Args: model (nn.Module): model for which we'd like to print node names tracer_kwargs (dict, optional): a dictionary of keyword arguments for ``NodePathTracer`` (they are eventually passed onto `torch.fx.Tracer `_). By default, it will be set to wrap and make leaf nodes all torchvision ops: {"autowrap_modules": (math, torchvision.ops,),"leaf_modules": _get_leaf_modules_for_ops(),} WARNING: In case the user provides tracer_kwargs, above default arguments will be appended to the user provided dictionary. suppress_diff_warning (bool, optional): whether to suppress a warning when there are discrepancies between the train and eval version of the graph. Defaults to False. concrete_args (Optional[Dict[str, any]]): Concrete arguments that should not be treated as Proxies. According to the `Pytorch docs `_, this parameter's API may not be guaranteed. Returns: tuple(list, list): a list of node names from tracing the model in train mode, and another from tracing the model in eval mode. Examples:: >>> model = torchvision.models.resnet18() >>> train_nodes, eval_nodes = get_graph_node_names(model) rrp) rtrainingr;tracetrainevalrr?rdr) rrrr is_trainingrrrrs &&&& rrrsd/}=M..K!2M2Lu{{}MB 1=1Kejjl-@|44;;=>Kk2299;)get_attrrF)rr GraphModulerrr1rrrrynodesr^r-rNr% TypeErrorrr rgraph _tracer_clsr3)rrrrrrOrs&&&&& rrDualGraphModule.__init__s~ bnnd,.",&$${0014 8H8H3IJDww55!$++s33#&STXY]YdYdTeSf$ghh4t{{3 K   ?? & &$*:*:*F*F FYZ^ZiZiZuZuYvwDEIEUEUEaEaDbblm   :: ! ! !j 8N8N8[8[&[#zz55D '\ !rc <V'd%VP'gVPVnM+V'g$VP'dVPVn\SV`VR7#)z Swap out the graph depending on the selected training mode. NOTE this should be safe when calling model.eval() because that just calls this with mode == False. )mode)rrrrrr)rrrs&&rrDualGraphModule.train@sD  ))DJ$---DJw}$}''rc"\P#ro)rr)rs&r_deepcopy_initDualGraphModule._deepcopy_initNs'''rc \V4P\V44pW!\V4&\\P !VP V44pVP4!W#VP R,VP R,4.ROpVFGpWPP 9gK\W%\P !VP V,V44KI \P !\VR/4V4Vn \VP9d;VP\,P4Fwre\W&V4K V#)rrmeta)_state_dict_hooks_load_state_dict_pre_hooks_load_state_dict_post_hooks _replace_hook_create_node_hooks_erase_node_hooks) r__new__idr copyr__dict__rsetattrgetattrrritems)rmemoresfake_modextra_preserved_attrsattr attr_names&& r __deepcopy__DualGraphModule.__deepcopy__Rs4j  d,RX"4==#EF cX->->}-MxO`O`amOno! *D}}$4==t1Dd#KL*==vr!:DA )SXX 5#&88,J#K#Q#Q#S -$T r)rrrr)rr,) r1r2r3r4r5rrrrr6r7r8r9s@@rrr s))6)6V ((rrc`V^8dQhR\PR\\\\ ,\ \ \ 3,3,,R\\\\ ,\ \ \ 3,3,,R\\\\ ,\ \ \ 3,3,,R\\ \ \3,,R\R\\ \ \3,,R\P/#) r r return_nodestrain_return_nodeseval_return_nodesrrrr#) r r$rr rr%rrr&r r)r's"rr)r)qsnn 99n5cDcN!:;<n!tCy$sCx.'@!ABn d3ic3h&? @A n DcN+ n  nDcN+n^^nrc  \V4pVPp\;QJdRWV34F 'dK RM RM!RWV344'd \R4hVRJVRJ, 'd \R4hVRJVRJ, 'g \R4hRR lpVf V!V4p\ V4p\ V4pMV!V4pV!V4p/p /p R VR V/p REFvp V R 8XdVP 4MV R 8XdVP 4\R/VBp V PWR 7p\V\P4'dVPPM VPp\P!V P W4p\#V P$P'44p\)\+V44\)V48wd \R 4hW,P-4FNp\/VUu.uF p\0P2!RV R2V4RJNK" up4'dKA\RV R24h .p\5VP6P84F'pVP:R8XgKVP=V4K) V'g \R4hVFpVP6P?V4K VP6P8Uu.uFpVNK pp\A4p\5V4FpV P$PCV4pVfK$W,FopVPER4pRPGVPIR4RV^,4V8XgKEVVW,V,&W,PKV4K K \A\5\#VPM4444pVP6POVR,4;_uu_4VP6PQV4RRR4VP6PS4VPU4WV &WV &EKy V'g\WV R ,V R ,4\YW R ,V R ,XR7pVP V4VP V4V#uupiuupi +'giL;i)a Creates a new graph module that returns intermediate nodes from a given model as dictionary with user specified keys as strings, and the requested outputs as values. This is achieved by re-writing the computation graph of the model via FX to return the desired nodes as outputs. All unused nodes are removed, together with their corresponding parameters. Desired output nodes must be specified as a ``.`` separated path walking the module hierarchy from top level module down to leaf operation or leaf module. For more details on the node naming conventions used here, please see the :ref:`relevant subheading ` in the `documentation `_. Not all models will be FX traceable, although with some massaging they can be made to cooperate. Here's a (not exhaustive) list of tips: - If you don't need to trace through a particular, problematic sub-module, turn it into a "leaf module" by passing a list of ``leaf_modules`` as one of the ``tracer_kwargs`` (see example below). It will not be traced through, but rather, the resulting graph will hold a reference to that module's forward method. - Likewise, you may turn functions into leaf functions by passing a list of ``autowrap_functions`` as one of the ``tracer_kwargs`` (see example below). - Some inbuilt Python functions can be problematic. For instance, ``int`` will raise an error during tracing. You may wrap them in your own function and then pass that in ``autowrap_functions`` as one of the ``tracer_kwargs``. For further information on FX see the `torch.fx documentation `_. Args: model (nn.Module): model on which we will extract the features return_nodes (list or dict, optional): either a ``List`` or a ``Dict`` containing the names (or partial names - see note above) of the nodes for which the activations will be returned. If it is a ``Dict``, the keys are the node names, and the values are the user-specified keys for the graph module's returned dictionary. If it is a ``List``, it is treated as a ``Dict`` mapping node specification strings directly to output names. In the case that ``train_return_nodes`` and ``eval_return_nodes`` are specified, this should not be specified. train_return_nodes (list or dict, optional): similar to ``return_nodes``. This can be used if the return nodes for train mode are different than those from eval mode. If this is specified, ``eval_return_nodes`` must also be specified, and ``return_nodes`` should not be specified. eval_return_nodes (list or dict, optional): similar to ``return_nodes``. This can be used if the return nodes for train mode are different than those from eval mode. If this is specified, ``train_return_nodes`` must also be specified, and `return_nodes` should not be specified. tracer_kwargs (dict, optional): a dictionary of keyword arguments for ``NodePathTracer`` (which passes them onto it's parent class `torch.fx.Tracer `_). By default, it will be set to wrap and make leaf nodes all torchvision ops: {"autowrap_modules": (math, torchvision.ops,),"leaf_modules": _get_leaf_modules_for_ops(),} WARNING: In case the user provides tracer_kwargs, above default arguments will be appended to the user provided dictionary. suppress_diff_warning (bool, optional): whether to suppress a warning when there are discrepancies between the train and eval version of the graph. Defaults to False. concrete_args (Optional[Dict[str, any]]): Concrete arguments that should not be treated as Proxies. According to the `Pytorch docs `_, this parameter's API may not be guaranteed. Examples:: >>> # Feature extraction with resnet >>> model = torchvision.models.resnet18() >>> # extract layer1 and layer3, giving as names `feat1` and feat2` >>> model = create_feature_extractor( >>> model, {'layer1': 'feat1', 'layer3': 'feat2'}) >>> out = model(torch.rand(1, 3, 224, 224)) >>> print([(k, v.shape) for k, v in out.items()]) >>> [('feat1', torch.Size([1, 64, 56, 56])), >>> ('feat2', torch.Size([1, 256, 14, 14]))] >>> # Specifying leaf modules and leaf functions >>> def leaf_function(x): >>> # This would raise a TypeError if traced through >>> return int(x) >>> >>> class LeafModule(torch.nn.Module): >>> def forward(self, x): >>> # This would raise a TypeError if traced through >>> int(x.shape[0]) >>> return torch.nn.functional.relu(x + 4) >>> >>> class MyModule(torch.nn.Module): >>> def __init__(self): >>> super().__init__() >>> self.conv = torch.nn.Conv2d(3, 1, 3) >>> self.leaf_module = LeafModule() >>> >>> def forward(self, x): >>> leaf_function(x.shape[0]) >>> x = self.conv(x) >>> return self.leaf_module(x) >>> >>> model = create_feature_extractor( >>> MyModule(), return_nodes=['leaf_module'], >>> tracer_kwargs={'leaf_modules': [LeafModule], >>> 'autowrap_functions': [leaf_function]}) c3("TFqRJxK R#5irorp)rqargs& rrt+create_feature_extractor..s X"W3$;"WsFTzcEither `return_nodes` or `train_return_nodes` and `eval_return_nodes` together, should be specifiedNzcIf any of `train_return_nodes` and `eval_return_nodes` are specified, then both should be specifiedz\If `train_return_nodes` and `eval_return_nodes` are specified, then both should be specifiedcFV^8dQhR\\\3,/#r)rr%)r's"rr).create_feature_extractor..__annotate__s66c3h6rc\V\4'd&VUu/uFp\V4\V4bK up#VP4UUu/uFwr#\V4\V4bK upp#uupiuuppiro)r-rr%r)nikvs& r to_strdict,create_feature_extractor..to_strdictsa a  ,-.AqCFCFNA. .+,7795941AA955/5s A3 A8rrrzYThere are duplicate nodes! Please raise an issue https://github.com/pytorch/vision/issues^z(\.|$)znode: 'z' is not present in model. Hint: use `get_graph_node_names` to make sure the `return_nodes` you specified are present. It may even be that you need to specify `train_return_nodes` and `eval_return_nodes` separately.outputz1No output nodes found in graph_module.graph.nodesr\)r)rrrp)-rrrz ValueErrorrrrr;rr-r r$rr1r rrrr?rdr_rkeysrvr`rarcrrr^r erase_nodergetcountjoinsplitrrinserting_afterreliminate_dead_code recompilerr)rrrrrrrrrtracersgraphsmode_return_nodesrtracerrrU graph_moduleavailable_nodesqueryrorig_output_nodesr output_nodesr"depths&&&&&&& rrrqsRj/}=M..K s XVZ[L KK {# ^*6&; :s&R? ( SS  S )NFN)NNNNFN)#rrrr`r collectionsrr itertoolsrtypingrrrr rCrr r torch.fx.graph_moduler r r__all__Tracerrr;r}rrrrrrrrprrrs  #11 ]] &'= >:BII:(`*`FK(> ">#BabnnaHnnr