+ j_^RIHt^RIt^RIHtHtHt^RIt^RIH t ^RIH t ^RI H t ]'d ^RI HtHtHtRRltR R ltR R ltR RltRRltR-RRlltRRltR-RRlltR.RRlltRRltRRlt!RR] P84t!RR ] P84tR-R!R"lltR-R#R$llt R%R&lt!R'R(lt"R/R)R*llt#R/R+R,llt$R#)0) annotationsN)AnyNoReturn TYPE_CHECKING)Tensor)NamedMemberAccessor)CallableIterableSequencecV^8dQhRR/#)returnr)formats"a/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/_functorch/make_functional.py __annotate__rsXc\R4h)a$make_functional(module): we don't yet support models that do parameter tying (also sometimes known as weight sharing). Please try to rewrite your model by replacing all instances of the tied parameter with another and/or comment your support in https://github.com/pytorch/functorch/issues/446) RuntimeErrorrrrraise_parameter_tying_errorrs  : rc$V^8dQhRRRRRR/#)r named_paramsz0dict[str, Tensor] | Iterable[tuple[str, Tensor]]tied_named_paramsrdict[str, list[str]]r)rs"rrr$s& , ,B ,G , ,rc\V4p\V4p\VP44p\VP44pVPV4'g\ RV RV 24h/pVP 4F wrxV.3Wh&K VP 4F8wrxW9d\ RV R24hWh,^,P V4K: \VP44#)aC named_params is a dictionary of tensors: {'A': A, 'B': B} tied_named_params is another dictionary of tensors {'A': A, 'B': B, 'B_tied': B} with potentially tied (or 'duplicated') tensors This function creates a mapping from the names in named_params to the names in tied_named_params: {'A': ['A'], 'B': ['B', 'B_tied']}. ztensors_dict_keys z+ is not a subset of tied_tensors_dict_keys ztensor for key 'z ' not found in tensor_to_mapping)dictsetkeysissubsetAssertionErroritemsappendvalues) rrnamed_params_dicttied_named_params_dicttensors_dict_keystied_tensors_dict_keystensor_to_mappingkeytensors && rcreate_names_mapr+$s \*!"34-2245 !7!?  % %&< = = !2 34&&<%= ?  >@(..0 %("I!1-335   * "3%'GH  !!$++C0 6 !((* ++rc(V^8dQhRRRRRRRR/#) r mod nn.Module named_membersz+Callable[..., Iterable[tuple[str, Tensor]]]subclasszCallable[[Tensor], Tensor]r@tuple[tuple[Tensor, ...], tuple[str, ...], dict[str, list[str]]]r)rs"rrrGs1$$ $>$)$F $rcT\V!RR74p\V!RR74p\WC4p/p\V4pVFDwrW9d!V!\P!V RR74Wi&Wi,p VP W4KF \ V4^8XdRRrM \V!wrWV3#)F)remove_duplicateTmeta)devicer)tupler+rtorch empty_like set_tensorlenzip) r-r/r0all_named_membersunique_named_members names_mapmemoaccessornamep replacementnamesparamss &&& r_extract_membersrFGs mUCD !EF !5II D"3'H$ =u//&ABDGg D. %   A%Bv12  ) ##rc V^8dQhRRRR/#r r-r.rr1r)rs"rrras E E  EE ErcJ\WP\P4#)a> This function removes all the Parameters from the model and return them as a tuple as well as their original attribute names. The weights must be re-loaded with `load_weights` before the model can be used again. Note that this function modifies the model in place and after this call, mod.parameters() will be empty. )rFnamed_parametersnn Parameterr-s&rextract_weightsrNas C!5!5r|| DDrc V^8dQhRRRR/#rHr)rs"rrros AA AEArc0\WPR4#)cV#Nr)xs&r!extract_buffers..rsar)rF named_buffersrMs&rextract_buffersrWos C!2!2K @@rc ,V^8dQhRRRRRRRRR R /#) r r-r.rD Sequence[str]rESequence[Tensor] as_paramsboolrNoner)rs"rrrus:(( ( ( ( (  (rc\V4pV'd&VUu.uFp\P!V4NK ppVPW4R#uupi)z Reload a set of weights so that `mod` can be used again to perform a forward pass. Note that the `params` are regular Tensors (that can have history) and so are left as Tensors. This means that mod.parameters() will still be empty after this call. N)rrKrL set_tensors)r-rDrEr[r@rBs&&&& r load_weightsr`us@#3'H+126a",,q/62 '3sA c(V^8dQhRRRRRRRR/#) r r-r.r>relemsIterable[Tensor]rz list[Tensor]r)rs"rrrs,    3 rbresultr@_ attr_nameselemi attr_names &&& r _swap_statermsvF"3'H!$Y__%6!>%j1LAAv h229CD##I4 2"? Mrc ,V^8dQhRRRRRRRRR R /#) r r-r.rDrYbuffersrZr[r\rr]r)rs"rrrs:)) ) )) )  )rc>\V4pVPW4R#rR)rr_)r-rDror[r@s&&&& r load_buffersrqs #3'H (rc 0V^8dQhRRRRRRRRRRR R/#) r modelr.weightsrZ weight_namesrYro buffer_namesrr)rs"rrrsD        rcF\V4\V48wd$\R\V4 R\V4 24h\WV4\V4^8dI\V4\V48wd$\R\V4 R\V4 24h\WV4V#)zload_state(model, weights, weight_names, buffers=(), buffer_names=()) -> model load_state takes `weights` and `buffers` and assigns them to the model. This is the inverse operation of `make_functional_deprecated_v1`. zlen(weight_names)=z != len(weights)=zlen(buffer_names)=z != len(buffers)=)r:r r`rq)rsrtrurorvs&&&&&r load_staterxs <CL( \!2 33DS\N S  g. 7|a | G , $S%6$77HWW  U'2 Lrc V^8dQhRRRR/#)r rsr.r>tuple[tuple[Tensor, ...], Callable[..., Any], tuple[str, ...]]r)rs"rrrs)%)% )%C)%rcaa\SP44p\V4^8d \R4h\ S4wpopRVV3RllpW$S3#)amake_functional_deprecated_v1(model) -> weights, func, weight_names Given an nn.Module, make_functional_deprecated_v1 extracts the state (weights) and returns a functional version of the model, `func`. This makes it so that it is possible use transforms over the parameters of `model`. `func` can be invoked as follows: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, func, _ = make_functional_deprecated_v1(model) func(weights, (x,)) ``` And here is an example of applying the grad transform: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, _, func = make_functional_deprecated_v1(model) grad_weights = grad(func)(weights, (x,)) ``` To put the state back into a model, use `load_state`. zmake_functional_deprecated_v1(model): `model` has buffers. Please use make_functional_with_buffers_deprecated_v1(model) instead.c$V^8dQhRRRRRR/#)r rttuple[Tensor, ...]datatuple[Any, ...]rrr)rs"rr3make_functional_deprecated_v1..__annotate__s"$$'$$3$rcV<\P!S4p\VSV4V!V!#rR)copydeepcopyr`)rtr~ mutable_model descriptorsrss&& rfun*make_functional_deprecated_v1..funs( e, ]K9d##r)listror:rrN)rsrortrhrrsf @rmake_functional_deprecated_v1rs^85==?#G 7|a I  .e4G[!$$  $$rc V^8dQhRRRR/#)r rsr.rzctuple[tuple[Tensor, ...], tuple[Tensor, ...], Callable[..., Any], tuple[str, ...], tuple[str, ...]]r)rs"rrrs".F.F .F.Frcfaaa\S4wpop\S4wpopRVVV3RllpWVSS3#)amake_functional_with_buffers_deprecated_v1(model) -> weights, buffers, func, weight_names, buffer_names Given an nn.Module, make_functional_with_buffers_deprecated_v1 extracts the state (weights and buffers) and returns a functional version of the model, `func`. `func` can be invoked as follows: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, buffers, func, _, _ = make_functional_with_buffers_deprecated_v1(model) func(weights, buffers, (x,)) ``` And here is an example of applying the grad transform: ``` x = torch.randn(4, 3) model = nn.Linear(3, 3) weights, buffers, func, _, _ = make_functional_with_buffers_deprecated_v1(model) func(weights, buffers, (x,)) grad_weights = grad(func)(weights, buffers, (x,)) ``` To put the state back into a model, use `load_state`. c(V^8dQhRRRRRRRR/#)r rtr}ror~rrrr)rs"rr@make_functional_with_buffers_deprecated_v1..__annotate__s0$$#$#$$ $rcp<\P!S4p\VSV4\VSV4V!V!#rR)rrr`rq)rtror~rbuf_descriptorsrsweight_descriptorss&&& rr7make_functional_with_buffers_deprecated_v1..funs7  e, ]$6@]OW=d##r)rNrW)rsrtrhrorrrsf @@r*make_functional_with_buffers_deprecated_v1rsEB&5U%;"G "1%"8G_a$$ S"4o EErcXa]tRtRtRtRV3Rllt]R RRll4tRRltR t V;t #) FunctionalModuleWithBuffersizO This is the callable object returned by :func:`make_functional_with_buffers`. c 0V^8dQhRRRRRRRRRRR R /#) r stateless_modelr. param_namestuple[str, ...]rvparam_names_maprbuffer_names_maprr]r)rs"rr(FunctionalModuleWithBuffers.__annotate__sF44"4%4& 4 . 4 / 4 4rc <\SV`4WnW nW0n\ V4VnVP PV4R#rR)super__init__rrrvr all_names_mapupdate)selfrrrvrr __class__s&&&&&&rr$FunctionalModuleWithBuffers.__init__sD .&(!/2 !!"23rc$V^8dQhRRRRRR/#r rsr.disable_autograd_trackingr\rJtuple[FunctionalModuleWithBuffers, tuple[Tensor, ...], tuple[Tensor, ...]]r)rs"rrr+s$   59 S rc \P!V4p\V4wr4p\V4wrgpV'dVFp V P R4K \ W$WuV4VV3#F)rrrNrWrequires_grad_r) rsr model_copyrErrrorvrparams && r _create_from(FunctionalModuleWithBuffers._create_from*sp ]]5) />z/J,_2A*2M// $$$U+  (HX      rc ,V^8dQhRRRRRRRRRR/#)r rErcroargsrkwargsrr)rs"rrr=sCMM M"M M  M Mrc *\VPVP\V4\V4,4pVP!V/VB\VPVPV4# \TPTPT4i;irR)rmrrr6)rrErorr old_states&&&*, rforward#FunctionalModuleWithBuffers.forward=s|     &ME'N *  M''88 ,,d.@.@) LK,,d.@.@) Ls A//#B)rrvrrr __name__ __module__ __qualname____firstlineno____doc__r staticmethodrr__static_attributes__ __classcell__rs@rrrs244   $MMrrcXa]tRtRtRtRV3Rllt]R RRll4tRRltR t V;t #) FunctionalModuleiQzB This is the callable object returned by :func:`make_functional`. c(V^8dQhRRRRRRRR/#) r rr.rrr>rrr]r)rs"rrFunctionalModule.__annotate__Vs2 # #" #% #( #  #rc H<\SV`4WnW nW0nR#rR)rrrrr>)rrrr>rs&&&&rrFunctionalModule.__init__Vs! .&"rc$V^8dQhRRRRRR/#r rsr.rr\rz+tuple[FunctionalModule, tuple[Tensor, ...]]r)rs"rrrbs) L L L59 L 4 Lrc \P!V4p\V4wr4pV'dVFpVPR4K \ W$V4V3#r)rrrNrr)rsrrrErr>rs&& rrFunctionalModule._create_fromasN ]]5) )8)D&Y $$$U+  CVKKrc(V^8dQhRRRRRRRR/#)r rErcrrrrr)rs"rrrms/II.IsIcIcIrc \VPVPV4pVP!V/VB\VPVPV4# \TPTPT4i;irR)rmrr>)rrErrrs&&*, rrFunctionalModule.forwardmsb 4 4dnnfM  I''88 ,,dnni HK,,dnni Hs A#A9)r>rrrrrs@rrrQs4 # # L LIIrrc$V^8dQhRRRRRR/#rr)rs"rrrws)EE E15E0Erc\VP44p\V4^8d \R4h\P WR7#)amake_functional(model, disable_autograd_tracking=False) -> func, params Given a ``torch.nn.Module``, :func:`make_functional` extracts the state (params) and returns a functional version of the model, ``func``. This makes it so that it is possible use transforms over the parameters of ``model``. ``func`` can be invoked as follows: .. code-block:: python import torch import torch.nn as nn from functorch import make_functional x = torch.randn(4, 3) model = nn.Linear(3, 3) func, params = make_functional(model) func(params, x) And here is an example of applying the grad transform over the parameters of a model. .. code-block:: python import torch import torch.nn as nn from functorch import make_functional, grad x = torch.randn(4, 3) t = torch.randn(4, 3) model = nn.Linear(3, 3) func, params = make_functional(model) def compute_loss(params, x, t): y = func(params, x) return nn.functional.mse_loss(y, t) grad_weights = grad(compute_loss)(params, x, t) If the model has any buffers, please use :func:`make_functional_with_buffers` instead. Args: model (torch.nn.Module): Input model. disable_autograd_tracking (bool): Flag to disable gradients tracking for output parameters. The returned params are unrelated to the set of params from the original model. If False (default), the params will have ``requires_grad=True`` on them (aka they will be trackable with regular PyTorch autograd), matching the requires_grad-ness of the params from the original model. Otherwise, the returned params will have ``requires_grad=False``. Default, False. If you plan on using regular PyTorch autograd (e.g., if you want to call ``.backward()`` or ``torch.autograd.grad()``, then set ``disable_autograd_tracking=False``. Otherwise, if you're only planning on using functorch's gradient transforms, then please set ``disable_autograd_tracking=True`` to avoid unnecessarily tracking history with PyTorch autograd. zdmake_functional(model): `model` has buffers. Please use make_functional_with_buffers(model) instead.r)rror:rrr)rsrros&& rmake_functionalrwsPz5==?#G 7|a ;    ( (  ) rc$V^8dQhRRRRRR/#rr)rs"rrrs$<< <15<O<rc.\PWR7#)a$make_functional_with_buffers(model, disable_autograd_tracking=False) -> func, params, buffers Given a ``torch.nn.Module``, make_functional_with_buffers extracts the state (params and buffers) and returns a functional version of the model ``func`` that can be invoked like a function. ``func`` can be invoked as follows: .. code-block:: python import torch import torch.nn as nn from functorch import make_functional_with_buffers x = torch.randn(4, 3) model = nn.Linear(3, 3) func, params, buffers = make_functional_with_buffers(model) func(params, buffers, x) And here is an example of applying the grad transform over the parameters of a model: .. code-block:: python import torch import torch.nn as nn from functorch import make_functional_with_buffers, grad x = torch.randn(4, 3) t = torch.randn(4, 3) model = nn.Linear(3, 3) func, params, buffers = make_functional_with_buffers(model) def compute_loss(params, buffers, x, t): y = func(params, buffers, x) return nn.functional.mse_loss(y, t) grad_weights = grad(compute_loss)(params, buffers, x, t) Args: model (torch.nn.Module): Input model. disable_autograd_tracking (bool): Flag to disable gradients tracking for output parameters. The returned params are unrelated to the set of params from the original model. If False (default), the params will have ``requires_grad=True`` on them (aka they will be trackable with regular PyTorch autograd), matching the requires_grad-ness of the params from the original model. Otherwise, the returned params will have ``requires_grad=False``. Default, False. If you plan on using regular PyTorch autograd (e.g., if you want to call ``.backward()`` or ``torch.autograd.grad()``, then set ``disable_autograd_tracking=False``. Otherwise, if you're only planning on using functorch's gradient transforms, then please set ``disable_autograd_tracking=True`` to avoid unnecessarily tracking history with PyTorch autograd. r)rr)rsrs&&rmake_functional_with_buffersrs t ' 3 3  4 rc V^8dQhRRRR/#)r tuple_of_tuple_of_tensorsztuple[tuple[Tensor, ...], ...]rr}r)rs"rrrs=rc\\V!4p\;QJd.RV4FNK 5pV#!RV44pV#)c3j"TF)p\P!V4P4xK+ R#5irRr7stackdetach.0shardss& r "transpose_stack..s)3L F""$$3L13)r6r;)rresultss& rtranspose_stackrsR!&c+D&E Fe3LeG N3LG Nrc V^8dQhRRRR/#)r modelszSequence[nn.Module]rrr)rs"rrrs;%;% ;%O;%rca\V4^8Xd \R4h\;QJdRV4F 'dK RM RM !RV44'gF\;QJdRV4F 'dK RM RM !RV44'g \R4h\V^,4o\;QJdV3RlV4F 'dK RM RM!V3RlV44'g \R4h\ VUu.uFp\ V4NK up!wr#p\ V4p\ V4pV^,W43#uupi) acombine_state_for_ensemble(models) -> func, params, buffers Prepares a list of torch.nn.Modules for ensembling with :func:`vmap`. Given a list of ``M`` ``nn.Modules`` of the same class, stacks all of their parameters and buffers together to make ``params`` and ``buffers``. Each parameter and buffer in the result will have an additional dimension of size ``M``. :func:`combine_state_for_ensemble` also returns ``func``, a functional version of one of the models in :attr:`models`. One cannot directly run ``func(params, buffers, *args, **kwargs)`` directly, you probably want to use ``vmap(func, ...)(params, buffers, *args, **kwargs)`` Here's an example of how to ensemble over a very simple model: .. code-block:: python num_models = 5 batch_size = 64 in_features, out_features = 3, 3 models = [torch.nn.Linear(in_features, out_features) for i in range(num_models)] data = torch.randn(batch_size, 3) fmodel, params, buffers = combine_state_for_ensemble(models) output = vmap(fmodel, (0, 0, None))(params, buffers, data) assert output.shape == (num_models, batch_size, out_features) .. warning:: All of the modules being stacked together must be the same (except for the values of their parameters/buffers). For example, they should be in the same mode (training vs eval). This API is subject to change -- we're investigating better ways to create ensembles and would love your feedback how to improve this. z?combine_state_for_ensemble: Expected at least one model, got 0.c38"TFqPxK R#5irRtrainingrms& rr-combine_state_for_ensemble..4s+Fq FsFTc3B"TFqP'*xK R#5irRrrs& rrr4s2R6azz>>6szTcombine_state_for_ensemble: Expected all models to have the same training/eval mode.c3><"TFp\V4SJxK R#5irR)type)rr model0_typs& rrr:s5ftAw*$fszHcombine_state_for_ensemble: Expected all models to be of the same class.)r:rallrr;rr)rrsfuncsrErors& @rcombine_state_for_ensemblers P 6{a M   C+F+CCC+F+ + +ss2R62Rsss2R62R/R/R 0  fQiJ 35f53335f5 5 5 V  !;A B6% &u -6 BE7V $Fg&G 8V $$ CsD?c(V^8dQhRRRRRRRR/#) r model_classtype[nn.Module]ensemble_shapetuple[()] | tuple[int, ...]r5torch.types.DevicerzMCallable[..., tuple[tuple[Tensor, ...], Callable[..., Any], tuple[str, ...]]]r)rs"rrrFs1 / S rc"aaaRVVV3RllpV#)c$V^8dQhRRRRRR/#)r rrrrrzr)rs"rr%functional_init..__annotate__Ks$"""!" G"rcx<aa\S 4^8d \R4h\S 4^8Xd#S !S/SBPS 4p\V4#S ^,pV^8:d\RV R24h\;QJd#.VV VV 3Rl\ V44FNK 5M!VV VV 3Rl\ V444p\S !S/SB4wrVp\;QJd.RV4FNK 5M !RV44p\ \ V!4p\;QJd.RV4FNK 5M !RV44pWV3#)r ,NYI: ensemble_shape with more than 1 element num_models  should be > 0c3R<"TFpS!S/SBPS4xK R#5irRtorrhrr5rrs& rr3functional_init..wrapped..W- =NK ( ( + +F 3 3=N$'c3F"TFp\V4^,xK R#5i)rN)rrrss& rrr[sTVE5e.wrappedKs ~ ! #KL L ~ ! #0033F;E07 7#A& ?{:,nEF F =B:=N  =B:=N  5[$5Q&5QR u%TVT%%TVTTW &%K7K%%K7KKE!!rrrrr5rsfff rfunctional_initrFs ""* Nrc(V^8dQhRRRRRRRR/#) r rrrrr5rrzCallable[..., tuple[tuple[Tensor, ...], tuple[Tensor, ...], Callable[..., Any], tuple[str, ...], tuple[str, ...]] | tuple[tuple[Tensor, ...], Callable[..., Any], tuple[str, ...]]]r)rs"rrrcs0:: :/: :  :rc"aaaRVVV3RllpV#)c$V^8dQhRRRRRR/#)r rrrrztuple[tuple[Tensor, ...], tuple[Tensor, ...], Callable[..., Any], tuple[str, ...], tuple[str, ...]] | tuple[tuple[Tensor, ...], Callable[..., Any], tuple[str, ...]]r)rs"rr2functional_init_with_buffers..__annotate__rs,)@)@)@!)@ I)@rc<aa\S 4^8d \R4h\S 4^8Xd#S !S/SBPS 4p\V4#S ^,pV^8:d\RV R24h\;QJd#.VV VV 3Rl\ V44FNK 5M!VV VV 3Rl\ V444p\ S !S/SB4wppppp\\;QJd.RV4FNK 5M !RV44!wr\ \V !4p \;QJd.RV 4FNK 5M !RV 44p \ \V !4p \;QJd.RV 4FNK 5M !RV 44p WWgV3#)r rrrc3R<"TFpS!S/SBPS4xK R#5irRrrs& rr@functional_init_with_buffers..wrapped..rrc3F"TFp\V4R,xK R#5i):Nr NN)rrs& rrrs$#E;5A"EE#r c3j"TF)p\P!V4P4xK+ R#5irRrrs& rrrr rc3j"TF)p\P!V4P4xK+ R#5irRrrs& rrrr r)r:r rrr6r rr;)rrrsrrrhrrurvrtror5rrsjl rr-functional_init_with_buffers..wrappedrsm ~ ! #KL L ~ ! #0033F;E07 7#A& ?{:,nEF F =B:=N  =B:=N   7{D7SF7S T     U#UU#  W &%K7K%%K7KKW &%K7K%%K7KK<??rrrsfff rfunctional_init_with_buffersrcs)@)@V Nrr)rr)rcpu)% __future__rrtypingrrrr7torch.nnrKr%torch.nn.utils._named_member_accessorrcollections.abcr r r rr+rFrNrWr`rmrqrxrrModulerrrrrrrrrrrr's# // E<< ,F$4 EA (" )4)%X.Fb9M"))9Mx#Iryy#ILEP<~;%|:::r