+ j1^RIt^RIHt^RIHtHt^RIt^RIHt!RR]4tRRlt R R lt R R lt R Rlt ]PRRl4tR#)N) defaultdict)Any NamedTuple)Tensorc0a]tRt^ toRtV3RltRtVtR#)_GroupSwapinInfoaPer-group bookkeeping for ``swap_in_optimizer_params_and_state``. Fields: live_group: live ``optimizer.param_groups[i]`` dict, mutated in place. swapin_group: per-group hyperparameter dict from the input state_dict (e.g. lr, betas, ...) plus packed ``params`` ids. swapin_params: replacement parameter tensors for ``live_group["params"]``. Same order as ``swapin_group["params"]`` (i.e. the i-th tensor corresponds to the i-th packed id), so the two are zippable. cz<V^8dQh/S[S[S[3,;R&S[S[S[3,;R&S[S[,;R&#) live_group swapin_group swapin_params)dictstrrlistr)format __classdict__s"W/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/optim/_stateless.py __annotate___GroupSwapinInfo.__annotate__ s@S#XsCx. <N)__name__ __module__ __qualname____firstlineno____doc____annotate_func____static_attributes____classdictcell__)rs@rrr s rrcRV^8dQhR\R\\\3,/#)r statereturn)rr)rs"rrrs"c3hrc\V\4'g#\R\V4P R24h\ ;QJdRV4F 'gK RM RM !RV44'd \R4h\ ;QJd*RVP 44F 'gK RM RM!RVP 444'd \R4hV#) zPCheck ``state`` is a dict mapping packed parameter ids to per-param state dicts.zswap_in_optimizer_params_and_state requires swapin_optim_state['state'] to be a dict mapping packed parameter ids to per-param state dicts, got .c3V"TFp\V\P4xK! R#5iN) isinstancetorchr).0ks& r (_validate_state_field..#s 61:a & &s')TFzmswap_in_optimizer_params_and_state requires optimizer.state_dict()-style state keyed by packed parameter ids.c3"TF4wr\V\4;'d\V\4'*xK6 R#5ir&)r'intr)r)r*vs& rr+r,(s. Tmda:a  9 9jD&9"9 9ms>>zeswap_in_optimizer_params_and_state requires per-parameter optimizer state entries to be dictionaries.)r'r RuntimeErrortyperanyitems)r!s&r_validate_state_fieldr4s eT " " ;''( +   s 6 6sss 6 666 3   s Tekkm Tsss Tekkm TTT 0   Lrc lV^8dQhRRR\R\\\\3,,/#)r optimizertorch.optim.Optimizer param_groupsr")rrrr)rs"rrr0s1&69 $sCx.rc\V\4'g#\R\V4P R24h\ VP 4\ V48wd \R4hV#)zW``param_groups`` must be a list whose length matches the live optimizer's param groups.zvswap_in_optimizer_params_and_state requires swapin_optim_state['param_groups'] to be a list of param-group dicts, got r$zVswapin_optim_state has a different number of parameter groups than the live optimizer.)r'rr0r1rlenr8)r6r8s&&r_validate_param_groups_fieldr;0sq lD ) ) 66:<6H6Q6Q5RRS U   9 ! !"c,&77 "   rc V^8dQhR\R\\\3,R\R\\,/#)r idxgroupr r")r.rrrr)rs"rrrAs<(( ( S>(( #Y (rc @\V\4'g \R4hVPR4p\V\4'd;\ ;QJdRV4F 'dK RM RM !RV44'g \R4h\ VR,4\ V48wd\RV R24hVUu.uF qDV9gK VNK ppVUu.uF qDV9gK VNK ppV'g V'd\R V R V R V R 24hV#uupiuupi) a>Validate a single swap-in param group against its live counterpart and return its packed parameter ids. Asserts: 1. swap-in group is a dict. 2. ``group['params']`` is a list of ints, and its length matches the params of the live optim group. 3. Keys inside swapin group should match the keys in the live optim group. zZswap_in_optimizer_params_and_state requires each optimizer param group to be a dictionary.paramsc3B"TFp\V\4xK R#5ir&)r'r.)r)pids& rr+/_validate_group_against_live..Ts9(8 3(8sFTzswap_in_optimizer_params_and_state requires optimizer.state_dict()-style param_groups[*]['params'] entries keyed by packed parameter ids.zswapin_optim_state param group zF has a different number of params than the live optimizer param group.zswap_in_optimizer_params_and_state requires optimizer.state_dict()-style param group keys to exactly match the live optimizer group keys for group z. Keys only in swap-in: z. Keys only in live: r$)r'rr0getrallr:)r=r>r swapin_param_idsr* swapin_only live_onlys&&& r_validate_group_against_liverIAs; lD ) ) "  $''1 & - -SS9(89SSS9(8966 O   5?s#344-cU3: :  +=lun11lK=!;Eql%:EI;i E%%0M1FykQR T  >;sDDD'Dc V^8dQhRRR\\\3,R\\\3,R\\\\3,\ \ ,3,/#)r r6r7swapin_parametersswapin_optim_stater")rrrrtuplerr)rs"rrrls_Q,Q,&Q,CK(Q,S#XQ, 4S>4 01 12 Q,rc VP'g \R4h\V\4'g \R4h\ VP R44p\ WP R44p\VP44p^p\4p.p\\VPVRR74Fuwp wr\WV 4p VPV 4V\V 4,p V \V48d \R4hWVV pT pVP!\#V V VR74Kw VUu.uF qV9gK VNK ppV'd\R V: R 24hW83#uupi) a Validate and normalize optimizer state for ``swap_in_optimizer_params_and_state``. This follows the same structural assumptions as DCP-compatible optimizers, but consumes the raw ``optimizer.state_dict()`` format: ``state`` is keyed by packed parameter ids and each param group contains the live optimizer group fields plus a packed ``params`` list whose order matches ``optimizer.param_groups``. zHswap_in_optimizer_params_and_state requires initialized optimizer state.zMswap_in_optimizer_params_and_state requires a DCP-style optimizer state_dict.r!r8Tstrictzrswap_in_optimizer_params_and_state requires the explicit parameter state to match optimizer.param_groups ordering.)r r r zswap_in_optimizer_params_and_state requires swapin_optim_state['state'] to be keyed only by packed parameter ids from param_groups[*]['params']; got extra keys r$)r!r0r'rr4rDr;rvaluesset enumeratezipr8rIupdater:appendr)r6rKrL swapin_stateswapin_param_groupsflat_parametersflat_param_offsetseen_param_idsgroup_swapin_infosr=r>r rF next_offsetr r* extra_keyss&&& r_prepare_swap_inr_ls ??? V   ($ / / [  ));)?)?)HIL6)).9,3356O"uN&/ I " "$7E'" "e8LQ./'#.>*?? _- -9 (+F ' !!  )+  ;'J*E\n-D!!\JE 99Ca I   ++Fs 8E(E(cxV^8dQhRRR\\\3,R\\\3,/#)r r6r7rKrL)rrrr)rs"rrrs<k)k)&k)CK(k)S#Xk)rc#H"\WV4wr4VPpVPUu.uFp\V4NK pp\ \4pVFp V P V P R&V PP4FwrV R8XdKWP V &K \V P V PR,RR7F"wr\VPV /44W&K$ K WnRx\VPVRR7F&wrVP4VPV4K( WPnR#uupi \TPTRR7F&wrTP4TPT4K( YPni;i5i)u Temporarily replace an optimizer's parameters and state with the supplied params and optim states, then restore them on exit. For the duration of the context, all optimizer APIs (including user hooks) see the swap-in values; the live optimizer is restored on exit. The difference between this API and ``optimizer.load_state_dict`` is that ``optimizer.load_state_dict`` only updates the optimizer's state and leaves the parameters in ``param_groups`` untouched. This API also swaps in the parameters, so that ``optimizer.step()`` acts on the swap-in parameter tensors you supply. Args: optimizer: the live optimizer; its state must already be initialized. swapin_parameters: tensors to use as parameters during the context, provided in the same order as the existing input parameters to the optimizer (most commonly in ``model.named_parameters()`` order). swapin_optim_state: an ``optimizer.state_dict()``-shaped dict (``{"state": ..., "param_groups": ...}``) holding the state to install. ``"state"`` is keyed by packed integer parameter ids and ``"param_groups"`` mirrors ``optimizer.param_groups``, with each ``"params"`` entry as a list of those packed ids and the remaining keys carrying per-group hyperparameters (``lr``, ``betas``, ``foreach``, ``capturable``, ...). Only in-place tensor edits propagate back to the user supplied ``swapin_optim_state``; all other side-effects (e.g., assigning a new tensor to the optim state) are ignored. Example: One use of this API is to run ``optimizer.step()`` against ``FakeTensor`` versions of the parameters and state for nonstrict tracing — capturing an FX graph of the step without touching the live optimizer:: from torch.fx.experimental.proxy_tensor import make_fx from torch._subclasses import FakeTensorMode from torch.utils import _pytree as pytree fake_mode = FakeTensorMode(allow_non_fake_inputs=True) with fake_mode: fake_params = { n: fake_mode.from_tensor(p) for n, p in model.named_parameters() } fake_osd = pytree.tree_map_only( torch.Tensor, fake_mode.from_tensor, optimizer.state_dict() ) def step_fn(params, osd): with swap_in_optimizer_params_and_state(optimizer, params, osd): optimizer.step() return params, osd gm = make_fx(step_fn)(fake_params, fake_osd) r@TrON) r_r!r8rrr r r r3rTrDclearrU)r6rKrLr!r\original_stategoriginal_group_snapshotsrWinfokeyvalue swapin_paramparam_idr>snapshots&&& r"swap_in_optimizer_params_and_staterlsF!1&8!E__N 2;1G1GH1GAQ1GH)1rqsS#"  z "*"(VQ,h k)k)r