+ j4>^RIHtRtRRltRt!RR]4tR#) )PushToHubMixincRp\VP44pTPP pTR8Xd2^RIHpT!4'd^RIp\TR4PeR#T# \d\RT 24hi;i)zw Infers the device type from the model parameters. Args: model: The model instance. Returns: The device type. z { "RMSNorm": { "cuda": "kernels-community/layer_norm:LlamaRMSNorm", ... }, ... } zPCannot determine model device, please provide a device to the mapping. Example: cuda)is_torch_availableNversionrocm) next parameters StopIteration ValueErrordevicetypeutilsrtorchgetattrhip)modelEXAMPLE_MAPPINGparamdev_typerrs& a/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/utils/kernel_config.py infer_devicers O U%%'( ||  H6.    ui(,,8 O!  ^_n^o p   s A--BNc ^RIHpVR9d\RV 24hVPR4^,p VPR4^,p VW8!V V VVVR7//W@&R#)LayerRepositoryBOnly cuda, rocm, xpu, npu, neuron and tpu devices supported, got: :)repo_id layer_namerrevisiontrust_remote_codeNrrxpunpuneurontpu)kernelsrr split) r r repo_namemodecompatible_mappingrr!r"rrepo_layer_namers &&&&&&&& radd_to_mappingr.:sw( DD]^d]efggooc*1-Oooc"1%G /*!"3   &"c^RIHp^RIHpVR9d\ RV 24hVP R4^,pVP R4^,pVW6!V!V4VR7//W@&R#)r)Path)LocalLayerRepositoryrr) repo_pathr Nr#)pathlibr1r(r2r r)) r r r*r+r,r1r2r-r3s &&&&& radd_to_mapping_localr5Pst, DD]^d]efggooc*1-O$Q'I &y/* &"r/cNa]tRt^ctoRtR RltR RltRtRtR Rlt Rt Vt R#) KernelConfigz] Kernel configuration class. This class is used to configure the kernel mapping for a model. NcFVeTM/Vn/VnW nW0nR#)N)kernel_mappingregistered_layer_namesuse_local_kernelinherit_mapping)selfr9r;r<s&&&&r__init__KernelConfig.__init__hs&0>0JnPR&(# 0.r/c L^RIHp VWY!VVVVVR7//VPV&R#)rr)rr r!rr"N)r(rr9) r=rregistered_namer r r+r!rr"rs &&&&&&&&& r update_kernelKernelConfig.update_kernelns< , o#)%#&7  0 O,r/c"VP4Fzwr#\VRR4e:VPP4FpW@PVRV 2,&K \ VR4'gKaVP VPV&K| R#) _kernel_funcsN.kernel_layer_name) named_modulesrrEkeysr:hasattrrG)r=rnamemodulerGs&& rstore_registered_layer_names)KernelConfig.store_registered_layer_namess!//1LDv5A)/)=)=)B)B)D%Rc//;L:M7N0NO*Ev2334:4L4L++D12r/c  ZRpVPV4\VP\4'g%\ RV R\ VP4 24hVPP 4EFwr4W0PP49d\ RV R24hRp\V\\34'dRV/pRp\V\4'Ed9VP 4EF wrgTpV'gVR9d\ R V 24h\V\4'dVwr\V \4'g\ R \ V 4: R V R 24hV PR R4;p e%\V \4'g\ RV 24hV PRR4;p f+V PRR4;p f\ RV : RV : 24h\V\4'dRV9g RV9gEK\ RV RV 24h EK\ RV RV 24h R#)a Validates the kernel_mapping to ensure that: 1. Each layer_name in the mapping is registered in the model (i.e., the model contains a module with a matching kernel_layer_name). 2. Each kernel value is - either a string of the form 'org/repo:layer_name' or a tuple with the same as string and a dict of {"revision"/"version/trust_remote_code": ...}, - or a dict mapping device types ("cuda", "rocm", "xpu", "npu") to such values as above. 3. Each device key in a dict is one of "cuda", "rocm", "xpu", or "npu". 5. Each trust remote code key must be a bool. 6. Each revision or version key must exist mutually exclusive if it has been passed explicitly. 7. Each repo_name is a valid repository and layer name in the format 'org/repo:layer_name' (i.e., a string containing both a slash and a colon). 8. If a local path is detected, it should be in the format '/abs/path:layer_name', where the absolute path points to the kernel repository, like "/home/user/layer_norm". Args: model: The model instance whose modules are checked for registered kernel_layer_name attributes. Raises: ValueError: If a layer_name is not registered in the model, if a device is not supported, or if a repo_name is not a valid 'org/repo:layer_name' string. a} For single device form remote { "RMSNorm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, For multiple devices form remote { "RMSNorm": { "cuda": "kernels-community/layer_norm:LlamaRMSNorm", "rocm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, ... } You can also pass metadata along to inform about specific kernel information { "RMSNorm": ("kernels-community/layer_norm:LlamaRMSNorm", {"version": 1, "trust_remote_code": True}), ... }, For single device form local { "RMSNorm": "/abs/path:LlamaRMSNorm", ... }, For multiple devices form local { "RMSNorm": { "cuda": "/abs/path:LlamaRMSNorm", "rocm": "/abs/path:LlamaRMSNorm", ... }, ... } z7Kernel mapping must be a dict of the following format: z, got: zLayer z[ is not registered in the model, please register it first using use_kernel_forward_from_hubFNTrz^The passed metadata as second entry in a tuple needs to be a dict but found: type(metadata) = z for rFr"z6Expected a bool value for `trust_remote_code` but got r!rzExpected valid combination for version/revision (mutually exclusive but one of them) to be passed when passed as tuple, but got revision= z and version= /rzKernel mapping for 'zl' must be a valid repo name with a layer name (e.g., 'org/repo:layer_name' or '/abs/path:layer_name'), got: z'Kernel mapping must follow the format: r#) rM isinstancer9dictr ritemsr:valuesstrtuplegetbool) r=rMAPPING_FORMATr kernelskip_device_checkr repor*metadatar"r!rs && rsanitize_kernel_mapping$KernelConfig.sanitize_kernel_mappingst() T ))%0$--t44I.IYY`aefjfyfyaz`{| #'"5"5";";"= J!"5U8*A!G# 2:>QSW1XX-e#-.?#F#F&0&\]n\o$p'"!")1 Z(FFHO'/||It'DDG!U"#-!YNV\Yh_f^i!k# &i55I9MQT\eQe(2:,?klukvw=%3D!#J>JZZabhai!jkk]#>r/c  ^RIHp/p\V4pVPP 4EFwrgVP 'd VP M VPpV'dWP,p\V\\34'dWW/pVP 4FwrW8wdK VP'd\WiWV4K.\V \4'd<V wrV PRR4p V PRR4pV PRR4pMT p Rp ^pRp\VV V VVVV VR7K EK W@nR#)ae Transforms a simple kernel_mapping of the form: { "RMSNorm": ("kernels-community/layer_norm:LlamaRMSNorm", {"version": 1, "trust_remote_code": True}), ... }, or for local path: { "RMSNorm": "/home/user/liger_kernels:LigerRMSNorm", ... }, into a nested mapping: { "RMSNorm": { "cuda": { Mode.INFERENCE: LayerRepository( repo_id="kernels-community/layer_norm", layer_name="LlamaRMSNorm", version=1, trust_remote_code=True, ) } } } or for local path: { "RMSNorm": { "cuda": { Mode.INFERENCE: LocalLayerRepository( repo_path=Path("/home/user/liger_kernels"), layer_name="LigerRMSNorm", ) } } } that's compatible with the kernels library. The device is inferred from the model's parameters if not provided. The Mode is inferred from the model's training state. )Moder!Nrr"F)r r*r+r,rr!r")r(rarr9rStrainingTRAINING INFERENCE TORCH_COMPILErQrUrVr;r5rWr.)r=rcompilerar,current_devicer rZr+r r\r*r]r!rr"s&&& rcreate_compatible_mapping&KernelConfig.create_compatible_mappings3d !%e,"&"5"5";";"= J$)NNN4==D000&3,//(1 &  +((((TI[\dE***.'I'||J=H&ll9d;G(0 5H%(P% $I#HG(-%!''9#%&7 )!/#>P1r/)r<r9r:r;)NFT)NF)F) __name__ __module__ __qualname____firstlineno____doc__r>rBrMr^rh__static_attributes____classdictcell__) __classdict__s@rr7r7cs./  " Mslj^1^1r/r7)rjNF)rrrr.r5r7r/rrts(#%P,&z1>z1r/