+ jORt^RIHt^RIHt^RIHtHt^RIH t H t ]P!] 4t ]!RR7]!RR]444t]!R R7]!R R ]444t]!R R7]!R R]444t]!RR7]!RR]444t]!RR7]!RR]444t]!RR7]!RR]444t]!RR7]!RR]444tR.tR#)zESMFold2 model configuration.)strict)PreTrainedConfig)auto_docstringlogging)CONFIG_MAPPING AutoConfiga Configuration for a sliding-window atom transformer with 3D RoPE. Used twice, with identical field names: once for the inputs embedder (`EsmFold2Config.atom_encoder`) and once for the diffusion denoiser (`EsmFold2Config.structure_head.diffusion_module.atom_encoder`). ) custom_introcvaa]tRt^toRt^tRt^t^tRt Rt Rt ^t ^ t RtV3RltV3RltV3R ltR tVtV;t#) EsmFold2AtomEncoderConfiga hidden_size (`int`, *optional*, defaults to 128): Per-atom hidden width of the atom transformer. output_dim (`int`, *optional*, defaults to 384): Width this stack aggregates to when scattering atoms back into tokens. num_hidden_layers (`int`, *optional*, defaults to 3): Number of sliding-window atom-transformer blocks. num_attention_heads (`int`, *optional*, defaults to 4): Number of attention heads in each atom-transformer block. head_dim (`int`, *optional*): Per-head width. Derived as `hidden_size // num_attention_heads` if unset. intermediate_size (`int`, *optional*, defaults to 256): SwiGLU feed-forward width. spatial_rope_base_frequency (`float`, *optional*, defaults to 20.0): Base frequency for the spatial (x/y/z) half of the 3D rotary embedding. num_spatial_rope_pairs_per_axis (`int`, *optional*, defaults to 2): Number of rotary frequency pairs allocated to each spatial axis. num_uid_rope_pairs (`int`, *optional*, defaults to 10): Number of rotary frequency pairs allocated to the per-atom space UID. uid_rope_base_frequency (`float`, *optional*, defaults to 10000.0): Base frequency for the space-UID half of the 3D rotary embedding. Ng4@g@c <VPf#VPVP,Vn\SV`!R/VBR#N)head_dim hidden_sizenum_attention_headssuper __post_init__selfkwargs __class__s&,t/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/esmfold2/configuration_esmfold2.pyr'EsmFold2AtomEncoderConfig.__post_init__Fs5 ==  ,,0H0HHDM ''c <\SV`4^VP,VP,pWP^,8d1\ RV RVP^, RVP R24hR#)zThe 3D rotary embedding needs z$ frequency pairs, which exceeds the z available in a head of width .N)rvalidate_architecturenum_spatial_rope_pairs_per_axisnum_uid_rope_pairsr ValueError)rnum_rope_pairsrs& rr/EsmFold2AtomEncoderConfig.validate_architectureKsx %'TAAADD[D[[ MMQ. .00@@d==A%&&DT]]OSTV  /rc<V^8dQh/S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R &S[R,;R &S[R,;R &#) Nr output_dimnum_hidden_layersrrintermediate_sizespatial_rope_base_frequencyr r!uid_rope_base_frequencyintfloat)format __classdict__s"r __annotate__&EsmFold2AtomEncoderConfig.__annotate__sBt!CDd  EFTz%GHt'IJDjKLTz'MN"'4OP&)4Z3QRd 'ST#T\3Ur)r)__name__ __module__ __qualname____firstlineno____doc__rr'r(rrr)r*r r!r+rr__annotate_func____static_attributes____classdictcell__ __classcell__rr0s@@rr r sW."K J$%&'H$'0423#%',3( crr zlConfiguration for the diffusion denoiser: an atom encoder/decoder around a token transformer with pair bias.czaa]tRt^VtoRtR]/tRtRtRt ^ t ^t Rt Rt RtRtV3R ltV3R ltV3R ltR tVtV;t#) EsmFold2DiffusionModuleConfigaP sigma_data (`float`, *optional*, defaults to 16.0): Data noise scale of the EDM preconditioning; sets the scale at which coordinates are normalized. hidden_size (`int`, *optional*, defaults to 768): Token-stream width of the diffusion transformer. Also the width the atom stack projects to and from, so `atom_encoder.output_dim` mirrors it. fourier_dim (`int`, *optional*, defaults to 256): Width of the Fourier noise-level embedding. num_hidden_layers (`int`, *optional*, defaults to 12): Number of token-transformer blocks (one attention + one transition each). num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads in the token transformer; also the width of each block's pair bias. head_dim (`int`, *optional*): Per-head width. Derived as `hidden_size // num_attention_heads` if unset. intermediate_size (`int`, *optional*, defaults to 1536): SwiGLU width of the token transitions, and of the conditioning's single transitions. pair_intermediate_size (`int`, *optional*, defaults to 512): SwiGLU width of the conditioning's pair transitions, which run at `EsmFold2Config.pairwise_hidden_size`. atom_encoder (`EsmFold2AtomEncoderConfig`, *optional*): Configuration for the denoiser's atom encoder/decoder stack; defaults to one whose `output_dim` is this module's `hidden_size`. atom_encoderg0@ir Nic P<VPf#VPVP,VnVPf\ VPR7VnM:\ VP\ 4'd\ R/VPBVn\SV` !R/VBR#)N)r'r) rrrr?r isinstancedictrrrs&,rr+EsmFold2DiffusionModuleConfig.__post_init__s} ==  ,,0H0HHDM    $ 9TEUEU VD  ))4 0 0 9 NDr>VsZ 0"#<=K#J!K!K$&&(H$(),<@L(grr>zrConfiguration for the structure-prediction head: the distogram output plus the diffusion sampler's noise schedule.czaa]tRt^toRtR]/tRt^t^t Rt Rt Rt Rt RtR tR t^DtR tV3R ltV3R ltRtVtV;t#)EsmFold2StructureHeadConfigai diffusion_module (`EsmFold2DiffusionModuleConfig`, *optional*): Configuration for the denoiser this head samples from. num_distogram_bins (`int`, *optional*, defaults to 128): Number of distance bins predicted by the distogram head. num_diffusion_samples (`int`, *optional*, defaults to 8): Number of parallel structure samples drawn by the diffusion sampler. gamma_0 (`float`, *optional*, defaults to 0.605): Churn factor applied at noise levels above `gamma_min` (extra noise re-injected before a step). gamma_min (`float`, *optional*, defaults to 1.107): Noise level below which no churn is applied. noise_scale (`float`, *optional*, defaults to 0.0): Scale of the noise added by the churn step. `0.0` makes sampling a deterministic ODE. step_scale (`float`, *optional*, defaults to 1.0): Scale applied to each denoising update. inference_sigma_max_ratio (`float`, *optional*, defaults to 160.0): Highest sigma of the Karras noise schedule, as a multiple of `sigma_data` (so the default is a sigma of `160 * sigma_data`). Shapes the schedule; it is not where sampling starts, because `inference_sigma_cap` truncates the top of the schedule -- see there. inference_sigma_min_ratio (`float`, *optional*, defaults to 4e-4): Lowest sigma of the Karras noise schedule, likewise a multiple of `sigma_data`. inference_exponent (`float`, *optional*, defaults to 8.0): Exponent shaping the Karras schedule between `inference_sigma_min_ratio` and `inference_sigma_max_ratio`. inference_num_steps (`int`, *optional*, defaults to 68): Length of the Karras schedule built before truncation. Note this is not the number of denoising steps run: `inference_sigma_cap` drops the entries above the cap without lengthening the schedule to compensate, so the sampler runs fewer steps than this (with the released checkpoint's values, 14 becomes 10). inference_sigma_cap (`float`, *optional*, defaults to 256.0): Cap on the schedule, as an *absolute* sigma rather than a multiple of `sigma_data`. The high-sigma tail above it is truncated and the cap re-prepended, so sampling starts from the cap. diffusion_moduleNg\(\?gʡE?gg?gd@g-C6:?g @gp@c <VPf\4VnM:\VP\4'd\R/VPBVn\SV`!R/VBR#r)rPr>rBrCrrrs&,rr)EsmFold2StructureHeadConfig.__post_init__sS  ($A$CD ! --t 4 4$A$ZDDYDY$ZD ! ''rcb<V^8dQh/S[S[,R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R &S[R,;R &S[R,;R &S[R,;R &S[R,;R &#)r&NrPnum_distogram_binsnum_diffusion_samplesgamma_0 gamma_min noise_scale step_scaleinference_sigma_max_ratioinference_sigma_min_ratioinference_exponentinference_num_stepsinference_sigma_cap)rCr>r-r.)r/r0s"rr1(EsmFold2StructureHeadConfig.__annotate__sR::TAHSTd (UV:)WXT\!YZt|#[\#]^ "_` %t|3ab %t|2cd *eft(gh-ir)rP)r3r4r5r6r7r>rMrPrTrUrVrWrXrYrZr[r\r]r^rr8r9r:r;r<s@@rrOrOsi B&'DEKDH%(()!G#I #K"J.3.2'*&((-(mrrOz|Configuration for the confidence head, which predicts pLDDT, PAE, PDE, resolved-atom probability and the pTM/ipTM summaries.cPa]tRt^toRt^t^2t^@t^@tRt Rt ^t Rt V3Rlt RtVtR#)EsmFold2ConfidenceHeadConfigu num_hidden_layers (`int`, *optional*, defaults to 4): Number of pair-update blocks in the head's own folding trunk. num_plddt_bins (`int`, *optional*, defaults to 50): Number of bins in the per-atom pLDDT distribution. num_pde_bins (`int`, *optional*, defaults to 64): Number of bins in the predicted-distance-error distribution. num_pae_bins (`int`, *optional*, defaults to 64): Number of bins in the predicted-aligned-error distribution. min_dist (`float`, *optional*, defaults to 2.0): Lower edge (Å) of the head's distance binning. max_dist (`float`, *optional*, defaults to 52.0): Upper edge (Å) of the head's distance binning. distogram_bins (`int`, *optional*, defaults to 128): Number of distance bins used to embed predicted inter-atom distances. eps (`float`, *optional*, defaults to 1e-6): Additive guard for masked-mean denominators (empty chains / all-padding rows). g@gJ@gư>c<V^8dQh/S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R &#) r&Nr(num_plddt_bins num_pde_bins num_pae_binsmin_distmax_distdistogram_binsepsr,)r/r0s"rr1)EsmFold2ConfidenceHeadConfig.__annotate__s2Tz%34$J#56*!78*!9:dl ;<dl!=>$J$?@ ArrN)r3r4r5r6r7r(rcrdrerfrgrhrir8r9r:r0s@rraras= &%&!#N!L!L H!H!$NCArrazConfiguration for the MSA encoder: outer-product-mean into the pair stream, pair-weighted averaging back into the MSA stream, and the triangle updates.cLa]tRt^toRt^t^ t^t^t^ t Rt Rt V3Rlt Rt VtR#)EsmFold2MsaEncoderConfiga hidden_size (`int`, *optional*, defaults to 128): Width of the MSA stream. outer_hidden_size (`int`, *optional*, defaults to 32): Per-side width of the outer-product-mean projection; the outer product is this value squared. num_hidden_layers (`int`, *optional*, defaults to 4): Number of MSA encoder blocks. The last one updates only the pair stream, since its MSA output would be discarded. num_attention_heads (`int`, *optional*, defaults to 8): Number of heads in the pair-weighted averaging. head_dim (`int`, *optional*, defaults to 32): Per-head width of the pair-weighted averaging. intermediate_size (`int`, *optional*, defaults to 512): SwiGLU width of the MSA-stream transition. outer_product_chunk_size (`int`, *optional*): Chunk size for the outer-product-mean einsum, off by default. Chunking this one is not always bit-exact in bf16, so it trades exactness for peak memory on long sequences. r@Nc<V^8dQh/S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&#) r&Nrouter_hidden_sizer(rrr)outer_product_chunk_size)r-)r/r0s"rr1%EsmFold2MsaEncoderConfig.__annotate__s|2t!34Tz&56Tz%78t'9:Dj;<Tz'=>"Dj/?rr)r3r4r5r6r7rror(rrr)rpr8r9r:rks@rrmrms; &"K$&$%&'H$'+/?rrmzWConfiguration for the language-model (ESMC) hidden-state encoder folded into the trunk.c<a]tRtRtoRt^tRtRtV3RltRt Vt R#)EsmFold2LmEncoderConfigia num_hidden_layers (`int`, *optional*, defaults to 4): Number of pair-update blocks refining the projected ESMC hidden states. lm_dropout (`float`, *optional*, defaults to 0.25): Dropout applied to the projected language-model pair representation. per_loop_lm_dropout (`bool`, *optional*, defaults to `True`): Whether to resample that dropout on every trunk loop rather than once per fold. g?Tch<V^8dQh/S[R,;R&S[R,;R&S[R,;R&#)r&Nr( lm_dropoutper_loop_lm_dropout)r-r.bool)r/r0s"rr1$EsmFold2LmEncoderConfig.__annotate__s8Tz% #+rrN) r3r4r5r6r7r(rurvr8r9r:rks@rrsrss$%&#J'+rrszbiohub/ESMFold2-hf) checkpointc aa]tRtRtoRtRtR]R]R]R] R] R ] /t R t R tR tR t^t^@t^ t^t^ t^!t^t^@t^t^tRt^t^tRtRtRt Rt!Rt"Rt#V3Rlt$V3Rlt%V3Rlt&Rt'Vt(V;t)#)EsmFold2Configi'a hidden_size (`int`, *optional*, defaults to 384): Single-representation width. pairwise_hidden_size (`int`, *optional*, defaults to 256): Pair-representation width. single_inputs_size (`int`, *optional*, defaults to 451): Width of the concatenated single-input features fed to the trunk and diffusion conditioning: `atom_encoder.output_dim` plus two residue-type one-hots and the profile scalar. pair_transition_intermediate_size (`int`, *optional*, defaults to 1024): SwiGLU width of the pair-stream transitions. sliding_window (`int`, *optional*, defaults to 128): Sliding-window size (token-index distance) for the atom-stack attention, as the total window width rather than the radius. chunk_size (`int`, *optional*, defaults to 64): Chunk size for the memory-heavy pair-/MSA-stream ops. `None` disables chunking. num_relative_residx_bins (`int`, *optional*, defaults to 32): Number of relative residue-index bins in the relative-position encoding. num_relative_chain_bins (`int`, *optional*, defaults to 2): Number of relative chain-index bins in the relative-position encoding. num_loops (`int`, *optional*, defaults to 10): Number of trunk refinement loops. num_res_types (`int`, *optional*, defaults to 33): Number of residue types. max_atomic_number (`int`, *optional*, defaults to 128): Size of the element one-hot in the atom features. char_vocab_size (`int`, *optional*, defaults to 64): Character-vocabulary size for the encoded atom names. max_chars (`int`, *optional*, defaults to 4): Number of characters per encoded atom name. max_atoms_per_token (`int`, *optional*, defaults to 23): Maximum number of atoms per token. atom_feature_dim (`int`, *optional*): Atom feature width: xyz, charge and mask, plus the element and atom-name-char one-hots. Derived from `max_atomic_number`, `char_vocab_size` and `max_chars` if unset. folding_trunk_num_hidden_layers (`int`, *optional*, defaults to 24): Number of pair-update blocks in the folding trunk. parcae_num_coda_layers (`int`, *optional*, defaults to 2): Number of pair-update blocks in the parcae coda. atom_encoder (`EsmFold2AtomEncoderConfig`, *optional*): Configuration for the inputs-embedder SWA atom encoder. structure_head (`EsmFold2StructureHeadConfig`, *optional*): Configuration for the diffusion structure-prediction head (holds the `diffusion_module` sub-config). confidence_head (`EsmFold2ConfidenceHeadConfig`, *optional*): Configuration for the confidence head. msa_encoder (`EsmFold2MsaEncoderConfig`, *optional*): Configuration for the MSA encoder. lm_encoder (`EsmFold2LmEncoderConfig`, *optional*): Configuration for the language-model hidden-state encoder. esmc_config ([`EsmcConfig`], *optional*): Configuration for the bundled ESMC backbone. esmfold2 esmc_configr?structure_headconfidence_head msa_encoder lm_encoderr r iiNc <RpV!\R,VP4VnV!\VP4VnV!\VP 4VnV!\ VP4VnV!\VP4Vn V!\VP4Vn VPf;^VP,VPVP,,Vn \ SV`D!R/VBR#)cXVfV!4#\V\4'd V!R/VB#V#r)rBrC)clsvals&&r _init_nested2EsmFold2Config.__post_init__.._init_nesteds,{u #t$$zSz!JresmcNr)rr}r r?rOr~rarrmrrsratom_feature_dimmax_atomic_numberchar_vocab_size max_charsrr)rrrrs&, rrEsmFold2Config.__post_init__s (v(>@P@PQ()BDDUDUV*+FH[H[\+,H$J^J^_'(@$BRBRS&'>P  ($-0F0F$FI]I]`d`n`nIn$nD ! ''rc  <\SV`4VP^VP,^,, pVPP V8wd4\ RVPP RVP RV R24hR#)zQChecks the width relations that span sub-configs; each sub-config checks its own.rFz) is not single_inputs_size (z) - (2 * num_res_types + 1) = rN)rrsingle_inputs_size num_res_typesr?r'r")rexpected_output_dimrs& rr$EsmFold2Config.validate_architectures %'#55T=O=O9ORS9ST    ' '+> >/0A0A0L0L/MN++,,JK^J__`b  ?rc<V^8dQh/S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R &S[R,;R &S[R,;R &S[R,;R &S[R,;R &S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[S[,R,;R&S[S[,R,;R&S[S[,R,;R&S[S[,R,;R&S[S[,R,;R&S[S[,R,;R&#)r&Nrpairwise_hidden_sizer!pair_transition_intermediate_sizesliding_window chunk_sizenum_relative_residx_binsnum_relative_chain_bins num_loopsrrrrmax_atoms_per_tokenrfolding_trunk_num_hidden_layersparcae_num_coda_layersr?r~rrrr})r-rCr rOrarmrsr)r/r0s"rr1EsmFold2Config.__annotate__'sBt!CD**EFd (GH(+Tz8IJ$J$KLd MN"Dj-OP!4Z+QRTzST:"UVTz'WX4Z$YZTz[\t(]^Dj'_`&)4Z4ab $J*cf22T9@gh66=Dij884?Fkl0047>mn..5K8rOrarmrsr{__all__rrrrs$.3,-   H %0 000f"5$45  5p*6("26(  6(r.#3  <R0/0  0:vw ,. ,x ,/0w%w1wt  r