+ œ°³j+ãó°€Rt^RIHt^RIHt^RIHt^RIHt]!RR7]!RR]444t ]!RR7]!R R ]444t R .t R #) zMpt configuration)ÚLiteral)Ústrict)ÚPreTrainedConfig)Úauto_docstringzmosaicml/mpt-7b)Ú checkpointcó\a€]tRt^toRtRtRt^tRtRt Rt Rt Rt Rt Rt^tV3RltR tVtR#) ÚMptAttentionConfigaÜ attn_type (`str`, *optional*, defaults to `"multihead_attention"`): type of attention to use. Options: `"multihead_attention"`, `"multiquery_attention"`. attn_pdrop (`float`, *optional*, defaults to `0.0`): The dropout probability for the attention layers. attn_impl (`str`, *optional*, defaults to `"torch"`): The attention implementation to use. One of `"torch"`, `"flash"`, or `"triton"`. clip_qkv (`float`, *optional*): If not `None`, clip the queries, keys, and values in the attention layer to this value. softmax_scale (`float`, *optional*): If not `None`, scale the softmax in the attention layer by this value. If `None`, will default to `1/sqrt(hidden_size)`. prefix_lm (`bool`, *optional*, defaults to `False`): Whether the model should operate as a Prefix LM. This requires passing an extra `prefix_mask` argument which indicates which tokens belong to the prefix. Tokens in the prefix can attend to one another bi-directionally. Tokens outside the prefix use causal attention. qk_ln (`bool`, *optional*, defaults to `False`): Whether to apply layer normalization to the queries and keys in the attention layer. attn_uses_sequence_id (`bool`, *optional*, defaults to `False`): Whether to restrict attention to tokens that have the same token_type_ids. When the model is in `train` mode, this requires passing an extra *token_type_ids* argument which indicates which sub-sequence each token belongs to. Defaults to `False` meaning any provided *token_type_ids* will be ignored. alibi (`bool`, *optional*, defaults to `True`): Whether or not to use the alibi bias instead of positional embedding. alibi_bias_max (`int`, *optional*, defaults to 8): The maximum value of the alibi bias. Ú attn_configÚmultihead_attentionÚtorchNFTcó¼<€V^8„dQh/S[R ,;R&S[;R&S[;R&S[R,;R&S[R,;R&S[;R&S[;R&S[;R &S[;R &S[;R &#) éÚ attn_typeÚ attn_pdropÚ attn_implNÚclip_qkvÚ softmax_scaleÚ prefix_lmÚqk_lnÚattn_uses_sequence_idÚalibiÚalibi_bias_max)r Úmultiquery_attention)rÚintÚstrÚfloatÚbool)ÚformatÚ __classdict__s"€Új/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/mpt/configuration_mpt.pyÚ __annotate__ÚMptAttentionConfig.__annotate__s©ø‡‚ñBÐDÕEÑ]ñCñDÑñEñFÑñGñHdlÑ!ñIñJ˜4•<Ñ&ñKñLÑñMñN ÑñOñP Ñ'ñQñR ÑñSñTÑòUó©)Ú__name__Ú __module__Ú __qualname__Ú__firstlineno__Ú__doc__Úbase_config_keyrrrrrrrrrrÚ__annotate_func__Ú__static_attributes__Ú__classdictcell__)rs@rrrsLø‡€ñð8$€OàH]€IØ€JØ€IØ!€HØ"&€MØ€IØ€EØ"'ÐØ€EØ€N÷Uƒr"rcó¶aa€]tRt^EtoRtRtR]/tRRRRRR /tR t ^t ^t ^t R t R tR tR tR tRtRtRtRtRtRtRtRtRtRtRtRtRtV3RltV3Rlt Rt!Vt"V;t##)Ú MptConfigaÚ expansion_ratio (`int`, *optional*, defaults to 4): The ratio of the up/down scale in the MLP. max_seq_len (`int`, *optional*, defaults to 2048): The maximum sequence length of the model. layer_norm_epsilon (`float`, *optional*, defaults to 1e-05): The epsilon to use in the layer normalization layers. learned_pos_emb (`bool`, *optional*, defaults to `True`): Whether to use learned positional embeddings. attn_config (`dict`, *optional*): A dictionary used to configure the model's attention module. init_device (`str`, *optional*, defaults to `"cpu"`): The device to use for parameter initialization. Defined for backward compatibility logit_scale (`float`, *optional*): If not None, scale the logits by this value. no_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in all linear layers. embedding_fraction (`float`, *optional*, defaults to 1.0): The fraction to scale the gradients of the embedding layer by. norm_type (`str`, *optional*, defaults to `"low_precision_layernorm"`): Type of layer norm to use. All MPT models uses the same layer norm implementation. Defined for backward compatibility. Example: ```python >>> from transformers import MptConfig, MptModel >>> # Initializing a Mpt configuration >>> configuration = MptConfig() >>> # Initializing a model (with random weights) from the configuration >>> model = MptModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` Úmptr Únum_attention_headsÚn_headsÚ hidden_sizeÚd_modelÚnum_hidden_layersÚn_layersiiÀÄggñh㈵øä>TNÚcpugð?Úlow_precision_layernormFg{®Gáz”?c óØ<€VPf\4VnM:\VP\4'd\R/VPBVn\SV`!R/VBR#)Nr#)r rÚ isinstanceÚdictÚsuperÚ __post_init__)ÚselfÚkwargsÚ __class__s&,€rr<ÚMptConfig.__post_init__ŽsSø€Ø × Ñ Ò #Ü1Ó3ˆDÕ Ü ˜×(Ñ(¬$× /Ò /Ü1ÑE°D×4DÑ4DÑEˆDÔ Ü ‰ÒÑ' Ô'r"cóÈ<€V^8„dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[S[,;R&S[;R&S[S[,;R &S[;R &S[S[,R ,;R &S[;R &S[S[,R ,;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[R ,;R&S[R ,;R&S[S[S[,,R ,;R&#)r r3r1r5Úexpansion_ratioÚ max_seq_lenÚ vocab_sizeÚ resid_pdropÚlayer_norm_epsilonÚ emb_pdropÚlearned_pos_embNr Ú init_deviceÚ logit_scaleÚno_biasÚembedding_fractionÚ norm_typeÚ use_cacheÚinitializer_rangeÚtie_word_embeddingsÚ pad_token_idÚ bos_token_idÚ eos_token_id)rrrr:rrÚlist)rrs"€rr ÚMptConfig.__annotate__Es€ø‡‚ñdÑñeñfÑñgñhÑñiñjÑñkñlÑñmñnÑñoñp™•Ñ"ñqñrÑ$ñsñt‘s{Ñ ñuñvÑ ñwñxÑ*Õ*¨TÕ1Ñ8ñyñzÑñ{ñ|™•˜tÕ#Ñ*ñ}ñ~Ñññ@Ñ#ñAñBÑ.ñCñDÑñEñFÑ#ñGñHÑ$ñIñJ˜•*Ñ#ñKñL˜•*Ñ#ñMñN™™S •/ DÕ(Ñ/òOr")r )$r$r%r&r'r(Ú model_typerÚ sub_configsÚ attribute_mapr3r1r5rBrCrDrErFrGrHr rIrJrKrLrMrNrOrPrQrRrSr<r*r+r,Ú __classcell__)r?rs@@rr.r.Es´ù‡€ñ%ðN€JØ Ð"4Ð5€Kà˜yØyؘZð€Mð €GØ€GØ€HØ€OØ€KØ€JØ"€KØ $ÐØ €IØ €OØ48€KØ€KØ&*€KØ€GØ #ÐØ.€IØ€IØ#ÐØ $ÐØ#€LØ#€LØ+/€Lõ(÷S…r"r.N) r(ÚtypingrÚhuggingface_hub.dataclassesrÚconfiguration_utilsrÚutilsrrr.Ú__all__r#r"rÚr_sxðñåå.å3Ý#ñÐ,Ô-Øô(Ð)ó(óó.ð(ñVÐ,Ô-ØôL(Ð óL(óó.ðL(ð^ ˆ-‚r"