+ j@V^RIHtHt^RIHt^RIHt^RIHtH t ^RI H t ] P!] 4t]!RR7]!RR ]444t]!RR7]!R R ]444t]!RR7]!R R ]444t]!RR7]!RR]444t.ROtR#))AnyLiteral)strict)PreTrainedConfig)auto_docstringlogging)intervalzgoogle/gemma-4-e2b-it) checkpointcaa]tRt^toRtRtRt^ t^tRt Rt ^t Rt ^ t ^ t^tRtRtRtRtR tR t]!R R R 7!RR7tV3RltV3RltRtVtV;t#)Gemma4AudioConfiga subsampling_conv_channels (`list[int]`, defaults to `[128, 32]`): Channel sizes for the convolutional layers in the Sub-sample Convolution Projection. residual_weight (`float`, defaults to `0.5`): Scaling applied to hidden_states prior to combining with the residual in the feedforward. attention_chunk_size (`int`, defaults to `12`): The sub-sequence size for attention processing. attention_context_left (`int`, defaults to `13`): The leftward context size for the attention chunk. attention_context_right (`int`, defaults to `0`): The rightward context size for the attention chunk. attention_logit_cap (`float`, defaults to `50.0`): Cap applied to attention weights. attention_invalid_logits_value (`float`, defaults to `1e-9`): Value to use for invalid logits in attention. use_clipped_linears (`bool`, defaults to `True`): If true, apply clipping to the Linear layers, drawing bounds from the model checkpoint. gradient_clipping (`float`, defaults to `1e10`): Clipping value used to stabilize extremely large gradient values. output_proj_dims (`int`, defaults to `1536`): Dimension of the final linear projection from `hidden_size` to the model's output. gemma4_audioisilug?gI@Tư>g _Big?)minmax{Gz?)defaultc <\VP\4'd\VP4Vn\SV`!R/VBR#)N) isinstancesubsampling_conv_channelstuplelistsuper __post_init__selfkwargs __class__s&,p/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/gemma4/configuration_gemma4.pyrGemma4AudioConfig.__post_init__Ns9 d44e < <-1$2P2P-QD * ''c<V^8dQh/S[;R&S[;R&S[;R&S[;R&S[S[,S[S[S[3,,;R&S[;R&S[;R&S[;R&S[;R &S[;R &S[;R &S[;R &S[;R &S[;R&S[;R&S[;R&S[;R&#) hidden_sizenum_hidden_layersnum_attention_heads hidden_actrconv_kernel_sizeresidual_weightattention_chunk_sizeattention_context_leftattention_context_rightattention_logit_capattention_invalid_logits_valueuse_clipped_linears rms_norm_epsgradient_clippingoutput_proj_dimsinitializer_range)intstrrrfloatbool)format __classdict__s"r! __annotate__Gemma4AudioConfig.__annotate__s 89:;< =>?D $Cy5c?:FEJKL MN"OP $QR!$ST%UV%*2WZ$[\]^#_` abGcr#)r) ge)__name__ __module__ __qualname____firstlineno____doc__ model_typer&r'r(r)rr*r+r,r-r.r/r0r1r2r3r4r r5r__annotate_func____static_attributes____classdictcell__ __classcell__r r;s@@r!r r s. JK J>G O ""$#$!%,2" $L# 'CS9$G(gr#r chaa]tRt^UtoRtRtR.tRRRRRRRR R R R R R RRRRR RRRR RR/ tR RRRRR RRRRRRRR/tRR.R.3RRR.R.3RR.R.3/t Rt R t R!t ^t ^t^tR"tR#tR$tR%tR&tR't^t^t^tR'tR(tR)tR*tR+tR(tR(tR(t Rt!R"t"R)t#^t$R)t%R)t&R(t'R(t(R(t)V3R,lt*V3R-lV3R.llt+R/t,V3R0lt-R1t.Vt/V;t0#)2Gemma4TextConfiga use_bidirectional_attention (`str`, *optional*): Controls bidirectional attention behavior. When set to `"vision"`, vision tokens attend bidirectionally while text tokens use causal attention. When set to `"all"`, all tokens use bidirectional attention. vocab_size_per_layer_input (`int`, defaults to 262144): Vocabulary size for the per-layer input embeddings (PLE). Used by models with per-layer residual streams where a smaller embedding is added at each decoder layer. hidden_size_per_layer_input (`int`, defaults to 256): Per-layer hidden dimension for the PLE system. The actual embedding weight has shape `[vocab_size_per_layer_input, num_hidden_layers * hidden_size_per_layer_input]` because all layers are packed into a single table. See the [Gemma4](https://huggingface.co/docs/transformers/main/en/model_doc/gemma4#per-layer-embeddings-ple) docs for a description of the full PLE pipeline. attention_k_eq_v (`bool`, defaults to `False`): Whether keys and values share the same projection weights. When `True`, the key projection output is reused as the value projection. num_kv_shared_layers (`int`, defaults to 0): Number of consecutive decoder layers that share the same key-value projections. A value of 0 means no sharing (each layer has independent KV projections). enable_moe_block (`bool`, defaults to `False`): Whether to enable Mixture-of-Experts (MoE) blocks in the decoder layers. When `True`, eligible layers will use a sparse MoE feed-forward network. use_double_wide_mlp (`bool`, defaults to `False`): Whether to use a double-width MLP with fused gate and up projections. top_k_experts (`int`, *optional*): Number of experts activated per token in MoE layers. Only used when `enable_moe_block=True`. moe_intermediate_size (`int`, *optional*): Intermediate (hidden) size of each expert's feed-forward network in MoE layers. Only used when `enable_moe_block=True`. gemma4_textpast_key_valueszlayers.*.self_attn.q_projcolwisezlayers.*.self_attn.k_projzlayers.*.self_attn.v_projzlayers.*.self_attn.q_normreplicated_with_grad_allreducezlayers.*.self_attn.k_normzlayers.*.self_attn.o_projrowwisezlayers.*.mlp.gate_projzlayers.*.mlp.up_projzlayers.*.mlp.down_projzlayers.*.experts.gate_up_projpacked_colwisezlayers.*.experts.down_projzlayers.*.expertsmoe_tp_expertszlayers.*.router ep_router grouped_gemm embed_tokens input_ids inputs_embedslayers hidden_statesattention_masknormii i$gelu_pytorch_tanhrrTNFrc X<VPR8Xd'RVnVP^,^,VnVPfN^p\ VP 4Uu.uF&p\ V^,V,4'dRMRNK( upVnVP'dCVPR,;pR8wd)\PRV R24RVPR&RRRR R /RRR R R R R//pVPfWPn VPRR4pVPRR4pRV9dYRV/p\VRR4'dTMRp V eWR&\VP4U U u/uFwrV R8XgKWbK up p VR&\S V`8!R/VBR#uupiuup p i)allFNsliding_attentionfull_attentionz/Last layer must use `full_attention`, but got `z*`. Forcing last layer to `full_attention`. rope_typer rope_thetag@ proportionalpartial_rotary_factorg?g.Aglobal_head_dimr`num_global_key_value_headsper_layer_confighead_dimattention_k_eq_vTnum_key_value_headsr)use_bidirectional_attention is_causalsliding_window layer_typesranger'r9loggerwarningrope_parameterspopgetattr enumeraterr) rrsliding_window_patternilast_layer_typedefault_rope_paramsrirjlayer_overridesrn layer_idx layer_typer s &, r!rGemma4TextConfig.__post_init__s  + +u 4"DN#'#6#6!#;q"@D     #%& "t556 6A(,QU6L,L'M'M#Scc6 D     D4D4DR4H!HM] ] NNA/ARR|} $4D  R  +y,!Q {N>> from transformers import ( >>> Gemma4AudioConfig, >>> Gemma4Config, >>> Gemma4ForConditionalGeneration, >>> Gemma4TextConfig, >>> Gemma4VisionConfig, >>> ) >>> # Initializing a Gemma 4 Audio config. >>> audio_config = Gemma4AudioConfig() >>> # Initializing a Gemma 4 Text config. >>> text_config = Gemma4TextConfig() >>> # Initializing a Gemma 4 vision config. >>> vision_config = Gemma4VisionConfig() >>> # Initializing a Gemma 4 config similar to google/gemma-4-e2b-it >>> configuration = Gemma4Config(text_config, vision_config, audio_config) >>> # Initializing a model from the google/gemma-4-e2b-it configuration >>> model = Gemma4ForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```gemma4 text_config vision_config audio_configNiiBi@iDiiCiArTc v<VPf&\4Vn\PR4M:\ VP\ 4'd\R/VPBVnVP f\PR4\ VP \ 4'd\R/VP BVnVPf\PR4\ VP\ 4'd\R/VPBVn\SV`,!R/VBR#)Nz4text_config is None. Using default Gemma4TextConfig.zHvision_config is None. Gemma4Model.vision_tower will not be initialized.zFaudio_config is None. Gemma4Model.audio_tower will not be initialized.r) rrLruinforrrrrr rrrs&,r!rGemma4Config.__post_init__ds    #/1D  KKN O (($ / //C$2B2BCD     % KKb c d(($ / /!3!Id6H6H!ID     $ KK` a d'' . . 1 FD4E4E FD  ''r#c<V^8dQh/S[S[S[S[3,,R,;R&S[S[S[S[3,,R,;R&S[S[S[S[3,,R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R&S[R,;R &S[R,;R &S[R,;R &S[R,;R &S[;R &#)r%Nrrr boi_token_id eoi_token_idimage_token_idvideo_token_id boa_token_ideoa_token_indexaudio_token_idr5r) rLrr7rrr r6r8r9)r:r;s"r!r<Gemma4Config.__annotate__%sd"DcN2T9@ef&S#X6=Dgh$d38n4t;Bij*&kl*&mn$J(op$J(qr*&st4Z)uv$J(wxt|*yz${r#)rrr)r@rArBrCrDrErLrr sub_configsrrrrrrrrrrr5rrrFrGrHrIrJs@@r!rr%sy& PJ'+)K =AK@DM>BL&L&L!(N!(N&L")O!(N&* $(r#rN)r rrLr)typingrrhuggingface_hub.dataclassesrconfiguration_utilsrutilsrrutils.type_validatorsr get_loggerr@rur rLrr__all__rr#r!rs .3,-   H %235((5(45(p23W'W4Wt23/$)/$4/$d23N(#N(4N(b Zr#