+ j~j^RIHt^RIt^RIHt^RIHtHtHt^RIH t ^RI H t H t ^RIHtHtHt^RIHt^RIHtHt^R IHt^R IHtHtHtHtHtHtH t H!t!^R I"H#t#H$t$^R I%H&t&^R I'H(t(^RI)H*t*H+t+H,t,^RI-H.t.H/t/^RI0H1t1^RI2H3t3],Ph!]54t6!RR]Pn4t8R@RRllt9!RR]Pn4t:!RR]Pn4t;!RR]Pn4t<!RR]Pn4t=!RR]Pn4t>!R R!]Pn4t?!R"R#]4t@!R$R%]Pn4tA]+!R&R']$44tB!R(R)]Pn4tC!R*R+]Pn4tD]+!R,R-7!R.R/]B44tE]+!R0R1]B44tF!R2R3]Pn4tG]+!R4R-7!R5R6]B44tH]+!R7R8]B44tI]+!R9R:]B44tJ]+!R;R<]B44tK]+!R=R-7!R>R?]B]44tL.RAOtMR#)B)CallableN)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)ACT2FNgelu)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)create_bidirectional_maskcreate_causal_mask)GradientCheckpointingLayer))BaseModelOutputWithPastAndCrossAttentions,BaseModelOutputWithPoolingAndCrossAttentions!CausalLMOutputWithCrossAttentionsMaskedLMOutputMultipleChoiceModelOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)apply_chunking_to_forward)TransformersKwargsauto_docstringlogging)can_return_tuplemerge_with_config_defaults)capture_outputs)CamembertConfigcpaa]tRt^8toRtV3RltRV3RlRllt]R4t]R Rl4t Rt Vt V;t #) CamembertEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.c<\SV`4\P!VPVP VP R7Vn\P!VPVP 4Vn \P!VP VPR7Vn \P!VP4Vn\P!\ P"!VP$4P'R4RR7Vn\P!\ P*!VP(P-4\ P.R7RR7VnVP Vn\P!VP$VP VP2R7VnR#)) padding_idxepsF) persistent)dtypeN)super__init__nn Embedding vocab_size hidden_size pad_token_idword_embeddingstype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutBuffertorcharangemax_position_embeddingsexpand position_idszerossizelongtoken_type_idsr'position_embeddingsselfconfig __class__s&&q/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/camembert/modeling_camembert.pyr0CamembertEmbeddings.__init__;s% !||F,=,=v?Q?Q_e_r_rs%'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<= IIell63Q3Q&R&Y&YZa&botu ii D4E4E4J4J4LTYT^T^(_lqr!..#%<<  * *F,>,>DL\L\$  c <V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[RS[P/#) input_idsNrGrC inputs_embedspast_key_values_lengthreturn)r? LongTensor FloatTensorintTensor)format __classdict__s"rM __annotate__ CamembertEmbeddings.__annotate__Ksw..##d*.((4/.&&- . ((4/ . !$ . .rOc4Vf=VeVPWPV4pMVPW@P4pVeVP4pMVP4RRpVwrxVf\ VR4'drVP P VP4PVP^,R4p \P!V ^VR7p V PWx4pM;\P!V\PVPPR7pVfVPV4pVP!V4p WJ,p VP#V4p W,p VP%V 4p VP'V 4p V #)NrG)dimindexr+devicer.)"create_position_ids_from_input_idsr'&create_position_ids_from_inputs_embedsrEhasattrrGtorbrBshaper?gatherrDrFrCr6r8rHr9r=) rJrRrGrCrSrT input_shape batch_size seq_lengthbuffered_token_type_idsr8 embeddingsrHs &&&&&& rMforwardCamembertEmbeddings.forwardKst  $#FF//1G   $JJ=ZjZjk  #..*K',,.s3K!,  !t-..*.*=*=*@*@ATAT*U*\*\]i]o]opq]rtv*w'*/,,7NTU]i*j'!8!?!? !W!&[ SWSdSdSkSk!l   00;M $ : :> J": "66|D5 ^^J/ \\*- rOc VP4RRpV^,p\P!V^,W1,^,\PVPR7pVP ^4P V4#)z We are provided embeddings directly. We cannot infer which are padded so just generate sequential position ids. Args: inputs_embeds: torch.Tensor Returns: torch.Tensor Nrar.)rEr?r@rFrb unsqueezerB)rSr'risequence_lengthrCs&& rMrd:CamembertEmbeddings.create_position_ids_from_inputs_embeds{sn$((*3B/ %a.|| !O_:Q>ejjYfYmYm %%a(// <L==((6??([L,,|3K''1-88:K  $$rOcLaa]tRt^toRV3RlltRV3RlRlltRtVtV;t#)CamembertSelfAttentionct<\SV`4VPVP,^8wd9\ VR4'g'\ RVP RVP R24hWnVPVn\VPVP, 4VnVPVP,Vn VPR,Vn \P!VPVP4Vn \P!VPVP4Vn\P!VPVP4Vn\P !VP"4VnVP&VnW nW0nR#rembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()Nr)r/r0r4num_attention_headsre ValueErrorrKrXattention_head_size all_head_sizerr1Linearrrrr;attention_probs_dropout_probr= is_decoder is_causal layer_idxrJrKrrrLs&&&&rMr0CamembertSelfAttention.__init__sM     : : :a ?PVXhHiHi#F$6$6#78 445Q8  #)#=#= #&v'9'9F??*9*N*N'&=%C%CI\`\j\j%k "I(?(M(M KK , ,.E) %8      % $}}}C$,,.. % LL %  % ! "));;;;FFH ((rO) rrrKr=rrrrrrrrFN)NN r|r}r~rr0rnrrrrs@@rMrrs#0')')')rOrcLaa]tRt^toRV3RlltRV3RlRlltRtVtV;t#)CamembertCrossAttentioncR<\SV`4VPVP,^8wd9\ VR4'g'\ RVP RVP R24hWnVPVn\VPVP, 4VnVPVP,Vn VPR,Vn \P!VPVP4Vn \P!VPVP4Vn\P!VPVP4Vn\P !VP"4VnW nW0nR#r)r/r0r4rrerrKrXrrrr1rrrrr;rr=rrrs&&&&rMr0 CamembertCrossAttention.__init__s@     : : :a ?PVXhHiHi#F$6$6#78 445Q8  #)#=#= #&v'9'9FB**4>>:(?(M(M KK , ,.E) %8      % $}}}C$,,.. % LL %  % ! "));;;;FFH ((rO) rrrKr=rrrrrrrr)NNNrrs@@rMrrs#.1)1)1)rOrcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)CamembertSelfOutputiIc0<\SV`4\P!VPVP4Vn\P !VPVPR7Vn\P!VP4Vn R#r(N) r/r0r1rr4denser9r:r;r<r=rIs&&rMr0CamembertSelfOutput.__init__Js` YYv1163E3EF f&8&8f>S>STzz&"<"<= rOch<V^8dQhRS[PRS[PRS[P/#rQr input_tensorrUr?rY)rZr[s"rMr\ CamembertSelfOutput.__annotate__P1U\\RWR^R^rOcxVPV4pVPV4pVPW,4pV#Nrr=r9rJrrs&&&rMrnCamembertSelfOutput.forwardP5 =1  ]3 }'CD rOr9rr=rrs@@rMrrI> rOrcLaa]tRtRtoRV3RlltRV3RlRlltRtVtV;t#)CamembertAttentioniWc<\SV`4W@nV'd\M\pV!WVR7Vn\ V4VnR#)rrN)r/r0is_cross_attentionrrrJroutput)rJrKrrrattention_classrLs&&&&& rMr0CamembertAttention.__init__Xs9 "45G1Mc#F9U )&1 rOc<V^8dQhRS[PRS[PR,RS[PR,RS[PR,RS[R,RS[S[,RS[S[P,/# rQrrNrencoder_attention_maskrrrUr)rZr[s"rMr\CamembertAttention.__annotate___s..||.))D0. %0047 . !& 1 1D 8 .  .+,. u|| .rOc VP'gTMTpVP!V3RVRVRV/VBwrxVPWq4pWx3#)rrr)rrJr) rJrrrrrrattention_outputrs &&&&&&, rMrnCamembertAttention.forward_sm04/F/F/FLb)- * "7* ** , *  * & ;;'7G--rO)rrrJ)FNFNNNNrrs@@rMrrWs2...rOrcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)CamembertIntermediateitc"<\SV`4\P!VPVP 4Vn\VP\4'd\VP,Vn R#VPVn R#r) r/r0r1rr4intermediate_sizerr hidden_actstrrintermediate_act_fnrIs&&rMr0CamembertIntermediate.__init__us` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $rOcN<V^8dQhRS[PRS[P/#rQrrUr)rZr[s"rMr\"CamembertIntermediate.__annotate__}s#U\\ellrOcJVPV4pVPV4pV#rrr)rJrs&&rMrnCamembertIntermediate.forward}s& =1 00? rOrrrs@@rMrrts9rOrcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)CamembertOutputic0<\SV`4\P!VPVP 4Vn\P!VP VPR7Vn\P!VP4Vn R#r) r/r0r1rrr4rr9r:r;r<r=rIs&&rMr0CamembertOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= rOch<V^8dQhRS[PRS[PRS[P/#rr)rZr[s"rMr\CamembertOutput.__annotate__rrOcxVPV4pVPV4pVPW,4pV#rrrs&&&rMrnCamembertOutput.forwardrrOrrrs@@rMrrrrOrcRaa]tRtRtoRV3RlltRV3RlRlltRtRtVtV;t #) CamembertLayeric<\SV`4VPVn^Vn\ WP VR7VnVP VnVPVnVP'd5VP 'g\V R24h\ VRVRR7Vn \V4Vn \V4Vn R#)r-rz> should be used as a decoder model if cross attention is addedFT)rrrN)r/r0chunk_size_feed_forward seq_len_dimrr attentionadd_cross_attentionrcrossattentionr intermediaterr)rJrKrrLs&&&rMr0CamembertLayer.__init__s '-'E'E$+F>O>O[de ++#)#=#=  # # #??? D6)g!hii"4##' #D  2&9%f- rOc<V^8dQhRS[PRS[PR,RS[PR,RS[PR,RS[R,RS[S[,RS[P/#r)r?rYrWr rr)rZr[s"rMr\CamembertLayer.__annotate__s%%||%))D0% %0047 % !& 1 1D 8 %  %+,% %rOc 8VP!VV3RV/VBwrxTp VP'dBVe>\VR4'g\RV R24hVP!VRVV3RV/VBwrT p \ VP VPVPV 4p V #)rNrz'If `encoder_hidden_states` are passed, z` has to be instantiated with cross-attention layers by setting `config.add_cross_attention=True`) rrrerrrfeed_forward_chunkrr) rJrrrrrrself_attention_output_rcross_attention_output layer_outputs &&&&&&, rMrnCamembertLayer.forwards$(>>  $ ,$  $   1 ???4@4!122 =dVDDD )-(;(;%%& ) !0 )  ) % " 6 0  # #T%A%A4CSCSUe rOcJVPV4pVPW!4pV#r)rr)rJrintermediate_outputr s&& rMr!CamembertLayer.feed_forward_chunks)"//0@A{{#6I rO)rrrrrrrrrr) r|r}r~rr0rnrrrrrs@@rMrrs.&%%NrOrc<aa]tRtRtoRtV3RltRtRtVtV;t #)CamembertLMHeadiz,Camembert Head for masked language modeling.c<\SV`4\P!VPVP4Vn\P !VPVPR7Vn\P!VPVP4Vn \P!\P!VP44VnR#r)r/r0r1rr4rr9r: layer_normr3decoder Parameterr?rDbiasrIs&&rMr0CamembertLMHead.__init__s YYv1163E3EF ,,v'9'9v?T?TUyy!3!3V5F5FG LLV->->!?@ rOc VPV4p\V4pVPV4pVPV4pV#r)rr r(r)rJfeaturesrxs&&, rMrnCamembertLMHead.forwards; JJx  G OOA  LLOrO)r+r)rr( r|r}r~rrr0rnrrrrs@@rMr&r&s6ArOr&caa]tRtRto]tRtRtRtRt Rt Rt R] R] R]/t]P"!4V3Rl4tRtVtV;t#) CamembertPreTrainedModelirobertaTr attentionscross_attentionsc <\SV`V4\V\4'd#\P !VP 4R#\V\4'd\P!VP\P!VPPR,4PR44\P !VP4R#R#)zInitialize the weightsNr.r,)r/ _init_weightsrr&initzeros_r+r%copy_rCr?r@rgrBrG)rJrrLs&&rMr9&CamembertPreTrainedModel._init_weightss f% fo . . KK $  3 4 4 JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h KK-- .5rO)r|r}r~rr# config_classbase_model_prefixsupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attn_supports_attention_backendrrr_can_record_outputsr?no_gradr9rrrrs@@rMr4r4se"L!&*#N"&,3  ]]_///rOr4cHaa]tRtRtoV3RltRV3RlRlltRtVtV;t#)CamembertEncoderic <\SV`4Wn\P!\ VP 4Uu.uFp\WR7NK up4VnR#uupi))rN) r/r0rKr1 ModuleListrangenum_hidden_layersrlayer)rJrKirLs&& rMr0CamembertEncoder.__init__sI  ]]QVW]WoWoQp#qQpAN6$GQp#qr #qsA c$<V^8dQhRS[PRS[PR,RS[PR,RS[PR,RS[R,RS[R,RS[S[,R S[S[P,S[,/#) rQrrNrrr use_cacherrU) r?rYrWr boolrrrr)rZr[s"rMr\CamembertEncoder.__annotate__s  || ))D0  %0047  !& 1 1D 8    $; +,  u|| H H rOc \VP4FwrV !VVV3RVRV/VBpK \TV'dVR7#RR7#)rrN)last_hidden_stater) enumeraterNr) rJrrrrrrRrrO layer_modules &&&&&&&, rMrnCamembertEncoder.forwardsn )4OA(%(>  !0   M 59+/8O  >B  rO)rKrN)NNNNNrrs@@rMrIrIss    rOrIcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)CamembertPooleri"c<\SV`4\P!VPVP4Vn\P !4VnR#r)r/r0r1rr4rTanh activationrIs&&rMr0CamembertPooler.__init__#s9 YYv1163E3EF '')rOcN<V^8dQhRS[PRS[P/#rr)rZr[s"rMr\CamembertPooler.__annotate__(s#U\\ellrOc\VR,pVPV4pVPV4pV#)NNN)rcr)rr^)rJrfirst_token_tensor pooled_outputs&& rMrnCamembertPooler.forward(s2+40 #56  6 rO)r^rrrs@@rMr[r["s$ rOr[a  The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between the self-attention layers, following the architecture described in [Attention is all you need](https://huggingface.co/papers/1706.03762) by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin. To behave as an decoder the model needs to be initialized with the `is_decoder` argument of the configuration set to `True`. To be used in a Seq2Seq model, the model needs to initialized with both `is_decoder` argument and `add_cross_attention` set to `True`; an `encoder_hidden_states` is then expected as an input to the forward pass. ) custom_introcaa]tRtRtoRR.tR V3RlltRtRt]] ] R V3RlRll444t R t R t VtV;t#) CamembertModeli1r%rc <\SV`V4WnRVn\ V4Vn\ V4VnV'd \V4MRVn VP4R#)z^ add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer FN) r/r0rKgradient_checkpointingr%rmrIencoderr[pooler post_init)rJrKadd_pooling_layerrLs&&&rMr0CamembertModel.__init__@sS   &+#-f5'/ 1Bof-  rOc.VPP#rrmr6rJs&rMget_input_embeddings#CamembertModel.get_input_embeddingsQs...rOc&WPnR#rrr)rJrs&&rMset_input_embeddings#CamembertModel.set_input_embeddingsTs */'rOc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[R,R S[R,R S[S[,R S[S[P,S[,/ #) rQrRNrrGrCrSrrrrRrrU)r?rYr rSrrrr)rZr[s"rMr\CamembertModel.__annotate__Zs? ? <<$&?  t+?  t+ ? llT) ? ||d* ?  %||d2? !& t 3? ? $;? +,?  u|| K K? rOc  VRJVRJ, 'd \R4hVPP'dV eT MVPPp MRp V 'doVfkVfVPP'd5\ \ VPR7\ VPR74M\ VPR7pVeVP4M^p VPVVVVV R7p VPVVV VVR7wr'VP!V 3RVRVRVR VR V R V/V Bp V PpVPeVPV4MRp\VVV PR 7#) Nz:You must specify exactly one of input_ids or inputs_embedsF)rK)rRrCrGrSrT)rrembedding_outputrrrrrrrRrC)rV pooler_outputr)rrKrrRis_encoder_decoderr r get_seq_lengthrm_create_attention_masksrlrVrmrr)rJrRrrGrCrSrrrrRrrTr|encoder_outputssequence_outputres&&&&&&&&&&, rMrnCamembertModel.forwardWs  -t"; < <YZ Z ;; ! ! !%.%:  @U@UII 0)4 8V8V8V$L $DlZ^ZeZeFfg!5  FUE`!?!?!Afg??%)'#9 + 261M1M)#9-"7+ 2N2 .,,   )  #8  $:  ,    &    *;;8< 8O O4UY ;-'+;;  rOcVPP'd\VPVVVR7pM\VPVVR7pVe\VPVVVR7pW3#))rKrSrr)rKrSr)rKrSrr)rKrrr)rJrrr|rrs&&&&&&rMr&CamembertModel._create_attention_maskssu ;; ! ! !/{{.- / N7{{.-N " -%>{{.5&; & "55rO)rKrmrlrkrm)T) NNNNNNNNN)r|r}r~r_no_split_modulesr0rtrwr!r"rrnrrrrrs@@rMriri1sV/0@A"/0 ? ?  ? B66rOrictaa]tRtRtoRRRR/tV3RltRtRt]] R V3R lR ll44t R t Vt V;t #) CamembertForMaskedLMilm_head.decoder.weight)roberta.embeddings.word_embeddings.weightlm_head.decoder.bias lm_head.biasc<\SV`V4VP'd\P R4\ V4Vn\VRR7VnVP4R#)zpIf you want to use `CamembertForMaskedLM` make sure `config.is_decoder=False` for bi-directional self-attention.FroN r/r0rloggerwarningr&lm_headrir5rnrIs&&rMr0CamembertForMaskedLM.__init__sU      NN1 'v. %fF  rOc.VPP#rrr)rss&rMget_output_embeddings*CamembertForMaskedLM.get_output_embeddings||###rOc&WPnR#rrrJnew_embeddingss&&rMset_output_embeddings*CamembertForMaskedLM.set_output_embeddings - rOc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[S[,R S[S[P ,S[,/ #) rQrRNrrGrCrSrrlabelsrrU)r?rVrWrrrrYr)rZr[s"rMr\!CamembertForMaskedLM.__annotate__s5 5 ##d*5 ))D05 ((4/ 5 &&- 5 ((4/ 5  %00475 !& 1 1D 85   4'5 +,5  u|| ~ -5 rOc  VP!V3RVRVRVRVRVRVRR/V Bp V ^,p VPV 4p R p VebVPV P4p\ 4pV!V P R VP P4VP R 44p \V V V PV PR 7#) a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` rrGrCrSrr return_dictTNlosslogitsrr6r.) r5rrfrbrrrKr3rrr6)rJrRrrGrCrSrrrroutputsrprediction_scoresmasked_lm_lossloss_fcts&&&&&&&&&, rMrnCamembertForMaskedLM.forwards:,,   )  *  &  (  #8  $:      "!* LL9  YY0778F')H%&7&<&\4p V !V PR VP 4VPR 44p M-VPP R 8Xd\4p V !W4p \!V V VP"VP$R 7#) a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rrGrCrSrTN regressionsingle_label_classificationmulti_label_classificationrr.)r5rrfrbrK problem_typerr+r?rFrXrsqueezerrrrrr6 rJrRrrGrCrSrrrrrrrs &&&&&&&, rMrn*CamembertForSequenceClassification.forward:s6,,  ) * &  (     "!*1  YYv}}-F{{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#F3D))-JJ+- B @&++b/R))-II,./'!//))   rO)rrKrr5NNNNNN r|r}r~rr0r rrnrrrrs@@rMrr(s4 C C C C rOrc\aa]tRtRtoV3Rlt]]RV3RlRll44tRtVt V;t #)CamembertForMultipleChoiceic<\SV`V4\P!VP4Vn\P !VP^4Vn\VRR7Vn VP4R#)r-TrN) r/r0r1r;r<r=rr4rrir5rnrIs&&rMr0#CamembertForMultipleChoice.__init__sX  zz&"<"<= ))F$6$6:%fE  rOcZ<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[S[,R S[S[P ,S[,/#) rQrRNrGrrrCrSrrU)r?rVrWrrrrYr)rZr[s"rMr\'CamembertForMultipleChoice.__annotate__sP P ##d*P ((4/P ))D0 P   4' P &&- P ((4/P +,P  u|| 8 8P rOc ^VeVP^,MVP^,pVe"VPR VPR 44MRp Ve"VPR VPR 44MRp Ve"VPR VPR 44MRp Ve"VPR VPR 44MRp Ve2VPR VPR 4VPR 44MRp VP!V 3RV RV RV RV RR/VBpV^,pVP V4pVP V4pVPR V4pRpVe/VP VP4p\4pV!VV4p\VVVPVPR7#) aO input_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) token_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the multiple choice classification loss. Indices should be in `[0, ..., num_choices-1]` where `num_choices` is the size of the second dimension of the input tensors. (See `input_ids` above) position_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_choices, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. NrCrGrrSrTrr.) rgrrEr5r=rrfrbrrrr6)rJrRrGrrrCrSr num_choicesflat_input_idsflat_position_idsflat_token_type_idsflat_attention_maskflat_inputs_embedsrrerreshaped_logitsrrs&&&&&&&, rMrn"CamembertForMultipleChoice.forwardsV-6,Aiooa(}GZGZ[\G] CLCXINN2,>?^bLXLdL--b,2C2CB2GHjnR`Rln11"n6I6I"6MNrvR`Rln11"n6I6I"6MNrv(   r=#5#5b#9=;M;Mb;Q R  ,,  * / /  -        ]3 / ++b+6  YY556F')HOV4D("!//))   rO)rr=r5rrrs@@rMrrs4P P P P rOrc\aa]tRtRtoV3Rlt]]RV3RlRll44tRtVt V;t #)CamembertForTokenClassificationicn<\SV`V4VPVnVPe VPM VPp\ P !V4Vn\ P!VPVP4Vn \VRR7Vn VP4R#)NFr)r/r0rrr<r1r;r=rr4rrir5rnrs&& rMr0(CamembertForTokenClassification.__init__s   ++)/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ%fF  rOcZ<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[S[,R S[S[P ,S[,/#r)r?rVrWrrrrYr)rZr[s"rMr\,CamembertForTokenClassification.__annotate__s2 2 ##d*2 ))D02 ((4/ 2 &&- 2 ((4/ 2   4'2 +,2  u|| 4 42 rOc VP!V3RVRVRVRVRR/VBpV^,p VPV 4p VPV 4p Rp VeXVPV P4p\ 4p V !V P R VP4VP R 44p \V V VPVPR7#) a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. rrGrCrSrTNrr.) r5r=rrfrbrrrrrr6rs &&&&&&&, rMrn'CamembertForTokenClassification.forwards2,,  ) * &  (     "!*,,71  YYv}}-F')HFKKDOO > ##d*> ))D0> ((4/ > &&- > ((4/ > ))D0> ''$.> +,>  u|| ; ;> rOc VP!V3RVRVRVRVRR/VBp V ^,p VPV 4p V P^R R7wrV PR 4P 4p V PR 4P 4p RpVeVe\ VP 44^8dVPR 4p\ VP 44^8dVPR 4pV P ^4pVP^V4pVP^V4p\VR 7pV!W4pV!W4pVV,^, p\VV V V PV PR 7#) a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) rrGrCrSrTruN) ignore_index)r start_logits end_logitsrr6r.) r5rsplitrrlenrEclamprrrr6)rJrRrrGrCrSrrrrrrrr total_loss ignored_indexr start_lossend_losss&&&&&&&&, rMrn%CamembertForQuestionAnswering.forward6s0,,  ) * &  (     "!*1#)<<r<#: #++B/::< ''+668   &=+D?'')*Q."1"9"9""==%%'(1, - 5 5b 9 (--a0M-33A}EO)//=AM']CH!,@J :H$x/14J+%!!//))   rO)rrr5)NNNNNNNrrs@@rMrr*s0> > > > rOrzU Camembert Model with a `language modeling` head on top for CLM fine-tuning. ctaa]tRtRtoRRRR/tV3RltRtRt]] R V3R lR ll44t R t Vt V;t #) CamembertForCausalLMiyrrrrc<\SV`V4VP'g\P R4\ V4Vn\VRR7VnVP4R#)zQIf you want to use `CamembertLMHeadModel` as a standalone, add `is_decoder=True.`FrNrrIs&&rMr0CamembertForCausalLM.__init__sO     NNn o&v. %fF  rOc.VPP#rrrss&rMr*CamembertForCausalLM.get_output_embeddingsrrOc&WPnR#rrrs&&rMr*CamembertForCausalLM.set_output_embeddingsrrOc0<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[S[S[P,,R,R S[R,R S[S[P ,R S[S[,RS[S[P ,S[ ,/ #)rQrRNrrGrCrSrrrrrRlogits_to_keeprrU) r?rVrWrrSrXrYrrr)rZr[s"rMr\!CamembertForCausalLM.__annotate__s+O O ##d*O ))D0O ((4/ O &&- O ((4/ O  %0047O !& 1 1D 8O   4'O uU%6%6784?O $;O ell*O +,O  u|| @ @O rOc  VeRp VP!V3RVRVRVRVRVRVR V R V R R / V Bp V Pp\V \4'd\ V )R4MT pVP VR VR 3,4pRpVe-VP !RRVRVRVPP/V Bp\VVV PV PV PV PR7#)a token_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0,1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. This parameter can only be used when the model is initialized with `type_vocab_size` parameter with value >= 2. All the value in this tensor should be always < type_vocab_size. [What are token type IDs?](../glossary#token-type-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the left-to-right language modeling loss (next word prediction). Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` Example: ```python >>> from transformers import AutoTokenizer, CamembertForCausalLM, AutoConfig >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("almanach/camembert-base") >>> config = AutoConfig.from_pretrained("almanach/camembert-base") >>> config.is_decoder = True >>> model = CamembertForCausalLM.from_pretrained("almanach/camembert-base", config=config) >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> prediction_logits = outputs.logits ```NFrrGrCrSrrrrRrTrcrrr3)rrrrr6r7r>)r5rVrrXslicer loss_functionrKr3rrrr6r7)rJrRrrGrCrSrrrrrRrrrr slice_indicesrrs&&&&&&&&&&&&, rMrnCamembertForCausalLM.forwardsB`  I@D  A ) A * A & A ( A #8 A $: A , A   A  A  A  11 8B>SV8W8W~ot4]k mA}a,?@A  %%pVpFpt{{OeOepiopD0#33!//))$55   rOr) NNNNNNNNNNrrrs@@rMrrysS !"M $.O O O O rOr)Nr)rrrrrrrir4)Ncollections.abcrr?torch.nnr1rrrrr: activationsrr cache_utilsr r r generationr masking_utilsrrmodeling_layersrmodeling_outputsrrrrrrrrmodeling_utilsrrprocessing_utilsr pytorch_utilsrutilsrrr utils.genericr r!utils.output_capturingr"configuration_camembertr# get_loggerr|rrr%rrrrrrrrr&r4rIr[rirrrrrrr__all__r>rOrMrsV,% AA&'CC)J9   G&6@@I54   H %c8"))c8L%8@)RYY@)FI)biiI)X ")) ..: BII  bii >/>Bbii,///2 ryy @ bii   |6-|6 |6~R 3R R j")), Q )AQ  Q h] !9] ] @C &>C C LK $<K K \ i 3_i  i X rO