+ jRt^RIt^RIHt^RIHtHtHt^RIHt ^RI H t H t H t ^RIHtHtHtHtHtHtHtHt^RIHt^RIHt^RIHtHtHt^R IH t H!t!^R I"H#t#^R I$H%t%H&t&H't'H(t(H)t)H*t*H+t+H,t,H-t-H.t.H/t/H0t0H1t1H2t2H3t3H4t4H5t5^R I6H7t7]Pp!]94t:!R R]&4t;!RR]54t<!RR]%4t=!RR]04t>!RR]44t?!RR]24t@!RR]'4tA]!RR]44tB!RR]34tC!RR ],4tD!R!R"]+4tE!R#R$]14tF!R%R&](4tG!R'R(]*4tH!R)R*].4tI!R+R,])4tJ!R-R.]/4tK!R/R0]-4tL.R1OtMR#)2zPyTorch ERNIE model.N)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)Cache DynamicCacheEncoderDecoderCache),BaseModelOutputWithPoolingAndCrossAttentions!CausalLMOutputWithCrossAttentionsMaskedLMOutputMultipleChoiceModelOutputNextSentencePredictorOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)PreTrainedModel)Unpack)TransformersKwargsauto_docstringlogging)can_return_tuplemerge_with_config_defaults)capture_outputs)BertCrossAttentionBertEmbeddings BertEncoderBertForMaskedLMBertForMultipleChoiceBertForNextSentencePredictionBertForPreTrainingBertForPreTrainingOutputBertForQuestionAnsweringBertForSequenceClassificationBertForTokenClassification BertLayerBertLMHeadModelBertLMPredictionHead BertModel BertPoolerBertSelfAttention) ErnieConfigcLaa]tRt^>toRtV3RltRV3RlRlltRtVtV;t #)ErnieEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.c<\SV`V4VPVnVP'd3\P!VP VP 4VnR#R#)N)super__init__ use_task_idnn Embeddingtask_type_vocab_size hidden_sizetask_type_embeddings)selfconfig __class__s&&h/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/ernie/modular_ernie.pyr/ErnieEmbeddings.__init__AsO  !--    (* V5P5PRXRdRd(eD % c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[RS[P/#)  input_idsNtoken_type_ids task_type_ids position_ids inputs_embedspast_key_values_lengthreturn)torch LongTensor FloatTensorintTensor)format __classdict__s"r9 __annotate__ErnieEmbeddings.__annotate__Hs33##d*3((4/3''$. 3 &&- 3 ((4/ 3!$3 3r;cVeVP4pMVP4RRpVwrVfVPRWiV,13,pVf\VR4'dYVPP VP ^,R4p \ P!V ^VR7p V P W4pM;\ P!V\ PVPPR7pVfVPV4pVPV4p VPV P4pW[,p VPV4p W,p VP'dYVf<\ P!V\ PVPPR7pVP!V4pW, p VP#V 4p VP%V 4p V #)NNNNr?)dimindex)dtypedevice)sizerAhasattrr?expandshaperEgatherzeroslongrSword_embeddingstoken_type_embeddingstoposition_embeddingsr0r5 LayerNormdropout)r6r>r?r@rArBrC input_shape batch_size seq_lengthbuffered_token_type_idsr] embeddingsr_r5s&&&&&&& r9forwardErnieEmbeddings.forwardHs  #..*K',,.s3K!,  ,,Q0FVlIl0l-lmL  !t-..*.*=*=*D*D\EWEWXYEZ\^*_'*/,,7NTU]i*j'!8!?!? !W!&[ SWSdSdSkSk!l   00;M $ : :> J&(()>)E)EF ": "66|D5     $ % KuzzRVRcRcRjRj k #'#<#<]#K  .J^^J/ \\*- r;)r5r0)NNNNN) __name__ __module__ __qualname____firstlineno____doc__r/rg__static_attributes____classdictcell__ __classcell__r8rKs@@r9r,r,>sQf333r;r,c]tRt^~tRtR#)ErnieSelfAttentionNrjrkrlrmrorur;r9rtrt~r;rtc]tRt^tRtR#)ErnieCrossAttentionruNrvrur;r9ryryrwr;ryc]tRt^tRtR#) ErnieLayerruNrvrur;r9r{r{rwr;r{c]tRt^tRtR#) ErniePoolerruNrvrur;r9r}r}rwr;r}c]tRt^tRtR#)ErnieLMPredictionHeadruNrvrur;r9rrrwr;rc]tRt^tRtR#) ErnieEncoderruNrvrur;r9rrrwr;rcaa]tRt^to]tRtRtRtRt Rt Rt R] R] R]/t]P"!4V3Rl4tRtVtV;t#)ErniePreTrainedModelernieT hidden_states attentionscross_attentionsc <\SV`V4\V\4'd#\P !VP 4R#\V\4'd\P!VP\P!VPPR,4PR44\P !VP4R#R#)zInitialize the weightsNrT)rT)r. _init_weights isinstancerinitzeros_biasr,copy_rArEarangerXrWr?)r6moduler8s&&r9r"ErniePreTrainedModel._init_weightss f% f3 4 4 KK $  0 0 JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h KK-- .1r;ru)rjrkrlrmr* config_classbase_model_prefixsupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attn_supports_attention_backendr{rtry_can_record_outputsrEno_gradrrorprqrrs@@r9rrseL&*#N"&(/  ]]_///r;rcpaa]tRt^toR.tRV3Rllt]]]RV3RlRll444t Rt Vt V;t #) ErnieModelr{c<\SV`W4WnRVn\ V4Vn\ V4VnV'd \V4MRVn VP4R#)FN) r.r/r7gradient_checkpointingr,rfrencoderr}pooler post_init)r6r7add_pooling_layerr8s&&&r9r/ErnieModel.__init__sQ & &+#)&1#F+ ->k&)D  r;c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[R,R S[R,R S[S[,R S[S[P,S[,/ #)r=r>Nattention_maskr?r@rArBencoder_hidden_statesencoder_attention_maskpast_key_values use_cachekwargsrD)rErIrboolrrtupler )rJrKs"r9rLErnieModel.__annotate__sH H <<$&H  t+H  t+ H ||d* H llT) H ||d*H  %||d2H !& t 3H H $;H +,H  u|| K KH r;c  VRJVRJ, 'd \R4hVPP'dV eT MVPPp MRp V 'doV fkVfVPP'd5\ \ VPR7\ VPR74M\ VPR7p V eV P4M^p VPVVVVVV R7p VPVVV VV R7wr(VP!V 3RVRVR VR V R V R V/V BpV^,pVPeVPV4MRp\VVVPR 7#) task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] Nz:You must specify exactly one of input_ids or inputs_embedsF)r7)r>rAr?r@rBrC)rrembedding_outputrrrrrrrrA)last_hidden_state pooler_outputr) ValueErrorr7 is_decoderris_encoder_decoderrrget_seq_lengthrf_create_attention_masksrrr r)r6r>rr?r@rArBrrrrrrCrencoder_outputssequence_output pooled_outputs&&&&&&&&&&&, r9rgErnieModel.forwards0  -t"; < <YZ Z ;; ! ! !%.%:  @U@UII 0)4 8V8V8V$L $DlZ^ZeZeFfg!5  FUE`!?!?!Afg??%)''#9+ 261M1M)#9-"7+ 2N2 .,,   )  #8  $:  ,    &    *!,8< 8O O4UY ;-'+;;  r;)r7rfrrr)T) NNNNNNNNNN) rjrkrlrm_no_split_modulesr/rrrrgrorprqrrs@@r9rrsD%  H H  H H r;rc]tRtRtRtR#)ErnieForPreTrainingOutputiruNrvrur;r9rrrwr;rcVa]tRtRtoRRRR/t]]R V3RlRll44tR tVt R#) ErnieForPreTrainingicls.predictions.decoder.biascls.predictions.biascls.predictions.decoder.weight'ernie.embeddings.word_embeddings.weightNc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[S[,R S[S[P,S[,/ #) r=r>Nrr?r@rArBlabelsnext_sentence_labelrrD)rErIrrrr)rJrKs"r9rL ErnieForPreTraining.__annotate__sH H <<$&H  t+H  t+ H ||d* H llT) H ||d*H  t#H #\\D0H +,H  u|| 8 8H r;c  VP!V3RVRVRVRVRVRR/V Bp V R,wrVPW4wrR pVe|Vex\4pV!V PR VPP 4VPR 44pV!VPR ^4VPR 44pVV,p\ VV VV PV PR 7#) aj task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` next_sentence_label (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the next sequence prediction (classification) loss. Input should be a sequence pair (see `input_ids` docstring) Indices should be in `[0, 1]`: - 0 indicates sequence B is a continuation of sequence A, - 1 indicates sequence B is a random sequence. Example: ```python >>> from transformers import AutoTokenizer, ErnieForPreTraining >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> model = ErnieForPreTraining.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> prediction_logits = outputs.prediction_logits >>> seq_relationship_logits = outputs.seq_relationship_logits ``` rr?r@rArB return_dictT:Nr=NN)lossprediction_logitsseq_relationship_logitsrrrT) rclsrviewr7 vocab_sizerrr)r6r>rr?r@rArBrrroutputsrrprediction_scoresseq_relationship_score total_lossloss_fctmasked_lm_lossnext_sentence_losss&&&&&&&&&, r9rgErnieForPreTraining.forwards^**   )  *  (  &  (      *1&48HH_4\1  "5"A')H%&7&<&(*S H H H r;rcJa]tRtRto]]RV3RlRll44tRtVtR#)ErnieForCausalLMieNcH<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[PR,R S[S[P,R,R S[R,R S[S[P,RS[S[,RS[S[P,S[,/#)r=r>Nrr?r@rArBrrrrrlogits_to_keeprrD) rErIlistrrHrrrr )rJrKs"r9rLErnieForCausalLM.__annotate__hs= = <<$&=  t+=  t+ = ||d* = llT) = ||d*=  %||d2= !& t 3=  t#= ell+d2= $;= ell*= +,=  u|| @ @= r;c  V eRp VP!V3RVRVRVRVRVRVR VR V R V R R / V BpVPp\V \4'd\ V )R4MT pVP VRVR3,4pRpV e-VP !RRVRV RVPP/V Bp\VVVPVPVPVPR7#)aA task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the left-to-right language modeling loss (next word prediction). Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` NFrr?r@rArBrrrrrTrOlogitsrr)rrrrrrru)rrrrHslicer loss_functionr7rr rrrr)r6r>rr?r@rArBrrrrrrrrr slice_indicesrrs&&&&&&&&&&&&&, r9rgErnieForCausalLM.forwardfsL:  I@D  A ) A * A ( A & A ( A #8 A $: A , A   A  A  A  11 8B>SV8W8W~ot4]k -=!(;<=  %%pVpFpt{{OeOepiopD0#33!//))$55   r;ru) NNNNNNNNNNNri rjrkrlrmrrrgrorprs@r9rres$= = = r;rcVa]tRtRtoRRRR/t]]R V3RlRll44tR tVt R#) ErnieForMaskedLMirrrrNc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[PR,R S[S[,R S[S[P,S[,/ #) r=r>Nrr?r@rArBrrrrrD)rErIrrrr )rJrKs"r9rLErnieForMaskedLM.__annotate__s2 2 <<$&2  t+2  t+ 2 ||d* 2 llT) 2 ||d*2  %||d22 !& t 32  t#2 +,2  u|| ~ -2 r;c  ZVP!V3RVRVRVRVRVRVRVRR /V Bp V ^,p VPV 4p R pV eG\4pV!V PR VPP 4V PR 44p\ VV V PV PR 7#) a# task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` rr?r@rArBrrrTNrrrrrT) rrrrr7rr rr)r6r>rr?r@rArBrrrrrrrrrs&&&&&&&&&&, r9rgErnieForMaskedLM.forwards4**   )  *  (  &  (  #8  $:      "!* HH_5  ')H%&7&<&(*S 2 2 2 r;rcJa]tRtRto]]RV3RlRll44tRtVtR#)ErnieForNextSentencePredictioniNc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[S[,R S[S[P,S[,/ # r=r>Nrr?r@rArBrrrD)rErIrrrr )rJrKs"r9rL+ErnieForNextSentencePrediction.__annotate__sD D <<$&D  t+D  t+ D ||d* D llT) D ||d*D  t#D +,D  u|| : :D r;c *VP!V3RVRVRVRVRVRR/VBp V ^,p VPV 4p Rp Ve3\4p V !V PR ^4VPR 44p \ V V V P V P R 7#) a task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the next sequence prediction (classification) loss. Input should be a sequence pair (see `input_ids` docstring). Indices should be in `[0, 1]`: - 0 indicates sequence B is a continuation of sequence A, - 1 indicates sequence B is a random sequence. Example: ```python >>> from transformers import AutoTokenizer, ErnieForNextSentencePrediction >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> model = ErnieForNextSentencePrediction.from_pretrained("nghuyong/ernie-1.0-base-zh") >>> prompt = "In Italy, pizza served in formal settings, such as at a restaurant, is presented unsliced." >>> next_sentence = "The sky is blue due to the shorter wavelength of blue light." >>> encoding = tokenizer(prompt, next_sentence, return_tensors="pt") >>> outputs = model(**encoding, labels=torch.LongTensor([1])) >>> logits = outputs.logits >>> assert logits[0, 0] < logits[0, 1] # next sentence was random ``` rr?r@rArBrTNrrT)rrrrr rr)r6r>rr?r@rArBrrrrseq_relationship_scoresrrs&&&&&&&&, r9rg&ErnieForNextSentencePrediction.forwardsZ**   )  *  (  &  (        "&((="9!  ')H!)*A*F*Fr1*Mv{{[]!_ *#*!//))   r;ruNNNNNNNrrs@r9rrs'D D D r;rcJa]tRtRto]]RV3RlRll44tRtVtR#)ErnieForSequenceClassificationi/Nc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[S[,R S[S[P,S[,/ #r)rErIrrrr)rJrKs"r9rL+ErnieForSequenceClassification.__annotate__2sB B <<$&B  t+B  t+ B ||d* B llT) B ||d*B  t#B +,B  u|| 7 7B r;c VP!V3RVRVRVRVRVRR/VBp V ^,p VPV 4p VPV 4p Rp VEeVPPfVP ^8XdR VPnMrVP ^8dQVP \P8Xg VP \P8XdR VPnMR VPnVPPR 8XdK\4p VP ^8Xd'V !V P4VP44p MV !W4p MVPPR 8Xd>\4p V !V PR VP 4VPR 44p M-VPPR 8Xd\4p V !W4p \V V V P V P"R 7#)a task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rr?r@rArBrTN regressionsingle_label_classificationmulti_label_classificationrrT)rra classifierr7 problem_type num_labelsrRrEr[rHrsqueezerrrrrr)r6r>rr?r@rArBrrrrrrrs&&&&&&&&, r9rg&ErnieForSequenceClassification.forward0s0**   )  *  (  &  (         ]3 /  {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#F3D))-JJ+- B @&++b/R))-II,./'!//))   r;rurrrs@r9rr/s'B B B r;rcJa]tRtRto]]RV3RlRll44tRtVtR#)ErnieForMultipleChoiceiwNc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[S[,R S[S[P,S[,/ #r)rErIrrrr )rJrKs"r9rL#ErnieForMultipleChoice.__annotate__zsU U <<$&U  t+U  t+ U ||d* U llT) U ||d*U  t#U +,U  u|| 8 8U r;c *VeVP^,MVP^,p Ve"VPR VPR 44MRpVe"VPR VPR 44MRpVe"VPR VPR 44MRpVe"VPR VPR 44MRpVe2VPR VPR 4VPR 44MRpVP!V3RVRVRVRVRVRR/VBp V ^,p VP V 4p VP V 4p V PR V 4p RpVe\ 4pV!W4p\VV V PV PR 7#) aQ input_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) token_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) task_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] position_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_choices, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the multiple choice classification loss. Indices should be in `[0, ..., num_choices-1]` where `num_choices` is the size of the second dimension of the input tensors. (See `input_ids` above) Nrr?r@rArBrTrrT) rXrrUrrarrr rr)r6r>rr?r@rArBrr num_choicesrrrreshaped_logitsrrs&&&&&&&&, r9rgErnieForMultipleChoice.forwardxs`-6,Aiooa(}GZGZ[\G] >G>SINN2y~~b'9:Y] M[Mg,,R1D1DR1HImqM[Mg,,R1D1DR1HImqGSG_|((\->->r-BCei (   r=#5#5b#9=;M;Mb;Q R  **   )  *  (  &  (         ]3 / ++b+6  ')HO4D("!//))   r;rurrrs@r9r r ws'U U U r;r cJa]tRtRto]]RV3RlRll44tRtVtR#)ErnieForTokenClassificationiNc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[S[,R S[S[P,S[,/ #r)rErIrrrr)rJrKs"r9rL(ErnieForTokenClassification.__annotate__s. . <<$&.  t+.  t+ . ||d* . llT) . ||d*.  t#. +,.  u|| 4 4. r;c `VP!V3RVRVRVRVRVRR/VBp V ^,p VPV 4p VPV 4p Rp Ve=\4p V !V P R VP 4VP R 44p \ V V V PV PR 7#) al task_type_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Task type embedding is a special embedding to represent the characteristic of different tasks, such as word-aware pre-training task, structure-aware pre-training task and semantic-aware pre-training task. We assign a `task_type_id` to each task and the `task_type_id` is in the range `[0, config.task_type_vocab_size-1] labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. rr?r@rArBrTNrrT) rrarrrrrrr)r6r>rr?r@rArBrrrrrrrs&&&&&&&&, r9rg#ErnieForTokenClassification.forwards,**   )  *  (  &  (      "!*,,71  ')HFKKDOONrr?r@rArBstart_positions end_positionsrrD)rErIrrrr)rJrKs"r9rL&ErnieForQuestionAnswering.__annotate__ s< < <<$&<  t+<  t+ < ||d* < llT) < ||d*< ,< ||d*< +,<  u|| ; ;< r;c  VP!V3RVRVRVRVRVRR/V Bp V ^,p VPV 4p V P^R R7wrV PR 4P 4p VPR 4P 4pR pVeVe\ VP 44^8dVPR 4p\ VP 44^8dVPR 4pV P ^4pVP^V4pVP^V4p\VR 7pV!W4pV!W4pVV,^, p\VV VV PV PR 7#) rrr?r@rArBrT)rPN) ignore_index)r start_logits end_logitsrrrT) r qa_outputssplitr contiguouslenrUclamprrrr)r6r>rr?r@rArBrrrrrrrr r ignored_indexr start_lossend_losss&&&&&&&&&, r9rg!ErnieForQuestionAnswering.forwards***   )  *  (  &  (      "!*1#)<<r<#: #++B/::< ''+668   &=+D?'')*Q."1"9"9""==%%'(1, - 5 5b 9 (--a0M-33A}EO)//=AM']CH!,@J :H$x/14J+%!!//))   r;rurrrs@r9rrs$< < < r;r) rrr rrrrrrr)NrnrEtorch.nnr1rrrrr cache_utilsrrrmodeling_outputsr r r r r rrrmodeling_utilsrprocessing_utilsrutilsrrr utils.genericrrutils.output_capturingrbert.modeling_bertrrrrrrrr r!r"r#r$r%r&r'r(r)configuration_ernier* get_loggerrjloggerr,rtryr{r}rrrrrrrrrrr rr__all__rur;r9r8s AA&CC   .&@@I5&-   H %=n=@ *  ,    *  0  ; /?//2[ [ | 8 P ,P f@ @ F: : zG %BG TE %BE PX 2X v1 "<1 h? 8? D r;