+ jCRt^RIt^RIt^RIHt^RIHt^RIHt^RI H t ^RI H t ^RI HtHtHt^RIHt^R IHt^R IHtHtHt^R IHt^R IHt^R IHtHtH t H!t!H"t"^RI#H$t$^RI%H&t&^RI'H(t(!RR]"4t)]!RR]44t*!RR] 4t+!RR]4t,]!RR]*44t-]!RR7!RR]*] 44t.!RR ]4t/!R!R"]$4t0!R#R$]!4t1.R%Ot2R#)&zPyTorch PLBART model.N)nn)CrossEntropyLoss)initialization)Cache)GenerationMixin)BaseModelOutputSeq2SeqLMOutputSeq2SeqModelOutput)PreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tuple)merge_with_config_defaults)capture_outputs)BartClassificationHead BartDecoder BartEncoderBartForCausalLMBartScaledWordEmbedding)'BigBirdPegasusForSequenceClassification)shift_tokens_right PLBartConfigc]tRt^/tRtR#)PLBartScaledWordEmbeddingN__name__ __module__ __qualname____firstlineno____static_attributes__rj/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/plbart/modular_plbart.pyrr/r#rcZaa]tRt^3toRtRtRR.tRtRtRt V3Rlt V3Rlt Rt Vt V;t#)PLBartPreTrainedModelmodelTPLBartDecoderLayerPLBartEncoderLayerc<\SV`V4\V\4'd#\P !VP 4R#R#N)super _init_weights isinstancePLBartForConditionalGenerationinitzeros_final_logits_bias)selfmodule __class__s&&r$r.#PLBartPreTrainedModel._init_weights=s5 f% f< = = KK00 1 >r#c&<V^8dQh/S[;R&#configr)format __classdict__s"r$ __annotate__"PLBartPreTrainedModel.__annotate__3s r#r)rrr r!base_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_flash_attn_supports_sdpa_supports_flex_attnr.__annotate_func__r"__classdictcell__ __classcell__r6r=s@@r$r'r'3s< &*#-/CDN2r#r'c]tRt^CtRtR#) PLBartEncoderrNrrr#r$rKrKCr%r#rKc]tRt^GtRtR#) PLBartDecoderrNrrr#r$rMrMGr%r#rMcaa]tRt^KtoRRRR/tV3RlV3RlltRtRt]] ] R V3RlR ll444t R t Vt V;t#) PLBartModelzencoder.embed_tokens.weightz shared.weightzdecoder.embed_tokens.weightc <V^8dQhRS[/#r9r)r<r=s"r$r>PLBartModel.__annotate__Rs  | r#cT<\SV`V4VPVPr2VP'd!\ P !VP4MRp\W1PW$R7Vn \V4Vn \V4Vn VP4R#)g?) embed_scaleN)r-__init__ pad_token_id vocab_sizescale_embeddingmathsqrtd_modelrsharedrKencoderrMdecoder post_init)r4r; padding_idxrVrSr6s&& r$rTPLBartModel.__init__Rsv  "("5"5v7H7HZ393I3I3Idii/s / NNKq $V, $V,  r#cVP#r,)r[)r4s&r$get_input_embeddings PLBartModel.get_input_embeddings^s {{r#c~WnVPVPnVPVPnR#r,)r[r\ embed_tokensr])r4values&&r$set_input_embeddings PLBartModel.set_input_embeddingsas' $(KK !$(KK !r#c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[S[P,R,RS[R,RS[PR,R S[PR,R S[R,R S[S[,R S[ S[P,S[ ,/ #) r: input_idsNattention_maskdecoder_input_idsdecoder_attention_maskencoder_outputspast_key_values inputs_embedsdecoder_inputs_embeds use_cachekwargsreturn) torch LongTensorTensorlist FloatTensorrboolr r tupler )r<r=s"r$r>rQisJ J ##d*J ((4/J !++d2 J !& t 3 J e//047 J J ((4/J  %0047J $;J +,J  u|| 1 1J r#c  XVf$Vf \WPP4pVfVP!R RVRVRV/V BpM]\ V\ 4'gH\ V^,\ V4^8d V^,MR\ V4^8d V^,MRR7pVP!R RVRVRV^,RVRVRVR V /V Bp \V PV PV PV PV PVPVPVPR 7#) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`] or [`PLBartMultiTokenizer`] depending on the checkpoint. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) PLBart uses a specific language id token as the starting token for `decoder_input_ids` generation that varies according to source and target language, *e.g.* 50003 for *en_XX*, and 50001 for *java*. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (: obj:*torch.LongTensor* of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. Nrjrkrp)last_hidden_state hidden_states attentionsencoder_hidden_statesencoder_attention_maskrorr)r}rodecoder_hidden_statesdecoder_attentionscross_attentionsencoder_last_hidden_staterencoder_attentionsr)rr;rUr\r/rlenr]r r}ror~rr) r4rjrkrlrmrnrorprqrrrsdecoder_outputss &&&&&&&&&&, r$forwardPLBartModel.forwardfsdP  $)>)F 29kk>V>V W   "/3||0#0-0,0 0O O_==-"1!"4474H14Loa0RV14_1E1I?1-tO ,,  '  1  #2!"4  $2  ,  0       "-??+;;"1"?"?.99,==&5&G&G"1"?"?.99  r#)r]r\r[) NNNNNNNNN)rrr r!_tied_weights_keysrTrbrgrrr rr"rGrHrIs@@r$rOrOKs_ &%  0  J J  J J r#rOzv The PLBART Model with a language modeling head. Can be used for code-to-text, text-to-code and code-to-code. ) custom_introcaa]tRt^toRtR.tRR/tV3RlV3RlltRV3RlV3RllltV3R lR lt ] ] ] RV3R lR ll444t V3R lRltRtVtV;t#)r0r(r3zlm_head.weightzmodel.shared.weightc <V^8dQhRS[/#r9r)r<r=s"r$r>+PLBartForConditionalGeneration.__annotate__s|r#c<\SV`V4\V4Vn\P !\ P!^VPPP344Vn \P!VPVPPPRR7Vn VP4R#)F)biasN)r-rTrOr(rBufferruzerosr[num_embeddingsr3LinearrZlm_headr^)r4r;r6s&&r$rT'PLBartForConditionalGeneration.__init__s~   ( !#5;;4::;L;L;[;[7\+]!^yy1B1B1Q1QX]^  r#cT<V^8dQhRS[RS[R,RS[RS[P/#)r:new_num_tokenspad_to_multiple_ofN mean_resizingrt)intrzr Embedding)r<r=s"r$r>rs5!7:TzY] r#c<\SV`WV4pVPVPP^,4V#))r-resize_token_embeddings_resize_final_logits_biasweightshape)r4rrrnew_embeddingsr6s&&&& r$r6PLBartForConditionalGeneration.resize_token_embeddingss<8]jk &&~'<'<'B'B1'EFr#c$<V^8dQhRS[RR/#)r:rrtN)r)r<r=s"r$r>rs5555r#cbVPPR,pW8:dVPRRV13,pMX\P!^W, 3VPPR7p\P !VPV.^R7p\ P!V4VnR#)r:NNNN)device)dim)r3rrurrcatrr)r4rold_num_tokensnew_bias extra_biass&& r$r8PLBartForConditionalGeneration._resize_final_logits_biass//55b9  +--a..@AHa)H%IRVRhRhRoRopJyy$"8"8*!E1MH!#8!4r#c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[S[P,R,RS[R,RS[PR,R S[PR,R S[PR,R S[R,R S[S[,R S[ S[P,S[ ,/ #)r:rjNrkrlrmrnrorprqlabelsrrrsrt) rurvrwrxryrrzr r r{r)r<r=s"r$r>rs_ _ ##d*_ ((4/_ !++d2 _ !& t 3 _ e//047 _ _ ((4/_  %0047_  t#_ $;_ +,_  u||  ._ r#c  vV e(Vf$Vf \WPP4pVP!V3RVRVRVRVRVRVRVR V /V Bp VP V P 4p WP PV P4,p RpV eG\4pV!V PR VPP4V PR 44p\VV V PV PV PV P V P"V P$V P&R 7 #) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`] or [`PLBartMultiTokenizer`] depending on the checkpoint. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) PLBart uses a specific language id token as the starting token for `decoder_input_ids` generation that varies according to source and target language, *e.g.* 50003 for *en_XX*, and 50001 for *java*. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (: obj:*torch.LongTensor* of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example Mask-filling: ```python >>> from transformers import AutoTokenizer, PLBartForConditionalGeneration >>> model = PLBartForConditionalGeneration.from_pretrained("uclanlp/plbart-base") >>> tokenizer = AutoTokenizer.from_pretrained("uclanlp/plbart-base") >>> # en_XX is the language symbol id for English >>> TXT = " Is 0 the Fibonacci number ? en_XX" >>> input_ids = tokenizer([TXT], add_special_tokens=False, return_tensors="pt").input_ids >>> logits = model(input_ids).logits >>> masked_index = (input_ids[0] == tokenizer.mask_token_id).nonzero().item() >>> probs = logits[0, masked_index].softmax(dim=0) >>> values, predictions = probs.topk(5) >>> tokenizer.decode(predictions).split() ['first', 'same', 'highest', 'result', 'number'] ``` Nrkrlrnrmrorprqrr) losslogitsrorrrrrrr)rr;rUr(rr}r3torrviewrVrrorrrrrr)r4rjrkrlrmrnrorprqrrrrsoutputs lm_logitsmasked_lm_lossloss_fcts&&&&&&&&&&&, r$r&PLBartForConditionalGeneration.forwards]@   (-B-J$6v{{?W?W$X!&*jj  ' ) ' 0 ' , ' $: ' , ' ( ' #8 '   '  ' LL!:!:;  6 6 9 9):J:J KK   ')H%innR9O9O&PRXR]R]^`RabN#33")"?"?&99$55&-&G&G")"?"?&99  r#c4<V^8dQhRS[P/#)r:r)rurw)r<r=s"r$r>r>sDDELLDr#c@\WPP4#r,)rr;rU)r4rs&&r$%prepare_decoder_input_ids_from_labelsDPLBartForConditionalGeneration.prepare_decoder_input_ids_from_labels>s!&++*B*BCCr#)r3rr()NT) NNNNNNNNNN)rrr r!r@_keys_to_ignore_on_load_missingrrTrrrrr rrr"rGrHrIs@@r$r0r0s  ':&;#/55 _ _  _ BDDDr#r0c]tRtRtRtR#)PLBartClassificationHeadiBrNrrr#r$rrBr%r#rc2aa]tRtRtoV3RltRtVtV;t#)PLBartForSequenceClassificationiFc <<\4P!R/VBR#)a  decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`] or [`PLBartMultiTokenizer`] depending on the checkpoint. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) PLBart uses a specific language id token as the starting token for `decoder_input_ids` generation that varies according to source and target language, *e.g.* 50003 for *en_XX*, and 50001 for *java*. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (: obj:*torch.LongTensor* of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr-r super_kwargsr6s,r$r'PLBartForSequenceClassification.forwardGs4 ','r#r)rrr r!rr"rGrHrIs@@r$rrFs((r#rcFaa]tRtRto]]V3Rl44tRtVtV;t #)PLBartForCausalLMidc <<\4P!R/VBR#)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, PLBartForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("uclanlp/plbart-base") >>> model = PLBartForCausalLM.from_pretrained("uclanlp/plbart-base") >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nrrrs,r$rPLBartForCausalLM.forwardes2 ','r#r) rrr r!rr rr"rGrHrIs@@r$rrds$(((r#r)rr0rrOr')3__doc__rXrurtorch.nnrrr1 cache_utilsr generationrmodeling_outputsrrr modeling_utilsr processing_utilsr utilsr r r utils.genericrutils.output_capturingrbart.modeling_bartrrrrr(bigbird_pegasus.modeling_bigbird_pegasusrmbart.modeling_mbartrconfiguration_plbartrrr'rKrMrOr0rrr__all__rr#r$rs %& ) .&II75_5. 7  2O 2 2 K  K g 'g g T DD%:ODD  DDN 5 (&M(<((: r#