+ j{{J^RIt^RIt^RIHt^RIHt^RIHt^RIH t ^RI H t H t H t ^RIHt^RIHtHt^R IHt^R IHtHtHt^R IHt^R IHt^R IHt^RIH t H!t!H"t"^RI#H$t$^RI%H&t&H't'^RI(H)t)]"PT!]+4t,!RR]PZ4t.!RR]PZ4t/!RR]PZ4t0!RR]PZ4t1!RR]PZ4t2!RR]PZ4t3!RR]4t4!R R!]PZ4t5!R"R#]PZ4t6!R$R%]PZ4t7!R&R']PZ4t8!R(R)]PZ4t9!R*R+]4t:!R,R-]:4t;!R.R/]:]4t<.R0Ot=R#)1N)nn)CrossEntropyLoss)initialization)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)create_bidirectional_maskcreate_causal_mask)GradientCheckpointingLayer))BaseModelOutputWithPastAndCrossAttentions,BaseModelOutputWithPoolingAndCrossAttentions!CausalLMOutputWithCrossAttentions)PreTrainedModel)Unpack)apply_chunking_to_forward)TransformersKwargscan_return_tuplelogging)merge_with_config_defaults)OutputRecordercapture_outputsBlipTextConfigcLaa]tRt^.toRtV3RltRV3RlRlltRtVtV;t #)BlipTextEmbeddingsz;Construct the embeddings from word and position embeddings.cH<\SV`4\P!VPVP VP R7Vn\P!VPVP 4Vn \P!VP VPR7Vn \P!VP4Vn\P!\ P"!VP4P%R4RR7VnWnR#)) padding_idxepsF) persistentN)super__init__r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutBuffertorcharangeexpand position_idsconfigselfr9 __class__s&&l/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/blip/modeling_blip_text.pyr'BlipTextEmbeddings.__init__1s !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c f&8&8f>S>STzz&"<"<= IIell63Q3Q&R&Y&YZa&botu c <V^8dQhRS[PR,RS[PR,RS[PR,RS[RS[P/#) input_idsNr8 inputs_embedspast_key_values_lengthreturn)r5 LongTensor FloatTensorintTensor)format __classdict__s"r= __annotate__BlipTextEmbeddings.__annotate__>sb##d*&&-((4/  !$   r?cNVeVP4pMVP4RRpV^,pVfVPRWFV,13,pVfVPV4pTpVPV4pWx, pVP V4pVP V4pV#)NNNNr%)sizer8r,r.r/r3) r;rBr8rCrD input_shape seq_length embeddingsr.s &&&&& r=forwardBlipTextEmbeddings.forward>s  #..*K',,.s3K ^  ,,Q0FVlIl0l-lmL   00;M" "66|D) ^^J/ \\*- r?)r/r9r3r.r8r,)NNNr) __name__ __module__ __qualname____firstlineno____doc__r'rT__static_attributes____classdictcell__ __classcell__r<rKs@@r=rr.sE r?rcdaa]tRt^]toR V3RlltRtRtRtRtR V3RlRllt Rt Vt V;t #) BlipTextSelfAttentionc<\SV`4WnVPVP,^8wd;\ VR4'g)\ RVPVP3,4hVPVn\VPVP, 4VnVPVP,Vn W0n \P!VPVP4Vn V'db\P!VPVP4Vn\P!VPVP4VnM`\P!VPVP4Vn\P!VPVP4Vn\P"!VP$4VnR#)rembedding_sizezLThe hidden size (%d) is not a multiple of the number of attention heads (%d)N)r&r'r9r*num_attention_headshasattr ValueErrorrHattention_head_size all_head_size layer_idxrLinearqueryencoder_hidden_sizekeyvaluer1attention_probs_dropout_probr3r;r9is_cross_attentionrhr<s&&&&r=r'BlipTextSelfAttention.__init__^s_     : : :a ?PVXhHiHi^%%v'A'ABC  $*#=#= #&v'9'9F3E/>  &/+>??,77;;DNNK %+:+P+P(+:+O+O('6$2D.- /"=*,33DNNCHHI.55dnnELLKQ--cr2QBQ8P8PQH055h?II!QOI**^499(CMMaQRSK*)=)D)DY]a]k]k)l& %*_FY*Z*ZAEO..t~~>!<< 5H5HR5PQ+dii8P8P.QQ  %/.2C2CDTD[D[2\\ **,-=>#',,"? %S"S%**,CD o--r?) rgrfr~rur9r3rlrhrcrjrmrsNNNN) rVrWrXrYr'rvrzrrrTr[r\r]r^s@@r=r`r`]s1G0-#+"D.D.D.r?r`cDaa]tRt^toV3RltV3RlRltRtVtV;t#)BlipTextSelfOutputc0<\SV`4\P!VPVP4Vn\P !VPVPR7Vn\P!VP4Vn R#r N) r&r'rrir*denser/r0r1r2r3r:s&&r=r'BlipTextSelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= r?ch<V^8dQhRS[PRS[PRS[P/#rAr input_tensorrEr5rI)rJrKs"r=rLBlipTextSelfOutput.__annotate__1U\\RWR^R^r?cxVPV4pVPV4pVPW,4pV#rsrr3r/r;rrs&&&r=rTBlipTextSelfOutput.forward5 =1  ]3 }'CD r?r/rr3 rVrWrXrYr'rTr[r\r]r^s@@r=rr> r?rcLaa]tRt^toRV3RlltRV3RlRlltRtVtV;t#)BlipTextAttentionch<\SV`4\WVR7Vn\ V4VnR#)rhN)r&r'r`r;routputros&&&&r=r'BlipTextAttention.__init__s) )&PYZ (0 r?c<V^8dQhRS[PRS[PR,RS[PR,RS[R,RS[S[,RS[S[PS[P3,/#)rArrNrrrrEr)rJrKs"r=rLBlipTextAttention.__annotate__s}11||1))D01 %0047 1  1 +, 1 u||U\\) *1r?c XVPVVVVR7wrgVPWa4pW3#)rrr)r;r) r;rrrrrrrattention_outputs &&&&&, r=rTBlipTextAttention.forwards@*. )"7+ *3* &  ;;}D00r?)rr;FN)NNNrr^s@@r=rrs1 111r?rcDaa]tRt^toV3RltV3RlRltRtVtV;t#)BlipTextIntermediatec"<\SV`4\P!VPVP 4Vn\VP\4'd\VP,Vn R#VPVn R#rs) r&r'rrir*intermediate_sizerr hidden_actstrrintermediate_act_fnr:s&&r=r'BlipTextIntermediate.__init__s` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $r?cN<V^8dQhRS[PRS[P/#rArrEr)rJrKs"r=rL!BlipTextIntermediate.__annotate__s#U\\ellr?cJVPV4pVPV4pV#rsrrr;rs&&r=rTBlipTextIntermediate.forwards& =1 00? r?rrr^s@@r=rrs9r?rcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)BlipTextOutputic0<\SV`4\P!VPVP 4Vn\P!VP VPR7Vn\P!VP4Vn R#r) r&r'rrirr*rr/r0r1r2r3r:s&&r=r'BlipTextOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= r?ch<V^8dQhRS[PRS[PRS[P/#rr)rJrKs"r=rLBlipTextOutput.__annotate__ rr?cxVPV4pVPV4pVPW,4pV#rsrrs&&&r=rTBlipTextOutput.forward rr?rrr^s@@r=rrrr?rcNaa]tRtRtoV3RltRV3RlRlltRtRtVtV;t #) BlipTextLayericR<\SV`4WnVPVn^Vn\ WR7VnW nVPP'd'\ WPPVR7Vn \V4Vn \V4Vn R#)r$r)rprhN)r&r'r9chunk_size_feed_forward seq_len_dimr attention layer_num is_decodercrossattentionr intermediaterr)r;r9rr<s&&&r=r'BlipTextLayer.__init__s  '-'E'E$*6G" ;; ! ! !"3;;+A+AY#D 18$V, r?c<V^8dQhRS[PRS[PR,RS[PR,RS[PR,RS[R,RS[S[,RS[P/#) rArrNrrrrrE)r5rIrGrrr)rJrKs"r=rLBlipTextLayer.__annotate__s|| %0047))D0  !& 1 1D 8   +, r?c VPVVVR7wrxVeVPVVVVR7wrx\VPVPVP V4p V #))rrr)rrrfeed_forward_chunkrr) r;rrrrrrr_ layer_outputs &&&&&&, r=rTBlipTextLayer.forwards#nn )+-  ! ,"&"5"5 5&; / #6#   1  # #T%A%A4CSCSUe r?cJVPV4pVPW!4pV#rs)rr)r;rintermediate_outputrs&& r=r BlipTextLayer.feed_forward_chunk:s)"//0@A{{#6I r?)rrr9rrrrrr) rVrWrXrYr'rTrr[r\r]r^s@@r=rrs -6r?rcHaa]tRtRtoV3RltRV3RlRlltRtVtV;t#)BlipTextEncoderiAc <\SV`4Wn\P!\ VP 4Uu.uFp\W4NK up4VnRVn R#uupir) r&r'r9r ModuleListrangenum_hidden_layersrlayergradient_checkpointing)r;r9ir<s&& r=r'BlipTextEncoder.__init__BsR  ]]eFLdLdFe#fFeM&$  !0   M'9++  r?)r9rr)NNNNNrr^s@@r=rrAs, ( ( ( r?rcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)BlipTextPooleritc<\SV`4\P!VPVP4Vn\P !4VnR#rs)r&r'rrir*rTanh activationr:s&&r=r'BlipTextPooler.__init__us9 YYv1163E3EF '')r?cN<V^8dQhRS[PRS[P/#rr)rJrKs"r=rLBlipTextPooler.__annotate__zs#U\\ellr?c\VR,pVPV4pVPV4pV#)rO)rOr)rr)r;rfirst_token_tensor pooled_outputs&& r=rTBlipTextPooler.forwardzs2+40 #56  6 r?)rrrr^s@@r=rrts$ r?rcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)BlipTextPredictionHeadTransformic<\SV`4\P!VPVP4Vn\ VP\4'd\VP,Vn MVPVn \P!VPVPR7Vn R#r) r&r'rrir*rrrrrtransform_act_fnr/r0r:s&&r=r'(BlipTextPredictionHeadTransform.__init__s~ YYv1163E3EF f'' - -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STr?cN<V^8dQhRS[PRS[P/#rr)rJrKs"r=rL,BlipTextPredictionHeadTransform.__annotate__s#U\\ellr?clVPV4pVPV4pVPV4pV#rs)rr$r/rs&&r=rT'BlipTextPredictionHeadTransform.forwards4 =1 --m< }5 r?)r/rr$rr^s@@r=r"r"sUr?r"c8aa]tRtRtoV3RltRtRtVtV;t#)BlipTextLMPredictionHeadic<\SV`4\V4Vn\P !VP VPRR7Vn\P!\P!VP44Vn R#)T)biasN) r&r'r" transformrrir*r)decoder Parameterr5zerosr-r:s&&r=r'!BlipTextLMPredictionHead.__init__s[ 8@yy!3!3V5F5FTR LLV->->!?@ r?cJVPV4pVPV4pV#rs)r.r/rs&&r=rT BlipTextLMPredictionHead.forwards$}5  ]3 r?)r-r/r.rr^s@@r=r+r+sAr?r+cDaa]tRtRtoV3RltV3RlRltRtVtV;t#)BlipTextOnlyMLMHeadicD<\SV`4\V4VnR#rs)r&r'r+ predictionsr:s&&r=r'BlipTextOnlyMLMHead.__init__s 3F;r?cN<V^8dQhRS[PRS[P/#)rAsequence_outputrEr)rJrKs"r=rL BlipTextOnlyMLMHead.__annotate__s#!!u||! !r?c(VPV4pV#rsr8)r;r;prediction_scoress&& r=rTBlipTextOnlyMLMHead.forwards ,,_=  r?r>rr^s@@r=r6r6s<!!!r?r6c aa]tRtRtoRtRt.tR]R]!] ^RR7.R]!] ^R R7./t V3R lt V3R lt R t VtV;t#) BlipTextPreTrainedModelizz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. bertr attentionsz .attention.)index layer_namecross_attentionsz.crossattention.c<\SV`V4\V\4'db\P !VP \P!VP PR,4PR44R#R#)r$Nr%r#) r& _init_weightsrrinitcopy_r8r5r6rr7)r;moduler<s&&r=rI%BlipTextPreTrainedModel._init_weightss^ f% f0 1 1 JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h 2r?c&<V^8dQh/S[;R&#)rAr9r)rJrKs"r=rL$BlipTextPreTrainedModel.__annotate__s  r?)rVrWrXrYrZbase_model_prefix_no_split_modulesrrr`_can_record_outputsrI__annotate_func__r[r\r]r^s@@r=rBrBs_  0m T   0FX Y i'r?rBcpaa]tRtRtoRtR V3RlltRtRt]] R V3RlRll44t Rt Vt V;t #) BlipTextModelia The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between the self-attention layers, following the architecture described in [Attention is all you need](https://huggingface.co/papers/1706.03762) by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin. argument and `is_decoder` set to `True`; an `encoder_hidden_states` is then expected as an input to the forward pass. c<\SV`V4Wn\V4Vn\ V4VnV'd \V4MRVnVP4R#rs) r&r'r9rrSrencoderrpooler post_init)r;r9add_pooling_layerr<s&&&r=r'BlipTextModel.__init__sG   ,V4&v. 0AnV,t  r?c.VPP#rsrSr,rys&r=get_input_embeddings"BlipTextModel.get_input_embeddingss...r?c&WPnR#rsr^)r;rms&&r=set_input_embeddings"BlipTextModel.set_input_embeddingss */'r?c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[R,R S[R,R S[R,R S[S[,R S[/ #)rArBNrr8rCencoder_embedsrrrr rrrE)r5rIrr rrr)rJrKs"r=rLBlipTextModel.__annotate__s[ [ <<$&[  t+[ llT) [ ||d* [  t+ [  %||d2[ !& t 3[ [ $;[ 4K[ +,[  6[ r?c TV 'gRp VeVe \R4hVeVP4M^p VfVPVVVV R7p MTp V 'd\VPV VVR7pM\ VPV VR7pVe\ VPV VVR7pVP !V 3RVR VR VR VR V /V BpVPpVPeVPV4MRp\VVVPVPVPVPR 7#)a encoder_hidden_states (`torch.FloatTensor`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention if the model is configured as a decoder. encoder_attention_mask (`torch.FloatTensor`, *optional*): Mask to avoid performing attention on the padding token indices of the encoder input. This mask is used in the cross-attention if the model is configured as a decoder. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. past_key_values (`Cache`, *optional*): Contains precomputed key and value hidden states of the attention blocks. Can be used to speed up decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). FNzDYou cannot specify both input_ids and inputs_embeds at the same time)rBr8rCrD)r9rCrr)r9rCr)r9rCrrrrrrr )r pooler_outputrrrDrG)reget_seq_lengthrSr r9r rXrrYrrrrDrG)r;rBrr8rCrerrrr rrrDembedding_outputencoder_outputsr;rs&&&&&&&&&&&, r=rTBlipTextModel.forwardsvDI  ]%>cd dETE`!?!?!Afg  !##)+'= /   .  /{{.- / N7{{.-N " -%>{{.5&; & "FJ\\ F )F #8F $: F , F  F F *;;8< 8O O4UY ;-'+;;)77&11,==   r?)r9rSrXrY)T) NNNNNNNNNF)rVrWrXrYrZr'r_rbrrrTr[r\r]r^s@@r=rVrVsC/0 [ [  [ [ r?rVcaa]tRtRtoRRRR/tV3RltRtRtR tR t ] RV3R lR ll4t RV3R llt Rt VtV;t#)BlipTextLMHeadModeliFzcls.predictions.decoder.biaszcls.predictions.biaszcls.predictions.decoder.weightz&bert.embeddings.word_embeddings.weightc<\SV`V4\VRR7Vn\ V4VnVP VnVP4R#)F)r[N)r&r'rVrCr6clslabel_smoothingrZr:s&&r=r'BlipTextLMHeadModel.__init__LsB  !&EB &v.%55 r?c6VPP4#rs)rCr_rys&r=r_(BlipTextLMHeadModel.get_input_embeddingsUsyy--//r?c<VPPV4R#rs)rCrbr;new_embeddingss&&r=rb(BlipTextLMHeadModel.set_input_embeddingsXs &&~6r?cBVPPP#rs)rpr8r/rys&r=get_output_embeddings)BlipTextLMHeadModel.get_output_embeddings[sxx##+++r?cWPPnVPVPPnR#rs)rpr8r/r-rvs&&r=set_output_embeddings)BlipTextLMHeadModel.set_output_embeddings^s*'5$$2$7$7!r?c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[R,R S[R,R S[R,R S[R,R S[R,RS[S[P,RS[S[,RS[/#)rArBNrr8rCrrlabelsrr  return_logitsr reductionlogits_to_keeprrE) r5rIrr rrHrrr)rJrKs"r=rL BlipTextLMHeadModel.__annotate__csW W <<$&W  t+W llT) W ||d* W %||d2 W !& t 3W  t#W W $;W d{W 4KW :W ell*W +,W +!W r?c VPRR4pVeRp VP!V3RVRVRVRVRVR VR V R V /VBpVPp\V \4'd\ V )R4MT pVP VR VR 3,4pV 'dVR RR1R 3,P4#RpVeVf5VR RR1R 3,P4pVR,P4pMVP4pVPVP4p\WPR 7pV!VPRVPP4VPR44pV R8Xd1VPVP^4R4P!^4p\#VVVP$VP&VP(VP*R7#)a encoder_hidden_states (`torch.FloatTensor`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention if the model is configured as a decoder. encoder_attention_mask (`torch.FloatTensor`, *optional*): Mask to avoid performing attention on the padding token indices of the encoder input. This mask is used in the cross-attention if the model is configured as a decoder. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. labels (`torch.LongTensor`, *optional*): Labels for computing the left-to-right language modeling loss (next word prediction). Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` past_key_values (`Cache`, *optional*): Contains precomputed key and value hidden states of the attention blocks. Can be used to speed up decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). shift_labelsNFrr8rCrrrr rrO)rrqnone)losslogitsrrrDrGr%)rO:r$NN)poprCrrrHslicerprrrrrqrr9r)rPsumrrrrDrG)r;rBrr8rCrrrrr rrrrrroutputsr slice_indicesr?lm_lossshifted_prediction_scoresloss_fcts&&&&&&&&&&&&&&, r=rTBlipTextLMHeadModel.forwardbsTzz.$7  I@D  A ) A & A ( A #8 A $: A , A   A " A  A  11 8B>SV8W8W~ot4]k  HH]1mQ3F%GH $QQY/::< <  #,=a"ai,H,S,S,U)%e}779 ,=,H,H,J)'??+D+K+KLL')MaMabH8==b$++BXBXY[g[l[lmo[pqGF"!,,'8'='=a'@"EII!L0$#33!//))$55   r?c <<\SV`!V3RVRV/VBpRVR&V#)rrTr)r&prepare_inputs_for_generation)r;rBrr model_kwargs model_inputsr<s&&&&, r=r1BlipTextLMHeadModel.prepare_inputs_for_generationsEw<  + *   &* \"r?)rCrprq) NNNNNNNNNFTmeanr)NN)rVrWrXrY_tied_weights_keysr'r_rbrzr}rrTrr[r\r]r^s@@r=rnrnFsY&(>(*R 07,8W W W r  r?rn)rVrnrB)>rr5rtorch.nnrrrJ activationsr cache_utilsrrr generationr masking_utilsr r modeling_layersr modeling_outputsrrrmodeling_utilsrprocessing_utilsr pytorch_utilsrutilsrrr utils.genericrutils.output_capturingrrconfiguration_blipr get_loggerrVrModulerr`rrrrrrrr"r+r6rBrVrn__all__rPr?r=rsY  %&!CC)J9 .&6BB7E.   H %++^i.BIIi.Z  1 12 299  RYY -.-b/ bii/ f RYY  bii$ ryy "!"))!ioi4v +v tA1?AH Nr?