+ jE^RIHt^RIt^RIHt^RIHt^RIHt^RIH t ^RI H t H t H t ^RIHt^R IHt^R IHtHtHt^R IHt^R IHt^R IHt]!RR7]!RR] 444t]!RR7]!RR] 444t!RR]P<4t]!RR]44t ]!RR7!RR] 44t!]!RR7!RR] ] 44t".ROt#R#) ) dataclassN)nn)ACT2FN)Cache)GenerationMixin)BaseModelOutputWithPastBaseModelOutputWithPooling ModelOutput)PreTrainedModel)Unpack)TransformersKwargsauto_docstringtorch_compilable_check)can_return_tuple) AutoModelVipLlavaConfigzM Base class for VipLlava outputs, with hidden states and attentions.  custom_introc4a]tRt^&toRtRtV3RltRtVtR#)VipLlavaModelOutputWithPasta past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. NcH<V^8dQh/S[PR,;R&#)Nimage_hidden_states)torch FloatTensor)format __classdict__s"o/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/vipllava/modeling_vipllava.py __annotate__(VipLlavaModelOutputWithPast.__annotate__&s$**T18%) __name__ __module__ __qualname____firstlineno____doc__r__annotate_func____static_attributes____classdictcell__rs@rrr&s 59%r"rzT Base class for VipLlava causal language model (or autoregressive) outputs. cHa]tRt^;toRtRtRtRtRtRt Rt V3Rlt Rt Vt R#)VipLlavaCausalLMOutputWithPasta loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nc:<V^8dQh/S[PR,;R&S[PR,;R&S[R,;R&S[S[P,R,;R&S[S[P,R,;R&S[PR,;R&#)rNlosslogitspast_key_values hidden_states attentionsr)rrrtuple)rrs"rr +VipLlavaCausalLMOutputWithPast.__annotate__;s,   d ")-.    $+/0T\(12**+d2934e''(4/656**T187r"r#)r$r%r&r'r(r0r1r2r3r4rr)r*r+r,s@rr.r.;s2 &*D'+F$(O59M26J487r"r.cDaa]tRt^YtoV3RlV3RlltRtRtVtV;t#)VipLlavaMultiModalProjectorc <V^8dQhRS[/#rconfigr)rrs"rr (VipLlavaMultiModalProjector.__annotate__Zs m m~ mr"cp<\SV`4\VP\4'd^M\ VP4p\ P!W!PP,VPR7Vn \ P!W!PP,VPPRR7Vn\VP ,Vn\ P!VPPVPPRR7VnR#))epsTbiasN)super__init__ isinstancevision_feature_layersintlenr LayerNorm vision_config hidden_sizeprojector_layernorm_epsprojector_layernormLinear text_configlinear_1rprojector_hidden_actactlinear_2)selfr;num_feature_layers __class__s&& rrC$VipLlavaMultiModalProjector.__init__Zs ",V-I-I3"O"OQUXY_YuYuUv#%<< !5!5!A!A AvGeGe$   !5!5!A!A A    * *  &556 &"4"4"@"@&BTBTB`B`gkl r"cVPV4pVPV4pVPV4pVPV4pV#N)rLrOrQrR)rSr3s&&rforward#VipLlavaMultiModalProjector.forwardisB00?  m4 /  m4 r")rQrOrRrL) r$r%r&r'rCrYr*r+ __classcell__rUrs@@rr8r8Ys m mr"r8cRa]tRt^qtoRtRtRtR.tRtRt Rt Rt Rt V3Rlt RtVtR#)VipLlavaPreTrainedModelmodelTr2c&<V^8dQh/S[;R&#r:r)rrs"rr $VipLlavaPreTrainedModel.__annotate__qs r"r#N)imagetext)r$r%r&r'base_model_prefixinput_modalitiessupports_gradient_checkpointing_skip_keys_device_placement_supports_flash_attn_supports_sdpa_can_compile_fullgraph_supports_flex_attn_supports_attention_backendr)r*r+r,s@rr^r^qsF (&*##4"5N!"&r"r^zx The VipLlava model which consists of a vision backbone and a language model, without a language modeling head. caa]tRt^toV3RlV3Rllt]]!RR7R V3RlRll44tV3RlRlt]]R V3R lR ll44t R t Vt V;t #) VipLlavaModelc <V^8dQhRS[/#r:r)rrs"rr VipLlavaModel.__annotate__s~r"c<\SV`V4\P!VP4Vn\ V4Vn\P!VP4Vn VP4R#rX) rBrCr from_configrI vision_towerr8multi_modal_projectorrNlanguage_model post_initrSr;rUs&&rrCVipLlavaModel.__init__sY  %11&2F2FG%@%H"'33F4F4FG r"zWObtains image last hidden states from the vision tower and apply multimodal projection.rc <V^8dQhRS[PRS[S[S[,,R,RS[S[,RS[S[,/#r pixel_valuesrENkwargsreturnrrrFlistr r r5r )rrs"rr rpsR""''" #T#Y5"+, " + + "r"c VeTMVPPpRVR&VP!V3/VBp\V\4'dVP V,R,pM@VUu.uFqdP V,R,NK pp\ P!VRR7pVPV4pWTn V#uupi)+ pixel_values (`torch.FloatTensor` of shape `(batch_size, channels, height, width)`): The tensors corresponding to the input images. vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. Toutput_hidden_states)dim)NNN:r>NN) r;rErsrDrFr3rcatrt pooler_output)rSr{rEr| image_outputsimage_featuresindexs&&&, rget_image_features VipLlavaModel.get_image_featuress$&;%F !DKKLmLm *.%&))    +S 1 1*889NOPUVNVkkUjE99%@GGUjNk"YY~2>N33NC&4# ls*"B?ch<V^8dQhRS[PRS[PRS[P/#)r input_ids inputs_embedsr)r LongTensorr)rrs"rr rps7""))":?:K:K"]b]n]n"r"c  XVfoW P4!\P!RVPP\P VP R748HpVPR4pMWPP8HpVP4pVP^,VP^,,pVPR4PVP 4p\WRPR,,VP48HRV RV 24V#)z Obtains multimodal placeholder mask from `input_ids` or `inputs_embeds`, and checks that the placeholder token count is equal to the length of multimodal features. If the lengths are different, an error is raised. )dtypedevicez6Image features and image tokens do not match, tokens: z , features: r#r)get_input_embeddingsrfullr;image_token_idlongrallsumshape unsqueezetornumel)rSrrrspecial_image_maskn_image_tokensn_image_featuress&&&& rget_placeholder_mask"VipLlavaModel.get_placeholder_masks  !.2K2K2M 2t{{99TaThThi3" "4!7!7!; !*kk.H.H!H +//1)//2^5I5I!5LL/99"=@@AUAUV 004 48L8L8N ND^DTT`aq`r s "!r"cj<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[PR,RS[S[S[,,R,R S[R,R S[S[ ,R S[ S[ ,/ #) rrNr{attention_mask position_idsr2rrE use_cache lm_kwargsr}) rrrTensorrrFrboolr r r5r)rrs"rr rps55##d*5''$.5 t+ 5 &&- 5  5((4/5 #T#Y55$;5./5 , ,5r"c 6VeTMVPPpVRJVRJ, 'd \R4hVfVP4!V4pVegVP W'R7P p V P VPVP4p VPWV R7p VPW4pVP!R RVRVRVRVR V/V Bp \V PV PV PV P VeX MRR 7p V #) z vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. Nz:You must specify exactly one of input_ids or inputs_embeds)r{rE)rrrrr2rr)last_hidden_stater2r3r4rr#)r;rE ValueErrorrrrrrrrmasked_scatterrurrr2r3r4)rSrr{rrr2rrErrrroutputsoutputs&&&&&&&&&, rrYVipLlavaModel.forwardsX(&;%F !DKKLmLm   -t"; < <YZ Z   557 BM  #!44)5m ,..}/C/C]EXEXYN!%!:!:~";" *889K\M+/+>+>, ), %, ,, ( ,  ,  , -%77#33!//))2>2JPT   r")rurtrsrX)NNNNNNNN) r$r%r&r'rCrrrrrYr*r+r[r\s@@rrnrnsi n"""H""05555r"rnzV The VIPLLAVA model which consists of a vision backbone and a language model. caa]tRtRtoRR/tV3RlV3RlltV3RlRlt]R V3RlR ll4t] ]RV3R lR ll44t R t Vt V;t #) VipLlavaForConditionalGenerationi zlm_head.weightz(model.language_model.embed_tokens.weightc <V^8dQhRS[/#r:r)rrs"rr -VipLlavaForConditionalGeneration.__annotate__s~r"c<\SV`V4\V4Vn\P !VP PVP PRR7Vn VP4R#)Fr@N) rBrCrnr_rrMrNrJ vocab_sizelm_headrvrws&&rrC)VipLlavaForConditionalGeneration.__init__sS  "6* yy!3!3!?!?ASASA^A^ejk  r"c4<V^8dQhRS[P/#)rr})rModule)rrs"rr rsryyr"cVP#rX)r)rSs&rget_output_embeddings6VipLlavaForConditionalGeneration.get_output_embeddingss ||r"c <V^8dQhRS[PRS[S[S[,,R,RS[S[,RS[S[,/#rzr~)rrs"rr rsR  ''  #T#Y5 +,  + +  r"c BVPP!RRVRV/VB#)rr{rEr#)r_r)rSr{rEr|s&&&,rr3VipLlavaForConditionalGeneration.get_image_featuress3zz,, % =R V\  r"c<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[PR,RS[S[S[,,R,R S[PR,R S[R,R S[S[P,R S[S[ ,R S[ S[ ,/ #)rrNr{rrr2rrElabelsrlogits_to_keeprr}) rrrrrrFrrr r r5r.)rrs"rr r.sT T ##d*T ''$.T  t+ T &&- T  T ((4/T  #T#Y5T   4'T $;T ell*T ./T  / /T r"c  VeTMVPPpVP!RRVRVRVRVRVRVRV R V/V Bp V Pp \ V \ 4'd\ V )R4MT pVPV R VR 3,4pRpVe7VP!RR VR VR VPPP/V Bp\VVV PV PV PV PR7#)a vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> import torch >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> from transformers import AutoProcessor, VipLlavaForConditionalGeneration >>> model = VipLlavaForConditionalGeneration.from_pretrained("llava-hf/vip-llava-7b-hf", device_map="auto", dtype=torch.float16) >>> processor = AutoProcessor.from_pretrained("llava-hf/vip-llava-7b-hf") >>> prompt = "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.###Human: \n{}###Assistant:" >>> question = "Can you please describe this image?" >>> prompt = prompt.format(question) >>> url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/compel-neg.png" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(text=text, images=image, return_tensors="pt").to(0, torch.float16) >>> # Generate >>> generate_ids = model.generate(**inputs, max_new_tokens=20) >>> processor.decode(generate_ids[0][len(inputs["input_ids"][0]):], skip_special_tokens=True) The image features a brown and white cat sitting on a green surface, with a red ball in its ```Nrr{rrr2rrrErr1rr)r0r1r2r3r4rr#)r;rEr_rrDrFslicer loss_functionrNrr.r2r3r4r)rSrr{rrr2rrErrrrrr3 slice_indicesr1r0s&&&&&&&&&&&, rrY(VipLlavaForConditionalGeneration.forward,sNj&;%F !DKKLmLm 04zz 0  0 % 0 * 0 & 0 , 0 ( 0   0 #8 0  0  11 8B>SV8W8W~ot4]k mA}a,?@A  %%&,9=9P9P9[9[_hD.#33!//)) ' ; ;   r")rr_rX) NNNNNNNNNr)r$r%r&r'_tied_weights_keysrCrrrrrYr*r+r[r\s@@rrr sj +,VW    "T T T T r"r)rnrr^)$ dataclassesrrr activationsr cache_utilsr generationrmodeling_outputsrr r modeling_utilsr processing_utilsr utilsr rr utils.genericrautorconfiguration_vipllavarrr.rr8r^rnr__all__r#r"rrs*" ! )``-&OO-2   9"9 9  9  9[9  90"))0 'o ' ' @+@  @F t '>t  t n [r"