+ j.Z^RIt^RIHt^RIHtHtHtHtHt^RIH t ^RI H t ^RI H t Ht^RIHt^RIHtHtHt^R IHt^R IHt]P2!]4t!R R ]4t!R R]4t!RR]P<4t!RR]4t !RR]4t!!RR]4t".ROt#R#)N)nn)LlavaCausalLMOutputWithPastLlavaForConditionalGeneration LlavaModelLlavaModelOutputWithPastLlavaPreTrainedModel)ACT2FN)Cache)BaseModelOutputWithPastBaseModelOutputWithPooling)Unpack)TransformersKwargsauto_docstringlogging)can_return_tupleVipLlavaConfigc]tRt^&tRtR#)VipLlavaModelOutputWithPastN__name__ __module__ __qualname____firstlineno____static_attributes__rn/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/vipllava/modular_vipllava.pyrr&rrc]tRt^*tRtR#)VipLlavaCausalLMOutputWithPastrNrrrrr!r!*rrr!cDaa]tRt^.toV3RlV3RlltRtRtVtV;t#)VipLlavaMultiModalProjectorc <V^8dQhRS[/#)configr)format __classdict__s"r __annotate__(VipLlavaMultiModalProjector.__annotate__/s m m~ mrcp<\SV`4\VP\4'd^M\ VP4p\ P!W!PP,VPR7Vn \ P!W!PP,VPPRR7Vn\VP ,Vn\ P!VPPVPPRR7VnR#))epsT)biasN)super__init__ isinstancevision_feature_layersintlenr LayerNorm vision_config hidden_sizeprojector_layernorm_epsprojector_layernormLinear text_configlinear_1r projector_hidden_actactlinear_2)selfr&num_feature_layers __class__s&& rr0$VipLlavaMultiModalProjector.__init__/s ",V-I-I3"O"OQUXY_YuYuUv#%<< !5!5!A!A AvGeGe$   !5!5!A!A A    * *  &556 &"4"4"@"@&BTBTB`B`gkl rcVPV4pVPV4pVPV4pVPV4pV#N)r9r<r>r?)r@ hidden_statess&&rforward#VipLlavaMultiModalProjector.forward>sB00?  m4 /  m4 r)r>r<r?r9) rrrrr0rGr__classdictcell__ __classcell__)rBr(s@@rr#r#.s m mrr#c]tRt^FtRtR#)VipLlavaPreTrainedModelrNrrrrrLrLFrrrLca]tRt^Jto]]!RR7R V3RlRll44t]]R V3RlRll44tRtVt R#) VipLlavaModelzWObtains image last hidden states from the vision tower and apply multimodal projection.) custom_introNc <V^8dQhRS[PRS[S[S[,,R,RS[S[,RS[S[,/#r% pixel_valuesr2Nkwargsreturntorch FloatTensorr3listr rtupler )r'r(s"rr)VipLlavaModel.__annotate__OsR""''" #T#Y5"+, " + + "rc VeTMVPPpRVR&VP!V3/VBp\V\4'dVP V,R,pM@VUu.uFqdP V,R,NK pp\ P!VRR7pVPV4pWTn V#uupi)+ pixel_values (`torch.FloatTensor` of shape `(batch_size, channels, height, width)`): The tensors corresponding to the input images. vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. Toutput_hidden_states)dim)NNN:r,NN) r&r2 vision_towerr1r3rFrVcatmulti_modal_projector pooler_output)r@rRr2rS image_outputsimage_featuresindexs&&&, rget_image_features VipLlavaModel.get_image_featuresKs$&;%F !DKKLmLm *.%&))    +S 1 1*889NOPUVNVkkUjE99%@GGUjNk"YY~2>N33NC&4# ls*"B?cj<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[PR,RS[S[S[,,R,R S[R,R S[S[ ,R S[ S[ ,/ #) r% input_idsNrRattention_mask position_idspast_key_values inputs_embedsr2 use_cache lm_kwargsrT) rV LongTensorrWTensorr r3rXboolr rrYr)r'r(s"rr)rZus55##d*5''$.5 t+ 5 &&- 5  5((4/5 #T#Y55$;5./5 , ,5rc 6VeTMVPPpVRJVRJ, 'd \R4hVfVP4!V4pVegVP W'R7P p V P VPVP4p VPWV R7p VPW4pVP!R RVRVRVRVR V/V Bp \V PV PV PV P VeX MRR 7p V #) z vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. Nz:You must specify exactly one of input_ids or inputs_embeds)rRr2)rorfrlrmrnrorp)last_hidden_staternrF attentionsimage_hidden_statesr)r&r2 ValueErrorget_input_embeddingsrhrdtodevicedtypeget_placeholder_maskmasked_scatterlanguage_modelrrvrnrFrw)r@rkrRrlrmrnror2rprqrfspecial_image_maskoutputsoutputs&&&&&&&&&, rrGVipLlavaModel.forwardssX(&;%F !DKKLmLm   -t"; < <YZ Z   557 BM  #!44)5m ,..}/C/C]EXEXYN!%!:!:~";" *889K\M+/+>+>, ), %, ,, ( ,  ,  , -%77#33!//))2>2JPT   rrrE)NNNNNNNN) rrrrrrrhrGrrIr(s@rrNrNJsNn"""H555rrNcja]tRt^to]RV3RlRll4t]]RV3RlRll44tRtVt R#) VipLlavaForConditionalGenerationNc <V^8dQhRS[PRS[S[S[,,R,RS[S[,RS[S[,/#rQrU)r'r(s"rr)-VipLlavaForConditionalGeneration.__annotate__sR  ''  #T#Y5 +,  + +  rc BVPP!RRVRV/VB#)r\rRr2r)modelrh)r@rRr2rSs&&&,rrh3VipLlavaForConditionalGeneration.get_image_featuress3zz,, % =R V\  rc<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[PR,RS[S[S[,,R,R S[PR,R S[R,R S[S[P,R S[S[ ,R S[ S[ ,/ #)r%rkNrRrlrmrnror2labelsrplogits_to_keeprqrT) rVrrrWrsr r3rXrtr rrYr!)r'r(s"rr)rsT T ##d*T ''$.T  t+ T &&- T  T ((4/T  #T#Y5T   4'T $;T ell*T ./T  / /T rc  VeTMVPPpVP!RRVRVRVRVRVRVRV R V/V Bp V Pp \ V \ 4'd\ V )R4MT pVPV R VR 3,4pRpVe7VP!RR VR VR VPPP/V Bp\VVV PV PV PV PR7#)a vision_feature_layers (`Union[int, list[int]]`, *optional*): The vision feature layer, or the list of indexes of the layers to select the vision feature. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> import torch >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> from transformers import AutoProcessor, VipLlavaForConditionalGeneration >>> model = VipLlavaForConditionalGeneration.from_pretrained("llava-hf/vip-llava-7b-hf", device_map="auto", dtype=torch.float16) >>> processor = AutoProcessor.from_pretrained("llava-hf/vip-llava-7b-hf") >>> prompt = "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.###Human: \n{}###Assistant:" >>> question = "Can you please describe this image?" >>> prompt = prompt.format(question) >>> url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/compel-neg.png" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(text=text, images=image, return_tensors="pt").to(0, torch.float16) >>> # Generate >>> generate_ids = model.generate(**inputs, max_new_tokens=20) >>> processor.decode(generate_ids[0][len(inputs["input_ids"][0]):], skip_special_tokens=True) The image features a brown and white cat sitting on a green surface, with a red ball in its ```NrkrRrlrmrnrorpr2r_logitsr vocab_size)lossrrnrFrwrxr)r&r2rrvr1r3slicelm_head loss_functionr;rr!rnrFrwrx)r@rkrRrlrmrnror2rrprrqrrF slice_indicesrrs&&&&&&&&&&&, rrG(VipLlavaForConditionalGeneration.forwardsNj&;%F !DKKLmLm 04zz 0  0 % 0 * 0 & 0 , 0 ( 0   0 #8 0  0  11 8B>SV8W8W~ot4]k mA}a,?@A  %%&,9=9P9P9[9[_hD.#33!//)) ' ; ;   rrrE) NNNNNNNNNr) rrrrrrhrrGrrIrs@rrrs@   "T T T rr)rNrrL)$rVr(transformers.models.llava.modeling_llavarrrrr activationsr cache_utilsr modeling_outputsr r processing_utilsr utilsrrr utils.genericrconfiguration_vipllavar get_loggerrloggerrr!Moduler#rLrNr__all__rrrrs " S&@@-2   H % ":  %@ "))0 2 `J`Fi 'Di X [r