+ jHRt^RIt^RIHt^RIt^RIHt^RIHt^RIH t ^RI H t ^RI HtHtHt^RIHt^R IHtHt^R IHt^R IHtHtHt^R IHt^R IHt^RI H!t!H"t"H#t#^RI$H%t%^RI&H't']"PP!])4t*!RR]PV4t,!RR]PV4t-!RR]PV4t.!RR]4t/]!!RR]44t0]!!RR]044t1]!!RR7!RR ]0]44t2]!!R!R7!R"R#]044t3.R$Ot4R#)%zPyTorch OpenAI ImageGPT model.N)Any)nn)CrossEntropyLoss)initialization)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)create_bidirectional_maskcreate_causal_mask)GradientCheckpointingLayer))BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentions SequenceClassifierOutputWithPast)PreTrainedModel)Conv1D)auto_docstringlogging torch_float)maybe_autocastImageGPTConfigcTaa]tRt^0toRV3RlV3RllltV3RlRltRtVtV;t#)ImageGPTLayerNormc6<V^8dQhRS[S[,RS[/#) hidden_sizeeps)tupleintfloat)format __classdict__s"o/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/imagegpt/modeling_imagegpt.py __annotate__ImageGPTLayerNorm.__annotate__1s>>E#J>U>c<\SV`4W n\P!\ P !V44VnR#N)super__init__rr ParametertorchTensorweight)selfrr __class__s&&&r$r+ImageGPTLayerNorm.__init__1s, ll5<< #<= r'cN<V^8dQhRS[PRS[P/#)rtensorreturnr-r.)r"r#s"r$r%r&6s#ellu||r'c V\P!\P!\P!V4RRR7VP,4, pWP ,pV#)T)axiskeepdim)r-sqrtmeansquarerr/)r0r4s&&r$forwardImageGPTLayerNorm.forward6sI%**UZZ V0D2W[%\_c_g_g%ghh++% r')rr/)gh㈵> __name__ __module__ __qualname____firstlineno__r+r?__static_attributes____classdictcell__ __classcell__r1r#s@@r$rr0s>> r'rcxaa]tRt^=toR V3RlV3RllltR RltR RltRtRtR V3RlRllt R t Vt V;t #) ImageGPTAttentioncB<V^8dQhRS[R,RS[R,/#)ris_cross_attentionN layer_idx)boolr )r"r#s"r$r%ImageGPTAttention.__annotate__>s&%<%<4$;%4Vn R#)dtypeF) persistentz=`embed_dim` must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).N)!r*r+configmax_position_embeddingsrBufferr-trilonesrOviewbiasr embed_dimnum_attention_heads num_headshead_dim split_size ValueErrorscale_attn_weightsrMscale_attn_by_inverse_layer_idxrNreorder_and_upcast_attnrc_attnq_attnc_projDropout attn_pdrop attn_dropout resid_pdrop resid_dropout)r0rUrMrN max_positionsr1s&&&& r$r+ImageGPTAttention.__init__>s  66 II JJuzz="@ S T Y Y1m     ++33$..8 .. ==4>> )T^^ ;OPTP^P^O_`NN#2'  #)";";"406/U/U,"'-'E'E$  " " " T^^!3T^^DDK @DK T^^!3T^^DDKT^^T^^< JJv'8'89ZZ(:(:;r'c\P!WPRR44pVP'd)V\ VP R4R,4, pVP 'd$V\VP^,4, pVP'gVP R4VP R4rvVPRRWv, V1RV13,p\P!VP4Pp \P!WPVPR7p \P !WV 4pVe WT,p\"P$!RR7!V4pVP'VP4pVP)V4p\P!WS4p W3#)r8?NNNNrSdevicedimr;)r-matmul transposerbrsizercr!rNrMr[finforSminr4rswhererSoftmaxtyperj) r0querykeyvalueattention_mask attn_weights query_length key_length causal_mask mask_value attn_outputs &&&&& r$_attnImageGPTAttention._attnesY||E==R+@A  " " "'+ejjn6K*LLL  / / /'%0B*CCL&&&',zz"~sxx|*))Aq**Cj*PR]S]R]$]^K\%7%78< &&&',zz"~sxx|*L))Aq*|*Cj*PR]S]R]$]^K\%7%78< = =s :A'--a  !*&9::'2266t~~F %+5+K+K(+5+J+J('1$2D.-  4** t %* M2*11$..AFF,33DNNCJJ M2![[8>>tTU>V UhhsB FPPQRTUV 3DNNDMMJTTUVXYZ $ N ; A A$//WX A Y E3((3DNNDMMBLLQPQRCJJsB FPPQRTUVE  !-44S%PJC!8< %%dnn5 3GQQRSUVW  ' ' '(,(G(GsTY[i(j %K(, 5#un(U %K'' T^^T]]S kk+. ((5 L((r')rjr[rergrUr\r_rMrNr^rfrdrlrcrbr`)FNr)NNNNFF) rBrCrDrEr+rrrrr?rFrGrHrIs@@r$rKrK=s8%<%>B<< > >OJJv001 ERXRjRjLklLkq f BLklm%dnn&:S:ST &+#  msD.cVP#r)r)r0s&r$get_input_embeddings"ImageGPTModel.get_input_embeddingss xxr'cWnR#r)r)r0new_embeddingss&&r$set_input_embeddings"ImageGPTModel.set_input_embeddingss!r'c<V^8dQhRS[PR,RS[R,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[R,R S[R,R S[R,R S[R,RS[RS[S[,/#)rrNpast_key_valuesrtoken_type_ids position_ids inputs_embedsrrrroutput_hidden_states return_dictrr5)r-r.rrOrrr)r"r#s"r$r%r sg g <<$&g g  t+ g  t+ g llT) g ||d*g  %||d2g !& t 3g $;g  $;g #Tkg D[g g  : :g r'c V eT MVPPp V eT MVPPp V eT MVPPp V eT MVPPp VeVe \ R4hVeOVP W4VP4pVPRVR,4pVP^,pM6Ve(VP4RRpVP^,pM \ R4hVe VPM VPpVP'd2VP'd V 'd\PR4Rp VeVPRVR,4pV 'dVf\VPR7pVfNVeVP!4M^p\"P$!VR,VR7V,pVP'^4pVeVPVR4pVfVP)V4pVP+V4pVVP-VP4,p\/VPRR4'd\1VPVVVR 7pM\3VPVVR 7pVe\3VPVVVR 7pVeVP)V4pVV,pVP5V4pVVPR43,pV 'dRMRpV 'dVPP6'dRMRpV 'dRMRp\9VP:4FzwppV 'd VV3,pV!VVVVVV V R 7pV^,pV 'gK:VV^,3,pVPP6'gKiVV^,3,pK| VP=V4pVP!V!pV 'd VV3,pV 'g7\>;QJd.R VVVVV34FNK 5#!R VVVVV344#\AVVVVVR7#)a input_ids (`torch.LongTensor` of shape `(batch_size, input_ids_length)`): `input_ids_length` = `sequence_length` if `past_key_values` is `None` else `past_key_values.get_seq_length()` (`sequence_length` of input past key value states). Indices of input sequence tokens in the vocabulary. If `past_key_values` is used, only `input_ids` that do not have their past calculated should be passed as `input_ids`. Indices can be obtained using [`AutoImageProcessor`]. See [`ImageGPTImageProcessor.__call__`] for details. Examples: ```python >>> from transformers import AutoImageProcessor, ImageGPTModel >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small") >>> model = ImageGPTModel.from_pretrained("openai/imagegpt-small") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_states = outputs.last_hidden_state ```NzDYou cannot specify both input_ids and inputs_embeds at the same timez5You have to specify either input_ids or inputs_embedszZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...F)rU)rs is_decoder)rUr*rr')rUr*r)rUr*rr)rrrc32"TF pVfK VxK R#5ir)r).0vs& r$ (ImageGPTModel.forward..<swAws )last_hidden_stater'r attentionscross_attentionsr;r)!rUrr+rr,ra%warn_if_padding_and_no_attention_maskryrZrrsrtraininglogger warning_oncerget_seq_lengthr-arange unsqueezerrtogetattrr r rr enumeraterrrr)r0rr'rr(r)r*rrrrr+r,r input_shape batch_sizerspast_seen_tokensposition_embedsrtoken_type_embeds output_shapeall_self_attentionsall_cross_attentionsall_hidden_statesrblockrs&&&&&&&&&&&&&, r$r?ImageGPTModel.forwards`2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKcd d  "  6 6y Q#..*K!r;r?;I"+J  &',,.s3K&,,Q/JTU U%.%:!! @T@T  & & &4===##p"  %+00[_EN 0*$++>O  CRC^==?de  << BGJZZL'11!4L  %+00R@N   HHY/M((<0%(:(:=;O;O(PP 4;; e 4 4/{{+- / N7{{+-N " -%>{{+5&; & "  % $ 8 ),==M -0 "m&8&8&<%>> $5b4%64;;;Z;Z;Zr`d"6BD!$&&)HAu#$58H$H!%'=#"3G$AJM  &9WQZM&I#;;222+?71:-+O(%*( -0 %**L9  1]4D D 5':KM`bvw5 5':KM`bvw  9+++*1   r')rr\rrrrr) NNNNNNNNNNNN) rBrCrDrEr+r r$rr?rFrGrHrIs@@r$r r s; "g g g g r'r z The ImageGPT Model transformer with a language modeling head on top (linear layer with weights tied to the input embeddings). ) custom_introcfaa]tRtRtoRR/tV3RlV3Rllt]R V3RlRll4tRtVt V;t #) ImageGPTForCausalImageModelingiKzlm_head.weightztransformer.wte.weightc <V^8dQhRS[/#rr)r"r#s"r$r%+ImageGPTForCausalImageModeling.__annotate__Ts~r'c<\SV`V4\V4Vn\P !VP VP^, RR7VnVP4R#)r8Fr[N) r*r+r rrLinearn_embdrlm_headrr0rUr1s&&r$r+'ImageGPTForCausalImageModeling.__init__TsL  (0yy0A0AA0EER  r'c<V^8dQhRS[PR,RS[R,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[PR,R S[PR,R S[R,R S[R,R S[R,RS[R,RS[RS[S[,/#)rrNr'rr(r)r*rrlabelsrrr+r,rr5)r-r.rrOrrr)r"r#s"r$r%rP]sl l <<$&l l  t+ l  t+ l llT) l ||d*l  %||d2l !& t 3l  t#l $;l  $;l #Tkl D[l l 2 2!l r'c `V eT MVPPp VPVVVVVVVVV V V V R7 pV^,pVPV4pRpV euVRRR1R3,P 4pV R,P 4p\ 4pV!VP RVPR44VP R44pV 'g"V3VR,,pVe V3V,#T#\VVVPVPVPVPR7#) a input_ids (`torch.LongTensor` of shape `(batch_size, input_ids_length)`): `input_ids_length` = `sequence_length` if `past_key_values` is `None` else `past_key_values.get_seq_length()` (`sequence_length` of input past key value states). Indices of input sequence tokens in the vocabulary. If `past_key_values` is used, only `input_ids` that do not have their past calculated should be passed as `input_ids`. Indices can be obtained using [`AutoImageProcessor`]. See [`ImageGPTImageProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size, input_ids_length)`, *optional*): Labels for language modeling. Note that the labels **are shifted** inside the model, i.e. you can set `labels = input_ids` Indices are selected in `[-100, 0, ..., config.vocab_size]` All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size]` Examples: ```python >>> from transformers import AutoImageProcessor, ImageGPTForCausalImageModeling >>> import torch >>> import matplotlib.pyplot as plt >>> import numpy as np >>> image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small") >>> model = ImageGPTForCausalImageModeling.from_pretrained("openai/imagegpt-small") >>> device = torch.device("cuda" if torch.cuda.is_available() else "cpu") >>> model.to(device) # doctest: +IGNORE_RESULT >>> # unconditional generation of 8 images >>> batch_size = 4 >>> context = torch.full((batch_size, 1), model.config.vocab_size - 1) # initialize with SOS token >>> context = context.to(device) >>> output = model.generate( ... input_ids=context, max_length=model.config.n_positions + 1, temperature=1.0, do_sample=True, top_k=40 ... ) >>> clusters = image_processor.clusters >>> height = image_processor.size["height"] >>> width = image_processor.size["width"] >>> samples = output[:, 1:].detach().cpu().numpy() >>> samples_img = [ ... np.reshape(np.rint(127.5 * (clusters[s] + 1.0)), [height, width, 3]).astype(np.uint8) for s in samples ... ] # convert color cluster tokens back to pixels >>> f, axes = plt.subplots(1, batch_size, dpi=300) >>> for img, ax in zip(samples_img, axes): # doctest: +IGNORE_RESULT ... ax.axis("off") ... ax.imshow(img) ```N) r'rr(r)r*rrrrr+r,.rqr)losslogitsr'rr5r6r;).r) rUr,rrUrrrZryrr'rr5r6)r0rr'rr(r)r*rrrYrrr+r,rtransformer_outputsr lm_logitsr[ shift_logits shift_labelsloss_fctoutputs&&&&&&&&&&&&&&, r$r?&ImageGPTForCausalImageModeling.forward\sDJ&1%L!'?557L')HL--b,2C2CB2GH,J[J[\^J_`D\$7$;;F)-)9TGf$ Ev E0/??-;;*550AA   r')rUr) NNNNNNNNNNNNN) rBrCrDrE_tied_weights_keysr+rr?rFrGrHrIs@@r$rNrNKs?+,DEl l l l r'rNz The ImageGPT Model transformer with an image classification head on top (linear layer). [`ImageGPTForImageClassification`] average-pools the hidden states in order to do the classification. c^aa]tRtRtoV3RlV3Rllt]RV3RlRll4tRtVtV;t #)ImageGPTForImageClassificationic <V^8dQhRS[/#rr)r"r#s"r$r%+ImageGPTForImageClassification.__annotate__s~r'c<\SV`V4VPVn\V4Vn\ P !VPVPRR7VnVP4R#)FrRN) r*r+ num_labelsr rrrSrTscorerrVs&&r$r+'ImageGPTForImageClassification.__init__sR   ++(0YYv}}dooEJ  r'c<V^8dQhRS[PR,RS[R,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,R S[R,R S[R,R S[R,R S[R,R S[RS[S[,/ #)rrNr'rr(r)r*rYrrr+r,rr5)r-r.rrOrrr)r"r#s"r$r%rhsT T <<$&T T  t+ T  t+ T llT) T ||d*T  t#T $;T  $;T #TkT D[T T  1 1T r'c V eT MVPPp VPVVVVVVVV V V R7 p V ^,pVP^R7pVP V4pRpVeVP VVVP4pV 'g"V3V R,,pVe V3V,#T#\ VVV PV PV PR7#)a input_ids (`torch.LongTensor` of shape `(batch_size, input_ids_length)`): `input_ids_length` = `sequence_length` if `past_key_values` is `None` else `past_key_values.get_seq_length()` (`sequence_length` of input past key value states). Indices of input sequence tokens in the vocabulary. If `past_key_values` is used, only `input_ids` that do not have their past calculated should be passed as `input_ids`. Indices can be obtained using [`AutoImageProcessor`]. See [`ImageGPTImageProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, ImageGPTForImageClassification >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> image_processor = AutoImageProcessor.from_pretrained("openai/imagegpt-small") >>> model = ImageGPTForImageClassification.from_pretrained("openai/imagegpt-small") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits ```N) r'rr(r)r*rrr+r,rtr)r[r\r'rr5) rUr,rr=rk loss_functionrr'rr5)r0rr'rr(r)r*rYrrr+r,rr]rpooled_hidden_statesr\r[rbs&&&&&&&&&&&&, r$r?&ImageGPTForImageClassification.forwardsf&1%rs@%  %&!CC)J9 .# ,2   H %  L) L)^"))"E*.E*PoD +  D x %