+ jP.r^RIHtHt^RIHt^RIHt!RR]RR7tRt]!R R ]44t R .t R #) )ProcessingKwargsProcessorMixin)make_flat_list_of_images)auto_docstringc"]tRt^tRRR//tRtR#)Llama4ProcessorKwargs text_kwargs padding_sideleftN)__name__ __module__ __qualname____firstlineno__ _defaults__static_attributes__r m/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/llama4/processing_llama4.pyrrs F IrrF)totala>{{- bos_token }} {%- if custom_tools is defined %} {%- set tools = custom_tools %} {%- endif %} {%- if not tools_in_user_message is defined %} {%- set tools_in_user_message = true %} {%- endif %} {%- if not date_string is defined %} {%- if strftime_now is defined %} {%- set date_string = strftime_now("%d %b %Y") %} {%- else %} {%- set date_string = "26 Jul 2024" %} {%- endif %} {%- endif %} {%- if not tools is defined %} {%- set tools = none %} {%- endif %} {#- This block extracts the system message, so we can slot it into the right place. #} {%- if messages[0]['role'] == 'system' %} {%- if messages[0]['content'] is string %} {%- set system_message = messages[0]['content']|trim %} {%- else %} {#- FIXME: The processor requires an array, always. #} {%- set system_message = messages[0]['content'][0]['text']|trim %} {%- endif %} {%- set messages = messages[1:] %} {%- set user_supplied_system_message = true %} {%- else %} {%- set system_message = "" %} {%- set user_supplied_system_message = false %} {%- endif %} {#- System message if the user supplied one #} {%- if user_supplied_system_message %} {{- "<|header_start|>system<|header_end|> " }} {%- if tools is not none %} {{- "Environment: ipython " }} {%- endif %} {%- if tools is not none and not tools_in_user_message %} {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }} {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }} {{- "Do not use variables. " }} {%- for t in tools %} {{- t | tojson(indent=4) }} {{- " " }} {%- endfor %} {%- endif %} {{- system_message }} {{- "<|eot|>" }} {%- endif %} {#- Custom tools are passed in a user message with some extra guidance #} {%- if tools_in_user_message and not tools is none %} {#- Extract the first user message so we can plug it in here #} {%- if messages | length != 0 %} {%- set first_user_message = messages[0]['content']|trim %} {%- set messages = messages[1:] %} {%- else %} {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }} {%- endif %} {{- '<|header_start|>user<|header_end|> ' -}} {{- "Given the following functions, please respond with a JSON for a function call " }} {{- "with its proper arguments that best answers the given prompt. " }} {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }} {{- "Do not use variables. " }} {%- for t in tools %} {{- t | tojson(indent=4) }} {{- " " }} {%- endfor %} {{- first_user_message + "<|eot|>"}} {%- endif %} {%- for message in messages %} {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %} {{- '<|header_start|>' + message['role'] + '<|header_end|> ' }} {%- if message['content'] is string %} {{- message['content'] }} {%- else %} {%- for content in message['content'] %} {%- if content['type'] == 'image' %} {{- '<|image|>' }} {%- elif content['type'] == 'text' %} {{- content['text'] }} {%- endif %} {%- endfor %} {%- endif %} {{- "<|eot|>" }} {%- elif 'tool_calls' in message and message.tool_calls|length > 0 %} {{- '<|header_start|>assistant<|header_end|> ' -}} {{- '<|python_start|>' }} {%- if message['content'] is string %} {{- message['content'] }} {%- else %} {%- for content in message['content'] %} {%- if content['type'] == 'image' %} {{- '<|image|>' }} {%- elif content['type'] == 'text' %} {{- content['text'] }} {%- endif %} {%- endfor %} {%- endif %} {{- '<|python_end|>' }} {%- for tool_call in message.tool_calls %} {{- '{"name": "' + tool_call.function.name + '", ' }} {{- '"parameters": ' }} {{- tool_call.function.arguments | tojson }} {{- "}" }} {%- endfor %} {{- "<|eot|>" }} {%- elif message.role == "tool" or message.role == "ipython" %} {{- "<|header_start|>ipython<|header_end|> " }} {%- if message.content is mapping or message.content is iterable %} {{- message.content | tojson }} {%- else %} {{- message.content }} {%- endif %} {{- "<|eot|>" }} {%- endif %} {%- endfor %} {%- if add_generation_prompt %} {{- '<|header_start|>assistant<|header_end|> ' }} {%- endif %} c aa]tRt^!to]tRR^RRRRRRRR]3 V3R lV3R llltR tRV3R llt RV3R llt V3RlRlt ] V3RlRl4t RtVtV;t#)Llama4ProcessorNg? <|image|><|image_start|> <|image_end|> <|patch|><|tile_x_separator|><|tile_y_separator|>c&<V^8dQhRS[RS[/#) patch_sizepixel_shuffle_ratio)intfloat)format __classdict__s"r __annotate__Llama4Processor.__annotate__%s!7878 78 # 78rc  <\SV`WV R7\\RV^,, 44VnW0nWPnW`nVPVP4Vn Wpn Wn Wn Wn WnR#)a patch_size (`int`, *optional*, defaults to 28): The size of image patches for tokenization. pixel_shuffle_ratio (`float`, *optional*, defaults to `0.5`): The ratio used for pixel shuffling when processing images. This controls the downsampling factor applied to image patches. The actual downsampling ratio is calculated as `1 / (pixel_shuffle_ratio^2)`. fake_image_token (`str`, *optional*, defaults to `"<|image|>"`): The placeholder token in the text that will be replaced with actual image tokens. This token serves as a marker indicating where images should be inserted in the text sequence. image_token (`str`, *optional*, defaults to `"<|image|>"`): The token to be used to represent an image in the text. start_of_image_token (`str`, *optional*, defaults to `"<|image_start|>"`): The special token that marks the beginning of an image sequence in the text. This token is prepended to image token sequences to delimit image boundaries. end_of_image_token (`str`, *optional*, defaults to `"<|image_end|>"`): The special token that marks the end of an image sequence in the text. This token is appended to image token sequences to delimit image boundaries. patch_token (`str`, *optional*, defaults to `"<|patch|>"`): The token used to represent individual image patches. Multiple patch tokens are used to represent the full image, with the number depending on the image size and patch configuration. tile_x_separator_token (`str`, *optional*, defaults to `"<|tile_x_separator|>"`): The token used to separate tiles (patches) horizontally within an image. This token is inserted between patches in the same row when images are split into multiple tiles. tile_y_separator_token (`str`, *optional*, defaults to `"<|tile_y_separator|>"`): The token used to separate tiles (patches) vertically within an image. This token is inserted between rows of patches when images are split into multiple tiles. ) chat_templateg?N)super__init__r"rounddownsample_ratior fake_image_token image_tokenconvert_tokens_to_idsimage_token_idstart_of_img_tokenend_of_img_tokenimg_patch_token tile_tokentile_global_token)selfimage_processor tokenizerr r!r.r/start_of_image_tokenend_of_image_token patch_tokentile_x_separator_tokentile_y_separator_tokenr)kwargs __class__s&&&&&&&&&&&&&,rr+Llama4Processor.__init__%s|V =Q #E#1Da1G*H$I J$ 0&'==d>N>NO"6 2*0!7rc RpVwrEWE,^8dV\V4FFp\V4F+pVRV,, pWu^, 8gK"VR, pK- VR, pKH VR, pVRV,, pVR, pV#)z Create a structured string representation of image tokens Args: num_patches: Number of patches in the image Returns: String with appropriate image tokens rrrrrr)range)r7 aspect_rationum_patches_per_chunk img_stringratio_hratio_wyyxxs&&& r_prompt_split_image#Llama4Processor._prompt_split_image^s' '  q Gn.B+0E"EEJaK'"&<< ) 44 % k! k$999 o% rc X<\SV`!RRVRV/VBvrpVe \V4pWRR3#)imagestextNr )r*prepare_inputs_layoutr)r7rNrOr?_r@s&&&, rrP%Llama4Processor.prepare_inputs_layoutysB 78\\T\U[\q  -f5FT4''rc <a\SS`!RRVRV/VBVf \R4hVeIVeC\V3RlV44pV\ V48wd\RV R\ V4 R24hR#R#R#) rNrONzYou have to specify text.c3X<"TFqPSP4xK! R#5i)N)countr.).0promptr7s& r 2Llama4Processor.validate_inputs..s#$\W[V\\$2G2G%H%HW[s'*zFound z) placeholders across the batch, but have z flattened images.r )r*validate_inputs ValueErrorsumlen)r7rNrOr?total_placeholdersr@sf&&, rrZLlama4Processor.validate_inputss CvCDCFC <89 9  $"2!$$\W[$\!\ !S[0 /01 #F },>@1#3 rc,<V^8dQhRS[RS[RS[/#)r image_inputs image_idxreturn)dictr"str)r$r%s"rr&r's(iiiiSVirc VR,^,PRRwrE\W@P,WPP,,VP,4pVP VR,V,V4#) pixel_valuesN aspect_ratios)shaper"r r-rK)r7rarbr? image_height image_widthrEs&&&, rreplace_image_token#Llama4Processor.replace_image_tokensn$0$@$C$I$I"#$N! # __ ,1O PTXTiTi i! '' _(Ei(PRghhrc0<V^8dQhRS[S[,/#)rrc)listre)r$r%s"rr&r's!!DI!rcR.#)rhr )r7s&runused_input_names"Llama4Processor.unused_input_namess   r) r-r3r.r/r1r4r r2r6r5)NN)r rrrrvalid_processor_kwargsr)r+rKrPrZrmpropertyrrr__classdictcell__ __classcell__)r@r%s@@rrr!sp2%($.*55#7878r6( ii!!!rrN) transformers.processing_utilsrr image_utilsrutilsrrr)r__all__r rrr|sR K3#,E]P s!ns!s!l  r