+ j^RIt^RIHtHtHt^RIHt^RIt^RIH t ^RI H t ^RI H t ^RIHtHtHtHt^RIHt^R IHt^R IHtHtHtHtR R ltR RltRRlt!RR4t!RR4t R#)N)floorgcdsqrt)Any)PreTrainedConfig)ContinuousBatchingConfig)is_flash_attention_requested) BlockManagerCacheAllocatorFullAttentionCacheAllocatorSlidingAttentionCacheAllocator)DistributedHelper)resolve_max_memory_percent) RequestState RequestStatusget_device_and_memory_breakdownloggerc0V^8dQhR\R\/#configreturnrint)formats"r/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/generation/continuous_batching/cache.py __annotate__rs p p. p3 pcj\VRR4pVeV#\VRR4pVeV#\RV 24h)z9Finds the number of key-value heads for the given config.num_key_value_headsNnum_attention_headszMnum_key_value_heads or num_attention_heads could not be found in the config: getattr ValueError)rkv_headss& rfind_num_kv_headsr&sMv4d;Hv4d;H eflemn oorc0V^8dQhR\R\/#rr)rs"rrr*s w w* ws wrc\VRR4pVeV#\VRR4p\VRR4pVe Ve W#,#\RV 24h)z.Finds the head dimension for the given config.head_dimN hidden_sizer!zThead_dim or (hidden_size and num_attention_heads) could not be found in the config: r")rr)r*r!s& r find_head_dimr+*sevz40H&-6K!&*?F#6#B11 lmsltu vvrcV^8dQhR\R\\\\,,\\,3,/#r)rtuplelistrstr)rs"rrr8s4%%&6%5d3iRVWZR[A[;\%rcV\VRR4pVf7\VRR4eRMRp\VP4Uu.uFq2NK pp/p\V4F!wrVVP V.4V.,WF&K# \ VP 4Uu.uFp\V4NK up!p.p VP4F;wrg\^\V4V4FpV PWuWX,4K K= V U u.uFqV ^,,NK p p W3#uupiuupiuup i)a| Group layers depending on the attention mix, according to VLLM's hybrid allocator rules: - Layers in each group need to have the same type of attention - All groups have the same number of layers For a model with the following layer types: ["sliding", "full", "full", "sliding", "full", "full", "full", "full"] We would get four groups: [0, 3], [1, 2], [4,5] and [6,7]. layer_typesNsliding_windowsliding_attentionfull_attention) r#rangenum_hidden_layers enumerategetrvalueslenitemsappend) rr1 attn_type_ layer_countsi layer_typeindices group_size layer_groupslg group_typess & rgroup_layers_by_attn_typerG8s2&-6K+26;KT+R+^'dt */0H0H*IJ*IQy*I JL";/ #/#3#3J#Cqc#I  0<3F3F3HI3Hs7|3HIJJL+113 q#g, 3A   AN ; <4 41== "r!u%% K=  $$#KJ>s DD!?D&ca]tRt^YtoRt^t]P3V3RlRlltV3RlRlt V3RlRlt V3RlR lt V3R lR lt V3R lR lt V3RlRltV3RlRltV3RlRltV3RlRltV3RlRltV3RlRltV3RlRltV3RlRltV3RlRltV3R lR!ltV3R"lR#ltV3R$lR%ltR&tVtR'#)(PagedAttentionCacheu Manages the cache for a paged attention mechanism, inspired by VLLM's hybrid allocator. The cache relies on making groups of layers to reduce the complexity of cache management and fragmentation. The cache uses a three-level hierarchy: - Pages: The smallest unit of cache, a page has a size of [num_heads, head_size], which is the space needed to store the key or value states for one token and one layer. For a model with only full-attention layers, to store the KV cache of one token, we need `2 * num_layers` pages: key and values each take `num_layers` pages. Pages are grouped into blocks: - Blocks: A block is a collection of `block_size` pages, serving as the allocation unit to reduce management complexity and fragmentation. Cache is allocated and freed block by block, not page by page. One block is allocated to one layer group, which only has one attention type, like full-attention or sliding-attention. If all layers in the model have the same attention type, then all layers will be in the same group. There is more than one group if and only if the model has a mixed attention types, like layers with full-attention and layers with sliding-attention. - Cache tensors: The physical supports for the cache. There are as many cache tensors as there are layer in a layer group, and the shape of the cache tensor is `[num_blocks * block_size, num_heads, head_size]`. Grouping layers into groups is useful because when we allocate one block to a group N, the block allocated is the same for all layers in group N, equivalently it is allocated across all cache tensors. This allows us to efficiently allocate and free blocks, and to efficiently read and write key and value states. For instance, imagine we have 8 blocks of cache and a model with two layer groups: a full-attention group with 3 layers and a sliding-attention group with 3 layers. At creation time, the physical cache tensors look like this: cache_tensor_0: □ □ □ □ □ □ □ □ cache_tensor_1: □ □ □ □ □ □ □ □ cache_tensor_2: □ □ □ □ □ □ □ □ where □ means the blocks is not allocated to any layer group yet. We have 3 cache tensors because there are 3 layers per group. We allocate 1 block to each group, after allocation, the cache tensors look like this: cache_tensor_0: ✖ ◉ □ □ □ □ □ □ cache_tensor_1: ✖ ◉ □ □ □ □ □ □ cache_tensor_2: ✖ ◉ □ □ □ □ □ □ where ✖ means the block is allocated to the full-attention group, and ◉ means the block is allocated to the sliding-attention group. Now, if we continue to generate, and the sliding window has been reached, we only need to allocate a new block for the full-attention group, and the cache tensors look like this: cache_tensor_0: ✖ ◉ ✖ □ □ □ □ □ cache_tensor_1: ✖ ◉ ✖ □ □ □ □ □ cache_tensor_2: ✖ ◉ ✖ □ □ □ □ □ And after further generation, when we need a new block allocated: cache_tensor_0: ✖ ◉ ✖ ✖ □ □ □ □ cache_tensor_1: ✖ ◉ ✖ ✖ □ □ □ □ cache_tensor_2: ✖ ◉ ✖ ✖ □ □ □ □ This would not have been possible if all layers were in the same group: we would have had to allocate a new block for the sliding-attention group, although it is not needed. c<V^8dQhRS[RS[RS[PS[,RS[RS[S[S[3,RS[PRR/#) rrcontinuous_batching_configdevicedistributed_helpertp_plandtyperN) rrtorchrLr/rdictrrO)r __classdict__s"rr PagedAttentionCache.__annotate__soZ%Z% Z%%=Z% s" Z% . Z% c3h Z%{{Z% Z%rc  DWnW`nW0n\V4Vn\ V4VnVPVnVPVP8d&\RVP RVP 24h\V4wrx\V^,4p \V4Vn /Vn /Vn\V4FRwrW,R8Xd VP M^p \V 4F#wrW3VPV&WPV&K% KT RpRFpVV9dK RV,V9dKRpM VP"pV^8dVV'dNVPV,^8wd\RVP RV R 24hV;PV,unVP$f\'VRR 7\)VVVPVV R 7P+4wppV^8d\,P.!VV.VP\,P0R 7pVP3V4\5V^,P744\5V^,P744ppVVnVVnVP:VP,Vn\>P@!RVP8: RVP:: RVP: 24.Vn!.Vn"V^,VPVPVP 3pV^,VP,VPVP 3Vn#VVP,Vn$VVP,^,Vn%V^,VP,Vn&\OV 4EF4p\,PP!VPFVPVPR7p\,PP!VPFVPVPR7p\,PRPUV4\,PRPUV4VPBPWV4VPDPWV4VPYV4V,P[^4VPYV4V,P[^4EK7 \>P@!RVPF: RVPB^,P\: RVPB^,P_4: 24VP`Vn0.Vn1^Vn2^Vn3^Vn4\V4Fwp pVR8Xd<\kWPVP`R7pV;Pd^, un2MvVR8Xdb\mWPVP VPJVPL4pV;Pf^, un3VPnVn4M\RV 24hVPbPWV4K VPppVPf^8d+VeV^8d\>Pr!R4^Vn8^pVf VPtpVVn8VP`;'dVR.8HVn;\yVVPV^8R7Vn=^Vn>R Vn?R #)aInitialize a paged attention cache for efficient memory usage. Also turns in prefix sharing if the model has only full attention layers. Args: config: Model configuration continuous_batching_config: Continuous batching configuration containing cache parameters device: Device for the cache tensors distributed_helper: TP-aware helper. Used to dispatch attention heads and ensure coherent cache size tp_plan: Tensor parallelism plan dtype: Data type of the activation and the cache (for now, these are the same) zBlock size must be at least z , but got r3Tzmodel.FzNumber of key value heads z+ must be divisible by tensor parallel size .N) cb_confighas_logit_processors)rrKrOrFrCrLrOz1Paged cache initialized: self.max_batch_tokens = z, self.num_blocks = z, self.block_size = )rOrLzself.cache_shape = z self.key_cache[0].shape = z self.key_cache[0].numel() = r4)allow_block_sharingzInvalid group type: zHSliding window attention groups detected: disabling block table support.)tp_on)zlayers.*.self_attn.k_projzlayers.*.self_attn.v_proj)@rrOrLr&r r+r) block_size_min_block_sizer$rGr: num_groupssliding_windowslayer_index_to_group_indicesr7r2tp_sizemax_memory_percentrPagedAttentionMemoryHandler%infer_max_batch_tokens_and_num_blocksrPtensorint64tp_all_reduce_minritemmax_batch_tokens num_blocks num_pagesrinfo key_cache value_cache cache_shaperead_trash_indexsentinel_indexwrite_trash_indexr5empty_dynamomark_static_addressr<viewfill_shapenumelrYgroup_cache_managersnum_full_attention_groupsnum_sliding_attention_groups%max_sliding_window_blocks_per_requestr r _max_blocks_per_requestmax_blocks_per_requestwarningfallback_max_blocks_per_requestuse_prefix_sharingr _block_manager_total_prefix_length_block_table_key)selfrrKrLrMrNrOrDrFrCr@groupr2jlayerkv_is_tpkeyr`rhrisyncblock_based_shaper>new_layer_key_cachenew_layer_value_cache group_typecmr~s&&&&&&& r__init__PagedAttentionCache.__init__s(   ):&(A *62 5?? ??T11 1;Df$E! a) l+!,.)!,/HA6AnH[6[V22abN%e,<=611%8.<$$U+-0MC7Nhn&? N%,, Q;8'''1Q6 01I1I0JJuv}u~~A  $ $ 0 $ & 8 8 @ &1Kbf g'B'A**#! ( 0 / 1 %* Q;<t{{Z_ZeZefD  0 0 6+.tAw||~+>DGLLN@Sj !1$4??: H0E0E/II^DOOK__tbfbqbqauvw.0/1 (!^T__d>V>VX\XeXef'!^t>@X@XZ^ZgZgh *T__ <(4??:Q>",q.DOO!Cz"A"'++d.>.>djjY]YdYd"e $)KK0@0@ [_[f[f$g ! MM - -.A B MM - -.C D NN ! !"5 6    # #$9 :  $ $%6 7 C I I! L ! & &'8 9* E K KA N#  *t''++GT^^A->-D-D,HHf$..YZJ[JaJaJcIghi$>#Q#Q :<!)*&,-)562&{3MAz--0OOY]YqYqr..!3.223(=(=t?R?RTXTjTj11Q61=?=W=W: #7 |!DEE  % % , ,R 04"4PagedAttentionCache.blocks_in_use..Is,_E^r3r~~))*b9::E^s25)sumry)rrs&fr blocks_in_use!PagedAttentionCache.blocks_in_useGs_TE^E^___rc @<V^8dQhRS[RS[RS[RS[R,/#)rn_blocksrrrN)rr/)rrRs"rrrSKs1    PS X[^bXb rc VPW4'gR#^pVPF?pVPWVP4pVf\ RV RV 24h\ WF4pKA V#)zAllocate cache blocks across all layer groups for a given request. Actual allocation is done by the cache managers, and this method only returns the maximum number of blocks actually allocated across all managers.NzFailed to allocate z blocks for request )rryallocate_blocksrr$r)rrrr max_allocatedrnum_allocated_blockss&&&& rr#PagedAttentionCache.allocate_blocksKsz11(MM ++B#%#5#5hDL_L_#` #+ #6xj@TU_T`!abb DM , rc$<V^8dQhRS[RR/#)rrrN)r/)rrRs"rrrSZs<%> !L B  !5!5j|!\ ]"M  !$'(A(A:$N ##B$7$7 Q]$^_%O "rc J<V^8dQhRS[RS[RS[RS[PRR/#)rrrrrrN)r/rrPTensor)rrRs"rrrSxsBWWW,/W?BWQVQ]Q]W Wrct\VP4FwrVVPWW4V,4K R#)N)r7ryfill_block_table)rrrrrr@rs&&&&& rr$PagedAttentionCache.fill_block_tablexs0t889EA    ST~ V:rcB<V^8dQhRS[RS[RS[S[S[3,/#)rrrr)rrQr/)rrRs"rrrS~s*   C DcN rc /pVP^8d W,VR&VP^8d1V\WPP^, 4,VR&V#)zRetrieve the key sequence length for the given request_id across all layer types. Returns a dictionary of layer types to their corresponding key sequence lengths.r4r3)rzr{rrr2)rrr seqlens_ks&&& r get_seqlens_k!PagedAttentionCache.get_seqlens_k~s^  ) )A -*5*DI& '  , ,q 0-9C [[MgMgjkMk;>LL;>ll;> ;> & ;> %,,' ;> u||U\\) *;>rc VPV,wrgWF,pWV,p VPV,p VPV,p VP^^4P ^4pVP^^4P ^4pVP 4^8Xd(V P ^W4V P ^W4W3#VPV,p V ^8XdXV P ^W4V P ^W4\P!V ^V4p \P!V ^V4pW3#WP8HPR4PR4p\P!V ^V4p V PW4\P!V ^V4pVPW4V P ^W4V P ^W4W3#)aUpdate the cache with new key-value states for a specific layer, and retrieves the relevant KV states from the cache for attention computation. The behavior differs based on the layer's attention type: - Full attention: New KV states are written to cache, then complete sequence is read from cache - Sliding window: Old KV is read from cache along with extra spaces for the new KV, then new KV is written to cache. This is because new KV might overwrite the old KV, so we need to read the old KV first. When the layer's read index is empty, the batch has no cache reads (all requests are non-chunked prefills): we only write to the cache and return the input KV states directly, skipping the index_select read-back. Returns the complete KV states (cached + new) for attention computation. ) r_rlrm transposesqueezerx index_copy_r^rP index_selectrp unsqueezemasked_scatter_)rrrrrr group_idxlayer_idx_in_grouplayer_read_indexlayer_write_indexk_cachev_cacher2key_states_with_cachevalue_states_with_cachemasks&&&&&& rupdatePagedAttentionCache.updates*)-(I(I)(T% %0'2..!34""#56))!Q/77: #--a3;;A>   ! ! #q (   #4 A   #4 C+ +--i8 Q    #4 A   #4 C$)$6$6wCS$T !&+&8&8!EU&V #"%==%(;(;;FFrJTTUWXD$)$6$6wCS$T ! ! 1 1$ C&+&8&8!EU&V # # 3 3D G   #4 A   #4 C%==rc&<V^8dQhRS[RS[/#)rflash_attn_with_kvcache_fnr)rr/)rrRs"rrrSs%%c%c%rc >VPf\P!V4PP 4pRV9dRVnVP#RV9dRVnVP#\ R\P!V4 24hVP#)zA function to get the name of the block table key for the given flash_attn_with_kvcache_fn. The function's signature is only inspected once. This is necessary because different version of flash have different names for the block table key.r page_tablezOflash_attn_with_kvcache_fn does not have a block_table or page_table argument: )rinspect signature parameterskeysr$)rr kwarg_namess&& rget_block_table_key'PagedAttentionCache.get_block_table_keys  (!++,FGRRWWYK +(5%$$$ ,(4% $$$!efmfwfwySgTfUV$$$rc<<V^8dQhRS[RS[S[,RS[/#)rr prompt_idsr)r/r.r)rrRs"rrrSs&ctCySrc Rp.p\\V4VP,4FpW%VP,V^,VP,pVPP W6^R7pVPP P V4pVe/VPV4VPPV4KM V'dH\P!RV R\V4 R24VP^,pWHPV&\V4VP,p V;PV , un V #)a Searches for a prefix match in the cache for the given (prompts_ids). If one is found, we reference the matching blocks in the (request_id), increase the reference count of the blocks and return the number of blocks that match. If no prefix match is found, we return 0.N)group_idzFound prefix match for request z with z blocks)r5r:r[r compute_hash _hash_to_idr8r<increase_ref_countrdebugryrr) rrr current_hashrbtokensblock_idr prefix_lengths &&& rsearch_prefix_match'PagedAttentionCache.search_prefix_matchs s:$//9:ADOO 3q1u6OPF..;;L[\;]L**66::<HH# ''1##66x@;  LL::,fSQaMbLccjk l**1-B)9NN: &,-?  !!]2!rc*<V^8dQhRS[RS[RR/#)rstatenum_complete_blocksrN)rr)rrRs"rrrSs$|Z]bfrc LV^8Xg VP\P8XdR#VPFkpVP'gKVP P VVPVP,VPVP,R7Km R#)aMarks the blocks allocated to a request (state) as complete if they are shareable and they have been computed in the forward pass. A complete block is a block where the KV cache has been fully computed: if the block has enough space to hold the cache for N tokens, the block is marked as complete when the cache data is present for the N tokens. If block sharing is off, this is a no-op.N)r rr) statusrFINISHEDryuses_block_sharingr!mark_shareable_blocks_as_completerrinitial_tokensgenerated_tokens)rr r rs&&& rr5PagedAttentionCache.mark_shareable_blocks_as_completes ! #u||}7M7M'M++B$$$##EE(;%'^^E4D4D%E % 4 4u7M7M MF,rcJ<V^8dQhRS[S[,RS[S[,RR/#)rlist_source_blockslist_forked_blocksrN)r.r)rrRs"rrrSs1DDT#YDDQTIDZ^Drc \P!WP\PR7p\P!W P\PR7p\ VP VP 4F}wrVVPRVPVPVP4pVPRVPVPVP4pWS,WT&Wc,Wd&K R#)z;Copy the cache from the source blocks to the forked blocks.rXNr) rPrdrLint32rrlrmrur[r r))rrr source_blocks forked_blocksrlrms&&& r copy_cachePagedAttentionCache.copy_caches %7 SXS^S^_  %7 SXS^S^_ &)$..$:J:J&K "I!r4??D>#6#6#8 9,)J   Z (*r)rrrrYr[rnrrLrOryr)rlr_rhr~r|rirzr]r rjr{rorpr^rrmrqN)__name__ __module__ __qualname____firstlineno____doc__r\rPfloat16rrrrrrrrrrrrr rrr'r0r6__static_attributes____classdictcell__rRs@rrIrIYs6pO#]]Z%Z%x&hh``  << 33``(WW  ;>;>z%% 4"DDDD$ 1 1))rrIca]tRtRtoRtRtRtV3RlRlt]V3RlRl4t V3R lR lt V3R lR lt V3R lRlt V3RlRlt V3RlRltV3RlRltV3RlRltV3RlRltRtVtR#)rbi5uDetermines the optimal max batch tokens (M) and number of blocks (N) for the paged attention cache, given available GPU memory. The relation between N and number of blocks is: num_blocks = N // block_size. The memory footprint is a polynomial in M and N, where each term maps to a tensor allocated in ``ContinuousBatchingIOs._setup_static_tensors`` or ``PagedAttentionCache.__init__``: memory(M, N) = coeff_m · M + coeff_n · N + coeff_mn · M·N + coeff_mm · M² See ``_equation_coefficients`` for the breakdown. All three solving modes (auto, fixed-N, fixed-M) reduce to solving this equation, which is at most quadratic in one variable. i c `<V^8dQhRS[RS[RS[PRS[S[,RS[RR/#)rrrKrOrFrCrN)rrrPrOr.r/r)rrRs"rr(PagedAttentionMemoryHandler.__annotate__EsO%<%< %<%=%<{{ %< #Y %<  %< %$N$N$NqTU"<"O"O"OQUV $ 9 9 ;rcJ<V^8dQhRS[S[S[S[R3,3,/#)rr.)rQr/r-r)rrRs"rrrDms&c5c?&:!;rcPVPP\VP4,pVPp/pVPPVP P ,pW@PP\PP ,, pV^^^3VR&VP P VPPV,^V,,,p^V,VP P ,pWE^^3VR&V#)rlm_head attention) rr!r+rHr*rGitemsize vocab_sizerPfloat32)rmem_per_q_tokenmem_per_k_or_v_tokenpeaksdelta_mdelta_ns& ractivation_peak+PagedAttentionMemoryHandler.activation_peakls++99M$++totalreserved allocatedrOs& rrN0PagedAttentionMemoryHandler.get_available_memorys`)H(I%( 3y#;;/..2S2SST =>NRY>Y=ZZ]^_rc6<V^8dQhRS[S[S[3,/#rr-r)rrRs"rrrDsCCuS#XCrc VPPpVPPpVeVeVPW4#VfVe'VP WRR7wrVPW4#VP RRRR7wr4\ VP V4p\WP4pVP WRR7wrVPW4#)aHInfers max_batch_tokens and num_blocks based on the available memory and the size of the activation peaks. If neither value is provided, we use a default value of 8192 for max_batch_tokens, apply bounds depending on the available VRAM, and solve for num_blocks. If one value is provided, the other is found using a linear solve.N)cache_fill_per_batchg?)rhrirh) rVrhri_check_footprint_solve_for_peaksr_default_max_batch_tokensr_min_max_batch_tokens)rrhriupper_bound_vramr>s& rrcAPagedAttentionMemoryHandler.infer_max_batch_tokens_and_num_blockss >>::^^..   'J,B(()9F F  ':+A+/+@+@ 4,A, ( (()9F F#33!!$4  t==?OP/1K1KL'+'<'<=Mpt'<'u$$$%5BBrc r<V^8dQhRS[R,RS[R,RS[R,RS[S[S[3,/#)rrhNrirhr)rfloatr-)rrRs"rrrDsG  * $J $dl  sCx  rc 2.pVPP4F)pVPWQW#4wrgVPWg34K+ \ VUu.uF q^,NK up4p \ VUu.uF q^,NK up4p W3#uupiuupi)a}Returns max_batch_tokens and num_blocks so that their memory footprint is within the available memory for all activation peaks. If neither value is given, a value must be provided for cache_fill_per_batch: this means we solve for both variables by saying each batch fill a certain percentage of the cache (eg, if cache_fill_per_batch is 0.01, each batch will fill 1% of the cache).)r]r9_solve_for_peakr<r) rrhrirh solutions peak_deltasmnsolutionfinal_mfinal_ns &&&& rrj,PagedAttentionMemoryHandler._solve_for_peakss //668K'' zhDA   aV $99=9x{{9=>9=9x{{9=>>=s B4Bc <V^8dQhRS[S[R3,RS[R,RS[R,RS[R,RS[S[S[3,/#)rpeak.rhNrirhr)r-rrp)rrRs"rrrDsY!,!,CHo!,*!,$J !, $dl !, sCx !,rc VPV4wrVrxVfVfVf \R4hTp VPWy,W^,,,WeV ,,VP)4p \ W,4p\ V 4VP ,pW#3#VfbTp \ VPW[,, W^,,, WgV ,,, 4p WP ,pW#3#VfRW0P ,p \ VPWW|,,Wl,VP, 44pW#3#)zvReturns a couple of `(max_batch_tokens, num_blocks)` that satisfy the memory constraint for the given activation peak.z>m must be provided if max_batch_tokens and num_blocks are None)_equation_coefficientsr$_solve_quadraticrOrr[r) rr|rhrirhrcncmncmmrurjMNs &&&&& rrr+PagedAttentionMemoryHandler._solve_for_peaks4 66t<  # (:#+ !abb$A--cgd .BBaKRVRgRgQghI"9=1 Y4??:J ++  At44rv=d JrZ[T[|\]I"oo5J ++  %__,A"4#8#837lBFUYUjUjLj#kl ++rcB<V^8dQhRS[RS[RS[S[S[3,/#rrhrir)rr-)rrRs"rrrDs- , , ,# ,%PSUXPX/ ,rc VPW4pW0P8d\RV RVP 24hV^8:gV^8:d\RV RV 24hW3#)zDChecks if the footprint of the cache is within the available memory.zMemory footprint z is more than available memory z#Invalid values: max_batch_tokens = z, num_blocks = )compute_memory_footprintrO MemoryErrorr$)rrhrimemory_footprints&&& rri,PagedAttentionMemoryHandler._check_footprints889IV 33 3#$4#55TUYUjUjTkl  q J!OBCSBTTcdncopq q++rc2<V^8dQhRS[RS[RS[RS[/#)rarcr)rpr)rrRs"rrrDs)   %  E  e    rc 4V^8Xd\V)V, 4#V^,^V,V,, pV^8d\RV R24hV)\V4,^V,, pV^8d\RV R24h\\V44#)u_Largest positive root of a·x² + b·x + c = 0. Falls back to linear when a == 0. Rounded down.z!No real solution (discriminant = )zNo positive solution (root = )rr$rr)rrrr discriminantroots&&&& rr,PagedAttentionMemoryHandler._solve_quadratics 6rAv; !ta!eai' ! @aPQ QT,''AE2 !8r?r@s@rrbrb5s   $%<%rs\!! 3F9tt*6ZZ p w%BY)Y)xy$y$r