+ jLp^RIt^RIHt^RIHtRRlt]P PRRl4tR#)N)PagedAttentionCache)!lazy_import_paged_flash_attentionc*V^8dQhR\PPR\PR\PR\PR\PR,R\R\PR \P\ \ \P3,,R \R \\ \ \3,,R \PR,R \\PR3,/ #)moduleqkvattention_maskNcache cu_seq_lens_q cu_seq_lens_k max_seqlen_q max_seqlen_k block_tablereturn) torchnnModuleTensorrdictstrinttuple)formats"f/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/integrations/flash_paged.py __annotate__rsZZ HHOOZ ||Z ||Z || Z LL4' Z  Z<<Z<<$sELL'8"99ZZS#X&Z$Z 5<< Zc pVPR,VPR,rW8wd3\PPP V^W, .4p\ VP P4wr\VRR4'gRMVP^, ^3pVR8XdRMRp\V\4'dVV,pV V,p V EfVPVVVPV R,V R,R7wr#R V 9dR V PR 4/M/pV!VP^^4P!^4P#4VP#4VP#4VP%\P&4VP%\P&4P)4VV 3R VP*R R R V/VBp\V\,4'd V^,pM$R V 9d R V R ,/M/p\/WW#WWVW3 /VBpW8wdVRRV 13,pVR3#)a Performs the forward pass of attention with paged key-value cache. This function handles the cache updates and performs the attention computation. For decode-only batches (when block_table is provided), uses `flash_attn_with_kvcache` for fused attention + cache update. Otherwise uses `flash_attn_varlen_func`. See the [paged attention guide](https://huggingface.co/docs/transformers/en/paged_attention) for more details. Args: q: (1, nheads, total_q, headdim), where total_q = total number of query tokens in the batch. k: (1, nheads_k, total_k, headdim), where total_k = total number of key tokens in the batch. v: (1, nheads_k, total_k, headdim), where total_k = total number of key tokens in the batch. cu_seq_lens_q: (batch_size + 1,), dtype torch.int32. The cumulative sequence lengths of the sequences in the batch, used to index into q. cu_seq_lens_k: (batch_size + 1,), dtype torch.int32. The cumulative sequence lengths of the sequences in the batch, used to index into kv. max_seqlen_q: int. Maximum query sequence length in the batch. max_seqlen_k: int. Maximum key sequence length in the batch. block_table: (num_groups, batch_size, max_blocks_per_seq), dtype int32. Block table for paged KV cache. If provided, uses flash_attn_with_kvcache for fused attention + cache update. For each request, the block table is a vector of size (max_blocks_per_seq,) with indices indicating the physical location of the cache to read from and write to. The kernel, using the cache_seqlens for that request, knows how much cache to read and dispatches the read using the block table. Same for the write. If a request has fewer than max_blocks_per_seq blocks, the block table is padded with -1s to indicate that the block is not allocated. sliding_windowFfull_attentionsliding_attentionN read_index write_index) key_states value_states layer_idxr#r$s_aux softmax_scalecausalT window_size.)r,r,)shaperr functionalpadrconfig_attn_implementationgetattrr isinstancerupdater'get transposesqueeze contiguoustoint32clonescalingr_paged_decode_forward)rrr r r r r rrrrkwargshead_dim v_head_dimflash_attn_varlen_funcflash_attn_with_kvcacher layer_type custom_kwargs attn_output flash_kwargss&&&&&&&&&&&, rpaged_attention_forwardrGs/L772; j HH   # #A8+@'A B7X **73 &-V5Eu%M%MXTZTiTilmTmopSqN%3x%?!EXJ-&&%j1 #J/ ||&&l+}-  ;BV:K&**W"56QS , KK1  % %a ( 3 3 5 LLN LLN   U[[ )   U[[ ) / / 1    !..    '     k5 ) )%a.K6=5F1B + qU>CZ kw !#{ {"23  rcFV^8dQhR\PPR\PR\PR\PR\R\PR\ \ \ 3,R\PR \P/ #) rrrr r r rr rr)rrrrrrr)rs"rrres."." HHOO." ||." ||." || ."  ." << ."#s(O."." \\."rc  VPVP,wrVPV ,PR VPVP VP 4p VPV ,PR VPVP VP 4p VP^^^^4P4pVP^^^^4P4pVP^^^^4P4pVP^4pV^V^,VRV, ^, P\P4pW,WPV4&V!R RVRV RV RVRVRVRVPR R R V/ V Bp\!V\"4'd V^,pVP%^4#)zaDecode fast path using flash_attn_with_kvcache. Disabled because FA3 has issue with tracing this.Nrk_cachev_cacher r cache_seqlensr)r*Tr+r,)layer_index_to_group_indicesr' key_cacheview block_sizenum_key_value_headsr? value_cachepermuter8sizer9rr:get_block_table_keyr<r3rr7)rrr r r rr rBrrF group_idxlayer_idx_in_grouprJrK batch_sizerLrEs&&&&&&&&&, rr=r=ds%*$F$FvGWGW$X!Ioo0166r5;K;KUMfMfhmhvhvwG 2388 E  e77G !Q1((*A !Q1((*A !Q1((*AJ"1zA~6{ 9SSVWW[[\a\g\ghMGRG]L**+BCD)        $ nn  #   K+u%%!!n   q !!r) rgeneration.continuous_batchingrmodeling_flash_attention_utilsrrGcompilerdisabler=rMrrr^s3 @NZz."."r