+ "ݳj1^RIt^RIt^RIHt^RIHtHtHtHt^RI t ^RI H t H t ^RI H t ^RIHtR RRlltRR R lltRR R lltR#)N)partial)CallableDictListOptional) LongTensorTensor) DataLoader)PreTrainedTokenizerc"V^8dQhR\\\\,3,R\R\R\R\\ ,R\ R\ R\R \R \R \\\\,3,/ #) samplesprompt_col_namelabel_col_name tokenizer preprocess_fnsample_max_len block_max_len add_eos_tokentruncate_promptmerge_prompt_labelreturn) rstrrr rrintboolr)formats"U/home/luca/screen-checking/.venv/lib/python3.14/site-packages/gptqmodel/utils/data.py __annotate__rs}} #tCy. !}}}# } H% }  }}}}} #tJ  }c a V'd V!V4pW,p W,p V!V RR7R,p V!V RR7R,p .p\\W44FwpwppV'dVVP., p\V4p\V4pVV,V, pV^8dV'dVVRpMVRV)pVW&VW&V'dKxVP V4K \ \\W44UUUu.uFwpwppW9gKVV3NK upppV 3RlR7p.p.p^p^pVFpVwpp\V4pS 'dV\V4, pVV8:dTp Tp!M"\V4VV, ,V,p Tp!VV ,V8d2VP \ P !V4V34.p^p^pTp!Tp VP V4\VV!4pVV , pK V'd(VP \ P !V4V34???R.R.R./p"VEFwp#p.p$.p%.p\V#U&u.uFp&\V&^,4NK up&4p'V#EF#p&V&wpp\V4p!S 'dV!\V4, p!VV!, p(S 'dcV$P VP.V(,V,V,4VP R .V(\V4,,V,4MZV$P VP.V(,V,4VP R .V'\V4, ,V,4V%P ^.V(,^.V!,,4EK& V"R,P V$4V"R,P V%4V"R,P V4EK V"#uupppiuup&i) aA simple implementation of text generation oriented smart batching to maximize VRAM usage when evaluation :param samples: Dict[str, List[str]], samples that used to make data blocks :param prompt_col_name: str, name of the key in samples whose value stores prompt :param label_col_name: str, name of the key in samples whose value stores label :param tokenizer: transformers.PretrainedTokenizer, tokenizer that used to tokenize samples :param preprocess_fn: Optional[Callable], optional function that used to preprocess samples such as refactor the data structure of samples, note the output of this function must be a dict whose keys at least contains `prompt_col_name` and `label_col_name` :param sample_max_len: int, defaults to 1024, max tokens number of each sample (before padding) :param block_max_len: int, defaults to 2048, max tokens number of each data block (after padding) :param add_eos_token: bool, defaults to False, whether add eos_token or not to the label :param truncate_prompt: bool, defaults to True, whether to truncate prompt if the sample's total tokens number exceeds `sample_max_len`, if not, will truncate label and drop this sample when all tokens in label are truncated :param merge_prompt_label: bool, defaults to False, will merge label into prompt if set to True, usually this only required when doing language modeling task :return: Dict[str, List[torch.LongTensor]], a dict whose keys are `input_ids`, `attention_mask` and `label` and values are a list of torch.LongTensor F) truncation input_idsNc<S'd*\V^,4\V^,4,#\V^,4#)r)len)xrs&r!make_data_block..Ps-1Cs1Q4y3qt9,RQqTRr)keyattention_masklabelsi) enumeratezip eos_token_idr$appendsortedcopymax pad_token_id))rrrrrrrrrrpromptsr*tokenized_promptstokenized_labelsdropped_indicesidxtokenized_prompttokenized_label len_prompt len_label exceed_lenpltokenized_samples sample_blocks sample_block blk_max_len blk_total_lentokenized_sample prompt_ids label_idsori_sample_lenadditional_len sample_len new_samplesblockr"r)sample label_max_lenpad_nums)&&&&&&&&&f rmake_data_blockrOs@(&G  $F"'e<[I E:;GO4=cBS>f4g0 0   6 67 7O)* ( )+n< >#3JK#@ "1,J;"?!1 /  " "3 '5h"#,S1B-U#Vu#VKC!QZ]Zt!Q#Vu RMLKM- 0 IZ  c)n ,N [ ((N$J ..;2NOR``N'J > )M 9  $))L"9;!G HLKM'J+N,-+z2 ' -.0dii 5{CD$4b(BGK+{  %@%S^%@A F06 - o-.J!c/22 !J.G!  )"8"8!9G!CFV!VYh!hi  $7S9I5J+J!Ko!]^  )"8"8!9G!CFV!VW  $=3;O+O!PSb!bc  ! !1#-1# 2B"B C K '' 2$%,,^<H$$Y/-,0  vVAsO) O) O0 c V^8dQhR\\\\\\,,3,,R\R\R\\\3,/#)r batchr2 padding_sider)rrrrr )rs"rrrsVTT S$tCy/)* +TTT #v+ Trc a.p.pVFpVR,pVR,p\V4\V48XgQR4h\\V44Fp\P!Wh,\PR7p \P!Wx,\P R7p V P 4V P 48wd \R4hVPV 4VPV 4K K V'd\RV44M^oRRV3Rlllp RR V3R lllp VR 8XdT p MVR 8XdT p M\R V R24hVUu.uFq!W\PR7NK ppVUu.uFq!VR\P R7NK ppV'd\P!V^R7M%\P!R\PR7pV'd\P!V^R7M%\P!R\P R7pRVRV/#uupiuupi)ay Collate an outer batch (size B) of items, where each item holds multiple rows. We flatten the rows across items, pad to a global max length, and stack into [total_rows, max_len] tensors for HF Transformers. Each element of `batch` looks like: { "input_ids": List[List[int]], # rows "attention_mask": List[List[int]], # rows (0/1 ints, cast to bool here) } r"r)z2input_ids and attention_mask row counts must match)dtypez?Row has mismatched lengths between input_ids and attention_maskc3@"TFqP4xK R#5iN)numel).0ts& r collate_data..s.X''))XscXV^8dQhR\PR\P/#r rowrtorchr )rs"rr"collate_data..__annotate__s"   u||  u||  rc <SVP4, pV^8:dV#\P!T\P!V3Y;'g VPVP R7.^R7#r)rTdevicedimrWr`catfullrTrdr^ pad_valuerTpad_lenmax_lens&&& r right_padcollate_data..right_padsbCIIK' a<Jyy G:y8J8JSVS]S]^    rcXV^8dQhR\PR\P/#r]r_)rs"rrras"   ell  ell  rc <SVP4, pV^8:dV#\P!\P!V3Y;'g VPVP R7V.^R7#rcrgrjs&&& rleft_padcollate_data..left_padsbCIIK' a<Jyy G:y8J8JSVS]S]^    rleftrightzUnsupported padding_side `z`. Expected `left` or `right`.FrerV)rr) r$ranger` as_tensorlongrrW ValueErrorr.r1stackempty)rQr2rRrows_ids rows_maskitemids_listmsk_listridsmskrnrrpad_fnrY padded_ids padded_mskr"r)rms&&& @r collate_datars"HI $()8}H -c/cc-s8}%A//(+UZZ@C//(+UZZ@Cyy{ciik) !bcc OOC   S !&&3;c.X..G      v  5l^CabccEMNX& ;XJN>GHi&E4iJH3= JA.%++f\a\f\fBgI7AU[[3u{{SYafakakGlN Y. OHs  H8!H=c$V^8dQhR\R\R\R\R\\,R\\,R\R\R \R \ R \ R \ R \\ ,R\\ ,R\/#)r data_path_or_namerrrload_fnr num_samplesrrrrrload_fn_kwargspreprocess_fn_kwargsr)rr rrrrdictr )rs"rrrs[[[[[# [ h  [ H% [[[[[[[TN[#4.[ ![rc ^RIHpHpHpV 'g/p V 'g/p V'd V!V3/V BpM V!V3/V Bp\ VVV34'd,RV9d VR,pMRV9d VR,pM VR,pVP \ P!\\V44\\V4V44RR7pVP\R\V4^VPRRRVR VR VR \V3/V BR VR VRV RV RV / R7p^VR&RVR&\\VP R7VR&\#V3/VBpV#)aload dataset and build dataloader :param data_path_or_name: str, dataset name in hf-hub or local file path :param prompt_col_name: str, see `make_data_block` :param label_col_name: str, see `make_data_block` :param tokenizer: str, see `make_data_block` :param load_fn: Optional[Callable], defaults to None, function used to load dataset, if not specified, use `datasets.load_dataset` :param preprocess_fn: Optional[Callable], see `make_data_block` :param num_samples: int, defaults to 128, total samples used to evaluation :param sample_max_len: int, see `make_data_block` :param block_max_len: int, see `make_data_block` :param add_eos_token: bool, see `make_data_block` :param truncate_prompt: bool, see `make_data_block` :param merge_prompt_label: bool, see `make_data_block` :param load_fn_kwargs: Optional[dict], defaults to None, keyword arguments used for `load_fn` or `datasets.load_dataset` :param preprocess_fn_kwargs: Optional[dict], defaults to None, keyword arguments used for `preprocess_fn` :param kwargs: additional keyword arguments will be passed to torch's `DataLoader` initialization, note values of `batch_size`, `shuffle` and `collate_fn` will always be overridden to fixed value :return: torch.utils.data.DataLoader ) DatasetDictIterableDatasetDict load_dataset evaluationtesttrainT)indiceskeep_in_memoryFrrrrrrrrr)batched batch_sizenum_procremove_columnsrload_from_cache_file fn_kwargsrshuffle)r2 collate_fn)datasetsrrr isinstanceselectrandomrLrvr$minmaprO column_namesrrr2r )rrrrrrrrrrrrrrkwargsrrrdsdls&&&&&&&&&&&&&&, rget_dataloaderrsfPHG  ! & 9. 9 + >~ >"{$7899 2 L!B r\FBGB  eCGnc#b';.GH  B r7"  n  W]K6JK n ] ]  "4    B,F<F9"T>TUF< B !& !B Ir)NFTF)ru) NNrrFTFNN)r0r functoolsrtypingrrrrr`rr torch.utils.datar transformersr rOrrrrrs<  11 $',}~Tn[[r