+ jRt^RIHt^RIHt^RIt^RIHt^RIHtH t H t ^RI H t ^RIHtHt]!4'd^RIHt^R IHt^R IHt^R IHtHtHtHtHtHtHtHtH t ^R I!H"t"^R I#H$t$^RIH%t%^RI&H't']%PP!])4t*Rt+Rt,Rt-!RR]P\4t/!RR]P\4t0!RR]P\4t1!RR]P\4t2!RR]P\4t3!RR]P\4t4!RR ]4t5!R!R"]P\4t6!R#R$]P\4t7!R%R&]P\4t8!R'R(]P\4t9!R)R*]P\4t:!R+R,]P\4t;!R-R.]P\4t<]!R/R0]"44t=]!R1R27]!R3R4]444t>]!R5R6]=44t?]!R7R27!R8R9]=44t@]!R:R;]=44tA]!R]=44tB]!R?R27!R@RA]=44tC]!RBRC]=44tD]!RDRE]=44tE]!RFRG]=44tF.RHOtGR#)IzPyTorch FNet model.) dataclass)partialN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)auto_docstringis_scipy_available)linalg)ACT2FN)GradientCheckpointingLayer) BaseModelOutputBaseModelOutputWithPoolingMaskedLMOutput ModelOutputMultipleChoiceModelOutputNextSentencePredictorOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)PreTrainedModel)apply_chunking_to_forward)logging FNetConfigcVP^,pVRV1RV13,pVP\P4p\P!RWV4#)z4Applies 2D matrix multiplication to 3D input arrays.Nzbij,jk,ni->bnk)shapetypetorch complex64einsum)xmatrix_dim_onematrix_dim_two seq_lengths&&& g/home/luca/screen-checking/.venv/lib/python3.14/site-packages/transformers/models/fnet/modeling_fnet.py_two_dim_matmulr'5sLJ#KZK*$<=N uA <<(!^ LLc\WV4#N)r')r"r#r$s&&&r&two_dim_matmulr+>s 1n ==r(cTp\\VP4R,4F#p\PP WR7pK% V#)z Applies n-dimensional Fast Fourier Transform (FFT) to input array. Args: x: Input n-dimensional array. Returns: n-dimensional Fourier transform of input n-dimensional array. NN)axis)reversedrangendimrfft)r"outr/s& r&fftnr5Cs> Cqvvr*+iimmCm+, Jr(c@aa]tRt^StoRtV3RltRRltRtVtV;t #)FNetEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.c<\SV`4\P!VPVP VP R7Vn\P!VPVP 4Vn \P!VPVP 4Vn \P!VP VPR7Vn \P!VP VP 4Vn\P !VP"4Vn\P&!\(P*!VP4P-R4RR7Vn\P&!\(P0!VP.P34\(P4R7RR7VnR#)) padding_idxepsF) persistentdtypeNr.)super__init__r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingstype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsLinear projectionDropouthidden_dropout_probdropoutBufferrarangeexpand position_idszerossizelongtoken_type_idsselfconfig __class__s&&r&rBFNetEmbeddings.__init__Vs- !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2H2H&J\J\%]"f&8&8f>S>ST))F$6$68J8JKzz&"<"<= IIell63Q3Q&R&Y&YZa&botu ii D4E4E4J4J4LTYT^T^(_lqrr(cVeVP4pMVP4RRpV^,pVfVPRRV13,pVf\VR4'd4VPRRV13,pVP V^,V4pTpM;\ P !V\ PVPPR7pVfVPV4pVPV4p WI,p VPV4p W, p VPV 4p VPV 4p VPV 4p V #)NNNNrZr>devicer@)rXrVhasattrrZrUrrWrYrcrGrKrIrLrOrR) r\ input_idsrZrV inputs_embeds input_shaper%buffered_token_type_ids buffered_token_type_ids_expandedrK embeddingsrIs &&&&& r&forwardFNetEmbeddings.forwardfs:  #..*K',,.s3K ^  ,,Q  ^ J": "66|D) ^^J/ __Z0 \\*- r()rLrRrIrVrOrKrZrG)NNNN) __name__ __module__ __qualname____firstlineno____doc__rBrk__static_attributes____classdictcell__ __classcell__r^ __classdict__s@@r&r7r7SsQs !!r(r7c>aa]tRt^toV3RltRtRtRtVtV;t #)FNetBasicFourierTransformcF<\SV`4VPV4R#r*)rArB_init_fourier_transformr[s&&r&rB"FNetBasicFourierTransform.__init__s  $$V,r(cVP'g-\\PPRR7VnR#VP R8:Ed\4'd\P!\P!\P!VP4\PR74Vn\P!\P!\P!VP 4\PR74Vn\\$VP"VPR7VnR#\&P(!R4\VnR#\VnR#)r.dimr=)r#r$zpSciPy is needed for DFT matrix calculation and is not found. Using TPU optimized fast fourier transform instead.N)r.)use_tpu_fourier_optimizationsrrr3r5fourier_transformrHr rrStensorr dftrEr dft_mat_hiddentpu_short_seq_length dft_mat_seqr+rwarning)r\r]s&&r&rz1FNetBasicFourierTransform._init_fourier_transforms333%,UYY^^%HD "  + +t 3!##&(ii VZZHZHZ=[chcrcr0s&t##%99LLF,G,G!HPUP_P_`$ *1"43C3CTXTgTg*&**.&%)D "r(c>VPV4PpV3#r*)rreal)r\ hidden_statesoutputss&& r&rk!FNetBasicFourierTransform.forwards" ((7<<zr()rrr) rmrnrorprBrzrkrrrsrtrus@@r&rxrxs-**r(rxc8aa]tRt^toV3RltRtRtVtV;t#)FNetBasicOutputc<\SV`4\P!VPVP R7VnR#r:N)rArBrrLrErMr[s&&r&rBFNetBasicOutput.__init__s, f&8&8f>S>STr(c4VPW!,4pV#r*rLr\r input_tensors&&&r&rkFNetBasicOutput.forwards|'CD r(r rmrnrorprBrkrrrsrtrus@@r&rrsUr(rc8aa]tRt^toV3RltRtRtVtV;t#)FNetFourierTransformcd<\SV`4\V4Vn\ V4VnR#r*)rArBrxr\routputr[s&&r&rBFNetFourierTransform.__init__s& -f5 %f- r(c`VPV4pVPV^,V4pV3pV#)r\r)r\r self_outputsfourier_outputrs&& r&rkFNetFourierTransform.forwards1yy/ \!_mD!#r()rr\rrus@@r&rrs. r(rcDaa]tRt^toV3RltV3RlRltRtVtV;t#)FNetIntermediatec"<\SV`4\P!VPVP 4Vn\VP\4'd\VP,Vn R#VPVn R#r*) rArBrrNrEintermediate_sizedense isinstance hidden_actstrr intermediate_act_fnr[s&&r&rBFNetIntermediate.__init__s` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $r(cN<V^8dQhRS[PRS[P/#rrreturnrTensor)formatrvs"r& __annotate__FNetIntermediate.__annotate__s#U\\ellr(cJVPV4pVPV4pV#r*rrr\rs&&r&rkFNetIntermediate.forwards& =1 00? r(rrrus@@r&rrs9r(rcDaa]tRt^toV3RltV3RlRltRtVtV;t#) FNetOutputc0<\SV`4\P!VPVP 4Vn\P!VP VPR7Vn\P!VP4Vn R#r) rArBrrNrrErrLrMrPrQrRr[s&&r&rBFNetOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= r(ch<V^8dQhRS[PRS[PRS[P/#)rrrrr)rrvs"r&rFNetOutput.__annotate__s1U\\RWR^R^r(cxVPV4pVPV4pVPW,4pV#r*)rrRrLrs&&&r&rkFNetOutput.forwards5 =1  ]3 }'CD r()rLrrRrrus@@r&rrs> r(rc>aa]tRt^toV3RltRtRtRtVtV;t #) FNetLayerc<\SV`4VPVn^Vn\ V4Vn\ V4Vn\V4Vn R#r.N) rArBchunk_size_feed_forward seq_len_dimrfourierr intermediaterrr[s&&r&rBFNetLayer.__init__sI '-'E'E$+F3 ,V4 ( r(cVPV4pV^,p\VPVPVPV4pV3pV#r)rrfeed_forward_chunkrr)r\rself_fourier_outputsr layer_outputrs&& r&rkFNetLayer.forwardsO#||M:-a00  # #T%A%A4CSCSUc  /r(cJVPV4pVPW!4pV#r*)rr)r\rintermediate_outputrs&& r&rFNetLayer.feed_forward_chunks("//?{{#6G r()rrrrr) rmrnrorprBrkrrrrsrtrus@@r&rrs) r(rc<aa]tRt^toV3RltRRltRtVtV;t#) FNetEncoderc<\SV`4Wn\P!\ VP 4Uu.uFp\V4NK up4VnRVn R#uupi)FN) rArBr]r ModuleListr1num_hidden_layersrlayergradient_checkpointing)r\r]_r^s&& r&rBFNetEncoder.__init__sR  ]]uVE]E]?^#_?^!If$5?^#_` &+#$`sA&c@V'dRMRp\VP4F'wrVV'd WA3,pV!V4pV^,pK) V'd WA3,pV'g/\;QJd.RW34FNK 5#!RW344#\WR7#)Nc30"TF qfKVxK R#5ir*).0vs& r& &FNetEncoder.forward..sX$Fq$Fs )last_hidden_staterr) enumeratertupler)r\routput_hidden_states return_dictall_hidden_statesi layer_module layer_outputss&&&& r&rkFNetEncoder.forwards"6BD(4OA#$58H$H!(7M)!,M 5 14D D 5X]$FX5 X5X]$FXX X``r()r]rr)FTrrus@@r&rrs, aar(rcDaa]tRtRtoV3RltV3RlRltRtVtV;t#) FNetPooleric<\SV`4\P!VPVP4Vn\P !4VnR#r*)rArBrrNrErTanh activationr[s&&r&rBFNetPooler.__init__s9 YYv1163E3EF '')r(cN<V^8dQhRS[PRS[P/#rr)rrvs"r&rFNetPooler.__annotate__!s#U\\ellr(c\VR,pVPV4pVPV4pV#)ra)rar)rr)r\rfirst_token_tensor pooled_outputs&& r&rkFNetPooler.forward!s2+40 #56  6 r()rrrrus@@r&rrs$ r(rcDaa]tRtRtoV3RltV3RlRltRtVtV;t#)FNetPredictionHeadTransformi+c<\SV`4\P!VPVP4Vn\ VP\4'd\VP,Vn MVPVn \P!VPVPR7Vn R#r) rArBrrNrErrrrr transform_act_fnrLrMr[s&&r&rB$FNetPredictionHeadTransform.__init__,s~ YYv1163E3EF f'' - -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STr(cN<V^8dQhRS[PRS[P/#rr)rrvs"r&r(FNetPredictionHeadTransform.__annotate__5s#U\\ellr(clVPV4pVPV4pVPV4pV#r*)rrrLrs&&r&rk#FNetPredictionHeadTransform.forward5s4 =1 --m< }5 r()rLrrrrus@@r&rr+sUr(rc8aa]tRtRtoV3RltRtRtVtV;t#)FNetLMPredictionHeadi<c<\SV`4\V4Vn\P !VP VP4Vn\P!\P!VP44Vn R#r*) rArBr transformrrNrErDdecoder ParameterrrWbiasr[s&&r&rBFNetLMPredictionHead.__init__=sW 4V<yy!3!3V5F5FG LLV->->!?@ r(cJVPV4pVPV4pV#r*)rrrs&&r&rkFNetLMPredictionHead.forwardCs$}5  ]3 r()r rrrrus@@r&rr<sA r(rc8aa]tRtRtoV3RltRtRtVtV;t#)FNetOnlyMLMHeadiIcD<\SV`4\V4VnR#r*)rArBr predictionsr[s&&r&rBFNetOnlyMLMHead.__init__Js /7r(c(VPV4pV#r*r)r\sequence_outputprediction_scoress&& r&rkFNetOnlyMLMHead.forwardNs ,,_=  r(rrrus@@r&rrIs8!!r(rc8aa]tRtRtoV3RltRtRtVtV;t#)FNetOnlyNSPHeadiTcp<\SV`4\P!VP^4VnR#rN)rArBrrNrEseq_relationshipr[s&&r&rBFNetOnlyNSPHead.__init__Us'  " &*<*boolrr)rrvs"r&rFNetModel.__annotate__sD D ##d*D ((4/D &&- D ((4/ D #Tk D D[D   D r(c VeTMVPPpVeTMVPPpVeVe \R4hVeVP 4pVwrM'VeVP 4RR pVwrM \R4hVPP 'd.V R8:d'VPP V 8wd \R4hVe VPM VPp Vfy\VPR4'd6VPPRRV 13,p V PW4p T pM'\P!V\PV R7pVPVVVVR7pVPVVVR 7pV^,pVP eVP!V4MRpV'gVV3VR ,,#\#VVVP$R 7#) NzDYou cannot specify both input_ids and inputs_embeds at the same timez5You have to specify either input_ids or inputs_embedsrzThe `tpu_short_seq_length` in FNetConfig should be set equal to the sequence length being passed to the model when using TPU optimizations.rZrarb)rerVrZrf)rrr-)r pooler_outputrr@)r]rr ValueErrorrXrrrcrdrjrZrUrrWrYrCrDrr)r\rerZrVrfrrkwargsrg batch_sizer%rcrhriembedding_outputencoder_outputsrrVs&&&&&&&, r&rkFNetModel.forwards%9$D $++JjJj &1%cd d  "#..*K%0 "J  &',,.s3K%0 "J TU U KK 5 5 5d" 00J>;  &/%:!! @T@T  !t(899*.//*H*HKZK*X'3J3Q3QR\3i0!A!&[ SY!Z??%)' +  ,, !5#'  *!,8< 8O O4UY #]3ob6II I)-')77  r()r]rjrCrD)T)NNNNNN) rmrnrorprqrBrKrOr rkrrrsrtrus@@r&rArAs; /0D D D D r(rAz FNet Model with two heads on top as done during the pretraining: a `masked language modeling` head and a `next sentence prediction (classification)` head. cjaa]tRtRtoRRRR/tV3RltRtRt]R V3R lR ll4t R t Vt V;t #) FNetForPreTrainingicls.predictions.decoder.biascls.predictions.biascls.predictions.decoder.weight&fnet.embeddings.word_embeddings.weightc<\SV`V4\V4Vn\ V4VnVP 4R#r*)rArBrAr*r"clsrEr[s&&r&rBFNetForPreTraining.__init__s4  f% '/ r(cBVPPP#r*rdrrrJs&r&get_output_embeddings(FNetForPreTraining.get_output_embeddings xx##+++r(cWPPnVPVPPnR#r*rdrrr r\new_embeddingss&&r&set_output_embeddings(FNetForPreTraining.set_output_embeddings*'5$$2$7$7!r(cH<V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[R,R S[R,R S[S[,/ #) rreNrZrVrflabelsnext_sentence_labelrrr)rrrSrr9)rrvs"r&rFNetForPreTraining.__annotate__sC C <<$&C  t+C llT) C ||d* C  t# C #\\D0C #TkC D[C  ) )C r(c *VeTMVPPpVPVVVVVVR7p V R,wrVPW4wrRpVe|Vex\ 4pV!V P RVPP 4VP R44pV!VP R^4VP R44pVV,pV'g"W3V R,,pVe V3V,#T#\VV VV PR7#)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` next_sentence_label (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the next sequence prediction (classification) loss. Input should be a sequence pair (see `input_ids` docstring) Indices should be in `[0, 1]`: - 0 indicates sequence B is a continuation of sequence A, - 1 indicates sequence B is a random sequence. Example: ```python >>> from transformers import AutoTokenizer, FNetForPreTraining >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("google/fnet-base") >>> model = FNetForPreTraining.from_pretrained("google/fnet-base") >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> prediction_logits = outputs.prediction_logits >>> seq_relationship_logits = outputs.seq_relationship_logits ```NrZrVrfrr:NrNrNN)r;r<r=rr@) r]rr*rdrviewrDr9r)r\rerZrVrfrsrtrrrXrrrrr total_lossloss_fctmasked_lm_lossnext_sentence_lossrs&&&&&&&&&, r&rkFNetForPreTraining.forwards'L&1%(*R ,8C C C C r(r^cjaa]tRtRtoRRRR/tV3RltRtRt]R V3R lR ll4t R t Vt V;t #) FNetForMaskedLMiZr_r`rarbc<\SV`V4\V4Vn\ V4VnVP 4R#r*)rArBrAr*rrdrEr[s&&r&rBFNetForMaskedLM.__init__a4  f% "6* r(cBVPPP#r*rgrJs&r&rh%FNetForMaskedLM.get_output_embeddingsjrjr(cWPPnVPVPPnR#r*rlrms&&r&ro%FNetForMaskedLM.set_output_embeddingsmrqr(c <V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[R,R S[S[,/# rreNrZrVrfrsrrr)rrrSrr)rrvs"r&rFNetForMaskedLM.__annotate__rs(r(r<<$&(r t+(rllT) (r ||d* (r  t# (r#Tk(rD[(r  (rr(c VeTMVPPpVPVVVVVVR7p V ^,p VPV 4p Rp VeG\ 4p V !V P RVPP 4VP R44p V'g"V 3V R,,pV e V 3V,#T#\WV PR7#)a{ labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Nrwrxr;logitsrr@) r]rr*rdrryrDrr)r\rerZrVrfrsrrrXrrrr|r{rs&&&&&&&&, r&rkFNetForMaskedLM.forwardqs$&1%[b[p[pqqr(rNNNNNNNrrus@@r&rrZsL '(>(*R ,8(r(r(r(rr(rzT FNet Model with a `next sentence prediction (classification)` head on top. cRaa]tRtRtoV3Rlt]RV3RlRll4tRtVtV;t #)FNetForNextSentencePredictionic<\SV`V4\V4Vn\ V4VnVP 4R#r*)rArBrAr*rrdrEr[s&&r&rB&FNetForNextSentencePrediction.__init__rr(c <V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[R,R S[S[,/#r)rrrSrr)rrvs"r&r*FNetForNextSentencePrediction.__annotate__s? ? <<$&?  t+? llT) ? ||d* ?  t# ? #Tk? D[?  , ,? r(c VeTMVPPpVPVVVVVVR7p V ^,p VPV 4p Rp Ve3\ 4p V !V P R^4VP R44p V'g"V 3V R,,pV e V 3V,#T#\ V V V PR7#)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the next sequence prediction (classification) loss. Input should be a sequence pair (see `input_ids` docstring). Indices should be in `[0, 1]`: - 0 indicates sequence B is a continuation of sequence A, - 1 indicates sequence B is a random sequence. Example: ```python >>> from transformers import AutoTokenizer, FNetForNextSentencePrediction >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("google/fnet-base") >>> model = FNetForNextSentencePrediction.from_pretrained("google/fnet-base") >>> prompt = "In Italy, pizza served in formal settings, such as at a restaurant, is presented unsliced." >>> next_sentence = "The sky is blue due to the shorter wavelength of blue light." >>> encoding = tokenizer(prompt, next_sentence, return_tensors="pt") >>> outputs = model(**encoding, labels=torch.LongTensor([1])) >>> logits = outputs.logits >>> assert logits[0, 0] < logits[0, 1] # next sentence was random ```Nrwrxrr@)r]rr*rdrryrr)r\rerZrVrfrsrrrXrrseq_relationship_scoresr}r{rs&&&&&&&&, r&rk%FNetForNextSentencePrediction.forwardsH&1%\4p V !V PRVP 4VPR44p M-VPP R8Xd\4p V !W4p V'g"V 3V R,,pV e V 3V,#T#\!WV P"R7#) ae labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrw regressionsingle_label_classificationmulti_label_classificationrxrr@)r]rr*rRr problem_typerr>rrYintrsqueezerryrrr)r\rerZrVrfrsrrrXrrrr;r{rs&&&&&&&&, r&rk%FNetForSequenceClassification.forwards$&1%VeTMVPPpVeVP^,MVP^,p Ve"VPRVP R44MRpVe"VPRVP R44MRpVe"VPRVP R44MRpVe2VPRVP R4VP R44MRpVP VVVVVVR7p V ^,p VP V 4p VPV 4p V PRV 4p RpVe\4pV!W4pV'g"V 3V R,,pVe V3V,#T#\WV PR7#)a input_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) token_type_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Segment token indices to indicate first and second portions of the inputs. Indices are selected in `[0, 1]`: - 0 corresponds to a *sentence A* token, - 1 corresponds to a *sentence B* token. [What are token type IDs?](../glossary#token-type-ids) position_ids (`torch.LongTensor` of shape `(batch_size, num_choices, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.max_position_embeddings - 1]`. [What are position IDs?](../glossary#position-ids) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, num_choices, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert *input_ids* indices into associated vectors than the model's internal embedding lookup matrix. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the multiple choice classification loss. Indices should be in `[0, ..., num_choices-1]` where `num_choices` is the size of the second dimension of the input tensors. (See `input_ids` above) Nrwrxrr@) r]rrryrXr*rRrrrr)r\rerZrVrfrsrrrX num_choicesrrrreshaped_logitsr;r{rs&&&&&&&&, r&rkFNetForMultipleChoice.forwardKsT&1%G>SINN2y~~b'9:Y] M[Mg,,R1D1DR1HImqGSG_|((\->->r-BCei (   r=#5#5b#9=;M;Mb;Q R  )) )%'!5#     ]3 / ++b+6  ')HO4D%''"+5F)-)9TGf$ Ev E(dZaZoZoppr()rrRr*rrrus@@r&rr?s0MqMqMqMqr(rcRaa]tRtRtoV3Rlt]RV3RlRll4tRtVtV;t #)FNetForTokenClassificationic2<\SV`V4VPVn\V4Vn\ P !VP4Vn\ P!VPVP4Vn VP4R#r*rr[s&&r&rB#FNetForTokenClassification.__init__si   ++f% zz&"<"<= ))F$6$68I8IJ r(c <V^8dQhRS[PR,RS[PR,RS[PR,RS[PR,RS[PR,RS[R,RS[R,R S[S[,/#r)rrrSrr)rrvs"r&r'FNetForTokenClassification.__annotate__s)d)d<<$&)d t+)dllT) )d ||d* )d  t# )d#Tk)dD[)d & &)dr(c VeTMVPPpVPVVVVVVR7p V ^,p VPV 4p VP V 4p Rp Ve=\ 4p V !V P RVP4VP R44p V'g"V 3V R,,pV e V 3V,#T#\WV PR7#)z labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. Nrwrxrr@) r]rr*rRrrryrrr)r\rerZrVrfrsrrrXrrrr;r{rs&&&&&&&&, r&rk"FNetForTokenClassification.forwards &1%rs! AA&7!9   .6*   H %M>  4RYY4n! !Hbii 299  ryy   *6a"))a8   "))" 299 !bii!&bii& 9299 9 // / /  :{:  :$c #c c L Z ,Z  Z z?r)?r?rD J $7J  J Z Gg$7Gg  GgTYq/YqYqx7d!47d7dtB 2B B J r(