+ jd^RIt^RIHt^RIHt^RIt^RIHuHt ^RIHt^RI H t ^RI Ht^RIHtHt^RIHt^R IHt^R IHt^R IHtHt^R IHtHt^R IHtH t ^RI!H"t"H#t#^RI$H%t%H&t&^RI'H(t(^RI)H*t*H+t+H,t,H-t-^RI.H/t/^RI0H1t1H2t2^RI3H4t4H5t5^RI6H7t7]-!4'd^RI8H9t9]!R4!RR]Pt44t;!RR]Pt4tRR lt?R9R!R"llt@R:R#R$lltA]%!4tB]A]BR%&!R&R']Pt4tC!R(R)]Pt4tD!R*R+]Pt4tE!R,R-]4tF]+!R.R/]&44tG]+!R0R1]G44tHR;R2R3lltI]+!R4R5]G]44tJ!R6R7]]G4tK.R% H?T?T4T(UU {{]--k:::r1c^\VPP4 RVP 2#)z, eps=)tupler8shaper9)r:s&r. extra_reprDogeRMSNorm.extra_reprGs* ))*+6$2G2G1HIIr1)r9r8)gư>) __name__ __module__ __qualname____firstlineno__r4rNrS__static_attributes____classdictcell__ __classcell__r<r-s@@r.r&r&6s%$$;;JJr1r&caa]tRt^Kto]!RRR7R V3RlV3Rlll4t]]!RRR7R V3RlRll44t]P!4] R44t R t Vt V;t#) DogeRotaryEmbeddingdevicez5.18)versionc <V^8dQhRS[/#r(configr!)r,r-s"r.r/ DogeRotaryEmbedding.__annotate__MsOOzOr1c<\SV`4VPVnVPVnWnVP P R,VnVPpVPR8wd\VP,pV!VP V4wq@n \P!VRR7Vn \P!VP4RR7VnR#) rope_typedefaultF) persistentN)r3r4max_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenrcrope_parametersrfcompute_default_rope_parametersrattention_scalingrBufferinv_freqcloneoriginal_inv_freq)r:rcr_ rope_init_fnrpr<s&&& r.r4DogeRotaryEmbedding.__init__Ls "("@"@$*$B$B! 44[A!%!E!E >>Y &.t~~>L+7 V+L(( (u= !#8>>+;!Nr1cP<V^8dQhRS[RS[S[PS[3,/#)r(rcr*)r"rQr6rBr+)r,r-s"r.r/rd_s.55 5V[\a\h\hjo\oVp5r1c 0VPR,p\VRR4;'gVPVP,pRpRV\P !^V^\P R7V, ,, pVPV4V3#)a~ Computes the inverse frequencies according to the original RoPE implementation Args: config ([`~transformers.PreTrainedConfig`]): The model configuration. Returns: Tuple of (`torch.Tensor`, `float`), containing the inverse frequencies for the RoPE embeddings and the post-processing scaling factor applied to the computed cos/sin (unused in this type of RoPE). rope_thetahead_dimN?rF)rlgetattrr;num_attention_headsr6aranger+rG)rcr_kwargsbasedimattention_factorrps&&, r.rm3DogeRotaryEmbedding.compute_default_rope_parameters]s~%%l3fj$/cc63E3EIcIc3c$5<<3#MPS#STU{{6"$444r1c^VPR,PVP^,R ^4P\P VP R7pVR ,P 4p\VP P\4'd2VP PR8wdVP PMRp\VRR7;_uu_4W4,P^^4p\P!Wf3R R7pVP4VP,pVP4VP,p RRR4XPVP R7X PVP R73# +'giLH;i) NrFr_mpscpuF) device_typeenabledrrz)NNNNNrE)rNr)rpexpandrRrGr6r+r_ isinstancetypestrr transposecatcosrnsinrF) r:x position_idsinv_freq_expandedposition_ids_expandedrfreqsembrrs &&& r.rNDogeRotaryEmbedding.forwardqs2 MM- ( / / 0B0B10Er1 M P PW\WbWbklksks P t !-Z 8 > > @'1!((--'E'E!((--[`J`ahhmmfk  U C C&>II!QOE))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs (A4F F, )rnrcrprjrrrkrfN)rUrVrWrXrr4 staticmethodrmr6no_gradrrNrYrZr[r\s@@r.r^r^KsuXv.OO/O Xv.55/5$ ]]_<<