+ jj^RIHtHt^RIt^RIHt^RIHtHt^RIH t H t H t H t ]'d/^RI Ht]]]R3,,]],,R,tM]t]],R,t.ROt]!]P*R 4t]!]P.R 4t]!]P2R 4tRR R llt]!]P8R4t]!]P<R4t]!]P@R4t!]!]PDR4t#!RR4t$Rt%R#))Any TYPE_CHECKINGN)Tensor) _add_docstr_sparse)SparseSemiStructuredTensor$SparseSemiStructuredTensorCUSPARSELT!SparseSemiStructuredTensorCUTLASSto_sparse_semi_structured)_dtype.check_sparse_tensor_invariantsaH sparse.addmm(mat, mat1, mat2, *, beta=1., alpha=1.) -> Tensor This function does exact same thing as :func:`torch.addmm` in the forward, except that it supports backward for sparse COO and CSR matrix :attr:`mat1`. When :attr:`mat1` is a COO tensor it must have `sparse_dim = 2`. Supports both CSR and COO storage formats. .. note:: **Gradient support:** - **COO @ Dense**: Backward is supported for both inputs. The gradient for the sparse input is returned as a sparse COO tensor. - **CSR @ Dense**: Backward is supported for both inputs. The gradient for the sparse input is returned as a sparse CSR tensor. - **CSC/BSR/BSC @ Dense**: Not supported. - **Sparse @ Sparse** (COO @ COO, CSR @ CSR): Forward works, but backward is not supported. Args: mat (Tensor): a dense matrix to be added mat1 (Tensor): a sparse matrix to be multiplied mat2 (Tensor): a dense matrix to be multiplied beta (Number, optional): multiplier for :attr:`mat` (:math:`\beta`) alpha (Number, optional): multiplier for :math:`mat1 @ mat2` (:math:`\alpha`) a Performs a matrix multiplication of the sparse matrix :attr:`mat1` and the (sparse or strided) matrix :attr:`mat2`. Similar to :func:`torch.mm`, if :attr:`mat1` is a :math:`(n \times m)` tensor, :attr:`mat2` is a :math:`(m \times p)` tensor, out will be a :math:`(n \times p)` tensor. When :attr:`mat1` is a COO tensor it must have `sparse_dim = 2`. Supports both CSR and COO storage formats. .. note:: **Gradient support:** - **COO @ Dense**: Backward is supported for both inputs. The gradient for the sparse input is returned as a sparse COO tensor. - **CSR @ Dense**: Backward is supported for both inputs. The gradient for the sparse input is returned as a sparse CSR tensor. - **CSC/BSR/BSC @ Dense**: Not supported. - **Sparse @ Sparse** (COO @ COO, CSR @ CSR): Forward works, but backward is not supported. - **Mixed formats** (COO @ CSR, CSR @ COO): Not supported. This function also additionally accepts an optional :attr:`reduce` argument that allows specification of an optional reduction operation, mathematically performs the following operation: .. math:: z_{ij} = \bigoplus_{k = 0}^{K - 1} x_{ik} y_{kj} where :math:`\bigoplus` defines the reduce operator. :attr:`reduce` is implemented only for CSR storage format on CPU device. Args: mat1 (Tensor): the first sparse matrix to be multiplied mat2 (Tensor): the second matrix to be multiplied, which could be sparse or dense reduce (str, optional): the reduction operation to apply for non-unique indices (:obj:`"sum"`, :obj:`"mean"`, :obj:`"amax"`, :obj:`"amin"`). Default :obj:`"sum"`. Shape: The format of the output tensor of this function follows: - sparse x sparse -> sparse - sparse x dense -> dense Example:: >>> a = torch.tensor([[1., 0, 2], [0, 3, 0]]).to_sparse().requires_grad_() >>> a tensor(indices=tensor([[0, 0, 1], [0, 2, 1]]), values=tensor([1., 2., 3.]), size=(2, 3), nnz=3, layout=torch.sparse_coo, requires_grad=True) >>> b = torch.tensor([[0, 1.], [2, 0], [0, 0]], requires_grad=True) >>> b tensor([[0., 1.], [2., 0.], [0., 0.]], requires_grad=True) >>> y = torch.sparse.mm(a, b) >>> y tensor([[0., 1.], [6., 0.]], grad_fn=) >>> y.sum().backward() >>> a.grad tensor(indices=tensor([[0, 0, 1], [0, 2, 1]]), values=tensor([1., 0., 2.]), size=(2, 3), nnz=3, layout=torch.sparse_coo) >>> c = a.detach().to_sparse_csr() >>> c tensor(crow_indices=tensor([0, 2, 3]), col_indices=tensor([0, 2, 1]), values=tensor([1., 2., 3.]), size=(2, 3), nnz=3, layout=torch.sparse_csr) >>> y1 = torch.sparse.mm(c, b, 'sum') >>> y1 tensor([[0., 1.], [6., 0.]], grad_fn=) >>> y2 = torch.sparse.mm(c, b, 'max') >>> y2 tensor([[0., 1.], [6., 0.]], grad_fn=) a sparse.sampled_addmm(input, mat1, mat2, *, beta=1., alpha=1., out=None) -> Tensor Performs a matrix multiplication of the dense matrices :attr:`mat1` and :attr:`mat2` at the locations specified by the sparsity pattern of :attr:`input`. The matrix :attr:`input` is added to the final result. Mathematically this performs the following operation: .. math:: \text{out} = \alpha\ (\text{mat1} \mathbin{@} \text{mat2})*\text{spy}(\text{input}) + \beta\ \text{input} where :math:`\text{spy}(\text{input})` is the sparsity pattern matrix of :attr:`input`, :attr:`alpha` and :attr:`beta` are the scaling factors. :math:`\text{spy}(\text{input})` has value 1 at the positions where :attr:`input` has non-zero values, and 0 elsewhere. .. note:: :attr:`input` must be a sparse CSR tensor. :attr:`mat1` and :attr:`mat2` must be dense tensors. Args: input (Tensor): a sparse CSR matrix of shape `(m, n)` to be added and used to compute the sampled matrix multiplication mat1 (Tensor): a dense matrix of shape `(m, k)` to be multiplied mat2 (Tensor): a dense matrix of shape `(k, n)` to be multiplied Keyword args: beta (Number, optional): multiplier for :attr:`input` (:math:`\beta`) alpha (Number, optional): multiplier for :math:`mat1 @ mat2` (:math:`\alpha`) out (Tensor, optional): output tensor. Ignored if `None`. Default: `None`. Examples:: >>> input = torch.eye(3, device='cuda').to_sparse_csr() >>> mat1 = torch.randn(3, 5, device='cuda') >>> mat2 = torch.randn(5, 3, device='cuda') >>> torch.sparse.sampled_addmm(input, mat1, mat2) tensor(crow_indices=tensor([0, 1, 2, 3]), col_indices=tensor([0, 1, 2]), values=tensor([ 0.2847, -0.7805, -0.1900]), device='cuda:0', size=(3, 3), nnz=3, layout=torch.sparse_csr) >>> torch.sparse.sampled_addmm(input, mat1, mat2).to_dense() tensor([[ 0.2847, 0.0000, 0.0000], [ 0.0000, -0.7805, 0.0000], [ 0.0000, 0.0000, -0.1900]], device='cuda:0') >>> torch.sparse.sampled_addmm(input, mat1, mat2, beta=0.5, alpha=0.5) tensor(crow_indices=tensor([0, 1, 2, 3]), col_indices=tensor([0, 1, 2]), values=tensor([ 0.1423, -0.3903, -0.0950]), device='cuda:0', size=(3, 3), nnz=3, layout=torch.sparse_csr) cVV^8dQhR\R\R\R,R\/#)inputdimdtypeNreturn)r DimOrDimsDType)formats"V/home/luca/screen-checking/.venv/lib/python3.14/site-packages/torch/sparse/__init__.py __annotate__rs2C9C9vC9IC9UT\C9VC9cVf1Ve\P!W4#\P!V4#Ve\P!WVR7#\P!WR7#)a1 Return the sum of each row of the given sparse tensor. Returns the sum of each row of the sparse tensor :attr:`input` in the given dimensions :attr:`dim`. If :attr:`dim` is a list of dimensions, reduce over all of them. When sum over all ``sparse_dim``, this method returns a dense tensor instead of a sparse tensor. All summed :attr:`dim` are squeezed (see :func:`torch.squeeze`), resulting an output tensor having :attr:`dim` fewer dimensions than :attr:`input`. During backward, only gradients at ``nnz`` locations of :attr:`input` will propagate back. Note that the gradients of :attr:`input` is coalesced. Args: input (Tensor): the input sparse tensor dim (int or tuple of ints): a dimension or a list of dimensions to reduce. Default: reduce over all dims. dtype (:class:`torch.dtype`, optional): the desired data type of returned Tensor. Default: dtype of :attr:`input`. Example:: >>> nnz = 3 >>> dims = [5, 5, 2, 3] >>> I = torch.cat([torch.randint(0, dims[0], size=(nnz,)), torch.randint(0, dims[1], size=(nnz,))], 0).reshape(2, nnz) >>> V = torch.randn(nnz, dims[2], dims[3]) >>> size = torch.Size(dims) >>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> with torch.sparse.check_sparse_tensor_invariants(): ... S = torch.sparse_coo_tensor(I, V, size) >>> S tensor(indices=tensor([[2, 0, 3], [2, 4, 1]]), values=tensor([[[-0.6438, -1.6467, 1.4004], [ 0.3411, 0.0918, -0.2312]], [[ 0.5348, 0.0634, -2.0494], [-0.7125, -1.0646, 2.1844]], [[ 0.1276, 0.1874, -0.6334], [-1.9682, -0.5340, 0.7483]]]), size=(5, 5, 2, 3), nnz=3, layout=torch.sparse_coo) # when sum over only part of sparse_dims, return a sparse tensor >>> torch.sparse.sum(S, [1, 3]) tensor(indices=tensor([[0, 2, 3]]), values=tensor([[-1.4512, 0.4073], [-0.8901, 0.2017], [-0.3183, -1.7539]]), size=(5, 2), nnz=3, layout=torch.sparse_coo) # when sum over all sparse dim, return a dense tensor # with summed dims squeezed >>> torch.sparse.sum(S, [0, 1, 3]) tensor([-2.6596, -1.1450]) )r)torch _sparse_sum)rrrs&&&rsumrsZt } ?$$U0 0$$U+ + ?$$Uu= =$$U8 8ra sparse.softmax(input, dim, *, dtype=None) -> Tensor Applies a softmax function. Softmax is defined as: :math:`\text{Softmax}(x_{i}) = \frac{exp(x_i)}{\sum_j exp(x_j)}` where :math:`i, j` run over sparse tensor indices and unspecified entries are ignores. This is equivalent to defining unspecified entries as negative infinity so that :math:`exp(x_k) = 0` when the entry with index :math:`k` has not specified. It is applied to all slices along `dim`, and will re-scale them so that the elements lie in the range `[0, 1]` and sum to 1. Args: input (Tensor): input dim (int): A dimension along which softmax will be computed. dtype (:class:`torch.dtype`, optional): the desired data type of returned tensor. If specified, the input tensor is casted to :attr:`dtype` before the operation is performed. This is useful for preventing data type overflows. Default: None a sparse.spsolve(input, other, *, left=True) -> Tensor Computes the solution of a square system of linear equations with a unique solution. Its purpose is similar to :func:`torch.linalg.solve`, except that the system is defined by a sparse CSR matrix with layout `sparse_csr`. Args: input (Tensor): a sparse CSR matrix of shape `(n, n)` representing the coefficients of the linear system. other (Tensor): a dense matrix of shape `(n, )` representing the right-hand side of the linear system. left (bool, optional): whether to solve the system for `input @ out = other` (default) or `out @ input = other`. Only `left=True` is supported. a sparse.log_softmax(input, dim, *, dtype=None) -> Tensor Applies a softmax function followed by logarithm. See :class:`~torch.sparse.softmax` for more details. Args: input (Tensor): input dim (int): A dimension along which softmax will be computed. dtype (:class:`torch.dtype`, optional): the desired data type of returned tensor. If specified, the input tensor is casted to :attr:`dtype` before the operation is performed. This is useful for preventing data type overflows. Default: None a( sparse.spdiags(diagonals, offsets, shape, layout=None) -> Tensor Creates a sparse 2D tensor by placing the values from rows of :attr:`diagonals` along specified diagonals of the output The :attr:`offsets` tensor controls which diagonals are set. - If :attr:`offsets[i]` = 0, it is the main diagonal - If :attr:`offsets[i]` < 0, it is below the main diagonal - If :attr:`offsets[i]` > 0, it is above the main diagonal The number of rows in :attr:`diagonals` must match the length of :attr:`offsets`, and an offset may not be repeated. Args: diagonals (Tensor): Matrix storing diagonals row-wise offsets (Tensor): The diagonals to be set, stored as a vector shape (2-tuple of ints): The desired shape of the result Keyword args: layout (:class:`torch.layout`, optional): The desired layout of the returned tensor. ``torch.sparse_coo``, ``torch.sparse_csc`` and ``torch.sparse_csr`` are supported. Default: ``torch.sparse_coo`` Examples: Set the main and first two lower diagonals of a matrix:: >>> diags = torch.arange(9).reshape(3, 3) >>> diags tensor([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) >>> s = torch.sparse.spdiags(diags, torch.tensor([0, -1, -2]), (3, 3)) >>> s tensor(indices=tensor([[0, 1, 2, 1, 2, 2], [0, 1, 2, 0, 1, 0]]), values=tensor([0, 1, 2, 3, 4, 6]), size=(3, 3), nnz=6, layout=torch.sparse_coo) >>> s.to_dense() tensor([[0, 0, 0], [3, 1, 0], [6, 4, 2]]) Change the output layout:: >>> diags = torch.arange(9).reshape(3, 3) >>> diags tensor([[0, 1, 2],[3, 4, 5], [6, 7, 8]) >>> s = torch.sparse.spdiags(diags, torch.tensor([0, -1, -2]), (3, 3), layout=torch.sparse_csr) >>> s tensor(crow_indices=tensor([0, 1, 3, 6]), col_indices=tensor([0, 0, 1, 0, 1, 2]), values=tensor([0, 3, 1, 6, 4, 2]), size=(3, 3), nnz=6, layout=torch.sparse_csr) >>> s.to_dense() tensor([[0, 0, 0], [3, 1, 0], [6, 4, 2]]) Set partial diagonals of a large output:: >>> diags = torch.tensor([[1, 2], [3, 4]]) >>> offsets = torch.tensor([0, -1]) >>> torch.sparse.spdiags(diags, offsets, (5, 5)).to_dense() tensor([[1, 0, 0, 0, 0], [3, 2, 0, 0, 0], [0, 4, 0, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]) .. note:: When setting the values along a given diagonal the index into the diagonal and the index into the row of :attr:`diagonals` is taken as the column index in the output. This has the effect that when setting a diagonal with a positive offset `k` the first value along that diagonal will be the value in position `k` of the row of :attr:`diagonals` Specifying a positive offset:: >>> diags = torch.tensor([[1, 2, 3], [1, 2, 3], [1, 2, 3]]) >>> torch.sparse.spdiags(diags, torch.tensor([0, 1, 2]), (5, 5)).to_dense() tensor([[1, 2, 3, 0, 0], [0, 2, 3, 0, 0], [0, 0, 3, 0, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]) cpa]tRtRtoRt]R4t]R4t]R4tR Rlt Rt Rt R t R t VtR #) r ia]A tool to control checking sparse tensor invariants. The following options exists to manage sparse tensor invariants checking in sparse tensor construction: 1. Using a context manager: .. code:: python with torch.sparse.check_sparse_tensor_invariants(): run_my_model() 2. Using a procedural approach: .. code:: python prev_checks_enabled = torch.sparse.check_sparse_tensor_invariants.is_enabled() torch.sparse.check_sparse_tensor_invariants.enable() run_my_model() if not prev_checks_enabled: torch.sparse.check_sparse_tensor_invariants.disable() 3. Using function decoration: .. code:: python @torch.sparse.check_sparse_tensor_invariants() def run_my_model(): ... run_my_model() 4. Using ``check_invariants`` keyword argument in sparse tensor constructor call. For example: >>> torch.sparse_csr_tensor([0, 1, 3], [0, 1], [1, 2], check_invariants=True) Traceback (most recent call last): File "", line 1, in RuntimeError: `crow_indices[..., -1] == nnz` is not satisfied. c >\PP4#)aReturn True if the sparse tensor invariants checking is enabled. .. note:: Use :func:`torch.sparse.check_sparse_tensor_invariants.enable` or :func:`torch.sparse.check_sparse_tensor_invariants.disable` to manage the state of the sparse tensor invariants checks. )r_C_check_sparse_tensor_invariantsrr is_enabled)check_sparse_tensor_invariants.is_enabledsxx7799rc D\PPR4R#)a(Enable sparse tensor invariants checking in sparse tensor constructors. .. note:: By default, the sparse tensor invariants checks are disabled. Use :func:`torch.sparse.check_sparse_tensor_invariants.is_enabled` to retrieve the current state of sparse tensor invariants checking. .. note:: The sparse tensor invariants check flag is effective to all sparse tensor constructors, both in Python and ATen. The flag can be locally overridden by the ``check_invariants`` optional argument of the sparse tensor constructor functions. TNrr #_set_check_sparse_tensor_invariantsr"rrenable%check_sparse_tensor_invariants.enables$ 44T:rc D\PPR4R#)zDisable sparse tensor invariants checking in sparse tensor constructors. See :func:`torch.sparse.check_sparse_tensor_invariants.enable` for more information. FNr&r"rrdisable&check_sparse_tensor_invariants.disables 44U;rc WnRVnR#N)state saved_state)selfr(s&&r__init__'check_sparse_tensor_invariants.__init__s (,rcVPe \R4hVP4Vn\PP VP 4R#)NzqThis context manager instance is already activated. Use a different context manager instance for context nesting.)r0 RuntimeErrorr#rr r'r/)r1s&r __enter__(check_sparse_tensor_invariants.__enter__sH    'Q  ??, 44TZZ@rcVPf \R4h\PP VP4RVnR#)Nz&saved_state should not be None on exit)r0AssertionErrorrr r')r1typevalue tracebacks&&&&r__exit__'check_sparse_tensor_invariants.__exit__(s:    # !IJ J 44T5E5EFrcaaVV3RlpV#)c<\S4!SP4;_uu_4S!V/VBuuRRR4# +'giR#;ir.)r:r/)argskwargsmthr1s*,rtest_mth9check_sparse_tensor_invariants.__call__..test_mth0s2dDJJ''D+F+('''s 8 A r")r1rCrDsff r__call__'check_sparse_tensor_invariants.__call__/s ,r)r0r/N)T)__name__ __module__ __qualname____firstlineno____doc__ staticmethodr#r(r+r2r6r=rF__static_attributes____classdictcell__) __classdict__s@rr r sa)V : :;;&<<-A rcaV3RlpV#)aDecorate function, to extend gradcheck for sparse tensors. Decorator for torch.autograd.gradcheck or its functools.partial variants that extends the gradcheck function with support to input functions that operate on or/and return sparse tensors. The specified gradcheck function itself is guaranteed to operate on strided tensors only. For example: >>> warnings.filterwarnings( ... "ignore", message=".*Sparse CSR tensor support is in beta state" ... ) # docs: hide >>> gradcheck = torch.sparse.as_sparse_gradcheck(torch.autograd.gradcheck) >>> x = ( ... torch.tensor([[0, 1], [2, 3]], dtype=torch.float64) ... .to_sparse_coo() ... .requires_grad_(True) ... ) >>> gradcheck(lambda x: x.to_sparse_csr(), x) True c<aaaaa a a VPRR4o\P\P\P\P \P 0o \P\P\P \P 0o \P \P 0o RoVVV V 3RlpVV 3RloVVVV 3RlpWC!V43pS !V/VB#)z Create gradcheck with support for sparse tensors. Same as :func:`torch.autograd.gradcheck` but with sparse tensors inputs and outputs support. maskedF__STRIDED_REPRESENTATION__c<\V\\34'gV3p.pVEFp\V\P4'EdqVP 'Ed^VP S 9EdLRVP RVP/pS 'gVPVP4, VP4, pVP S 9d,VP4PV^,V^,MRp\P!VPVP\PR7PVP VVP4R7pVP!4P#V4pVP \P$JdAVP'VP)4VP+4R7VP-4pMVP \P.\P009dAVP'VP34VP54R7VP4pM?VP'VP74VP94R7VP4pVP;SW7P=R434EKVP?V4EK \V4#) ziConvert differentiable non-strided tensors to a representation containing differentiable strided tensors.layoutshapeN)devicer)rV blocksize dense_dim)indices is_coalesced)compressed_indices plain_indicesT) isinstancelisttuplerr requires_gradrVrWndimrZ sparse_dimvaluesonesrXbool to_sparseto_dense sparse_mask sparse_cooupdate_indicesr\_values sparse_csr sparse_bsr crow_indices col_indices ccol_indices row_indicesextendrequires_grad_append) rAnew_argsobjd batch_dimrY full_maskreSTRIDED_REPRESENTATIONrSsparse_block_layoutssparse_layoutss & r!convert_to_strided_representationeas_sparse_gradcheck..gradcheck_with_sparse_support..convert_to_strided_representationgs1dT5M22w"$HsELL11))) n4!#**A"$'HHs}}$>AQ$Q  #zz-AA JJL..y1}y1}M!%" %*JJIIcjj %#)#&::&/&)mmo$""lln88CzzU%5%55$'LLN),)9)9); ! "%(8(8%:J:J'KK/2/?/?/A*-//*; ! "%/2/?/?/A*-//*; ! "%OO/4I4I$4OPOOC(st? "rc*<.p\V4pV'dVP^4pVS8XdVP^4VP^4rCVR,\PJd1\P!VR,VVR,VR,R7pM\VR,S9d9\P !VR,VR,VVR,VR,R7pM\ R VR, R 24hVPV4K\V4#) zNRestore non-strided differentiable tensors from their strided representations.rVr[rWr\)sizer\r]r^)rrVzconversion of z! strided representation to tensor) r`poprrksparse_coo_tensorsparse_compressed_tensorNotImplementedErrorrwra)rArxarzrer}sparse_compressed_layoutss& r#restore_from_strided_representationgas_sparse_gradcheck..gradcheck_with_sparse_support..restore_from_strided_representationsH:DHHQK.. $ TXXa[v{e&6&66!33iL"!"7)*>):  8(AA!::23o."!"7#$X; 2,Qx[M9Z["? "rc.<S!V4pS!V/VBp\V\\34'd \V4MV3p\;QJd.VV3RlV4FNK 5M!VV3RlV44p\V\\34'dV#V^,#)c3<"TF\p\V\P4'd6VP'd$VPS9dVP SR7MTxK^ R#5i)) masked_gradN)r_rrrbrVri).0orSrs& r cas_sparse_gradcheck..gradcheck_with_sparse_support..func_wrapper..sY $)A "!U\\22N2JJ6J2  )s 6A'-A')r_r`ra) rArB restored_argsoutputsstrided_outputsfuncrSrrs *, r func_wrapperPas_sparse_gradcheck..gradcheck_with_sparse_support..func_wrappers?EMM4V4G#-WtUm"D"Dg7* $e $) $ee $) $ Oge}55  %Q' r)rrrkro sparse_cscrp sparse_bsc) rinputsrBrrrAr}rSrr~rr gradchecks f&, @@@@@@rgradcheck_with_sparse_support:as_sparse_gradcheck..gradcheck_with_sparse_supportPs He,                              % ! !& 0 0%2B2BC!=? #? #B #<  6?GH$)&))rr")rrsf ras_sparse_gradcheckr7s2S*j )(r) addmmr mmrsoftmaxsolve log_softmaxrr r r r)NN)&typingrrrrtorch._Crrsemi_structuredrr r r torch.typesr rintrar`r__all__ _sparse_addmmr _sparse_mmrsparse_sampled_addmm sampled_addmmr_sparse_softmaxr_spsolvespsolve_sparse_log_softmaxr_spdiagsspdiagsr rr"rrrs9& )+eCHo%S 1D8I Ec T!I    @ ORj   14 nC9L   >   (  *  Y\ ~nnbn)r