X[ω1,…,ωd]=∑n1=0N1−1⋯∑nd=0Nd−1x[n1,…,nd]e−j 2π∑i=0dωiniNi,X[\omega_1, \dots, \omega_d] = \sum_{n_1=0}^{N_1-1} \dots \sum_{n_d=0}^{N_d-1} x[n_1, \dots, n_d] e^{-j\ 2 \pi \sum_{i=0}^d \frac{\omega_i n_i}{N_i}}, X[ω1,…,ωd]=n1=0∑N1−1⋯nd=0∑Nd−1x[n1,…,nd]e−j 2π∑i=0dNiωini,

where ddd = signal_ndim is number of dimensions for the signal, and NiN_iNi is the size of signal dimension iii .

This method supports 1D, 2D and 3D complex-to-complex transforms, indicated by signal_ndim. input must be a tensor with last dimension of size 2, representing the real and imaginary components of complex numbers, and should have at least signal_ndim + 1 dimensions with optionally arbitrary number of leading batch dimensions. If normalized is set to True, this normalizes the result by dividing it with ∏i=1KNi\sqrt{\prod_{i=1}^K N_i}∏i=1KNi

so that the operator is unitary.

Returns the real and the imaginary parts together as one tensor of the same shape of input.

The inverse of this function is ifft().

Note

For CUDA tensors, an LRU cache is used for cuFFT plans to speed up repeatedly running FFT methods on tensors of same geometry with same configuration. See cuFFT plan cache for more details on how to monitor and control the cache.

Warning

For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available() to check if MKL is installed.

Parameters

input (Tensor) – the input tensor of at least signal_ndim + 1 dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False

Returns

A tensor containing the complex-to-complex Fourier transform result

Return type

Tensor

Example:

>>> # unbatched 2D FFT
>>> x = torch.randn(4, 3, 2)
>>> torch.fft(x, 2)
tensor([[[-0.0876,  1.7835],
         [-2.0399, -2.9754],
         [ 4.4773, -5.0119]],

        [[-1.5716,  2.7631],
         [-3.8846,  5.2652],
         [ 0.2046, -0.7088]],

        [[ 1.9938, -0.5901],
         [ 6.5637,  6.4556],
         [ 2.9865,  4.9318]],

        [[ 7.0193,  1.1742],
         [-1.3717, -2.1084],
         [ 2.0289,  2.9357]]])
>>> # batched 1D FFT
>>> torch.fft(x, 1)
tensor([[[ 1.8385,  1.2827],
         [-0.1831,  1.6593],
         [ 2.4243,  0.5367]],

        [[-0.9176, -1.5543],
         [-3.9943, -2.9860],
         [ 1.2838, -2.9420]],

        [[-0.8854, -0.6860],
         [ 2.4450,  0.0808],
         [ 1.3076, -0.5768]],

        [[-0.1231,  2.7411],
         [-0.3075, -1.7295],
         [-0.5384, -2.0299]]])
>>> # arbitrary number of batch dimensions, 2D FFT
>>> x = torch.randn(3, 3, 5, 5, 2)
>>> y = torch.fft(x, 2)
>>> y.shape
torch.Size([3, 3, 5, 5, 2])

`torch.ifft`(input, signal_ndim, normalized=False) → Tensor

Complex-to-complex Inverse Discrete Fourier Transform

This method computes the complex-to-complex inverse discrete Fourier transform. Ignoring the batch dimensions, it computes the following expression:

X[ω1,…,ωd]=1∏i=1dNi∑n1=0N1−1⋯∑nd=0Nd−1x[n1,…,nd]e j 2π∑i=0dωiniNi,X[\omega_1, \dots, \omega_d] = \frac{1}{\prod_{i=1}^d N_i} \sum_{n_1=0}^{N_1-1} \dots \sum_{n_d=0}^{N_d-1} x[n_1, \dots, n_d] e^{\ j\ 2 \pi \sum_{i=0}^d \frac{\omega_i n_i}{N_i}}, X[ω1,…,ωd]=∏i=1dNi1n1=0∑N1−1⋯nd=0∑Nd−1x[n1,…,nd]e j 2π∑i=0dNiωini,

where ddd = signal_ndim is number of dimensions for the signal, and NiN_iNi is the size of signal dimension iii .

The argument specifications are almost identical with fft(). However, if normalized is set to True, this instead returns the results multiplied by ∏i=1dNi\sqrt{\prod_{i=1}^d N_i}∏i=1dNi

, to become a unitary operator. Therefore, to invert a fft(), the normalized argument should be set identically for fft().

Returns the real and the imaginary parts together as one tensor of the same shape of input.

The inverse of this function is fft().

Note

Warning

For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available() to check if MKL is installed.

Parameters

input (Tensor) – the input tensor of at least signal_ndim + 1 dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False

Returns

A tensor containing the complex-to-complex inverse Fourier transform result

Return type

Tensor

Example:

>>> x = torch.randn(3, 3, 2)
>>> x
tensor([[[ 1.2766,  1.3680],
         [-0.8337,  2.0251],
         [ 0.9465, -1.4390]],

        [[-0.1890,  1.6010],
         [ 1.1034, -1.9230],
         [-0.9482,  1.0775]],

        [[-0.7708, -0.8176],
         [-0.1843, -0.2287],
         [-1.9034, -0.2196]]])
>>> y = torch.fft(x, 2)
>>> torch.ifft(y, 2)  # recover x
tensor([[[ 1.2766,  1.3680],
         [-0.8337,  2.0251],
         [ 0.9465, -1.4390]],

        [[-0.1890,  1.6010],
         [ 1.1034, -1.9230],
         [-0.9482,  1.0775]],

        [[-0.7708, -0.8176],
         [-0.1843, -0.2287],
         [-1.9034, -0.2196]]])

`torch.rfft`(input, signal_ndim, normalized=False, onesided=True) → Tensor

Real-to-complex Discrete Fourier Transform

This method computes the real-to-complex discrete Fourier transform. It is mathematically equivalent with fft() with differences only in formats of the input and output.

This method supports 1D, 2D and 3D real-to-complex transforms, indicated by signal_ndim. input must be a tensor with at least signal_ndim dimensions with optionally arbitrary number of leading batch dimensions. If normalized is set to True, this normalizes the result by dividing it with ∏i=1KNi\sqrt{\prod_{i=1}^K N_i}∏i=1KNi

so that the operator is unitary, where NiN_iNi is the size of signal dimension iii .

The real-to-complex Fourier transform results follow conjugate symmetry:

X[ω1,…,ωd]=X∗[N1−ω1,…,Nd−ωd],X[\omega_1, \dots, \omega_d] = X^*[N_1 - \omega_1, \dots, N_d - \omega_d], X[ω1,…,ωd]=X∗[N1−ω1,…,Nd−ωd],

where the index arithmetic is computed modulus the size of the corresponding dimension, ∗\ ^* ∗ is the conjugate operator, and ddd = signal_ndim. onesided flag controls whether to avoid redundancy in the output results. If set to True (default), the output will not be full complex result of shape (∗,2)(*, 2)(∗,2) , where ∗*∗ is the shape of input, but instead the last dimension will be halfed as of size ⌊Nd2⌋+1\lfloor \frac{N_d}{2} \rfloor + 1⌊2Nd⌋+1 .

The inverse of this function is irfft().

Note

Warning

For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available() to check if MKL is installed.

Parameters

input (Tensor) – the input tensor of at least signal_ndim dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False
onesided (bool, optional) – controls whether to return half of results to avoid redundancy. Default: True

Returns

A tensor containing the real-to-complex Fourier transform result

Return type

Tensor

Example:

>>> x = torch.randn(5, 5)
>>> torch.rfft(x, 2).shape
torch.Size([5, 3, 2])
>>> torch.rfft(x, 2, onesided=False).shape
torch.Size([5, 5, 2])

`torch.irfft`(input, signal_ndim, normalized=False, onesided=True, signal_sizes=None) → Tensor

Complex-to-real Inverse Discrete Fourier Transform

This method computes the complex-to-real inverse discrete Fourier transform. It is mathematically equivalent with ifft() with differences only in formats of the input and output.

The argument specifications are almost identical with ifft(). Similar to ifft(), if normalized is set to True, this normalizes the result by multiplying it with ∏i=1KNi\sqrt{\prod_{i=1}^K N_i}∏i=1KNi

so that the operator is unitary, where NiN_iNi is the size of signal dimension iii .

Note

Due to the conjugate symmetry, input do not need to contain the full complex frequency values. Roughly half of the values will be sufficient, as is the case when input is given by rfft() with rfft(signal, onesided=True). In such case, set the onesided argument of this method to True. Moreover, the original signal shape information can sometimes be lost, optionally set signal_sizes to be the size of the original signal (without the batch dimensions if in batched mode) to recover it with correct shape.

Therefore, to invert an rfft(), the normalized and onesided arguments should be set identically for irfft(), and preferrably a signal_sizes is given to avoid size mismatch. See the example below for a case of size mismatch.

See rfft() for details on conjugate symmetry.

The inverse of this function is rfft().

Warning

Generally speaking, input to this function should contain values following conjugate symmetry. Note that even if onesided is True, often symmetry on some part is still needed. When this requirement is not satisfied, the behavior of irfft() is undefined. Since torch.autograd.gradcheck() estimates numerical Jacobian with point perturbations, irfft() will almost certainly fail the check.

Note

Warning

For CPU tensors, this method is currently only available with MKL. Use torch.backends.mkl.is_available() to check if MKL is installed.

Parameters

input (Tensor) – the input tensor of at least signal_ndim + 1 dimensions
signal_ndim (int) – the number of dimensions in each signal. signal_ndim can only be 1, 2 or 3
normalized (bool, optional) – controls whether to return normalized results. Default: False
onesided (bool, optional) – controls whether input was halfed to avoid redundancy, e.g., by rfft(). Default: True
signal_sizes (list or torch.Size, optional) – the size of the original signal (without batch dimension). Default: None

Returns

A tensor containing the complex-to-real inverse Fourier transform result

Return type

Tensor

Example:

>>> x = torch.randn(4, 4)
>>> torch.rfft(x, 2, onesided=True).shape
torch.Size([4, 3, 2])
>>>
>>> # notice that with onesided=True, output size does not determine the original signal size
>>> x = torch.randn(4, 5)

>>> torch.rfft(x, 2, onesided=True).shape
torch.Size([4, 3, 2])
>>>
>>> # now we use the original shape to recover x
>>> x
tensor([[-0.8992,  0.6117, -1.6091, -0.4155, -0.8346],
        [-2.1596, -0.0853,  0.7232,  0.1941, -0.0789],
        [-2.0329,  1.1031,  0.6869, -0.5042,  0.9895],
        [-0.1884,  0.2858, -1.5831,  0.9917, -0.8356]])
>>> y = torch.rfft(x, 2, onesided=True)
>>> torch.irfft(y, 2, onesided=True, signal_sizes=x.shape)  # recover x
tensor([[-0.8992,  0.6117, -1.6091, -0.4155, -0.8346],
        [-2.1596, -0.0853,  0.7232,  0.1941, -0.0789],
        [-2.0329,  1.1031,  0.6869, -0.5042,  0.9895],
        [-0.1884,  0.2858, -1.5831,  0.9917, -0.8356]])

`torch.stft`(input, n_fft, hop_length=None, win_length=None, window=None, center=True, pad_mode='reflect', normalized=False, onesided=True)[source]

Short-time Fourier transform (STFT).

Ignoring the optional batch dimension, this method computes the following expression:

X[m,ω]=∑k=0win_length-1window[k] input[m×hop_length+k] exp⁡(−j2π⋅ωkwin_length),X[m, \omega] = \sum_{k = 0}^{\text{win\_length-1}}% \text{window}[k]\ \text{input}[m \times \text{hop\_length} + k]\ % \exp\left(- j \frac{2 \pi \cdot \omega k}{\text{win\_length}}\right), X[m,ω]=k=0∑win_length-1window[k] input[m×hop_length+k] exp(−jwin_length2π⋅ωk),

where mmm is the index of the sliding window, and ω\omegaω is the frequency that 0≤ω<n_fft0 \leq \omega < \text{n\_fft}0≤ω<n_fft . When onesided is the default value True,

input must be either a 1-D time sequence or a 2-D batch of time sequences.
If hop_length is None (default), it is treated as equal to floor(n_fft / 4).
If win_length is None (default), it is treated as equal to n_fft.
window can be a 1-D tensor of size win_length, e.g., from torch.hann_window(). If window is None (default), it is treated as if having 111 everywhere in the window. If win_length<n_fft\text{win\_length} < \text{n\_fft}win_length<n_fft , window will be padded on both sides to length n_fft before being applied.
If center is True (default), input will be padded on both sides so that the ttt -th frame is centered at time t×hop_lengtht \times \text{hop\_length}t×hop_length . Otherwise, the ttt -th frame begins at time t×hop_lengtht \times \text{hop\_length}t×hop_length .
pad_mode determines the padding method used on input when center is True. See torch.nn.functional.pad() for all available options. Default is "reflect".
If onesided is True (default), only values for ω\omegaω in [0,1,2,…,⌊n_fft2⌋+1]\left[0, 1, 2, \dots, \left\lfloor \frac{\text{n\_fft}}{2} \right\rfloor + 1\right][0,1,2,…,⌊2n_fft⌋+1] are returned because the real-to-complex Fourier transform satisfies the conjugate symmetry, i.e., X[m,ω]=X[m,n_fft−ω]∗X[m, \omega] = X[m, \text{n\_fft} - \omega]^*X[m,ω]=X[m,n_fft−ω]∗ .
If normalized is True (default is False), the function returns the normalized STFT results, i.e., multiplied by (frame_length)−0.5(\text{frame\_length})^{-0.5}(frame_length)−0.5 .

Returns the real and the imaginary parts together as one tensor of size (∗×N×T×2)(* \times N \times T \times 2)(∗×N×T×2) , where ∗*∗ is the optional batch size of input, NNN is the number of frequencies where STFT is applied, TTT is the total number of frames used, and each pair in the last dimension represents a complex number as the real part and the imaginary part.

Warning

This function changed signature at version 0.4.1. Calling with the previous signature may cause error or return incorrect result.

Parameters

input (Tensor) – the input tensor
n_fft (int) – size of Fourier transform
hop_length (int, optional) – the distance between neighboring sliding window frames. Default: None (treated as equal to floor(n_fft / 4))
win_length (int, optional) – the size of window frame and STFT filter. Default: None (treated as equal to n_fft)
window (Tensor, optional) – the optional window function. Default: None (treated as window of all 111 s)
center (bool, optional) – whether to pad input on both sides so that the ttt -th frame is centered at time t×hop_lengtht \times \text{hop\_length}t×hop_length . Default: True
pad_mode (string, optional) – controls the padding method used when center is True. Default: "reflect"
normalized (bool, optional) – controls whether to return the normalized STFT results Default: False
onesided (bool, optional) – controls whether to return half of results to avoid redundancy Default: True

Returns

A tensor containing the STFT result with shape described above

Return type

Tensor

`torch.bartlett_window`(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

Bartlett window function.

w[n]=1−∣2nN−1−1∣={2nN−1if 0≤n≤N−122−2nN−1if N−12<n<N,w[n] = 1 - \left| \frac{2n}{N-1} - 1 \right| = \begin{cases} \frac{2n}{N - 1} & \text{if } 0 \leq n \leq \frac{N - 1}{2} \\ 2 - \frac{2n}{N - 1} & \text{if } \frac{N - 1}{2} < n < N \\ \end{cases}, w[n]=1−∣∣∣∣∣N−12n−1∣∣∣∣∣={N−12n2−N−12nif 0≤n≤2N−1if 2N−1<n<N,

where NNN is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.bartlett_window(L, periodic=True) equal to torch.bartlett_window(L + 1, periodic=False)[:-1]).

Note

If window_length =1=1=1 , the returned window contains a single value 1.

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window

Return type

Tensor

`torch.blackman_window`(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

Blackman window function.

w[n]=0.42−0.5cos⁡(2πnN−1)+0.08cos⁡(4πnN−1)w[n] = 0.42 - 0.5 \cos \left( \frac{2 \pi n}{N - 1} \right) + 0.08 \cos \left( \frac{4 \pi n}{N - 1} \right) w[n]=0.42−0.5cos(N−12πn)+0.08cos(N−14πn)

where NNN is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.blackman_window(L, periodic=True) equal to torch.blackman_window(L + 1, periodic=False)[:-1]).

Note

If window_length =1=1=1 , the returned window contains a single value 1.

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window

Return type

Tensor

`torch.hamming_window`(window_length, periodic=True, alpha=0.54, beta=0.46, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

Hamming window function.

w[n]=α−β cos⁡(2πnN−1),w[n] = \alpha - \beta\ \cos \left( \frac{2 \pi n}{N - 1} \right), w[n]=α−β cos(N−12πn),

where NNN is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.hamming_window(L, periodic=True) equal to torch.hamming_window(L + 1, periodic=False)[:-1]).

Note

If window_length =1=1=1 , the returned window contains a single value 1.

Note

This is a generalized version of torch.hann_window().

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.
alpha (float, optional) – The coefficient α\alphaα in the equation above
beta (float, optional) – The coefficient β\betaβ in the equation above
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window

Return type

Tensor

`torch.hann_window`(window_length, periodic=True, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor

Hann window function.

w[n]=12 [1−cos⁡(2πnN−1)]=sin⁡2(πnN−1),w[n] = \frac{1}{2}\ \left[1 - \cos \left( \frac{2 \pi n}{N - 1} \right)\right] = \sin^2 \left( \frac{\pi n}{N - 1} \right), w[n]=21 [1−cos(N−12πn)]=sin2(N−1πn),

where NNN is the full window size.

The input window_length is a positive integer controlling the returned window size. periodic flag determines whether the returned window trims off the last duplicate value from the symmetric window and is ready to be used as a periodic window with functions like torch.stft(). Therefore, if periodic is true, the NNN in above formula is in fact window_length+1\text{window\_length} + 1window_length+1 . Also, we always have torch.hann_window(L, periodic=True) equal to torch.hann_window(L + 1, periodic=False)[:-1]).

Note

If window_length =1=1=1 , the returned window contains a single value 1.

Parameters

window_length (int) – the size of returned window
periodic (bool, optional) – If True, returns a window to be used as periodic function. If False, return a symmetric window.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, uses a global default (see torch.set_default_tensor_type()). Only floating point types are supported.
layout (torch.layout, optional) – the desired layout of returned window tensor. Only torch.strided (dense layout) is supported.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

A 1-D tensor of size (window_length,)(\text{window\_length},)(window_length,) containing the window

Return type

Tensor

Other Operations

`torch.bincount`(input, weights=None, minlength=0) → Tensor

Count the frequency of each value in an array of non-negative ints.

The number of bins (size 1) is one larger than the largest value in input unless input is empty, in which case the result is a tensor of size 0. If minlength is specified, the number of bins is at least minlength and if input is empty, then the result is tensor of size minlength filled with zeros. If n is the value at position i, out[n] += weights[i] if weights is specified else out[n] += 1.

Note

When using the CUDA backend, this operation may induce nondeterministic behaviour that is not easily switched off. Please see the notes on Reproducibility for background.

Parameters

input (Tensor) – 1-d int tensor
weights (Tensor) – optional, weight for each value in the input tensor. Should be of same size as input tensor.
minlength (int) – optional, minimum number of bins. Should be non-negative.

Returns

a tensor of shape Size([max(input) + 1]) if input is non-empty, else Size(0)

Return type

output (Tensor)

Example:

>>> input = torch.randint(0, 8, (5,), dtype=torch.int64)
>>> weights = torch.linspace(0, 1, steps=5)
>>> input, weights
(tensor([4, 3, 6, 3, 4]),
 tensor([ 0.0000,  0.2500,  0.5000,  0.7500,  1.0000])

>>> torch.bincount(input)
tensor([0, 0, 0, 2, 2, 0, 1])

>>> input.bincount(weights)
tensor([0.0000, 0.0000, 0.0000, 1.0000, 1.0000, 0.0000, 0.5000])

`torch.broadcast_tensors`(*tensors) → List of Tensors[source]

Broadcasts the given tensors according to Broadcasting semantics.

Parameters

*tensors – any number of tensors of the same type

Warning

More than one element of a broadcasted tensor may refer to a single memory location. As a result, in-place operations (especially ones that are vectorized) may result in incorrect behavior. If you need to write to the tensors, please clone them first.

Example:

>>> x = torch.arange(3).view(1, 3)
>>> y = torch.arange(2).view(2, 1)
>>> a, b = torch.broadcast_tensors(x, y)
>>> a.size()
torch.Size([2, 3])
>>> a
tensor([[0, 1, 2],
        [0, 1, 2]])

`torch.cartesian_prod`(*tensors)[source]

Do cartesian product of the given sequence of tensors. The behavior is similar to python’s itertools.product.

Parameters

*tensors – any number of 1 dimensional tensors.

Returns

A tensor equivalent to converting all the input tensors into lists,

do itertools.product on these lists, and finally convert the resulting list into tensor.

Return type

Tensor

Example:

>>> a = [1, 2, 3]
>>> b = [4, 5]
>>> list(itertools.product(a, b))
[(1, 4), (1, 5), (2, 4), (2, 5), (3, 4), (3, 5)]
>>> tensor_a = torch.tensor(a)
>>> tensor_b = torch.tensor(b)
>>> torch.cartesian_prod(tensor_a, tensor_b)
tensor([[1, 4],
        [1, 5],
        [2, 4],
        [2, 5],
        [3, 4],
        [3, 5]])

`torch.cdist`(x1, x2, p=2) → Tensor

Computes the p-norm distance between each pair of the two collections of row vectors.

If x1 has shape P×MP \times MP×M and x2 has shape R×MR \times MR×M then the output will have shape P×RP \times RP×R .

This function is equivalent to scipy.spatial.distance.cdist(input,’minkowski’, p=p) if p∈(0,∞)p \in (0, \infty)p∈(0,∞) . When p=0p = 0p=0 it is equivalent to scipy.spatial.distance.cdist(input, ‘hamming’) * M. When p=∞p = \inftyp=∞ , the closest scipy function is scipy.spatial.distance.cdist(xn, lambda x, y: np.abs(x - y).max()).

Parameters

x1 (Tensor) – input tensor of shape P×MP \times MP×M .
x2 (Tensor) – input tensor of shape R×MR \times MR×M .
p – p value for the p-norm distance to calculate between each vector pair ∈[0,∞]\in [0, \infty]∈[0,∞] .

Example:

>>> a = torch.tensor([[0.9041,  0.0196], [-0.3108, -2.4423], [-0.4821,  1.059]])
>>> a
tensor([[ 0.9041,  0.0196],
        [-0.3108, -2.4423],
        [-0.4821,  1.0590]])
>>> b = torch.tensor([[-2.1763, -0.4713], [-0.6986,  1.3702]])
>>> b
tensor([[-2.1763, -0.4713],
        [-0.6986,  1.3702]])
>>> torch.cdist(a, b, p=2)
tensor([[3.1193, 2.0959],
        [2.7138, 3.8322],
        [2.2830, 0.3791]])

`torch.combinations`(input, r=2, with_replacement=False) → seq

Compute combinations of length rrr of the given tensor. The behavior is similar to python’s itertools.combinations when with_replacement is set to False, and itertools.combinations_with_replacement when with_replacement is set to True.

Parameters

input (Tensor) – 1D vector.
r (int, optional) – number of elements to combine
with_replacement (boolean, optional) – whether to allow duplication in combination

Returns

A tensor equivalent to converting all the input tensors into lists, do itertools.combinations or itertools.combinations_with_replacement on these lists, and finally convert the resulting list into tensor.

Return type

Tensor

Example:

>>> a = [1, 2, 3]
>>> list(itertools.combinations(a, r=2))
[(1, 2), (1, 3), (2, 3)]
>>> list(itertools.combinations(a, r=3))
[(1, 2, 3)]
>>> list(itertools.combinations_with_replacement(a, r=2))
[(1, 1), (1, 2), (1, 3), (2, 2), (2, 3), (3, 3)]
>>> tensor_a = torch.tensor(a)
>>> torch.combinations(tensor_a)
tensor([[1, 2],
        [1, 3],
        [2, 3]])
>>> torch.combinations(tensor_a, r=3)
tensor([[1, 2, 3]])
>>> torch.combinations(tensor_a, with_replacement=True)
tensor([[1, 1],
        [1, 2],
        [1, 3],
        [2, 2],
        [2, 3],
        [3, 3]])

`torch.cross`(input, other, dim=-1, out=None) → Tensor

Returns the cross product of vectors in dimension dim of input and other.

input and other must have the same size, and the size of their dim dimension should be 3.

If dim is not given, it defaults to the first dimension found with the size 3.

Parameters

input (Tensor) – the input tensor.
other (Tensor) – the second input tensor
dim (int, optional) – the dimension to take the cross-product in.
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(4, 3)
>>> a
tensor([[-0.3956,  1.1455,  1.6895],
        [-0.5849,  1.3672,  0.3599],
        [-1.1626,  0.7180, -0.0521],
        [-0.1339,  0.9902, -2.0225]])
>>> b = torch.randn(4, 3)
>>> b
tensor([[-0.0257, -1.4725, -1.2251],
        [-1.1479, -0.7005, -1.9757],
        [-1.3904,  0.3726, -1.1836],
        [-0.9688, -0.7153,  0.2159]])
>>> torch.cross(a, b, dim=1)
tensor([[ 1.0844, -0.5281,  0.6120],
        [-2.4490, -1.5687,  1.9792],
        [-0.8304, -1.3037,  0.5650],
        [-1.2329,  1.9883,  1.0551]])
>>> torch.cross(a, b)
tensor([[ 1.0844, -0.5281,  0.6120],
        [-2.4490, -1.5687,  1.9792],
        [-0.8304, -1.3037,  0.5650],
        [-1.2329,  1.9883,  1.0551]])

`torch.diag`(input, diagonal=0, out=None) → Tensor

If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal.
If input is a matrix (2-D tensor), then returns a 1-D tensor with the diagonal elements of input.

The argument diagonal controls which diagonal to consider:

If diagonal = 0, it is the main diagonal.
If diagonal > 0, it is above the main diagonal.
If diagonal < 0, it is below the main diagonal.

Parameters

input (Tensor) – the input tensor.
diagonal (int, optional) – the diagonal to consider
out (Tensor, optional) – the output tensor.

`torch.diag_embed`(input, offset=0, dim1=-2, dim2=-1) → Tensor

Creates a tensor whose diagonals of certain 2D planes (specified by dim1 and dim2) are filled by input. To facilitate creating batched diagonal matrices, the 2D planes formed by the last two dimensions of the returned tensor are chosen by default.

The argument offset controls which diagonal to consider:

If offset = 0, it is the main diagonal.
If offset > 0, it is above the main diagonal.
If offset < 0, it is below the main diagonal.

The size of the new matrix will be calculated to make the specified diagonal of the size of the last input dimension. Note that for offset other than 000 , the order of dim1 and dim2 matters. Exchanging them is equivalent to changing the sign of offset.

Applying torch.diagonal() to the output of this function with the same arguments yields a matrix identical to input. However, torch.diagonal() has different default dimensions, so those need to be explicitly specified.

Parameters

input (Tensor) – the input tensor. Must be at least 1-dimensional.
offset (int, optional) – which diagonal to consider. Default: 0 (main diagonal).
dim1 (int, optional) – first dimension with respect to which to take diagonal. Default: -2.
dim2 (int, optional) – second dimension with respect to which to take diagonal. Default: -1.

Example:

>>> a = torch.randn(2, 3)
>>> torch.diag_embed(a)
tensor([[[ 1.5410,  0.0000,  0.0000],
         [ 0.0000, -0.2934,  0.0000],
         [ 0.0000,  0.0000, -2.1788]],

        [[ 0.5684,  0.0000,  0.0000],
         [ 0.0000, -1.0845,  0.0000],
         [ 0.0000,  0.0000, -1.3986]]])

>>> torch.diag_embed(a, offset=1, dim1=0, dim2=2)
tensor([[[ 0.0000,  1.5410,  0.0000,  0.0000],
         [ 0.0000,  0.5684,  0.0000,  0.0000]],

        [[ 0.0000,  0.0000, -0.2934,  0.0000],
         [ 0.0000,  0.0000, -1.0845,  0.0000]],

        [[ 0.0000,  0.0000,  0.0000, -2.1788],
         [ 0.0000,  0.0000,  0.0000, -1.3986]],

        [[ 0.0000,  0.0000,  0.0000,  0.0000],
         [ 0.0000,  0.0000,  0.0000,  0.0000]]])

`torch.diagflat`(input, offset=0) → Tensor

If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal.
If input is a tensor with more than one dimension, then returns a 2-D tensor with diagonal elements equal to a flattened input.

The argument offset controls which diagonal to consider:

If offset = 0, it is the main diagonal.
If offset > 0, it is above the main diagonal.
If offset < 0, it is below the main diagonal.

Parameters

input (Tensor) – the input tensor.
offset (int, optional) – the diagonal to consider. Default: 0 (main diagonal).

Examples:

>>> a = torch.randn(3)
>>> a
tensor([-0.2956, -0.9068,  0.1695])
>>> torch.diagflat(a)
tensor([[-0.2956,  0.0000,  0.0000],
        [ 0.0000, -0.9068,  0.0000],
        [ 0.0000,  0.0000,  0.1695]])
>>> torch.diagflat(a, 1)
tensor([[ 0.0000, -0.2956,  0.0000,  0.0000],
        [ 0.0000,  0.0000, -0.9068,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.1695],
        [ 0.0000,  0.0000,  0.0000,  0.0000]])

>>> a = torch.randn(2, 2)
>>> a
tensor([[ 0.2094, -0.3018],
        [-0.1516,  1.9342]])
>>> torch.diagflat(a)
tensor([[ 0.2094,  0.0000,  0.0000,  0.0000],
        [ 0.0000, -0.3018,  0.0000,  0.0000],
        [ 0.0000,  0.0000, -0.1516,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  1.9342]])

`torch.diagonal`(input, offset=0, dim1=0, dim2=1) → Tensor

Returns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape.

The argument offset controls which diagonal to consider:

If offset = 0, it is the main diagonal.
If offset > 0, it is above the main diagonal.
If offset < 0, it is below the main diagonal.

Applying torch.diag_embed() to the output of this function with the same arguments yields a diagonal matrix with the diagonal entries of the input. However, torch.diag_embed() has different default dimensions, so those need to be explicitly specified.

Parameters

input (Tensor) – the input tensor. Must be at least 2-dimensional.
offset (int, optional) – which diagonal to consider. Default: 0 (main diagonal).
dim1 (int, optional) – first dimension with respect to which to take diagonal. Default: 0.
dim2 (int, optional) – second dimension with respect to which to take diagonal. Default: 1.

Note

To take a batch diagonal, pass in dim1=-2, dim2=-1.

Examples:

>>> a = torch.randn(3, 3)
>>> a
tensor([[-1.0854,  1.1431, -0.1752],
        [ 0.8536, -0.0905,  0.0360],
        [ 0.6927, -0.3735, -0.4945]])


>>> torch.diagonal(a, 0)
tensor([-1.0854, -0.0905, -0.4945])


>>> torch.diagonal(a, 1)
tensor([ 1.1431,  0.0360])


>>> x = torch.randn(2, 5, 4, 2)
>>> torch.diagonal(x, offset=-1, dim1=1, dim2=2)
tensor([[[-1.2631,  0.3755, -1.5977, -1.8172],
         [-1.1065,  1.0401, -0.2235, -0.7938]],

        [[-1.7325, -0.3081,  0.6166,  0.2335],
         [ 1.0500,  0.7336, -0.3836, -1.1015]]])

`torch.einsum`(equation, *operands) → Tensor[source]

This function provides a way of computing multilinear expressions (i.e. sums of products) using the Einstein summation convention.

Parameters

equation (string) – The equation is given in terms of lower case letters (indices) to be associated with each dimension of the operands and result. The left hand side lists the operands dimensions, separated by commas. There should be one index letter per tensor dimension. The right hand side follows after -> and gives the indices for the output. If the -> and right hand side are omitted, it implicitly defined as the alphabetically sorted list of all indices appearing exactly once in the left hand side. The indices not apprearing in the output are summed over after multiplying the operands entries. If an index appears several times for the same operand, a diagonal is taken. Ellipses … represent a fixed number of dimensions. If the right hand side is inferred, the ellipsis dimensions are at the beginning of the output.
operands (list of Tensors) – The operands to compute the Einstein sum of.

Examples:

>>> x = torch.randn(5)
>>> y = torch.randn(4)
>>> torch.einsum('i,j->ij', x, y)  # outer product
tensor([[-0.0570, -0.0286, -0.0231,  0.0197],
        [ 1.2616,  0.6335,  0.5113, -0.4351],
        [ 1.4452,  0.7257,  0.5857, -0.4984],
        [-0.4647, -0.2333, -0.1883,  0.1603],
        [-1.1130, -0.5588, -0.4510,  0.3838]])


>>> A = torch.randn(3,5,4)
>>> l = torch.randn(2,5)
>>> r = torch.randn(2,4)
>>> torch.einsum('bn,anm,bm->ba', l, A, r) # compare torch.nn.functional.bilinear
tensor([[-0.3430, -5.2405,  0.4494],
        [ 0.3311,  5.5201, -3.0356]])


>>> As = torch.randn(3,2,5)
>>> Bs = torch.randn(3,5,4)
>>> torch.einsum('bij,bjk->bik', As, Bs) # batch matrix multiplication
tensor([[[-1.0564, -1.5904,  3.2023,  3.1271],
         [-1.6706, -0.8097, -0.8025, -2.1183]],

        [[ 4.2239,  0.3107, -0.5756, -0.2354],
         [-1.4558, -0.3460,  1.5087, -0.8530]],

        [[ 2.8153,  1.8787, -4.3839, -1.2112],
         [ 0.3728, -2.1131,  0.0921,  0.8305]]])

>>> A = torch.randn(3, 3)
>>> torch.einsum('ii->i', A) # diagonal
tensor([-0.7825,  0.8291, -0.1936])

>>> A = torch.randn(4, 3, 3)
>>> torch.einsum('...ii->...i', A) # batch diagonal
tensor([[-1.0864,  0.7292,  0.0569],
        [-0.9725, -1.0270,  0.6493],
        [ 0.5832, -1.1716, -1.5084],
        [ 0.4041, -1.1690,  0.8570]])

>>> A = torch.randn(2, 3, 4, 5)
>>> torch.einsum('...ij->...ji', A).shape # batch permute
torch.Size([2, 3, 5, 4])

`torch.flatten`(input, start_dim=0, end_dim=-1) → Tensor

Flattens a contiguous range of dims in a tensor.

Parameters

input (Tensor) – the input tensor.
start_dim (int) – the first dim to flatten
end_dim (int) – the last dim to flatten

Example:

>>> t = torch.tensor([[[1, 2],
                       [3, 4]],
                      [[5, 6],
                       [7, 8]]])
>>> torch.flatten(t)
tensor([1, 2, 3, 4, 5, 6, 7, 8])
>>> torch.flatten(t, start_dim=1)
tensor([[1, 2, 3, 4],
        [5, 6, 7, 8]])

`torch.flip`(input, dims) → Tensor

Reverse the order of a n-D tensor along given axis in dims.

Parameters

input (Tensor) – the input tensor.
dims (a list or tuple) – axis to flip on

Example:

>>> x = torch.arange(8).view(2, 2, 2)
>>> x
tensor([[[ 0,  1],
         [ 2,  3]],

        [[ 4,  5],
         [ 6,  7]]])
>>> torch.flip(x, [0, 1])
tensor([[[ 6,  7],
         [ 4,  5]],

        [[ 2,  3],
         [ 0,  1]]])

`torch.rot90`(input, k, dims) → Tensor

Rotate a n-D tensor by 90 degrees in the plane specified by dims axis. Rotation direction is from the first towards the second axis if k > 0, and from the second towards the first for k < 0.

Parameters

input (Tensor) – the input tensor.
k (int) – number of times to rotate
dims (a list or tuple) – axis to rotate

Example:

>>> x = torch.arange(4).view(2, 2)
>>> x
tensor([[0, 1],
        [2, 3]])
>>> torch.rot90(x, 1, [0, 1])
tensor([[1, 3],
        [0, 2]])

>>> x = torch.arange(8).view(2, 2, 2)
>>> x
tensor([[[0, 1],
         [2, 3]],

        [[4, 5],
         [6, 7]]])
>>> torch.rot90(x, 1, [1, 2])
tensor([[[1, 3],
         [0, 2]],

        [[5, 7],
         [4, 6]]])

`torch.histc`(input, bins=100, min=0, max=0, out=None) → Tensor

Computes the histogram of a tensor.

The elements are sorted into equal width bins between min and max. If min and max are both zero, the minimum and maximum values of the data are used.

Parameters

input (Tensor) – the input tensor.
bins (int) – number of histogram bins
min (int) – lower end of the range (inclusive)
max (int) – upper end of the range (inclusive)
out (Tensor, optional) – the output tensor.

Returns

Histogram represented as a tensor

Return type

Tensor

Example:

>>> torch.histc(torch.tensor([1., 2, 1]), bins=4, min=0, max=3)
tensor([ 0.,  2.,  1.,  0.])

`torch.meshgrid`(*tensors, **kwargs)[source]

Take NNN tensors, each of which can be either scalar or 1-dimensional vector, and create NNN N-dimensional grids, where the iii th grid is defined by expanding the iii th input over dimensions defined by other inputs.

Args: tensors (list of Tensor): list of scalars or 1 dimensional tensors. Scalars will be treated as tensors of size (1,)(1,)(1,) automatically Returns: seq (sequence of Tensors): If the input has kkk tensors of size (N1,),(N2,),…,(Nk,)(N_1,), (N_2,), \ldots , (N_k,)(N1,),(N2,),…,(Nk,) , then the output would also have kkk tensors, where all tensors are of size (N1,N2,…,Nk)(N_1, N_2, \ldots , N_k)(N1,N2,…,Nk) . Example: >>> x = torch.tensor([1, 2, 3]) >>> y = torch.tensor([4, 5, 6]) >>> grid_x, grid_y = torch.meshgrid(x, y) >>> grid_x tensor([[1, 1, 1], [2, 2, 2], [3, 3, 3]]) >>> grid_y tensor([[4, 5, 6], [4, 5, 6], [4, 5, 6]])

`torch.renorm`(input, p, dim, maxnorm, out=None) → Tensor

Returns a tensor where each sub-tensor of input along dimension dim is normalized such that the p-norm of the sub-tensor is lower than the value maxnorm

Note

If the norm of a row is lower than maxnorm, the row is unchanged

Parameters

input (Tensor) – the input tensor.
p (float) – the power for the norm computation
dim (int) – the dimension to slice over to get the sub-tensors
maxnorm (float) – the maximum norm to keep each sub-tensor under
out (Tensor, optional) – the output tensor.

Example:

>>> x = torch.ones(3, 3)
>>> x[1].fill_(2)
tensor([ 2.,  2.,  2.])
>>> x[2].fill_(3)
tensor([ 3.,  3.,  3.])
>>> x
tensor([[ 1.,  1.,  1.],
        [ 2.,  2.,  2.],
        [ 3.,  3.,  3.]])
>>> torch.renorm(x, 1, 0, 5)
tensor([[ 1.0000,  1.0000,  1.0000],
        [ 1.6667,  1.6667,  1.6667],
        [ 1.6667,  1.6667,  1.6667]])

`torch.repeat_interleave`()

torch.repeat_interleave(input, repeats, dim=None) → Tensor

Repeat elements of a tensor.

Warning

This is different from torch.repeat() but similar to numpy.repeat.

Parameters

input (Tensor) – the input tensor.
repeats (Tensor or int) – The number of repetitions for each element. repeats is broadcasted to fit the shape of the given axis.
dim (int, optional) – The dimension along which to repeat values. By default, use the flattened input array, and return a flat output array.

Returns

Repeated tensor which has the same shape as input, except along the

given axis.

Return type

Tensor

Example:

>>> x = torch.tensor([1, 2, 3])
>>> x.repeat_interleave(2)
tensor([1, 1, 2, 2, 3, 3])
>>> y = torch.tensor([[1, 2], [3, 4]])
>>> torch.repeat_interleave(y, 2)
tensor([1, 1, 2, 2, 3, 3, 4, 4])
>>> torch.repeat_interleave(y, 3, dim=1)
tensor([[1, 1, 1, 2, 2, 2],
        [3, 3, 3, 4, 4, 4]])
>>> torch.repeat_interleave(y, torch.tensor([1, 2]), dim=0)
tensor([[1, 2],
        [3, 4],
        [3, 4]])

`torch.repeat_interleave`(repeats) → Tensor

If the repeats is tensor([n1, n2, n3, …]), then the output will be tensor([0, 0, …, 1, 1, …, 2, 2, …, …]) where 0 appears n1 times, 1 appears n2 times, 2 appears n3 times, etc.

torch.roll(input, shifts, dims=None) → Tensor

Roll the tensor along the given dimension(s). Elements that are shifted beyond the last position are re-introduced at the first position. If a dimension is not specified, the tensor will be flattened before rolling and then restored to the original shape.

Parameters

input (Tensor) – the input tensor.
shifts (int or tuple of python:ints) – The number of places by which the elements of the tensor are shifted. If shifts is a tuple, dims must be a tuple of the same size, and each dimension will be rolled by the corresponding value
dims (int or tuple of python:ints) – Axis along which to roll

Example:

>>> x = torch.tensor([1, 2, 3, 4, 5, 6, 7, 8]).view(4, 2)
>>> x
tensor([[1, 2],
        [3, 4],
        [5, 6],
        [7, 8]])
>>> torch.roll(x, 1, 0)
tensor([[7, 8],
        [1, 2],
        [3, 4],
        [5, 6]])
>>> torch.roll(x, -1, 0)
tensor([[3, 4],
        [5, 6],
        [7, 8],
        [1, 2]])
>>> torch.roll(x, shifts=(2, 1), dims=(0, 1))
tensor([[6, 5],
        [8, 7],
        [2, 1],
        [4, 3]])

`torch.tensordot`(a, b, dims=2)[source]

Returns a contraction of a and b over multiple dimensions.

tensordot implements a generalized matrix product.

Parameters

a (Tensor) – Left tensor to contract
b (Tensor) – Right tensor to contract
dims (int or tuple of two lists of python:integers) – number of dimensions to contract or explicit lists of dimensions for a and b respectively

When called with an integer argument dims = ddd , and the number of dimensions of a and b is mmm and nnn , respectively, it computes

ri0,...,im−d,id,...,in=∑k0,...,kd−1ai0,...,im−d,k0,...,kd−1×bk0,...,kd−1,id,...,in.r_{i_0,...,i_{m-d}, i_d,...,i_n} = \sum_{k_0,...,k_{d-1}} a_{i_0,...,i_{m-d},k_0,...,k_{d-1}} \times b_{k_0,...,k_{d-1}, i_d,...,i_n}. ri0,...,im−d,id,...,in=k0,...,kd−1∑ai0,...,im−d,k0,...,kd−1×bk0,...,kd−1,id,...,in.

When called with dims of the list form, the given dimensions will be contracted in place of the last ddd of a and the first ddd of bbb . The sizes in these dimensions must match, but tensordot will deal with broadcasted dimensions.

Examples:

>>> a = torch.arange(60.).reshape(3, 4, 5)
>>> b = torch.arange(24.).reshape(4, 3, 2)
>>> torch.tensordot(a, b, dims=([1, 0], [0, 1]))
tensor([[4400., 4730.],
        [4532., 4874.],
        [4664., 5018.],
        [4796., 5162.],
        [4928., 5306.]])

>>> a = torch.randn(3, 4, 5, device='cuda')
>>> b = torch.randn(4, 5, 6, device='cuda')
>>> c = torch.tensordot(a, b, dims=2).cpu()
tensor([[ 8.3504, -2.5436,  6.2922,  2.7556, -1.0732,  3.2741],
        [ 3.3161,  0.0704,  5.0187, -0.4079, -4.3126,  4.8744],
        [ 0.8223,  3.9445,  3.2168, -0.2400,  3.4117,  1.7780]])

`torch.trace`(input) → Tensor

Returns the sum of the elements of the diagonal of the input 2-D matrix.

Example:

>>> x = torch.arange(1., 10.).view(3, 3)
>>> x
tensor([[ 1.,  2.,  3.],
        [ 4.,  5.,  6.],
        [ 7.,  8.,  9.]])
>>> torch.trace(x)
tensor(15.)

`torch.tril`(input, diagonal=0, out=None) → Tensor

Returns the lower triangular part of the matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.

The lower triangular part of the matrix is defined as the elements on and below the diagonal.

The argument diagonal controls which diagonal to consider. If diagonal = 0, all elements on and below the main diagonal are retained. A positive value includes just as many diagonals above the main diagonal, and similarly a negative value excludes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min⁡{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.

Parameters

input (Tensor) – the input tensor.
diagonal (int, optional) – the diagonal to consider
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(3, 3)
>>> a
tensor([[-1.0813, -0.8619,  0.7105],
        [ 0.0935,  0.1380,  2.2112],
        [-0.3409, -0.9828,  0.0289]])
>>> torch.tril(a)
tensor([[-1.0813,  0.0000,  0.0000],
        [ 0.0935,  0.1380,  0.0000],
        [-0.3409, -0.9828,  0.0289]])

>>> b = torch.randn(4, 6)
>>> b
tensor([[ 1.2219,  0.5653, -0.2521, -0.2345,  1.2544,  0.3461],
        [ 0.4785, -0.4477,  0.6049,  0.6368,  0.8775,  0.7145],
        [ 1.1502,  3.2716, -1.1243, -0.5413,  0.3615,  0.6864],
        [-0.0614, -0.7344, -1.3164, -0.7648, -1.4024,  0.0978]])
>>> torch.tril(b, diagonal=1)
tensor([[ 1.2219,  0.5653,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.4785, -0.4477,  0.6049,  0.0000,  0.0000,  0.0000],
        [ 1.1502,  3.2716, -1.1243, -0.5413,  0.0000,  0.0000],
        [-0.0614, -0.7344, -1.3164, -0.7648, -1.4024,  0.0000]])
>>> torch.tril(b, diagonal=-1)
tensor([[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.4785,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 1.1502,  3.2716,  0.0000,  0.0000,  0.0000,  0.0000],
        [-0.0614, -0.7344, -1.3164,  0.0000,  0.0000,  0.0000]])

`torch.tril_indices`(row, col, offset=0, dtype=torch.long, device='cpu', layout=torch.strided) → Tensor

Returns the indices of the lower triangular part of a row-by- col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates. Indices are ordered based on rows and then columns.

The lower triangular part of the matrix is defined as the elements on and below the diagonal.

The argument offset controls which diagonal to consider. If offset = 0, all elements on and below the main diagonal are retained. A positive value includes just as many diagonals above the main diagonal, and similarly a negative value excludes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min⁡{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.

NOTE: when running on ‘cuda’, row * col must be less than 2592^{59}259 to prevent overflow during calculation.

Parameters

row (int) – number of rows in the 2-D matrix.
col (int) – number of columns in the 2-D matrix.
offset (int) – diagonal offset from the main diagonal. Default: if not provided, 0.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, torch.long.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
layout (torch.layout, optional) – currently only support torch.strided.

Example::

>>> a = torch.tril_indices(3, 3)
>>> a
tensor([[0, 1, 1, 2, 2, 2],
        [0, 0, 1, 0, 1, 2]])

>>> a = torch.tril_indices(4, 3, -1)
>>> a
tensor([[1, 2, 2, 3, 3, 3],
        [0, 0, 1, 0, 1, 2]])

>>> a = torch.tril_indices(4, 3, 1)
>>> a
tensor([[0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3],
        [0, 1, 0, 1, 2, 0, 1, 2, 0, 1, 2]])

`torch.triu`(input, diagonal=0, out=None) → Tensor

Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0.

The upper triangular part of the matrix is defined as the elements on and above the diagonal.

The argument diagonal controls which diagonal to consider. If diagonal = 0, all elements on and below the main diagonal are retained. A positive value excludes just as many diagonals above the main diagonal, and similarly a negative value includes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min⁡{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.

Parameters

input (Tensor) – the input tensor.
diagonal (int, optional) – the diagonal to consider
out (Tensor, optional) – the output tensor.

Example:

>>> a = torch.randn(3, 3)
>>> a
tensor([[ 0.2309,  0.5207,  2.0049],
        [ 0.2072, -1.0680,  0.6602],
        [ 0.3480, -0.5211, -0.4573]])
>>> torch.triu(a)
tensor([[ 0.2309,  0.5207,  2.0049],
        [ 0.0000, -1.0680,  0.6602],
        [ 0.0000,  0.0000, -0.4573]])
>>> torch.triu(a, diagonal=1)
tensor([[ 0.0000,  0.5207,  2.0049],
        [ 0.0000,  0.0000,  0.6602],
        [ 0.0000,  0.0000,  0.0000]])
>>> torch.triu(a, diagonal=-1)
tensor([[ 0.2309,  0.5207,  2.0049],
        [ 0.2072, -1.0680,  0.6602],
        [ 0.0000, -0.5211, -0.4573]])

>>> b = torch.randn(4, 6)
>>> b
tensor([[ 0.5876, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
        [-0.2447,  0.9556, -1.2919,  1.3378, -0.1768, -1.0857],
        [ 0.4333,  0.3146,  0.6576, -1.0432,  0.9348, -0.4410],
        [-0.9888,  1.0679, -1.3337, -1.6556,  0.4798,  0.2830]])
>>> torch.triu(b, diagonal=1)
tensor([[ 0.0000, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
        [ 0.0000,  0.0000, -1.2919,  1.3378, -0.1768, -1.0857],
        [ 0.0000,  0.0000,  0.0000, -1.0432,  0.9348, -0.4410],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.4798,  0.2830]])
>>> torch.triu(b, diagonal=-1)
tensor([[ 0.5876, -0.0794, -1.8373,  0.6654,  0.2604,  1.5235],
        [-0.2447,  0.9556, -1.2919,  1.3378, -0.1768, -1.0857],
        [ 0.0000,  0.3146,  0.6576, -1.0432,  0.9348, -0.4410],
        [ 0.0000,  0.0000, -1.3337, -1.6556,  0.4798,  0.2830]])

`torch.triu_indices`(row, col, offset=0, dtype=torch.long, device='cpu', layout=torch.strided) → Tensor

Returns the indices of the upper triangular part of a row by col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates. Indices are ordered based on rows and then columns.

The upper triangular part of the matrix is defined as the elements on and above the diagonal.

The argument offset controls which diagonal to consider. If offset = 0, all elements on and above the main diagonal are retained. A positive value excludes just as many diagonals above the main diagonal, and similarly a negative value includes just as many diagonals below the main diagonal. The main diagonal are the set of indices {(i,i)}\lbrace (i, i) \rbrace{(i,i)} for i∈[0,min⁡{d1,d2}−1]i \in [0, \min\{d_{1}, d_{2}\} - 1]i∈[0,min{d1,d2}−1] where d1,d2d_{1}, d_{2}d1,d2 are the dimensions of the matrix.

NOTE: when running on ‘cuda’, row * col must be less than 2592^{59}259 to prevent overflow during calculation.

Parameters

row (int) – number of rows in the 2-D matrix.
col (int) – number of columns in the 2-D matrix.
offset (int) – diagonal offset from the main diagonal. Default: if not provided, 0.
dtype (torch.dtype, optional) – the desired data type of returned tensor. Default: if None, torch.long.
device (torch.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type (see torch.set_default_tensor_type()). device will be the CPU for CPU tensor types and the current CUDA device for CUDA tensor types.
layout (torch.layout, optional) – currently only support torch.strided.

Example::

>>> a = torch.triu_indices(3, 3)
>>> a
tensor([[0, 0, 0, 1, 1, 2],
        [0, 1, 2, 1, 2, 2]])

>>> a = torch.triu_indices(4, 3, -1)
>>> a
tensor([[0, 0, 0, 1, 1, 1, 2, 2, 3],
        [0, 1, 2, 0, 1, 2, 1, 2, 2]])

>>> a = torch.triu_indices(4, 3, 1)
>>> a
tensor([[0, 0, 1],
        [1, 2, 2]])

BLAS and LAPACK Operations

`torch.addbmm`(beta=1, input, alpha=1, batch1, batch2, out=None) → Tensor

Performs a batch matrix-matrix product of matrices stored in batch1 and batch2, with a reduced add step (all matrix multiplications get accumulated along the first dimension). input is added to the final result.

batch1 and batch2 must be 3-D tensors each containing the same number of matrices.

If batch1 is a (b×n×m)(b \times n \times m)(b×n×m) tensor, batch2 is a (b×m×p)(b \times m \times p)(b×m×p) tensor, input must be broadcastable with a (n×p)(n \times p)(n×p) tensor and out will be a (n×p)(n \times p)(n×p) tensor.

out=β input+α (∑i=0b−1batch1i@batch2i)out = \beta\ \text{input} + \alpha\ (\sum_{i=0}^{b-1} \text{batch1}_i \mathbin{@} \text{batch2}_i) out=β input+α (i=0∑b−1batch1i@batch2i)

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

Parameters

beta (Number, optional) – multiplier for input (β\betaβ )
input (Tensor) – matrix to be added
alpha (Number, optional) – multiplier for batch1 @ batch2 (α\alphaα )
batch1 (Tensor) – the first batch of matrices to be multiplied
batch2 (Tensor) – the second batch of matrices to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(3, 5)
>>> batch1 = torch.randn(10, 3, 4)
>>> batch2 = torch.randn(10, 4, 5)
>>> torch.addbmm(M, batch1, batch2)
tensor([[  6.6311,   0.0503,   6.9768, -12.0362,  -2.1653],
        [ -4.8185,  -1.4255,  -6.6760,   8.9453,   2.5743],
        [ -3.8202,   4.3691,   1.0943,  -1.1109,   5.4730]])

`torch.addmm`(beta=1, input, alpha=1, mat1, mat2, out=None) → Tensor

Performs a matrix multiplication of the matrices mat1 and mat2. The matrix input is added to the final result.

If mat1 is a (n×m)(n \times m)(n×m) tensor, mat2 is a (m×p)(m \times p)(m×p) tensor, then input must be broadcastable with a (n×p)(n \times p)(n×p) tensor and out will be a (n×p)(n \times p)(n×p) tensor.

alpha and beta are scaling factors on matrix-vector product between mat1 and mat2 and the added matrix input respectively.

out=β input+α (mat1i@mat2i)\text{out} = \beta\ \text{input} + \alpha\ (\text{mat1}_i \mathbin{@} \text{mat2}_i) out=β input+α (mat1i@mat2i)

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

Parameters

beta (Number, optional) – multiplier for input (β\betaβ )
input (Tensor) – matrix to be added
alpha (Number, optional) – multiplier for mat1@mat2mat1 @ mat2mat1@mat2 (α\alphaα )
mat1 (Tensor) – the first matrix to be multiplied
mat2 (Tensor) – the second matrix to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(2, 3)
>>> mat1 = torch.randn(2, 3)
>>> mat2 = torch.randn(3, 3)
>>> torch.addmm(M, mat1, mat2)
tensor([[-4.8716,  1.4671, -1.3746],
        [ 0.7573, -3.9555, -2.8681]])

`torch.addmv`(beta=1, input, alpha=1, mat, vec, out=None) → Tensor

Performs a matrix-vector product of the matrix mat and the vector vec. The vector input is added to the final result.

If mat is a (n×m)(n \times m)(n×m) tensor, vec is a 1-D tensor of size m, then input must be broadcastable with a 1-D tensor of size n and out will be 1-D tensor of size n.

alpha and beta are scaling factors on matrix-vector product between mat and vec and the added tensor input respectively.

out=β input+α (mat@vec)\text{out} = \beta\ \text{input} + \alpha\ (\text{mat} \mathbin{@} \text{vec}) out=β input+α (mat@vec)

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers

Parameters

beta (Number, optional) – multiplier for input (β\betaβ )
input (Tensor) – vector to be added
alpha (Number, optional) – multiplier for mat@vecmat @ vecmat@vec (α\alphaα )
mat (Tensor) – matrix to be multiplied
vec (Tensor) – vector to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(2)
>>> mat = torch.randn(2, 3)
>>> vec = torch.randn(3)
>>> torch.addmv(M, mat, vec)
tensor([-0.3768, -5.5565])

`torch.addr`(beta=1, input, alpha=1, vec1, vec2, out=None) → Tensor

Performs the outer-product of vectors vec1 and vec2 and adds it to the matrix input.

Optional values beta and alpha are scaling factors on the outer product between vec1 and vec2 and the added matrix input respectively.

out=β input+α (vec1⊗vec2)\text{out} = \beta\ \text{input} + \alpha\ (\text{vec1} \otimes \text{vec2}) out=β input+α (vec1⊗vec2)

If vec1 is a vector of size n and vec2 is a vector of size m, then input must be broadcastable with a matrix of size (n×m)(n \times m)(n×m) and out will be a matrix of size (n×m)(n \times m)(n×m) .

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers

Parameters

beta (Number, optional) – multiplier for input (β\betaβ )
input (Tensor) – matrix to be added
alpha (Number, optional) – multiplier for vec1⊗vec2\text{vec1} \otimes \text{vec2}vec1⊗vec2 (α\alphaα )
vec1 (Tensor) – the first vector of the outer product
vec2 (Tensor) – the second vector of the outer product
out (Tensor, optional) – the output tensor.

Example:

>>> vec1 = torch.arange(1., 4.)
>>> vec2 = torch.arange(1., 3.)
>>> M = torch.zeros(3, 2)
>>> torch.addr(M, vec1, vec2)
tensor([[ 1.,  2.],
        [ 2.,  4.],
        [ 3.,  6.]])

`torch.baddbmm`(beta=1, input, alpha=1, batch1, batch2, out=None) → Tensor

Performs a batch matrix-matrix product of matrices in batch1 and batch2. input is added to the final result.

batch1 and batch2 must be 3-D tensors each containing the same number of matrices.

If batch1 is a (b×n×m)(b \times n \times m)(b×n×m) tensor, batch2 is a (b×m×p)(b \times m \times p)(b×m×p) tensor, then input must be broadcastable with a (b×n×p)(b \times n \times p)(b×n×p) tensor and out will be a (b×n×p)(b \times n \times p)(b×n×p) tensor. Both alpha and beta mean the same as the scaling factors used in torch.addbmm().

outi=β inputi+α (batch1i@batch2i)\text{out}_i = \beta\ \text{input}_i + \alpha\ (\text{batch1}_i \mathbin{@} \text{batch2}_i) outi=β inputi+α (batch1i@batch2i)

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

Parameters

beta (Number, optional) – multiplier for input (β\betaβ )
input (Tensor) – the tensor to be added
alpha (Number, optional) – multiplier for batch1@batch2\text{batch1} \mathbin{@} \text{batch2}batch1@batch2 (α\alphaα )
batch1 (Tensor) – the first batch of matrices to be multiplied
batch2 (Tensor) – the second batch of matrices to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> M = torch.randn(10, 3, 5)
>>> batch1 = torch.randn(10, 3, 4)
>>> batch2 = torch.randn(10, 4, 5)
>>> torch.baddbmm(M, batch1, batch2).size()
torch.Size([10, 3, 5])

`torch.bmm`(input, mat2, out=None) → Tensor

Performs a batch matrix-matrix product of matrices stored in input and mat2.

input and mat2 must be 3-D tensors each containing the same number of matrices.

If input is a (b×n×m)(b \times n \times m)(b×n×m) tensor, mat2 is a (b×m×p)(b \times m \times p)(b×m×p) tensor, out will be a (b×n×p)(b \times n \times p)(b×n×p) tensor.

outi=inputi@mat2i\text{out}_i = \text{input}_i \mathbin{@} \text{mat2}_i outi=inputi@mat2i

Note

This function does not broadcast. For broadcasting matrix products, see torch.matmul().

Parameters

input (Tensor) – the first batch of matrices to be multiplied
mat2 (Tensor) – the second batch of matrices to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> input = torch.randn(10, 3, 4)
>>> mat2 = torch.randn(10, 4, 5)
>>> res = torch.bmm(input, mat2)
>>> res.size()
torch.Size([10, 3, 5])

`torch.chain_matmul`(*matrices)[source]

Returns the matrix product of the NNN 2-D tensors. This product is efficiently computed using the matrix chain order algorithm which selects the order in which incurs the lowest cost in terms of arithmetic operations ([CLRS]). Note that since this is a function to compute the product, NNN needs to be greater than or equal to 2; if equal to 2 then a trivial matrix-matrix product is returned. If NNN is 1, then this is a no-op - the original matrix is returned as is.

Parameters

matrices (Tensors...) – a sequence of 2 or more 2-D tensors whose product is to be determined.

Returns

if the ithi^{th}ith tensor was of dimensions pi×pi+1p_{i} \times p_{i + 1}pi×pi+1 , then the product would be of dimensions p1×pN+1p_{1} \times p_{N + 1}p1×pN+1 .

Return type

Tensor

Example:

>>> a = torch.randn(3, 4)
>>> b = torch.randn(4, 5)
>>> c = torch.randn(5, 6)
>>> d = torch.randn(6, 7)
>>> torch.chain_matmul(a, b, c, d)
tensor([[ -2.3375,  -3.9790,  -4.1119,  -6.6577,   9.5609, -11.5095,  -3.2614],
        [ 21.4038,   3.3378,  -8.4982,  -5.2457, -10.2561,  -2.4684,   2.7163],
        [ -0.9647,  -5.8917,  -2.3213,  -5.2284,  12.8615, -12.2816,  -2.5095]])

`torch.cholesky`(input, upper=False, out=None) → Tensor

Computes the Cholesky decomposition of a symmetric positive-definite matrix AAA or for batches of symmetric positive-definite matrices.

If upper is True, the returned matrix U is upper-triangular, and the decomposition has the form:

A=UTUA = U^TUA=UTU

If upper is False, the returned matrix L is lower-triangular, and the decomposition has the form:

A=LLTA = LL^TA=LLT

If upper is True, and AAA is a batch of symmetric positive-definite matrices, then the returned tensor will be composed of upper-triangular Cholesky factors of each of the individual matrices. Similarly, when upper is False, the returned tensor will be composed of lower-triangular Cholesky factors of each of the individual matrices.

Parameters

input (Tensor) – the input tensor AAA of size (∗,n,n)(*, n, n)(∗,n,n) where * is zero or more batch dimensions consisting of symmetric positive-definite matrices.
upper (bool, optional) – flag that indicates whether to return a upper or lower triangular matrix. Default: False
out (Tensor, optional) – the output matrix

Example:

>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) # make symmetric positive-definite
>>> l = torch.cholesky(a)
>>> a
tensor([[ 2.4112, -0.7486,  1.4551],
        [-0.7486,  1.3544,  0.1294],
        [ 1.4551,  0.1294,  1.6724]])
>>> l
tensor([[ 1.5528,  0.0000,  0.0000],
        [-0.4821,  1.0592,  0.0000],
        [ 0.9371,  0.5487,  0.7023]])
>>> torch.mm(l, l.t())
tensor([[ 2.4112, -0.7486,  1.4551],
        [-0.7486,  1.3544,  0.1294],
        [ 1.4551,  0.1294,  1.6724]])
>>> a = torch.randn(3, 2, 2)
>>> a = torch.matmul(a, a.transpose(-1, -2)) + 1e-03 # make symmetric positive-definite
>>> l = torch.cholesky(a)
>>> z = torch.matmul(l, l.transpose(-1, -2))
>>> torch.max(torch.abs(z - a)) # Max non-zero
tensor(2.3842e-07)

`torch.cholesky_inverse`(input, upper=False, out=None) → Tensor

Computes the inverse of a symmetric positive-definite matrix AAA using its Cholesky factor uuu : returns matrix inv. The inverse is computed using LAPACK routines dpotri and spotri (and the corresponding MAGMA routines).

If upper is False, uuu is lower triangular such that the returned tensor is

inv=(uuT)−1inv = (uu^{{T}})^{{-1}} inv=(uuT)−1

If upper is True or not provided, uuu is upper triangular such that the returned tensor is

inv=(uTu)−1inv = (u^T u)^{{-1}} inv=(uTu)−1

Parameters

input (Tensor) – the input 2-D tensor uuu , a upper or lower triangular Cholesky factor
upper (bool, optional) – whether to return a lower (default) or upper triangular matrix
out (Tensor, optional) – the output tensor for inv

Example:

>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) + 1e-05 * torch.eye(3) # make symmetric positive definite
>>> u = torch.cholesky(a)
>>> a
tensor([[  0.9935,  -0.6353,   1.5806],
        [ -0.6353,   0.8769,  -1.7183],
        [  1.5806,  -1.7183,  10.6618]])
>>> torch.cholesky_inverse(u)
tensor([[ 1.9314,  1.2251, -0.0889],
        [ 1.2251,  2.4439,  0.2122],
        [-0.0889,  0.2122,  0.1412]])
>>> a.inverse()
tensor([[ 1.9314,  1.2251, -0.0889],
        [ 1.2251,  2.4439,  0.2122],
        [-0.0889,  0.2122,  0.1412]])

`torch.cholesky_solve`(input, input2, upper=False, out=None) → Tensor

Solves a linear system of equations with a positive semidefinite matrix to be inverted given its Cholesky factor matrix uuu .

If upper is False, uuu is and lower triangular and c is returned such that:

c=(uuT)−1bc = (u u^T)^{{-1}} b c=(uuT)−1b

If upper is True or not provided, uuu is upper triangular and c is returned such that:

c=(uTu)−1bc = (u^T u)^{{-1}} b c=(uTu)−1b

torch.cholesky_solve(b, u) can take in 2D inputs b, u or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs c

Note

The out keyword only supports 2D matrix inputs, that is, b, u must be 2D matrices.

Parameters

input (Tensor) – input matrix bbb of size (∗,m,k)(*, m, k)(∗,m,k) , where ∗*∗ is zero or more batch dimensions
input2 (Tensor) – input matrix uuu of size (∗,m,m)(*, m, m)(∗,m,m) , where ∗*∗ is zero of more batch dimensions composed of upper or lower triangular Cholesky factor
upper (bool, optional) – whether to consider the Cholesky factor as a lower or upper triangular matrix. Default: False.
out (Tensor, optional) – the output tensor for c

Example:

>>> a = torch.randn(3, 3)
>>> a = torch.mm(a, a.t()) # make symmetric positive definite
>>> u = torch.cholesky(a)
>>> a
tensor([[ 0.7747, -1.9549,  1.3086],
        [-1.9549,  6.7546, -5.4114],
        [ 1.3086, -5.4114,  4.8733]])
>>> b = torch.randn(3, 2)
>>> b
tensor([[-0.6355,  0.9891],
        [ 0.1974,  1.4706],
        [-0.4115, -0.6225]])
>>> torch.cholesky_solve(b, u)
tensor([[ -8.1625,  19.6097],
        [ -5.8398,  14.2387],
        [ -4.3771,  10.4173]])
>>> torch.mm(a.inverse(), b)
tensor([[ -8.1626,  19.6097],
        [ -5.8398,  14.2387],
        [ -4.3771,  10.4173]])

`torch.dot`(input, tensor) → Tensor

Computes the dot product (inner product) of two tensors.

Note

This function does not broadcast.

Example:

>>> torch.dot(torch.tensor([2, 3]), torch.tensor([2, 1]))
tensor(7)

`torch.eig`(input, eigenvectors=False, out=None) -> (Tensor, Tensor)

Computes the eigenvalues and eigenvectors of a real square matrix.

Note

Since eigenvalues and eigenvectors might be complex, backward pass is supported only for torch.symeig()

Parameters

input (Tensor) – the square matrix of shape (n×n)(n \times n)(n×n) for which the eigenvalues and eigenvectors will be computed
eigenvectors (bool) – True to compute both eigenvalues and eigenvectors; otherwise, only eigenvalues will be computed
out (tuple, optional) – the output tensors

Returns

A namedtuple (eigenvalues, eigenvectors) containing

eigenvalues (Tensor): Shape (n×2)(n \times 2)(n×2) . Each row is an eigenvalue of input, where the first element is the real part and the second element is the imaginary part. The eigenvalues are not necessarily ordered.
eigenvectors (Tensor): If eigenvectors=False, it’s an empty tensor. Otherwise, this tensor of shape (n×n)(n \times n)(n×n) can be used to compute normalized (unit length) eigenvectors of corresponding eigenvalues as follows. If the corresponding eigenvalues[j] is a real number, column eigenvectors[:, j] is the eigenvector corresponding to eigenvalues[j]. If the corresponding eigenvalues[j] and eigenvalues[j + 1] form a complex conjugate pair, then the true eigenvectors can be computed as true eigenvector[j]=eigenvectors[:,j]+i×eigenvectors[:,j+1]\text{true eigenvector}[j] = eigenvectors[:, j] + i \times eigenvectors[:, j + 1]true eigenvector[j]=eigenvectors[:,j]+i×eigenvectors[:,j+1] , true eigenvector[j+1]=eigenvectors[:,j]−i×eigenvectors[:,j+1]\text{true eigenvector}[j + 1] = eigenvectors[:, j] - i \times eigenvectors[:, j + 1]true eigenvector[j+1]=eigenvectors[:,j]−i×eigenvectors[:,j+1] .

Return type

(Tensor, Tensor)

torch.gels(input, A, out=None)[source]

Computes the solution to the least squares and least norm problems for a full rank matrix AAA of size (m×n)(m \times n)(m×n) and a matrix BBB of size (m×k)(m \times k)(m×k) .

For more information regarding torch.gels(), please check torch.lstsq().

Warning

torch.gels() is deprecated in favour of torch.lstsq() and will be removed in the next release. Please use torch.lstsq() instead.

torch.geqrf(input, out=None) -> (Tensor, Tensor)

This is a low-level function for calling LAPACK directly. This function returns a namedtuple (a, tau) as defined in LAPACK documentation for geqrf .

You’ll generally want to use torch.qr() instead.

Computes a QR decomposition of input, but without constructing QQQ and RRR as explicit separate matrices.

Rather, this directly calls the underlying LAPACK function ?geqrf which produces a sequence of ‘elementary reflectors’.

See LAPACK documentation for geqrf for further details.

Parameters

input (Tensor) – the input matrix
out (tuple, optional) – the output tuple of (Tensor, Tensor)

`torch.ger`(input, vec2, out=None) → Tensor

Outer product of input and vec2. If input is a vector of size nnn and vec2 is a vector of size mmm , then out must be a matrix of size (n×m)(n \times m)(n×m) .

Note

This function does not broadcast.

Parameters

input (Tensor) – 1-D input vector
vec2 (Tensor) – 1-D input vector
out (Tensor, optional) – optional output matrix

Example:

>>> v1 = torch.arange(1., 5.)
>>> v2 = torch.arange(1., 4.)
>>> torch.ger(v1, v2)
tensor([[  1.,   2.,   3.],
        [  2.,   4.,   6.],
        [  3.,   6.,   9.],
        [  4.,   8.,  12.]])

`torch.inverse`(input, out=None) → Tensor

Takes the inverse of the square matrix input. input can be batches of 2D square tensors, in which case this function would return a tensor composed of individual inverses.

Note

Irrespective of the original strides, the returned tensors will be transposed, i.e. with strides like input.contiguous().transpose(-2, -1).stride()

Parameters

input (Tensor) – the input tensor of size (∗,n,n)(*, n, n)(∗,n,n) where * is zero or more batch dimensions
out (Tensor, optional) – the output tensor.

Example:

>>> x = torch.rand(4, 4)
>>> y = torch.inverse(x)
>>> z = torch.mm(x, y)
>>> z
tensor([[ 1.0000, -0.0000, -0.0000,  0.0000],
        [ 0.0000,  1.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  1.0000,  0.0000],
        [ 0.0000, -0.0000, -0.0000,  1.0000]])
>>> torch.max(torch.abs(z - torch.eye(4))) # Max non-zero
tensor(1.1921e-07)
>>> # Batched inverse example
>>> x = torch.randn(2, 3, 4, 4)
>>> y = torch.inverse(x)
>>> z = torch.matmul(x, y)
>>> torch.max(torch.abs(z - torch.eye(4).expand_as(x))) # Max non-zero
tensor(1.9073e-06)

`torch.det`(input) → Tensor

Calculates determinant of a square matrix or batches of square matrices.

Note

Backward through det() internally uses SVD results when input is not invertible. In this case, double backward through det() will be unstable in when input doesn’t have distinct singular values. See svd() for details.

Parameters

input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.

Example:

>>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(3.7641)

>>> A = torch.randn(3, 2, 2)
>>> A
tensor([[[ 0.9254, -0.6213],
         [-0.5787,  1.6843]],

        [[ 0.3242, -0.9665],
         [ 0.4539, -0.0887]],

        [[ 1.1336, -0.4025],
         [-0.7089,  0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])

`torch.logdet`(input) → Tensor

Calculates log determinant of a square matrix or batches of square matrices.

Note

Result is -inf if input has zero log determinant, and is nan if input has negative determinant.

Note

Backward through logdet() internally uses SVD results when input is not invertible. In this case, double backward through logdet() will be unstable in when input doesn’t have distinct singular values. See svd() for details.

Parameters

input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.

Example:

>>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(0.2611)
>>> torch.logdet(A)
tensor(-1.3430)
>>> A
tensor([[[ 0.9254, -0.6213],
         [-0.5787,  1.6843]],

        [[ 0.3242, -0.9665],
         [ 0.4539, -0.0887]],

        [[ 1.1336, -0.4025],
         [-0.7089,  0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])
>>> A.det().log()
tensor([ 0.1815, -0.8917, -0.3031])

`torch.slogdet`(input) -> (Tensor, Tensor)

Calculates the sign and log absolute value of the determinant(s) of a square matrix or batches of square matrices.

Note

If input has zero determinant, this returns (0, -inf).

Note

Backward through slogdet() internally uses SVD results when input is not invertible. In this case, double backward through slogdet() will be unstable in when input doesn’t have distinct singular values. See svd() for details.

Parameters

input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.

Returns

A namedtuple (sign, logabsdet) containing the sign of the determinant, and the log value of the absolute determinant.

Example:

>>> A = torch.randn(3, 3)
>>> A
tensor([[ 0.0032, -0.2239, -1.1219],
        [-0.6690,  0.1161,  0.4053],
        [-1.6218, -0.9273, -0.0082]])
>>> torch.det(A)
tensor(-0.7576)
>>> torch.logdet(A)
tensor(nan)
>>> torch.slogdet(A)
torch.return_types.slogdet(sign=tensor(-1.), logabsdet=tensor(-0.2776))

`torch.lstsq`(input, A, out=None) → Tensor

Computes the solution to the least squares and least norm problems for a full rank matrix AAA of size (m×n)(m \times n)(m×n) and a matrix BBB of size (m×k)(m \times k)(m×k) .

If m≥nm \geq nm≥n , lstsq() solves the least-squares problem:

min⁡X∥AX−B∥2.\begin{array}{ll} \min_X & \|AX-B\|_2. \end{array}minX∥AX−B∥2.

If m<nm < nm<n , lstsq() solves the least-norm problem:

min⁡X∥X∥2subject toAX=B.\begin{array}{ll} \min_X & \|X\|_2 & \text{subject to} & AX = B. \end{array}minX∥X∥2subject toAX=B.

Returned tensor XXX has shape (max⁡(m,n)×k)(\max(m, n) \times k)(max(m,n)×k) . The first nnn rows of XXX contains the solution. If m≥nm \geq nm≥n , the residual sum of squares for the solution in each column is given by the sum of squares of elements in the remaining m−nm - nm−n rows of that column.

Note

The case when m<nm < nm<n is not supported on the GPU.

Parameters

input (Tensor) – the matrix BBB
A (Tensor) – the mmm by nnn matrix AAA
out (tuple, optional) – the optional destination tensor

Returns

A namedtuple (solution, QR) containing:

solution (Tensor): the least squares solution
QR (Tensor): the details of the QR factorization

Return type

(Tensor, Tensor)

Note

The returned matrices will always be transposed, irrespective of the strides of the input matrices. That is, they will have stride (1, m) instead of (m, 1).

Example:

>>> A = torch.tensor([[1., 1, 1],
                      [2, 3, 4],
                      [3, 5, 2],
                      [4, 2, 5],
                      [5, 4, 3]])
>>> B = torch.tensor([[-10., -3],
                      [ 12, 14],
                      [ 14, 12],
                      [ 16, 16],
                      [ 18, 16]])
>>> X, _ = torch.lstsq(B, A)
>>> X
tensor([[  2.0000,   1.0000],
        [  1.0000,   1.0000],
        [  1.0000,   2.0000],
        [ 10.9635,   4.8501],
        [  8.9332,   5.2418]])

`torch.lu`(A, pivot=True, get_infos=False, out=None)[source]

Computes the LU factorization of a square matrix or batches of square matrices A. Returns a tuple containing the LU factorization and pivots of A. Pivoting is done if pivot is set to True.

Note

The pivots returned by the function are 1-indexed. If pivot is False, then the returned pivots is a tensor filled with zeros of the appropriate size.

Note

LU factorization with pivot = False is not available for CPU, and attempting to do so will throw an error. However, LU factorization with pivot = False is available for CUDA.

Note

This function does not check if the factorization was successful or not if get_infos is True since the status of the factorization is present in the third element of the return tuple.

Parameters

A (Tensor) – the tensor to factor of size (∗,m,m)(*, m, m)(∗,m,m)
pivot (bool, optional) – controls whether pivoting is done. Default: True
get_infos (bool, optional) – if set to True, returns an info IntTensor. Default: False
out (tuple, optional) – optional output tuple. If get_infos is True, then the elements in the tuple are Tensor, IntTensor, and IntTensor. If get_infos is False, then the elements in the tuple are Tensor, IntTensor. Default: None

Returns

A tuple of tensors containing

factorization (Tensor): the factorization of size (∗,m,m)(*, m, m)(∗,m,m)
pivots (IntTensor): the pivots of size (∗,m)(*, m)(∗,m)
infos (IntTensor, optional): if get_infos is True, this is a tensor of size (∗)(*)(∗) where non-zero values indicate whether factorization for the matrix or each minibatch has succeeded or failed

Return type

(Tensor, IntTensor, IntTensor (optional))

Example:

>>> A = torch.randn(2, 3, 3)
>>> A_LU, pivots = torch.lu(A)
>>> A_LU
tensor([[[ 1.3506,  2.5558, -0.0816],
         [ 0.1684,  1.1551,  0.1940],
         [ 0.1193,  0.6189, -0.5497]],

        [[ 0.4526,  1.2526, -0.3285],
         [-0.7988,  0.7175, -0.9701],
         [ 0.2634, -0.9255, -0.3459]]])
>>> pivots
tensor([[ 3,  3,  3],
        [ 3,  3,  3]], dtype=torch.int32)
>>> A_LU, pivots, info = torch.lu(A, get_infos=True)
>>> if info.nonzero().size(0) == 0:
...   print('LU factorization succeeded for all samples!')
LU factorization succeeded for all samples!

`torch.lu_solve`(input, LU_data, LU_pivots, out=None) → Tensor

Returns the LU solve of the linear system Ax=bAx = bAx=b using the partially pivoted LU factorization of A from torch.lu().

Parameters

b (Tensor) – the RHS tensor of size (∗,m,k)(*, m, k)(∗,m,k) , where ∗*∗ is zero or more batch dimensions.
LU_data (Tensor) – the pivoted LU factorization of A from torch.lu() of size (∗,m,m)(*, m, m)(∗,m,m) , where ∗*∗ is zero or more batch dimensions.
LU_pivots (IntTensor) – the pivots of the LU factorization from torch.lu() of size (∗,m)(*, m)(∗,m) , where ∗*∗ is zero or more batch dimensions. The batch dimensions of LU_pivots must be equal to the batch dimensions of LU_data.
out (Tensor, optional) – the output tensor.

Example:

>>> A = torch.randn(2, 3, 3)
>>> b = torch.randn(2, 3, 1)
>>> A_LU = torch.lu(A)
>>> x = torch.lu_solve(b, *A_LU)
>>> torch.norm(torch.bmm(A, x) - b)
tensor(1.00000e-07 *
       2.8312)

torch.lu_unpack(LU_data, LU_pivots, unpack_data=True, unpack_pivots=True)[source]

Unpacks the data and pivots from a LU factorization of a tensor.

Returns a tuple of tensors as (the pivots, the L tensor, the U tensor).

Parameters

LU_data (Tensor) – the packed LU factorization data
LU_pivots (Tensor) – the packed LU factorization pivots
unpack_data (bool) – flag indicating if the data should be unpacked
unpack_pivots (bool) – flag indicating if the pivots should be unpacked

Example:

>>> A = torch.randn(2, 3, 3)
>>> A_LU, pivots = A.lu()
>>> P, A_L, A_U = torch.lu_unpack(A_LU, pivots)
>>>
>>> # can recover A from factorization
>>> A_ = torch.bmm(P, torch.bmm(A_L, A_U))

`torch.matmul`(input, other, out=None) → Tensor

Matrix product of two tensors.

The behavior depends on the dimensionality of the tensors as follows:

If both tensors are 1-dimensional, the dot product (scalar) is returned.
If both arguments are 2-dimensional, the matrix-matrix product is returned.
If the first argument is 1-dimensional and the second argument is 2-dimensional, a 1 is prepended to its dimension for the purpose of the matrix multiply. After the matrix multiply, the prepended dimension is removed.
If the first argument is 2-dimensional and the second argument is 1-dimensional, the matrix-vector product is returned.
If both arguments are at least 1-dimensional and at least one argument is N-dimensional (where N > 2), then a batched matrix multiply is returned. If the first argument is 1-dimensional, a 1 is prepended to its dimension for the purpose of the batched matrix multiply and removed after. If the second argument is 1-dimensional, a 1 is appended to its dimension for the purpose of the batched matrix multiple and removed after. The non-matrix (i.e. batch) dimensions are broadcasted (and thus must be broadcastable). For example, if input is a (j×1×n×m)(j \times 1 \times n \times m)(j×1×n×m) tensor and other is a (k×m×p)(k \times m \times p)(k×m×p) tensor, out will be an (j×k×n×p)(j \times k \times n \times p)(j×k×n×p) tensor.

Note

The 1-dimensional dot product version of this function does not support an out parameter.

Parameters

input (Tensor) – the first tensor to be multiplied
other (Tensor) – the second tensor to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> # vector x vector
>>> tensor1 = torch.randn(3)
>>> tensor2 = torch.randn(3)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([])
>>> # matrix x vector
>>> tensor1 = torch.randn(3, 4)
>>> tensor2 = torch.randn(4)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([3])
>>> # batched matrix x broadcasted vector
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(4)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3])
>>> # batched matrix x batched matrix
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(10, 4, 5)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3, 5])
>>> # batched matrix x broadcasted matrix
>>> tensor1 = torch.randn(10, 3, 4)
>>> tensor2 = torch.randn(4, 5)
>>> torch.matmul(tensor1, tensor2).size()
torch.Size([10, 3, 5])

`torch.matrix_power`(input, n) → Tensor

Returns the matrix raised to the power n for square matrices. For batch of matrices, each individual matrix is raised to the power n.

If n is negative, then the inverse of the matrix (if invertible) is raised to the power n. For a batch of matrices, the batched inverse (if invertible) is raised to the power n. If n is 0, then an identity matrix is returned.

Parameters

input (Tensor) – the input tensor.
n (int) – the power to raise the matrix to

Example:

>>> a = torch.randn(2, 2, 2)
>>> a
tensor([[[-1.9975, -1.9610],
         [ 0.9592, -2.3364]],

        [[-1.2534, -1.3429],
         [ 0.4153, -1.4664]]])
>>> torch.matrix_power(a, 3)
tensor([[[  3.9392, -23.9916],
         [ 11.7357,  -0.2070]],

        [[  0.2468,  -6.7168],
         [  2.0774,  -0.8187]]])

`torch.matrix_rank`(input, tol=None, bool symmetric=False) → Tensor

Returns the numerical rank of a 2-D tensor. The method to compute the matrix rank is done using SVD by default. If symmetric is True, then input is assumed to be symmetric, and the computation of the rank is done by obtaining the eigenvalues.

tol is the threshold below which the singular values (or the eigenvalues when symmetric is True) are considered to be 0. If tol is not specified, tol is set to S.max() * max(S.size()) * eps where S is the singular values (or the eigenvalues when symmetric is True), and eps is the epsilon value for the datatype of input.

Parameters

input (Tensor) – the input 2-D tensor
tol (float, optional) – the tolerance value. Default: None
symmetric (bool, optional) – indicates whether input is symmetric. Default: False

Example:

>>> a = torch.eye(10)
>>> torch.matrix_rank(a)
tensor(10)
>>> b = torch.eye(10)
>>> b[0, 0] = 0
>>> torch.matrix_rank(b)
tensor(9)

`torch.mm`(input, mat2, out=None) → Tensor

Performs a matrix multiplication of the matrices input and mat2.

If input is a (n×m)(n \times m)(n×m) tensor, mat2 is a (m×p)(m \times p)(m×p) tensor, out will be a (n×p)(n \times p)(n×p) tensor.

Note

This function does not broadcast. For broadcasting matrix products, see torch.matmul().

Parameters

input (Tensor) – the first matrix to be multiplied
mat2 (Tensor) – the second matrix to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> mat1 = torch.randn(2, 3)
>>> mat2 = torch.randn(3, 3)
>>> torch.mm(mat1, mat2)
tensor([[ 0.4851,  0.5037, -0.3633],
        [-0.0760, -3.6705,  2.4784]])

`torch.mv`(input, vec, out=None) → Tensor

Performs a matrix-vector product of the matrix input and the vector vec.

If input is a (n×m)(n \times m)(n×m) tensor, vec is a 1-D tensor of size mmm , out will be 1-D of size nnn .

Note

This function does not broadcast.

Parameters

input (Tensor) – matrix to be multiplied
vec (Tensor) – vector to be multiplied
out (Tensor, optional) – the output tensor.

Example:

>>> mat = torch.randn(2, 3)
>>> vec = torch.randn(3)
>>> torch.mv(mat, vec)
tensor([ 1.0404, -0.6361])

`torch.orgqr`(input, input2) → Tensor

Computes the orthogonal matrix Q of a QR factorization, from the (input, input2) tuple returned by torch.geqrf().

This directly calls the underlying LAPACK function ?orgqr. See LAPACK documentation for orgqr for further details.

Parameters

input (Tensor) – the a from torch.geqrf().
input2 (Tensor) – the tau from torch.geqrf().

`torch.ormqr`(input, input2, input3, left=True, transpose=False) → Tensor

Multiplies mat (given by input3) by the orthogonal Q matrix of the QR factorization formed by torch.geqrf() that is represented by (a, tau) (given by (input, input2)).

This directly calls the underlying LAPACK function ?ormqr. See LAPACK documentation for ormqr for further details.

Parameters

input (Tensor) – the a from torch.geqrf().
input2 (Tensor) – the tau from torch.geqrf().
input3 (Tensor) – the matrix to be multiplied.

`torch.pinverse`(input, rcond=1e-15) → Tensor

Calculates the pseudo-inverse (also known as the Moore-Penrose inverse) of a 2D tensor. Please look at Moore-Penrose inverse for more details

Note

This method is implemented using the Singular Value Decomposition.

Note

The pseudo-inverse is not necessarily a continuous function in the elements of the matrix [1]. Therefore, derivatives are not always existent, and exist for a constant rank only [2]. However, this method is backprop-able due to the implementation by using SVD results, and could be unstable. Double-backward will also be unstable due to the usage of SVD internally. See svd() for more details.

Parameters

input (Tensor) – The input 2D tensor of dimensions m×nm \times nm×n
rcond (float) – A floating point value to determine the cutoff for small singular values. Default: 1e-15

Returns

The pseudo-inverse of input of dimensions n×mn \times mn×m

Example:

>>> input = torch.randn(3, 5)
>>> input
tensor([[ 0.5495,  0.0979, -1.4092, -0.1128,  0.4132],
        [-1.1143, -0.3662,  0.3042,  1.6374, -0.9294],
        [-0.3269, -0.5745, -0.0382, -0.5922, -0.6759]])
>>> torch.pinverse(input)
tensor([[ 0.0600, -0.1933, -0.2090],
        [-0.0903, -0.0817, -0.4752],
        [-0.7124, -0.1631, -0.2272],
        [ 0.1356,  0.3933, -0.5023],
        [-0.0308, -0.1725, -0.5216]])

`torch.qr`(input, some=True, out=None) -> (Tensor, Tensor)

Computes the QR decomposition of a matrix or a batch of matrices input, and returns a namedtuple (Q, R) of tensors such that input=QR\text{input} = Q Rinput=QR with QQQ being an orthogonal matrix or batch of orthogonal matrices and RRR being an upper triangular matrix or batch of upper triangular matrices.

If some is True, then this function returns the thin (reduced) QR factorization. Otherwise, if some is False, this function returns the complete QR factorization.

Note

precision may be lost if the magnitudes of the elements of input are large

Note

While it should always give you a valid decomposition, it may not give you the same one across platforms - it will depend on your LAPACK implementation.

Parameters

input (Tensor) – the input tensor of size (∗,m,n)(*, m, n)(∗,m,n) where * is zero or more batch dimensions consisting of matrices of dimension m×nm \times nm×n .
some (bool, optional) – Set to True for reduced QR decomposition and False for complete QR decomposition.
out (tuple, optional) – tuple of Q and R tensors satisfying input = torch.matmul(Q, R). The dimensions of Q and R are (∗,m,k)(*, m, k)(∗,m,k) and (∗,k,n)(*, k, n)(∗,k,n) respectively, where k=min⁡(m,n)k = \min(m, n)k=min(m,n) if some: is True and k=mk = mk=m otherwise.

Example:

>>> a = torch.tensor([[12., -51, 4], [6, 167, -68], [-4, 24, -41]])
>>> q, r = torch.qr(a)
>>> q
tensor([[-0.8571,  0.3943,  0.3314],
        [-0.4286, -0.9029, -0.0343],
        [ 0.2857, -0.1714,  0.9429]])
>>> r
tensor([[ -14.0000,  -21.0000,   14.0000],
        [   0.0000, -175.0000,   70.0000],
        [   0.0000,    0.0000,  -35.0000]])
>>> torch.mm(q, r).round()
tensor([[  12.,  -51.,    4.],
        [   6.,  167.,  -68.],
        [  -4.,   24.,  -41.]])
>>> torch.mm(q.t(), q).round()
tensor([[ 1.,  0.,  0.],
        [ 0.,  1., -0.],
        [ 0., -0.,  1.]])
>>> a = torch.randn(3, 4, 5)
>>> q, r = torch.qr(a, some=False)
>>> torch.allclose(torch.matmul(q, r), a)
True
>>> torch.allclose(torch.matmul(q.transpose(-2, -1), q), torch.eye(5))
True

`torch.solve`(input, A, out=None) -> (Tensor, Tensor)

This function returns the solution to the system of linear equations represented by AX=BAX = BAX=B and the LU factorization of A, in order as a namedtuple solution, LU.

LU contains L and U factors for LU factorization of A.

torch.solve(B, A) can take in 2D inputs B, A or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs solution, LU.

Note

Irrespective of the original strides, the returned matrices solution and LU will be transposed, i.e. with strides like B.contiguous().transpose(-1, -2).stride() and A.contiguous().transpose(-1, -2).stride() respectively.

Parameters

input (Tensor) – input matrix BBB of size (∗,m,k)(*, m, k)(∗,m,k) , where ∗*∗ is zero or more batch dimensions.
A (Tensor) – input square matrix of size (∗,m,m)(*, m, m)(∗,m,m) , where ∗*∗ is zero or more batch dimensions.
out ((Tensor, Tensor), optional) – optional output tuple.

Example:

>>> A = torch.tensor([[6.80, -2.11,  5.66,  5.97,  8.23],
                      [-6.05, -3.30,  5.36, -4.44,  1.08],
                      [-0.45,  2.58, -2.70,  0.27,  9.04],
                      [8.32,  2.71,  4.35,  -7.17,  2.14],
                      [-9.67, -5.14, -7.26,  6.08, -6.87]]).t()
>>> B = torch.tensor([[4.02,  6.19, -8.22, -7.57, -3.03],
                      [-1.56,  4.00, -8.67,  1.75,  2.86],
                      [9.81, -4.09, -4.57, -8.61,  8.99]]).t()
>>> X, LU = torch.solve(B, A)
>>> torch.dist(B, torch.mm(A, X))
tensor(1.00000e-06 *
       7.0977)

>>> # Batched solver example
>>> A = torch.randn(2, 3, 1, 4, 4)
>>> B = torch.randn(2, 3, 1, 4, 6)
>>> X, LU = torch.solve(B, A)
>>> torch.dist(B, A.matmul(X))
tensor(1.00000e-06 *
   3.6386)

`torch.svd`(input, some=True, compute_uv=True, out=None) -> (Tensor, Tensor, Tensor)

This function returns a namedtuple (U, S, V) which is the singular value decomposition of a input real matrix or batches of real matrices input such that input=U×diag(S)×VTinput = U \times diag(S) \times V^Tinput=U×diag(S)×VT .

If some is True (default), the method returns the reduced singular value decomposition i.e., if the last two dimensions of input are m and n, then the returned U and V matrices will contain only min(n,m)min(n, m)min(n,m) orthonormal columns.

If compute_uv is False, the returned U and V matrices will be zero matrices of shape (m×m)(m \times m)(m×m) and (n×n)(n \times n)(n×n) respectively. some will be ignored here.

Note

The implementation of SVD on CPU uses the LAPACK routine ?gesdd (a divide-and-conquer algorithm) instead of ?gesvd for speed. Analogously, the SVD on GPU uses the MAGMA routine gesdd as well.

Note

Irrespective of the original strides, the returned matrix U will be transposed, i.e. with strides U.contiguous().transpose(-2, -1).stride()

Note

Extra care needs to be taken when backward through U and V outputs. Such operation is really only stable when input is full rank with all distinct singular values. Otherwise, NaN can appear as the gradients are not properly defined. Also, notice that double backward will usually do an additional backward through U and V even if the original backward is only on S.

Note

When some = False, the gradients on U[..., :, min(m, n):] and V[..., :, min(m, n):] will be ignored in backward as those vectors can be arbitrary bases of the subspaces.

Note

When compute_uv = False, backward cannot be performed since U and V from the forward pass is required for the backward operation.

Parameters

input (Tensor) – the input tensor of size (∗,m,n)(*, m, n)(∗,m,n) where * is zero or more batch dimensions consisting of m×nm \times nm×n matrices.
some (bool, optional) – controls the shape of returned U and V
compute_uv (bool, optional) – option whether to compute U and V or not
out (tuple, optional) – the output tuple of tensors

Example:

>>> a = torch.randn(5, 3)
>>> a
tensor([[ 0.2364, -0.7752,  0.6372],
        [ 1.7201,  0.7394, -0.0504],
        [-0.3371, -1.0584,  0.5296],
        [ 0.3550, -0.4022,  1.5569],
        [ 0.2445, -0.0158,  1.1414]])
>>> u, s, v = torch.svd(a)
>>> u
tensor([[ 0.4027,  0.0287,  0.5434],
        [-0.1946,  0.8833,  0.3679],
        [ 0.4296, -0.2890,  0.5261],
        [ 0.6604,  0.2717, -0.2618],
        [ 0.4234,  0.2481, -0.4733]])
>>> s
tensor([2.3289, 2.0315, 0.7806])
>>> v
tensor([[-0.0199,  0.8766,  0.4809],
        [-0.5080,  0.4054, -0.7600],
        [ 0.8611,  0.2594, -0.4373]])
>>> torch.dist(a, torch.mm(torch.mm(u, torch.diag(s)), v.t()))
tensor(8.6531e-07)
>>> a_big = torch.randn(7, 5, 3)
>>> u, s, v = torch.svd(a_big)
>>> torch.dist(a_big, torch.matmul(torch.matmul(u, torch.diag_embed(s)), v.transpose(-2, -1)))
tensor(2.6503e-06)

`torch.symeig`(input, eigenvectors=False, upper=True, out=None) -> (Tensor, Tensor)

This function returns eigenvalues and eigenvectors of a real symmetric matrix input or a batch of real symmetric matrices, represented by a namedtuple (eigenvalues, eigenvectors).

This function calculates all eigenvalues (and vectors) of input such that input=Vdiag(e)VT\text{input} = V \text{diag}(e) V^Tinput=Vdiag(e)VT .

The boolean argument eigenvectors defines computation of both eigenvectors and eigenvalues or eigenvalues only.

If it is False, only eigenvalues are computed. If it is True, both eigenvalues and eigenvectors are computed.

Since the input matrix input is supposed to be symmetric, only the upper triangular portion is used by default.

If upper is False, then lower triangular portion is used.

Note

Irrespective of the original strides, the returned matrix V will be transposed, i.e. with strides V.contiguous().transpose(-1, -2).stride().

Note

Extra care needs to be taken when backward through outputs. Such operation is really only stable when all eigenvalues are distinct. Otherwise, NaN can appear as the gradients are not properly defined.

Parameters

input (Tensor) – the input tensor of size (∗,n,n)(*, n, n)(∗,n,n) where * is zero or more batch dimensions consisting of symmetric matrices.
eigenvectors (boolean, optional) – controls whether eigenvectors have to be computed
upper (boolean, optional) – controls whether to consider upper-triangular or lower-triangular region
out (tuple, optional) – the output tuple of (Tensor, Tensor)

Returns

A namedtuple (eigenvalues, eigenvectors) containing

eigenvalues (Tensor): Shape (∗,m)(*, m)(∗,m) . The eigenvalues in ascending order.
eigenvectors (Tensor): Shape (∗,m,m)(*, m, m)(∗,m,m) . If eigenvectors=False, it’s a tensor filled with zeros. Otherwise, this tensor contains the orthonormal eigenvectors of the input.

Return type

(Tensor, Tensor)

Examples:

>>> a = torch.randn(5, 5)
>>> a = a + a.t()  # To make a symmetric
>>> a
tensor([[-5.7827,  4.4559, -0.2344, -1.7123, -1.8330],
        [ 4.4559,  1.4250, -2.8636, -3.2100, -0.1798],
        [-0.2344, -2.8636,  1.7112, -5.5785,  7.1988],
        [-1.7123, -3.2100, -5.5785, -2.6227,  3.1036],
        [-1.8330, -0.1798,  7.1988,  3.1036, -5.1453]])
>>> e, v = torch.symeig(a, eigenvectors=True)
>>> e
tensor([-13.7012,  -7.7497,  -2.3163,   5.2477,   8.1050])
>>> v
tensor([[ 0.1643,  0.9034, -0.0291,  0.3508,  0.1817],
        [-0.2417, -0.3071, -0.5081,  0.6534,  0.4026],
        [-0.5176,  0.1223, -0.0220,  0.3295, -0.7798],
        [-0.4850,  0.2695, -0.5773, -0.5840,  0.1337],
        [ 0.6415, -0.0447, -0.6381, -0.0193, -0.4230]])
>>> a_big = torch.randn(5, 2, 2)
>>> a_big = a_big + a_big.transpose(-2, -1)  # To make a_big symmetric
>>> e, v = a_big.symeig(eigenvectors=True)
>>> torch.allclose(torch.matmul(v, torch.matmul(e.diag_embed(), v.transpose(-2, -1))), a_big)
True

`torch.trapz`()

torch.trapz(y, x, *, dim=-1) → Tensor

Estimate ∫y dx\int y\,dx∫ydx along dim, using the trapezoid rule.

Parameters

y (Tensor) – The values of the function to integrate
x (Tensor) – The points at which the function y is sampled. If x is not in ascending order, intervals on which it is decreasing contribute negatively to the estimated integral (i.e., the convention ∫abf=−∫baf\int_a^b f = -\int_b^a f∫abf=−∫baf is followed).
dim (int) – The dimension along which to integrate. By default, use the last dimension.

Returns

A Tensor with the same shape as the input, except with dim removed. Each element of the returned tensor represents the estimated integral ∫y dx\int y\,dx∫ydx along dim.

Example:

>>> y = torch.randn((2, 3))
>>> y
tensor([[-2.1156,  0.6857, -0.2700],
        [-1.2145,  0.5540,  2.0431]])
>>> x = torch.tensor([[1, 3, 4], [1, 2, 3]])
>>> torch.trapz(y, x)
tensor([-1.2220,  0.9683])

`torch.trapz`(y, , dx=1, dim=-1*) → Tensor

As above, but the sample points are spaced uniformly at a distance of dx.

Parameters

y (Tensor) – The values of the function to integrate
dx (float) – The distance between points at which y is sampled.
dim (int) – The dimension along which to integrate. By default, use the last dimension.

Returns

A Tensor with the same shape as the input, except with dim removed. Each element of the returned tensor represents the estimated integral ∫y dx\int y\,dx∫ydx along dim.

torch.triangular_solve(input, A, upper=True, transpose=False, unitriangular=False) -> (Tensor, Tensor)

Solves a system of equations with a triangular coefficient matrix AAA and multiple right-hand sides bbb .

In particular, solves AX=bAX = bAX=b and assumes AAA is upper-triangular with the default keyword arguments.

torch.triangular_solve(b, A) can take in 2D inputs b, A or inputs that are batches of 2D matrices. If the inputs are batches, then returns batched outputs X

Note

The out keyword only supports 2D matrix inputs, that is, b, A must be 2D matrices.

Parameters

input (Tensor) – multiple right-hand sides of size (∗,m,k)(*, m, k)(∗,m,k) where ∗*∗ is zero of more batch dimensions (bbb )
A (Tensor) – the input triangular coefficient matrix of size (∗,m,m)(*, m, m)(∗,m,m) where ∗*∗ is zero or more batch dimensions
upper (bool, optional) – whether to solve the upper-triangular system of equations (default) or the lower-triangular system of equations. Default: True.
transpose (bool, optional) – whether AAA should be transposed before being sent into the solver. Default: False.
unitriangular (bool, optional) – whether AAA is unit triangular. If True, the diagonal elements of AAA are assumed to be 1 and not referenced from AAA . Default: False.

Returns

A namedtuple (solution, cloned_coefficient) where cloned_coefficient is a clone of AAA and solution is the solution XXX to AX=bAX = bAX=b (or whatever variant of the system of equations, depending on the keyword arguments.)

Examples:

>>> A = torch.randn(2, 2).triu()
>>> A
tensor([[ 1.1527, -1.0753],
        [ 0.0000,  0.7986]])
>>> b = torch.randn(2, 3)
>>> b
tensor([[-0.0210,  2.3513, -1.5492],
        [ 1.5429,  0.7403, -1.0243]])
>>> torch.triangular_solve(b, A)
torch.return_types.triangular_solve(
solution=tensor([[ 1.7841,  2.9046, -2.5405],
        [ 1.9320,  0.9270, -1.2826]]),
cloned_coefficient=tensor([[ 1.1527, -1.0753],
        [ 0.0000,  0.7986]]))

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

原始发表：2019年09月23日，如有侵权请联系 cloudcommunity@tencent.com 删除

编程算法

批量计算

本文分享自作者个人站点/博客前往查看

如有侵权，请联系 cloudcommunity@tencent.com 删除。

本文参与腾讯云自媒体同步曝光计划，欢迎热爱写作的你一起参与！

编程算法

批量计算