# gpytorch.distributions¶

GPyTorch distribution objects are essentially the same as torch distribution objects. For the most part, GpyTorch relies on torch’s distribution library. However, we offer two custom distributions.

We implement a custom MultivariateNormal that accepts LinearOperator objects for covariance matrices. This allows us to use custom linear algebra operations, which makes this more efficient than PyTorch’s MVN implementation.

In addition, we implement a MultitaskMultivariateNormal which can be used with multi-output Gaussian process models.

Note

If Pyro is available, all GPyTorch distribution objects inherit Pyro’s distribution methods as well.

## Distribution¶

class gpytorch.distributions.Distribution(batch_shape=torch.Size([]), event_shape=torch.Size([]), validate_args=None)[source]

## MultivariateNormal¶

class gpytorch.distributions.MultivariateNormal(mean, covariance_matrix, validate_args=False)[source]

Constructs a multivariate normal random variable, based on mean and covariance. Can be multivariate, or a batch of multivariate normals

Passing a vector mean corresponds to a multivariate normal. Passing a matrix mean corresponds to a batch of multivariate normals.

Parameters
• mean (torch.Tensor) – … x N mean of mvn distribution.

• covariance_matrix (torch.Tensor or LinearOperator) – … x N X N covariance matrix of mvn distribution.

• validate_args (bool) – If True, validate mean anad covariance_matrix arguments. (Default: False.)

Variables
• base_sample_shape (torch.Size) – The shape of a base sample (without batching) that is used to generate a single sample.

• covariance_matrix (torch.Tensor) – The covariance matrix, represented as a dense torch.Tensor

• lazy_covariance_matrix (LinearOperator) – The covariance matrix, represented as a LinearOperator.

• mean (torch.Tensor) – The mean.

• stddev (torch.Tensor) – The standard deviation.

• variance (torch.Tensor) – The variance.

__getitem__(idx)[source]

Constructs a new MultivariateNormal that represents a random variable modified by an indexing operation.

The mean and covariance matrix arguments are indexed accordingly.

Parameters

idx – Index to apply.

Return type

MultivariateNormal

Adds a small constant diagonal to the MVN covariance matrix for numerical stability.

Parameters

noise (float) – The size of the constant diagonal.

Return type

MultivariateNormal

confidence_region()[source]

Returns 2 standard deviations above and below the mean.

Return type
Returns

Pair of tensors of size … x N, where N is the dimensionality of the random variable. The first (second) Tensor is the lower (upper) end of the confidence region.

expand(batch_size)[source]
Parameters

batch_size (torch.Size) –

Return type

MultivariateNormal

get_base_samples(sample_shape=torch.Size([]))[source]

Returns i.i.d. standard Normal samples to be used with MultivariateNormal.rsample(base_samples=base_samples).

Parameters

sample_shape (torch.Size) – The number of samples to generate. (Default: torch.Size([]).)

Return type

torch.Tensor

Returns

A *sample_shape x *batch_shape x N tensor of i.i.d. standard Normal samples.

log_prob(value)[source]
Parameters

value (torch.Tensor) –

Return type

torch.Tensor

rsample(sample_shape=torch.Size([]), base_samples=None)[source]

Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.

For the MultivariateNormal distribution, this is accomplished through:

$\boldsymbol \mu + \mathbf L \boldsymbol \epsilon$

where $$\boldsymbol \mu \in \mathcal R^N$$ is the MVN mean, $$\mathbf L \in \mathcal R^{N \times N}$$ is a “root” of the covariance matrix $$\mathbf K$$ (i.e. $$\mathbf L \mathbf L^\top = \mathbf K$$), and $$\boldsymbol \epsilon \in \mathcal R^N$$ is a vector of (approximately) i.i.d. standard Normal random variables.

Parameters
• sample_shape (torch.Size) – The number of samples to generate. (Default: torch.Size([]).)

• base_samples (torch.Tensor, optional) – The *sample_shape x *batch_shape x N tensor of i.i.d. (or approximately i.i.d.) standard Normal samples to reparameterize. (Defualt: None.)

Return type

torch.Tensor

Returns

A *sample_shape x *batch_shape x N tensor of i.i.d. reparameterized samples.

sample(sample_shape=torch.Size([]), base_samples=None)[source]

Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.

Note that these samples are not reparameterized and therefore cannot be backpropagated through.

Parameters
• sample_shape (torch.Size) – The number of samples to generate. (Default: torch.Size([]).)

• base_samples (torch.Tensor, optional) – The *sample_shape x *batch_shape x N tensor of i.i.d. (or approximately i.i.d.) standard Normal samples to reparameterize. (Defualt: None.)

Return type

torch.Tensor

Returns

A *sample_shape x *batch_shape x N tensor of i.i.d. samples.

to_data_independent_dist()[source]

Convert a … x N MVN distribution into a batch of independent Normal distributions. Essentially, this throws away all covariance information and treats all dimensions as batch dimensions.

Return type

torch.distributions.normal.Normal

Returns

A (data-independent) Normal distribution with batch shape *batch_shape x N.

Constructs a multi-output multivariate Normal random variable, based on mean and covariance Can be multi-output multivariate, or a batch of multi-output multivariate Normal

Passing a matrix mean corresponds to a multi-output multivariate Normal Passing a matrix mean corresponds to a batch of multivariate Normals

Parameters
• mean (torch.Tensor) – An n x t or batch b x n x t matrix of means for the MVN distribution.

• covar (LinearOperator) – An … x NT x NT (batch) matrix. covariance matrix of MVN distribution.

• validate_args (bool) – (default=False) If True, validate mean anad covariance_matrix arguments.

• interleaved (bool) – (default=True) If True, covariance matrix is interpreted as block-diagonal w.r.t. inter-task covariances for each observation. If False, it is interpreted as block-diagonal w.r.t. inter-observation covariance for each task.

property base_sample_shape

Returns the shape of a base sample (without batching) that is used to generate a single sample.

Reinterprate a batch of multivariate normal distributions as an (independent) multitask multivariate normal distribution.

Parameters
• batch_mvn (MultivariateNormal) – The base MVN distribution. (This distribution should have at least one batch dimension).

• task_dim (int) – Which batch dimension should be interpreted as the dimension for the independent tasks.

Returns

Return type

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 2, 3)
>>> covar_factor = torch.randn(4, 2, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar)
>>> print(mvn.event_shape, mvn.batch_shape)
>>> # torch.Size([3]), torch.Size([4, 2])
>>>
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])

classmethod from_independent_mvns(mvns)[source]

Convert an iterable of MVNs into a MultitaskMultivariateNormal. The resulting distribution will have len(mvns) tasks, and the tasks will be independent.

Parameters

mvn (MultitaskNormal) – The base MVN distributions.

Returns

Return type

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn1 = gpytorch.distributions.MultivariateNormal(mean, covar)
>>>
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn2 = gpytorch.distributions.MultivariateNormal(mean, covar)
>>>
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])


Convert a single MVN into a MultitaskMultivariateNormal, where each task shares the same mean and covariance.

Parameters
• mvn (MultitaskNormal) – The base MVN distribution.

Returns

Return type

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar)
>>> print(mvn.event_shape, mvn.batch_shape)
>>> # torch.Size([3]), torch.Size([4])
>>>
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])

to_data_independent_dist(jitter_val=0.0001)[source]

Convert a multitask MVN into a batched (non-multitask) MVNs The result retains the intertask covariances, but gets rid of the inter-data covariances. The resulting distribution will have len(mvns) tasks, and the tasks will be independent.

Returns

the bached data-independent MVN

Return type

gpytorch.distributions.MultivariateNormal

## Delta¶

class gpytorch.distributions.Delta(v, log_density=0.0, event_dim=0, validate_args=None)[source]

(Borrowed from Pyro.) Degenerate discrete distribution (a single point).

Discrete distribution that assigns probability one to the single element in its support. Delta distribution parameterized by a random choice should not be used with MCMC based inference, as doing so produces incorrect results.

Parameters
• v (torch.Tensor) – The single support element.

• log_density (torch.Tensor) – An optional density for this Delta. This is useful to keep the class of Delta distributions closed under differentiable transformation.

• event_dim (int) – Optional event dimension, defaults to zero.