gpytorch.distributions

GPyTorch distribution objects are essentially the same as torch distribution objects. For the most part, GPyTorch relies on torch’s distribution library. However, we offer two custom distributions.

We implement a custom MultivariateNormal that accepts LinearOperator objects for covariance matrices. This allows us to use custom linear algebra operations, which makes this more efficient than PyTorch’s MVN implementation.

In addition, we implement a MultitaskMultivariateNormal which can be used with multi-output Gaussian process models.

Note

If Pyro is available, all GPyTorch distribution objects inherit Pyro’s distribution methods as well.

Distribution

class gpytorch.distributions.Distribution(batch_shape=torch.Size([]), event_shape=torch.Size([]), validate_args=None)[source]
Parameters:
  • batch_shape (torch.Size) –

  • event_shape (torch.Size) –

  • validate_args (bool, optional) –

MultivariateNormal

class gpytorch.distributions.MultivariateNormal(mean, covariance_matrix, validate_args=False)[source]

Constructs a multivariate normal random variable, based on mean and covariance. Can be multivariate, or a batch of multivariate normals

Passing a vector mean corresponds to a multivariate normal. Passing a matrix mean corresponds to a batch of multivariate normals.

Parameters:
  • mean (torch.Tensor) – … x N mean of mvn distribution.

  • covariance_matrix (torch.Tensor or LinearOperator) – … x N X N covariance matrix of mvn distribution.

  • validate_args (bool) – If True, validate mean anad covariance_matrix arguments. (Default: False.)

Variables:
  • base_sample_shape (torch.Size) – The shape of a base sample (without batching) that is used to generate a single sample.

  • covariance_matrix (torch.Tensor) – The covariance matrix, represented as a dense torch.Tensor

  • lazy_covariance_matrix (LinearOperator) – The covariance matrix, represented as a LinearOperator.

  • mean (torch.Tensor) – The mean.

  • stddev (torch.Tensor) – The standard deviation.

  • variance (torch.Tensor) – The variance.

__getitem__(idx)[source]

Constructs a new MultivariateNormal that represents a random variable modified by an indexing operation.

The mean and covariance matrix arguments are indexed accordingly.

Parameters:

idx – Index to apply to the mean. The covariance matrix is indexed accordingly.

Return type:

MultivariateNormal

add_jitter(noise=0.0001)[source]

Adds a small constant diagonal to the MVN covariance matrix for numerical stability.

Parameters:

noise (float) – The size of the constant diagonal.

Return type:

MultivariateNormal

confidence_region()[source]

Returns 2 standard deviations above and below the mean.

Return type:

(torch.Tensor, torch.Tensor)

Returns:

Pair of tensors of size … x N, where N is the dimensionality of the random variable. The first (second) Tensor is the lower (upper) end of the confidence region.

expand(batch_size)[source]

See torch.distributions.Distribution.expand.

Parameters:

batch_size (torch.Size) –

Return type:

MultivariateNormal

get_base_samples(sample_shape=torch.Size([]))[source]

Returns i.i.d. standard Normal samples to be used with MultivariateNormal.rsample(base_samples=base_samples).

Parameters:

sample_shape (torch.Size) – The number of samples to generate. (Default: torch.Size([]).)

Return type:

torch.Tensor

Returns:

A *sample_shape x *batch_shape x N tensor of i.i.d. standard Normal samples.

log_prob(value)[source]

See torch.distributions.Distribution.log_prob.

Parameters:

value (torch.Tensor) –

Return type:

torch.Tensor

rsample(sample_shape=torch.Size([]), base_samples=None)[source]

Generates a sample_shape shaped reparameterized sample or sample_shape shaped batch of reparameterized samples if the distribution parameters are batched.

For the MultivariateNormal distribution, this is accomplished through:

\[\boldsymbol \mu + \mathbf L \boldsymbol \epsilon\]

where \(\boldsymbol \mu \in \mathcal R^N\) is the MVN mean, \(\mathbf L \in \mathcal R^{N \times N}\) is a “root” of the covariance matrix \(\mathbf K\) (i.e. \(\mathbf L \mathbf L^\top = \mathbf K\)), and \(\boldsymbol \epsilon \in \mathcal R^N\) is a vector of (approximately) i.i.d. standard Normal random variables.

Parameters:
  • sample_shape (torch.Size) – The number of samples to generate. (Default: torch.Size([]).)

  • base_samples (torch.Tensor, optional) – The *sample_shape x *batch_shape x N tensor of i.i.d. (or approximately i.i.d.) standard Normal samples to reparameterize. (Default: None.)

Return type:

torch.Tensor

Returns:

A *sample_shape x *batch_shape x N tensor of i.i.d. reparameterized samples.

sample(sample_shape=torch.Size([]), base_samples=None)[source]

Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.

Note that these samples are not reparameterized and therefore cannot be backpropagated through.

Parameters:
  • sample_shape (torch.Size) – The number of samples to generate. (Default: torch.Size([]).)

  • base_samples (torch.Tensor, optional) – The *sample_shape x *batch_shape x N tensor of i.i.d. (or approximately i.i.d.) standard Normal samples to reparameterize. (Default: None.)

Return type:

torch.Tensor

Returns:

A *sample_shape x *batch_shape x N tensor of i.i.d. samples.

to_data_independent_dist()[source]

Convert a … x N MVN distribution into a batch of independent Normal distributions. Essentially, this throws away all covariance information and treats all dimensions as batch dimensions.

Return type:

torch.distributions.normal.Normal

Returns:

A (data-independent) Normal distribution with batch shape *batch_shape x N.

MultitaskMultivariateNormal

class gpytorch.distributions.MultitaskMultivariateNormal(mean, covariance_matrix, validate_args=False, interleaved=True)[source]

Constructs a multi-output multivariate Normal random variable, based on mean and covariance Can be multi-output multivariate, or a batch of multi-output multivariate Normal

Passing a matrix mean corresponds to a multi-output multivariate Normal Passing a matrix mean corresponds to a batch of multivariate Normals

Parameters:
  • mean (torch.Tensor) – An n x t or batch b x n x t matrix of means for the MVN distribution.

  • covar (LinearOperator) – An … x NT x NT (batch) matrix. covariance matrix of MVN distribution.

  • validate_args (bool) – (default=False) If True, validate mean and covariance_matrix arguments.

  • interleaved (bool) – (default=True) If True, covariance matrix is interpreted as block-diagonal w.r.t. inter-task covariances for each observation. If False, it is interpreted as block-diagonal w.r.t. inter-observation covariance for each task.

__getitem__(idx)[source]

Constructs a new MultivariateNormal that represents a random variable modified by an indexing operation.

The mean and covariance matrix arguments are indexed accordingly.

Parameters:

idx (Any) – Index to apply to the mean. The covariance matrix is indexed accordingly.

Return type:

MultivariateNormal

Returns:

If indices specify a slice for samples and tasks, returns a MultitaskMultivariateNormal, else returns a MultivariateNormal.

property base_sample_shape

Returns the shape of a base sample (without batching) that is used to generate a single sample.

classmethod from_batch_mvn(batch_mvn, task_dim=-1)[source]

Reinterprate a batch of multivariate normal distributions as an (independent) multitask multivariate normal distribution.

Parameters:
  • batch_mvn (MultivariateNormal) – The base MVN distribution. (This distribution should have at least one batch dimension).

  • task_dim (int) – Which batch dimension should be interpreted as the dimension for the independent tasks.

Returns:

the independent multitask distribution

Return type:

gpytorch.distributions.MultitaskMultivariateNormal

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 2, 3)
>>> covar_factor = torch.randn(4, 2, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar)
>>> print(mvn.event_shape, mvn.batch_shape)
>>> # torch.Size([3]), torch.Size([4, 2])
>>>
>>> mmvn = MultitaskMultivariateNormal.from_batch_mvn(mvn, task_dim=-1)
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])
classmethod from_independent_mvns(mvns)[source]

Convert an iterable of MVNs into a MultitaskMultivariateNormal. The resulting distribution will have len(mvns) tasks, and the tasks will be independent.

Parameters:

mvn (MultitaskNormal) – The base MVN distributions.

Returns:

the independent multitask distribution

Return type:

gpytorch.distributions.MultitaskMultivariateNormal

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn1 = gpytorch.distributions.MultivariateNormal(mean, covar)
>>>
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn2 = gpytorch.distributions.MultivariateNormal(mean, covar)
>>>
>>> mmvn = MultitaskMultivariateNormal.from_independent_mvns([mvn1, mvn2])
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])
classmethod from_repeated_mvn(mvn, num_tasks)[source]

Convert a single MVN into a MultitaskMultivariateNormal, where each task shares the same mean and covariance.

Parameters:
  • mvn (MultitaskNormal) – The base MVN distribution.

  • num_tasks (int) – How many tasks to create.

Returns:

the independent multitask distribution

Return type:

gpytorch.distributions.MultitaskMultivariateNormal

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar)
>>> print(mvn.event_shape, mvn.batch_shape)
>>> # torch.Size([3]), torch.Size([4])
>>>
>>> mmvn = MultitaskMultivariateNormal.from_repeated_mvn(mvn, num_tasks=2)
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])
to_data_independent_dist(jitter_val=0.0001)[source]

Convert a multitask MVN into a batched (non-multitask) MVNs The result retains the intertask covariances, but gets rid of the inter-data covariances. The resulting distribution will have len(mvns) tasks, and the tasks will be independent.

Returns:

the bached data-independent MVN

Return type:

gpytorch.distributions.MultivariateNormal

Delta

class gpytorch.distributions.Delta(v, log_density=0.0, event_dim=0, validate_args=None)[source]

(Borrowed from Pyro.) Degenerate discrete distribution (a single point).

Discrete distribution that assigns probability one to the single element in its support. Delta distribution parameterized by a random choice should not be used with MCMC based inference, as doing so produces incorrect results.

Parameters:
  • v (torch.Tensor) – The single support element.

  • log_density (torch.Tensor) – An optional density for this Delta. This is useful to keep the class of Delta distributions closed under differentiable transformation.

  • event_dim (int) – Optional event dimension, defaults to zero.