gpytorch.distributions

GPyTorch distribution objects are essentially the same as torch distribution objects. For the most part, GpyTorch relies on torch’s distribution library. However, we offer two custom distributions.

We implement a custom MultivariateNormal that accepts LinearOperator objects for covariance matrices. This allows us to use custom linear algebra operations, which makes this more efficient than PyTorch’s MVN implementation.

In addition, we implement a MultitaskMultivariateNormal which can be used with multi-output Gaussian process models.

Note

If Pyro is available, all GPyTorch distribution objects inherit Pyro’s distribution methods as well.

Distribution

class gpytorch.distributions.Distribution(batch_shape=torch.Size([]), event_shape=torch.Size([]), validate_args=None)[source]

MultivariateNormal

class gpytorch.distributions.MultivariateNormal(mean, covariance_matrix, validate_args=False)[source]

Constructs a multivariate normal random variable, based on mean and covariance. Can be multivariate, or a batch of multivariate normals

Passing a vector mean corresponds to a multivariate normal. Passing a matrix mean corresponds to a batch of multivariate normals.

Parameters:
  • mean (torch.tensor) – Vector n or matrix b x n mean of mvn distribution.
  • covar (LinearOperator) – … x N X N covariance matrix of mvn distribution.
base_sample_shape

Returns the shape of a base sample (without batching) that is used to generate a single sample.

confidence_region()[source]

Returns 2 standard deviations above and below the mean.

Return type:(torch.Tensor, torch.Tensor)
Returns:pair of tensors of size (b x d) or (d), where b is the batch size and d is the dimensionality of the random variable. The first (second) Tensor is the lower (upper) end of the confidence region.
get_base_samples(sample_shape=torch.Size([]))[source]

Get i.i.d. standard Normal samples (to be used with rsample(base_samples=base_samples))

lazy_covariance_matrix

The covariance_matrix, represented as a LinearOperator

to_data_independent_dist()[source]

Convert a MVN into a batched Normal distribution

Returns:the bached data-independent Normal
Return type:gpytorch.distributions.Normal

MultitaskMultivariateNormal

class gpytorch.distributions.MultitaskMultivariateNormal(mean, covariance_matrix, validate_args=False, interleaved=True)[source]

Constructs a multi-output multivariate Normal random variable, based on mean and covariance Can be multi-output multivariate, or a batch of multi-output multivariate Normal

Passing a matrix mean corresponds to a multi-output multivariate Normal Passing a matrix mean corresponds to a batch of multivariate Normals

Parameters:
  • mean (torch.Tensor) – An n x t or batch b x n x t matrix of means for the MVN distribution.
  • covar (LinearOperator) – An … x NT x NT (batch) matrix. covariance matrix of MVN distribution.
  • validate_args (bool) – (default=False) If True, validate mean anad covariance_matrix arguments.
  • interleaved (bool) – (default=True) If True, covariance matrix is interpreted as block-diagonal w.r.t. inter-task covariances for each observation. If False, it is interpreted as block-diagonal w.r.t. inter-observation covariance for each task.
base_sample_shape

Returns the shape of a base sample (without batching) that is used to generate a single sample.

classmethod from_batch_mvn(batch_mvn, task_dim=-1)[source]

Reinterprate a batch of multivariate normal distributions as an (independent) multitask multivariate normal distribution.

Parameters:
  • batch_mvn (MultivariateNormal) – The base MVN distribution. (This distribution should have at least one batch dimension).
  • task_dim (int) – Which batch dimension should be interpreted as the dimension for the independent tasks.
Returns:

the independent multitask distribution

Return type:

gpytorch.distributions.MultitaskMultivariateNormal

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 2, 3)
>>> covar_factor = torch.randn(4, 2, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar)
>>> print(mvn.event_shape, mvn.batch_shape)
>>> # torch.Size([3]), torch.Size([4, 2])
>>>
>>> mmvn = MultitaskMultivariateNormal.from_batch_mvn(mvn, task_dim=-1)
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])
classmethod from_independent_mvns(mvns)[source]

Convert an iterable of MVNs into a MultitaskMultivariateNormal. The resulting distribution will have len(mvns) tasks, and the tasks will be independent.

Parameters:mvn (MultitaskNormal) – The base MVN distributions.
Returns:the independent multitask distribution
Return type:gpytorch.distributions.MultitaskMultivariateNormal

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn1 = gpytorch.distributions.MultivariateNormal(mean, covar)
>>>
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn2 = gpytorch.distributions.MultivariateNormal(mean, covar)
>>>
>>> mmvn = MultitaskMultivariateNormal.from_independent_mvns([mvn1, mvn2])
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])
classmethod from_repeated_mvn(mvn, num_tasks)[source]

Convert a single MVN into a MultitaskMultivariateNormal, where each task shares the same mean and covariance.

Parameters:
  • mvn (MultitaskNormal) – The base MVN distribution.
  • num_tasks (int) – How many tasks to create.
Returns:

the independent multitask distribution

Return type:

gpytorch.distributions.MultitaskMultivariateNormal

Example

>>> # model is a gpytorch.models.VariationalGP
>>> # likelihood is a gpytorch.likelihoods.Likelihood
>>> mean = torch.randn(4, 3)
>>> covar_factor = torch.randn(4, 3, 3)
>>> covar = covar_factor @ covar_factor.transpose(-1, -2)
>>> mvn = gpytorch.distributions.MultivariateNormal(mean, covar)
>>> print(mvn.event_shape, mvn.batch_shape)
>>> # torch.Size([3]), torch.Size([4])
>>>
>>> mmvn = MultitaskMultivariateNormal.from_repeated_mvn(mvn, num_tasks=2)
>>> print(mmvn.event_shape, mmvn.batch_shape)
>>> # torch.Size([3, 2]), torch.Size([4])
to_data_independent_dist()[source]

Convert a multitask MVN into a batched (non-multitask) MVNs The result retains the intertask covariances, but gets rid of the inter-data covariances. The resulting distribution will have len(mvns) tasks, and the tasks will be independent.

Returns:the bached data-independent MVN
Return type:gpytorch.distributions.MultivariateNormal

Delta

class gpytorch.distributions.Delta(v, log_density=0.0, event_dim=0, validate_args=None)[source]

(Borrowed from Pyro.) Degenerate discrete distribution (a single point).

Discrete distribution that assigns probability one to the single element in its support. Delta distribution parameterized by a random choice should not be used with MCMC based inference, as doing so produces incorrect results.

Parameters:
  • v (torch.Tensor) – The single support element.
  • log_density (torch.Tensor) – An optional density for this Delta. This is useful to keep the class of Delta distributions closed under differentiable transformation.
  • event_dim (int) – Optional event dimension, defaults to zero.