gpytorch.utils

gpytorch.utils.cached(method=None, name=None, ignore_args=False)[source]

A decorator allowing for specifying the name of a cache, allowing it to be modified elsewhere.

class gpytorch.utils.NNUtil(k, dim, batch_shape=torch.Size([]), preferred_nnlib='faiss', device='cpu')[source]

Utility for nearest neighbor search. It would first try to use faiss (requiring separate pacakge installment) as the backend for better computational performance. Otherwise, scikit-learn would be used as it is pre-installed with gpytorch.

Parameters:
  • k (int) – number of nearest neighbors
  • dim (int) – dimensionality of data
  • batch_shape (torch.Size) – batch shape for train data
  • preferred_nnlib (str) – currently supports faiss and scikit-learn (default: faiss).
  • device (torch.device) – device that the NN search will be performed on.

Example

>>> train_x = torch.randn(10, 5)
>>> nn_util = NNUtil(k=3, dim=train_x.size(-1), device=train_x.device)
>>> nn_util.set_nn_idx(train_x)
>>> test_x = torch.randn(2, 5)
>>> test_nn_indices = nn_util.find_nn_idx(test_x) # finding 3 nearest neighbors for test_x
>>> test_nn_indices = nn_util.find_nn_idx(test_x, k=2) # finding 2 nearest neighbors for test_x
>>> sequential_nn_idx = nn_util.build_sequential_nn_idx(train_x) # build up sequential nearest neighbor
>>>     # structure for train_x
build_sequential_nn_idx(x)[source]

Build the sequential \(k\) nearest neighbor structure within training data in the following way: for the \(i\)-th data point \(x_i\), find its \(k\) nearest neighbors among preceding training data \(x_1, \cdots, x_{i-1}\), for i=k+1:N where N is the size of training data.

Parameters:x – training data. Shape (N, D)
Return type:torch.LongTensor
Returns:indices of nearest neighbors. Shape: (N-k, k)
find_nn_idx(test_x, k=None)[source]

Find \(k\) nearest neighbors for test data test_x among the training data stored in this utility

Parameters:
  • test_x – test data, shape (… x N x D)
  • k (int) – number of nearest neighbors. Default is the value used in utility initialization.
Return type:

torch.LongTensor

Returns:

the indices of nearest neighbors in the training data

set_nn_idx(train_x)[source]

Set the indices of training data to facilitate nearest neighbor search. This function needs to be called every time that the data changes.

Parameters:train_x (torch.Tensor) – training data points (… x N x D)
to(device)[source]

Put the utility to a cpu or gpu device.

Parameters:device (torch.device) – Target device.

Grid Utilities

class gpytorch.utils.grid.ScaleToBounds(lower_bound, upper_bound)[source]

Scale the input data so that it lies in between the lower and upper bounds.

In training (self.train()), this module adjusts the scaling factor to the minibatch of data. During evaluation (self.eval()), this module uses the scaling factor from the previous minibatch of data.

Parameters:
  • lower_bound (float) – lower bound of scaled data
  • upper_bound (float) – upper bound of scaled data

Example

>>> train_x = torch.randn(10, 5)
>>> module = gpytorch.utils.grid.ScaleToBounds(lower_bound=-1., upper_bound=1.)
>>>
>>> module.train()
>>> scaled_train_x = module(train_x)  # Data should be between -0.95 and 0.95
>>>
>>> module.eval()
>>> test_x = torch.randn(10, 5)
>>> scaled_test_x = module(test_x)  # Scaling is based on train_x
gpytorch.utils.grid.choose_grid_size(train_inputs, ratio=1.0, kronecker_structure=True)[source]

Given some training inputs, determine a good grid size for KISS-GP.

Parameters:
  • x (torch.Tensor (.. x n x d)) – the input data
  • ratio (float, optional) – Amount of grid points per data point (default: 1.)
  • kronecker_structure (bool, optional) – Whether or not the model will use Kronecker structure in the grid (set to True unless there is an additive or product decomposition in the prior)
Returns:

Grid size

Return type:

int

gpytorch.utils.grid.create_data_from_grid(grid: List[torch.Tensor]) → torch.Tensor[source]
Parameters:grid (List[torch.Tensor]) – Each Tensor is a 1D set of increments for the grid in that dimension
Returns:The set of points on the grid going by column-major order
Return type:torch.Tensor
gpytorch.utils.grid.create_grid(grid_sizes: List[int], grid_bounds: List[Tuple[float, float]], extend: bool = True, device='cpu', dtype=torch.float32) → List[torch.Tensor][source]

Creates a grid represented by a list of 1D Tensors representing the projections of the grid into each dimension

If extend, we extend the grid by two points past the specified boundary which can be important for getting good grid interpolations.

Parameters:
  • grid_sizes (List[Tuple[float, float]]) – Sizes of each grid dimension
  • grid_bounds – Lower and upper bounds of each grid dimension
  • device (torch.device, optional) – target device for output (default: cpu)
  • dtype (torch.dtype, optional) – target dtype for output (default: torch.float)
Returns:

Grid points for each dimension. Grid points are stored in a torch.Tensor with shape grid_sizes[i].

Return type:

List[torch.Tensor]

gpytorch.utils.grid.scale_to_bounds(x, lower_bound, upper_bound)[source]

DEPRECATRED: Use ScaleToBounds instead.

Parameters:
  • x (torch.Tensor (.. x n x d)) – the input data
  • lower_bound (float) – lower bound of scaled data
  • upper_bound (float) – upper bound of scaled data
Returns:

scaled data

Return type:

torch.Tensor (.. x n x d)

Interpolation Utilities

Quadrature Utilities

class gpytorch.utils.quadrature.GaussHermiteQuadrature1D(num_locs=None)[source]

Implements Gauss-Hermite quadrature for integrating a function with respect to several 1D Gaussian distributions in batch mode. Within GPyTorch, this is useful primarily for computing expected log likelihoods for variational inference.

This is implemented as a Module because Gauss-Hermite quadrature has a set of locations and weights that it should initialize one time, but that should obey parent calls to .cuda(), .double() etc.

forward(func, gaussian_dists)[source]

Runs Gauss-Hermite quadrature on the callable func, integrating against the Gaussian distributions specified by gaussian_dists.

Parameters:
  • func (-) – Function to integrate
  • gaussian_dists (-) – Either a MultivariateNormal whose covariance is assumed to be diagonal or a torch.distributions.Normal.
Returns:

  • Result of integrating func against each univariate Gaussian in gaussian_dists.

Transform Utilities

Nearest Neighbors Utilities

class gpytorch.utils.nearest_neighbors.NNUtil(k, dim, batch_shape=torch.Size([]), preferred_nnlib='faiss', device='cpu')[source]

Utility for nearest neighbor search. It would first try to use faiss (requiring separate pacakge installment) as the backend for better computational performance. Otherwise, scikit-learn would be used as it is pre-installed with gpytorch.

Parameters:
  • k (int) – number of nearest neighbors
  • dim (int) – dimensionality of data
  • batch_shape (torch.Size) – batch shape for train data
  • preferred_nnlib (str) – currently supports faiss and scikit-learn (default: faiss).
  • device (torch.device) – device that the NN search will be performed on.

Example

>>> train_x = torch.randn(10, 5)
>>> nn_util = NNUtil(k=3, dim=train_x.size(-1), device=train_x.device)
>>> nn_util.set_nn_idx(train_x)
>>> test_x = torch.randn(2, 5)
>>> test_nn_indices = nn_util.find_nn_idx(test_x) # finding 3 nearest neighbors for test_x
>>> test_nn_indices = nn_util.find_nn_idx(test_x, k=2) # finding 2 nearest neighbors for test_x
>>> sequential_nn_idx = nn_util.build_sequential_nn_idx(train_x) # build up sequential nearest neighbor
>>>     # structure for train_x
build_sequential_nn_idx(x)[source]

Build the sequential \(k\) nearest neighbor structure within training data in the following way: for the \(i\)-th data point \(x_i\), find its \(k\) nearest neighbors among preceding training data \(x_1, \cdots, x_{i-1}\), for i=k+1:N where N is the size of training data.

Parameters:x – training data. Shape (N, D)
Return type:torch.LongTensor
Returns:indices of nearest neighbors. Shape: (N-k, k)
find_nn_idx(test_x, k=None)[source]

Find \(k\) nearest neighbors for test data test_x among the training data stored in this utility

Parameters:
  • test_x – test data, shape (… x N x D)
  • k (int) – number of nearest neighbors. Default is the value used in utility initialization.
Return type:

torch.LongTensor

Returns:

the indices of nearest neighbors in the training data

set_nn_idx(train_x)[source]

Set the indices of training data to facilitate nearest neighbor search. This function needs to be called every time that the data changes.

Parameters:train_x (torch.Tensor) – training data points (… x N x D)
to(device)[source]

Put the utility to a cpu or gpu device.

Parameters:device (torch.device) – Target device.