GP Regression with Uncertain Inputs

Introduction

In this notebook, we’re going to demonstrate one way of dealing with uncertainty in our training data. Let’s say that we’re collecting training data that models the following function.

\begin{align} y &= \sin(2\pi x) + \epsilon \\ \epsilon &\sim \mathcal{N}(0, 0.2) \end{align}

However, now assume that we’re a bit uncertain about our features. In particular, we’re going to assume that every x_i value is not a point but a distribution instead. E.g.

\[x_i \sim \mathcal{N}(\mu_i, \sigma_i).\]

Using a distributional kernel to deal with uncertain inputs

Rather than using a variational method (see the GP Regression with Uncertian Inputs tutorial in the variational examples), if we explicitly know the type of uncertainty in our inputs we can pass that into our kernel.

More specifically, assuming Gaussian inputs, we will compute the symmetrized KL divergence between the Gaussian inputs.

[1]:
import math
import torch
import tqdm
import gpytorch
from matplotlib import pyplot as plt

%matplotlib inline
%load_ext autoreload
%autoreload 2
[2]:
# Training data is 100 points in [0,1] inclusive regularly spaced
train_x_mean = torch.linspace(0, 1, 20)
# We'll assume the variance shrinks the closer we get to 1
train_x_stdv = torch.linspace(0.03, 0.01, 20)

# True function is sin(2*pi*x) with Gaussian noise
train_y = torch.sin(train_x_mean * (2 * math.pi)) + torch.randn(train_x_mean.size()) * 0.2

To effectively pass in the training distributional data, we will need to stack the mean and log variances.

[3]:
train_x_distributional = torch.stack((train_x_mean, (train_x_stdv**2).log()), dim=1)
[4]:
f, ax = plt.subplots(1, 1, figsize=(8, 3))
ax.errorbar(train_x_mean, train_y, xerr=(train_x_stdv * 2), fmt="k*", label="Train Data")
ax.legend()
[4]:
<matplotlib.legend.Legend at 0x7fc3069399d0>
../../_images/examples_01_Exact_GPs_GP_Regression_DistributionalKernel_6_1.png

We train the hyperparameters of the resulting distributional GP via type-II gradient descent, as is standard in many settings. We could also do fully Bayesian inference.

[5]:
from gpytorch.models import ExactGP
from gpytorch.kernels import GaussianSymmetrizedKLKernel, ScaleKernel
from gpytorch.means import ConstantMean

class ExactGPModel(ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = ConstantMean()
        self.covar_module = ScaleKernel(GaussianSymmetrizedKLKernel())

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

# initialize likelihood and model
likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(train_x_distributional, train_y, likelihood)
[7]:
# this is for running the notebook in our testing framework
import os
smoke_test = ('CI' in os.environ)
training_iter = 2 if smoke_test else 500


# Find optimal model hyperparameters
model.train()
likelihood.train()

# Use the adam optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=0.25)  # Includes GaussianLikelihood parameters

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

for i in range(training_iter):
    # Zero gradients from previous iteration
    optimizer.zero_grad()
    # Output from model
    output = model(train_x_distributional)
    # Calc loss and backprop gradients
    loss = -mll(output, train_y)
    loss.backward()
    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (
        i + 1, training_iter, loss.item(),
        model.covar_module.base_kernel.lengthscale.item(),
        model.likelihood.noise.item()
    ))
    optimizer.step()
Iter 1/500 - Loss: 1.274   lengthscale: 0.693   noise: 0.693
Iter 2/500 - Loss: 1.242   lengthscale: 0.826   noise: 0.576
Iter 3/500 - Loss: 1.173   lengthscale: 0.955   noise: 0.475
Iter 4/500 - Loss: 1.141   lengthscale: 1.102   noise: 0.389
Iter 5/500 - Loss: 1.121   lengthscale: 1.266   noise: 0.317
Iter 6/500 - Loss: 1.097   lengthscale: 1.442   noise: 0.261
Iter 7/500 - Loss: 1.093   lengthscale: 1.621   noise: 0.220
Iter 8/500 - Loss: 1.112   lengthscale: 1.801   noise: 0.192
Iter 9/500 - Loss: 1.112   lengthscale: 1.988   noise: 0.177
Iter 10/500 - Loss: 1.095   lengthscale: 2.188   noise: 0.172
Iter 11/500 - Loss: 1.081   lengthscale: 2.401   noise: 0.172
Iter 12/500 - Loss: 1.068   lengthscale: 2.624   noise: 0.177
Iter 13/500 - Loss: 1.049   lengthscale: 2.856   noise: 0.183
Iter 14/500 - Loss: 1.031   lengthscale: 3.095   noise: 0.188
Iter 15/500 - Loss: 1.020   lengthscale: 3.340   noise: 0.191
Iter 16/500 - Loss: 1.014   lengthscale: 3.589   noise: 0.189
Iter 17/500 - Loss: 1.007   lengthscale: 3.842   noise: 0.183
Iter 18/500 - Loss: 0.997   lengthscale: 4.098   noise: 0.173
Iter 19/500 - Loss: 0.983   lengthscale: 4.356   noise: 0.159
Iter 20/500 - Loss: 0.968   lengthscale: 4.616   noise: 0.144
Iter 21/500 - Loss: 0.951   lengthscale: 4.877   noise: 0.128
Iter 22/500 - Loss: 0.933   lengthscale: 5.139   noise: 0.112
Iter 23/500 - Loss: 0.913   lengthscale: 5.402   noise: 0.097
Iter 24/500 - Loss: 0.891   lengthscale: 5.667   noise: 0.083
Iter 25/500 - Loss: 0.870   lengthscale: 5.933   noise: 0.070
Iter 26/500 - Loss: 0.850   lengthscale: 6.202   noise: 0.059
Iter 27/500 - Loss: 0.832   lengthscale: 6.474   noise: 0.050
Iter 28/500 - Loss: 0.814   lengthscale: 6.748   noise: 0.042
Iter 29/500 - Loss: 0.794   lengthscale: 7.025   noise: 0.035
Iter 30/500 - Loss: 0.775   lengthscale: 7.304   noise: 0.029
Iter 31/500 - Loss: 0.756   lengthscale: 7.586   noise: 0.024
Iter 32/500 - Loss: 0.737   lengthscale: 7.868   noise: 0.020
Iter 33/500 - Loss: 0.719   lengthscale: 8.152   noise: 0.017
Iter 34/500 - Loss: 0.701   lengthscale: 8.437   noise: 0.014
Iter 35/500 - Loss: 0.684   lengthscale: 8.722   noise: 0.012
Iter 36/500 - Loss: 0.668   lengthscale: 9.007   noise: 0.010
Iter 37/500 - Loss: 0.653   lengthscale: 9.291   noise: 0.008
Iter 38/500 - Loss: 0.637   lengthscale: 9.575   noise: 0.007
Iter 39/500 - Loss: 0.621   lengthscale: 9.859   noise: 0.006
Iter 40/500 - Loss: 0.606   lengthscale: 10.142   noise: 0.005
Iter 41/500 - Loss: 0.591   lengthscale: 10.424   noise: 0.005
Iter 42/500 - Loss: 0.577   lengthscale: 10.705   noise: 0.004
Iter 43/500 - Loss: 0.564   lengthscale: 10.985   noise: 0.003
Iter 44/500 - Loss: 0.551   lengthscale: 11.263   noise: 0.003
Iter 45/500 - Loss: 0.539   lengthscale: 11.540   noise: 0.003
Iter 46/500 - Loss: 0.526   lengthscale: 11.815   noise: 0.002
Iter 47/500 - Loss: 0.514   lengthscale: 12.088   noise: 0.002
Iter 48/500 - Loss: 0.503   lengthscale: 12.359   noise: 0.002
Iter 49/500 - Loss: 0.492   lengthscale: 12.627   noise: 0.002
Iter 50/500 - Loss: 0.482   lengthscale: 12.892   noise: 0.001
Iter 51/500 - Loss: 0.472   lengthscale: 13.154   noise: 0.001
Iter 52/500 - Loss: 0.462   lengthscale: 13.412   noise: 0.001
Iter 53/500 - Loss: 0.453   lengthscale: 13.668   noise: 0.001
Iter 54/500 - Loss: 0.445   lengthscale: 13.919   noise: 0.001
Iter 55/500 - Loss: 0.437   lengthscale: 14.167   noise: 0.001
Iter 56/500 - Loss: 0.429   lengthscale: 14.410   noise: 0.001
Iter 57/500 - Loss: 0.422   lengthscale: 14.649   noise: 0.001
Iter 58/500 - Loss: 0.415   lengthscale: 14.883   noise: 0.001
Iter 59/500 - Loss: 0.409   lengthscale: 15.111   noise: 0.001
Iter 60/500 - Loss: 0.403   lengthscale: 15.335   noise: 0.001
Iter 61/500 - Loss: 0.398   lengthscale: 15.552   noise: 0.001
Iter 62/500 - Loss: 0.393   lengthscale: 15.764   noise: 0.001
Iter 63/500 - Loss: 0.389   lengthscale: 15.970   noise: 0.001
Iter 64/500 - Loss: 0.385   lengthscale: 16.170   noise: 0.000
Iter 65/500 - Loss: 0.381   lengthscale: 16.363   noise: 0.000
Iter 66/500 - Loss: 0.378   lengthscale: 16.551   noise: 0.000
Iter 67/500 - Loss: 0.375   lengthscale: 16.731   noise: 0.000
Iter 68/500 - Loss: 0.373   lengthscale: 16.905   noise: 0.000
Iter 69/500 - Loss: 0.371   lengthscale: 17.072   noise: 0.000
Iter 70/500 - Loss: 0.369   lengthscale: 17.231   noise: 0.000
Iter 71/500 - Loss: 0.367   lengthscale: 17.383   noise: 0.000
Iter 72/500 - Loss: 0.366   lengthscale: 17.527   noise: 0.000
Iter 73/500 - Loss: 0.365   lengthscale: 17.662   noise: 0.000
Iter 74/500 - Loss: 0.364   lengthscale: 17.791   noise: 0.000
Iter 75/500 - Loss: 0.363   lengthscale: 17.911   noise: 0.000
Iter 76/500 - Loss: 0.363   lengthscale: 18.024   noise: 0.000
Iter 77/500 - Loss: 0.362   lengthscale: 18.129   noise: 0.000
Iter 78/500 - Loss: 0.362   lengthscale: 18.227   noise: 0.000
Iter 79/500 - Loss: 0.362   lengthscale: 18.317   noise: 0.000
Iter 80/500 - Loss: 0.362   lengthscale: 18.400   noise: 0.000
Iter 81/500 - Loss: 0.362   lengthscale: 18.477   noise: 0.000
Iter 82/500 - Loss: 0.362   lengthscale: 18.546   noise: 0.000
Iter 83/500 - Loss: 0.362   lengthscale: 18.608   noise: 0.000
Iter 84/500 - Loss: 0.362   lengthscale: 18.664   noise: 0.000
Iter 85/500 - Loss: 0.362   lengthscale: 18.712   noise: 0.000
Iter 86/500 - Loss: 0.362   lengthscale: 18.755   noise: 0.000
Iter 87/500 - Loss: 0.362   lengthscale: 18.791   noise: 0.000
Iter 88/500 - Loss: 0.362   lengthscale: 18.822   noise: 0.000
Iter 89/500 - Loss: 0.362   lengthscale: 18.848   noise: 0.000
Iter 90/500 - Loss: 0.363   lengthscale: 18.868   noise: 0.000
Iter 91/500 - Loss: 0.363   lengthscale: 18.884   noise: 0.000
Iter 92/500 - Loss: 0.363   lengthscale: 18.896   noise: 0.000
Iter 93/500 - Loss: 0.363   lengthscale: 18.904   noise: 0.000
Iter 94/500 - Loss: 0.363   lengthscale: 18.909   noise: 0.000
Iter 95/500 - Loss: 0.363   lengthscale: 18.910   noise: 0.000
Iter 96/500 - Loss: 0.363   lengthscale: 18.907   noise: 0.000
Iter 97/500 - Loss: 0.363   lengthscale: 18.902   noise: 0.000
Iter 98/500 - Loss: 0.363   lengthscale: 18.894   noise: 0.000
Iter 99/500 - Loss: 0.362   lengthscale: 18.884   noise: 0.000
Iter 100/500 - Loss: 0.362   lengthscale: 18.873   noise: 0.000
Iter 101/500 - Loss: 0.362   lengthscale: 18.859   noise: 0.000
Iter 102/500 - Loss: 0.362   lengthscale: 18.844   noise: 0.000
Iter 103/500 - Loss: 0.362   lengthscale: 18.828   noise: 0.000
Iter 104/500 - Loss: 0.362   lengthscale: 18.811   noise: 0.000
Iter 105/500 - Loss: 0.362   lengthscale: 18.794   noise: 0.000
Iter 106/500 - Loss: 0.362   lengthscale: 18.776   noise: 0.000
Iter 107/500 - Loss: 0.362   lengthscale: 18.758   noise: 0.000
Iter 108/500 - Loss: 0.362   lengthscale: 18.740   noise: 0.000
Iter 109/500 - Loss: 0.362   lengthscale: 18.722   noise: 0.000
Iter 110/500 - Loss: 0.362   lengthscale: 18.704   noise: 0.000
Iter 111/500 - Loss: 0.362   lengthscale: 18.687   noise: 0.000
Iter 112/500 - Loss: 0.362   lengthscale: 18.670   noise: 0.000
Iter 113/500 - Loss: 0.362   lengthscale: 18.653   noise: 0.000
Iter 114/500 - Loss: 0.362   lengthscale: 18.638   noise: 0.000
Iter 115/500 - Loss: 0.362   lengthscale: 18.622   noise: 0.000
Iter 116/500 - Loss: 0.362   lengthscale: 18.608   noise: 0.000
Iter 117/500 - Loss: 0.362   lengthscale: 18.595   noise: 0.000
Iter 118/500 - Loss: 0.362   lengthscale: 18.583   noise: 0.000
Iter 119/500 - Loss: 0.362   lengthscale: 18.572   noise: 0.000
Iter 120/500 - Loss: 0.362   lengthscale: 18.561   noise: 0.000
Iter 121/500 - Loss: 0.362   lengthscale: 18.552   noise: 0.000
Iter 122/500 - Loss: 0.362   lengthscale: 18.543   noise: 0.000
Iter 123/500 - Loss: 0.362   lengthscale: 18.536   noise: 0.000
Iter 124/500 - Loss: 0.362   lengthscale: 18.529   noise: 0.000
Iter 125/500 - Loss: 0.362   lengthscale: 18.524   noise: 0.000
Iter 126/500 - Loss: 0.362   lengthscale: 18.519   noise: 0.000
Iter 127/500 - Loss: 0.362   lengthscale: 18.516   noise: 0.000
Iter 128/500 - Loss: 0.362   lengthscale: 18.513   noise: 0.000
Iter 129/500 - Loss: 0.362   lengthscale: 18.511   noise: 0.000
Iter 130/500 - Loss: 0.362   lengthscale: 18.510   noise: 0.000
Iter 131/500 - Loss: 0.362   lengthscale: 18.509   noise: 0.000
Iter 132/500 - Loss: 0.362   lengthscale: 18.510   noise: 0.000
Iter 133/500 - Loss: 0.362   lengthscale: 18.511   noise: 0.000
Iter 134/500 - Loss: 0.362   lengthscale: 18.512   noise: 0.000
Iter 135/500 - Loss: 0.362   lengthscale: 18.514   noise: 0.000
Iter 136/500 - Loss: 0.362   lengthscale: 18.517   noise: 0.000
Iter 137/500 - Loss: 0.362   lengthscale: 18.520   noise: 0.000
Iter 138/500 - Loss: 0.362   lengthscale: 18.524   noise: 0.000
Iter 139/500 - Loss: 0.362   lengthscale: 18.528   noise: 0.000
Iter 140/500 - Loss: 0.362   lengthscale: 18.532   noise: 0.000
Iter 141/500 - Loss: 0.362   lengthscale: 18.537   noise: 0.000
Iter 142/500 - Loss: 0.362   lengthscale: 18.542   noise: 0.000
Iter 143/500 - Loss: 0.362   lengthscale: 18.547   noise: 0.000
Iter 144/500 - Loss: 0.362   lengthscale: 18.552   noise: 0.000
Iter 145/500 - Loss: 0.362   lengthscale: 18.558   noise: 0.000
Iter 146/500 - Loss: 0.362   lengthscale: 18.563   noise: 0.000
Iter 147/500 - Loss: 0.362   lengthscale: 18.569   noise: 0.000
Iter 148/500 - Loss: 0.362   lengthscale: 18.574   noise: 0.000
Iter 149/500 - Loss: 0.362   lengthscale: 18.580   noise: 0.000
Iter 150/500 - Loss: 0.362   lengthscale: 18.586   noise: 0.000
Iter 151/500 - Loss: 0.362   lengthscale: 18.592   noise: 0.000
Iter 152/500 - Loss: 0.362   lengthscale: 18.597   noise: 0.000
Iter 153/500 - Loss: 0.362   lengthscale: 18.603   noise: 0.000
Iter 154/500 - Loss: 0.362   lengthscale: 18.608   noise: 0.000
Iter 155/500 - Loss: 0.362   lengthscale: 18.614   noise: 0.000
Iter 156/500 - Loss: 0.362   lengthscale: 18.619   noise: 0.000
Iter 157/500 - Loss: 0.362   lengthscale: 18.625   noise: 0.000
Iter 158/500 - Loss: 0.362   lengthscale: 18.630   noise: 0.000
Iter 159/500 - Loss: 0.362   lengthscale: 18.635   noise: 0.000
Iter 160/500 - Loss: 0.362   lengthscale: 18.640   noise: 0.000
Iter 161/500 - Loss: 0.362   lengthscale: 18.645   noise: 0.000
Iter 162/500 - Loss: 0.362   lengthscale: 18.650   noise: 0.000
Iter 163/500 - Loss: 0.362   lengthscale: 18.655   noise: 0.000
Iter 164/500 - Loss: 0.362   lengthscale: 18.659   noise: 0.000
Iter 165/500 - Loss: 0.361   lengthscale: 18.664   noise: 0.000
Iter 166/500 - Loss: 0.361   lengthscale: 18.669   noise: 0.000
Iter 167/500 - Loss: 0.361   lengthscale: 18.673   noise: 0.000
Iter 168/500 - Loss: 0.361   lengthscale: 18.678   noise: 0.000
Iter 169/500 - Loss: 0.361   lengthscale: 18.682   noise: 0.000
Iter 170/500 - Loss: 0.361   lengthscale: 18.686   noise: 0.000
Iter 171/500 - Loss: 0.361   lengthscale: 18.691   noise: 0.000
Iter 172/500 - Loss: 0.361   lengthscale: 18.695   noise: 0.000
Iter 173/500 - Loss: 0.361   lengthscale: 18.699   noise: 0.000
Iter 174/500 - Loss: 0.361   lengthscale: 18.704   noise: 0.000
Iter 175/500 - Loss: 0.361   lengthscale: 18.708   noise: 0.000
Iter 176/500 - Loss: 0.361   lengthscale: 18.713   noise: 0.000
Iter 177/500 - Loss: 0.361   lengthscale: 18.717   noise: 0.000
Iter 178/500 - Loss: 0.361   lengthscale: 18.721   noise: 0.000
Iter 179/500 - Loss: 0.361   lengthscale: 18.726   noise: 0.000
Iter 180/500 - Loss: 0.361   lengthscale: 18.731   noise: 0.000
Iter 181/500 - Loss: 0.361   lengthscale: 18.735   noise: 0.000
Iter 182/500 - Loss: 0.361   lengthscale: 18.740   noise: 0.000
Iter 183/500 - Loss: 0.361   lengthscale: 18.745   noise: 0.000
Iter 184/500 - Loss: 0.361   lengthscale: 18.750   noise: 0.000
Iter 185/500 - Loss: 0.361   lengthscale: 18.755   noise: 0.000
Iter 186/500 - Loss: 0.361   lengthscale: 18.760   noise: 0.000
Iter 187/500 - Loss: 0.361   lengthscale: 18.765   noise: 0.000
Iter 188/500 - Loss: 0.361   lengthscale: 18.771   noise: 0.000
Iter 189/500 - Loss: 0.361   lengthscale: 18.776   noise: 0.000
Iter 190/500 - Loss: 0.361   lengthscale: 18.782   noise: 0.000
Iter 191/500 - Loss: 0.361   lengthscale: 18.788   noise: 0.000
Iter 192/500 - Loss: 0.361   lengthscale: 18.794   noise: 0.000
Iter 193/500 - Loss: 0.361   lengthscale: 18.800   noise: 0.000
Iter 194/500 - Loss: 0.361   lengthscale: 18.806   noise: 0.000
Iter 195/500 - Loss: 0.361   lengthscale: 18.813   noise: 0.000
Iter 196/500 - Loss: 0.361   lengthscale: 18.820   noise: 0.000
Iter 197/500 - Loss: 0.361   lengthscale: 18.827   noise: 0.000
Iter 198/500 - Loss: 0.361   lengthscale: 18.834   noise: 0.000
Iter 199/500 - Loss: 0.361   lengthscale: 18.841   noise: 0.000
Iter 200/500 - Loss: 0.361   lengthscale: 18.848   noise: 0.000
Iter 201/500 - Loss: 0.361   lengthscale: 18.856   noise: 0.000
Iter 202/500 - Loss: 0.361   lengthscale: 18.863   noise: 0.000
Iter 203/500 - Loss: 0.361   lengthscale: 18.871   noise: 0.000
Iter 204/500 - Loss: 0.361   lengthscale: 18.880   noise: 0.000
Iter 205/500 - Loss: 0.361   lengthscale: 18.888   noise: 0.000
Iter 206/500 - Loss: 0.361   lengthscale: 18.897   noise: 0.001
Iter 207/500 - Loss: 0.361   lengthscale: 18.905   noise: 0.001
Iter 208/500 - Loss: 0.361   lengthscale: 18.914   noise: 0.001
Iter 209/500 - Loss: 0.361   lengthscale: 18.924   noise: 0.001
Iter 210/500 - Loss: 0.361   lengthscale: 18.933   noise: 0.001
Iter 211/500 - Loss: 0.361   lengthscale: 18.943   noise: 0.001
Iter 212/500 - Loss: 0.361   lengthscale: 18.953   noise: 0.001
Iter 213/500 - Loss: 0.361   lengthscale: 18.963   noise: 0.001
Iter 214/500 - Loss: 0.361   lengthscale: 18.974   noise: 0.001
Iter 215/500 - Loss: 0.361   lengthscale: 18.985   noise: 0.001
Iter 216/500 - Loss: 0.361   lengthscale: 18.996   noise: 0.001
Iter 217/500 - Loss: 0.361   lengthscale: 19.007   noise: 0.001
Iter 218/500 - Loss: 0.361   lengthscale: 19.019   noise: 0.001
Iter 219/500 - Loss: 0.361   lengthscale: 19.030   noise: 0.001
Iter 220/500 - Loss: 0.361   lengthscale: 19.043   noise: 0.001
Iter 221/500 - Loss: 0.360   lengthscale: 19.055   noise: 0.001
Iter 222/500 - Loss: 0.360   lengthscale: 19.068   noise: 0.001
Iter 223/500 - Loss: 0.360   lengthscale: 19.081   noise: 0.001
Iter 224/500 - Loss: 0.360   lengthscale: 19.095   noise: 0.001
Iter 225/500 - Loss: 0.360   lengthscale: 19.109   noise: 0.001
Iter 226/500 - Loss: 0.360   lengthscale: 19.123   noise: 0.001
Iter 227/500 - Loss: 0.360   lengthscale: 19.138   noise: 0.001
Iter 228/500 - Loss: 0.360   lengthscale: 19.153   noise: 0.001
Iter 229/500 - Loss: 0.360   lengthscale: 19.169   noise: 0.001
Iter 230/500 - Loss: 0.360   lengthscale: 19.185   noise: 0.001
Iter 231/500 - Loss: 0.360   lengthscale: 19.202   noise: 0.001
Iter 232/500 - Loss: 0.360   lengthscale: 19.219   noise: 0.001
Iter 233/500 - Loss: 0.360   lengthscale: 19.236   noise: 0.001
Iter 234/500 - Loss: 0.360   lengthscale: 19.254   noise: 0.001
Iter 235/500 - Loss: 0.360   lengthscale: 19.273   noise: 0.001
Iter 236/500 - Loss: 0.360   lengthscale: 19.292   noise: 0.001
Iter 237/500 - Loss: 0.360   lengthscale: 19.311   noise: 0.001
Iter 238/500 - Loss: 0.359   lengthscale: 19.332   noise: 0.001
Iter 239/500 - Loss: 0.359   lengthscale: 19.353   noise: 0.001
Iter 240/500 - Loss: 0.359   lengthscale: 19.374   noise: 0.001
Iter 241/500 - Loss: 0.359   lengthscale: 19.396   noise: 0.001
Iter 242/500 - Loss: 0.359   lengthscale: 19.419   noise: 0.001
Iter 243/500 - Loss: 0.359   lengthscale: 19.443   noise: 0.001
Iter 244/500 - Loss: 0.359   lengthscale: 19.467   noise: 0.001
Iter 245/500 - Loss: 0.359   lengthscale: 19.492   noise: 0.001
Iter 246/500 - Loss: 0.359   lengthscale: 19.518   noise: 0.001
Iter 247/500 - Loss: 0.358   lengthscale: 19.545   noise: 0.001
Iter 248/500 - Loss: 0.358   lengthscale: 19.572   noise: 0.001
Iter 249/500 - Loss: 0.358   lengthscale: 19.600   noise: 0.001
Iter 250/500 - Loss: 0.358   lengthscale: 19.630   noise: 0.001
Iter 251/500 - Loss: 0.358   lengthscale: 19.660   noise: 0.001
Iter 252/500 - Loss: 0.358   lengthscale: 19.690   noise: 0.001
Iter 253/500 - Loss: 0.358   lengthscale: 19.722   noise: 0.001
Iter 254/500 - Loss: 0.357   lengthscale: 19.755   noise: 0.001
Iter 255/500 - Loss: 0.357   lengthscale: 19.789   noise: 0.001
Iter 256/500 - Loss: 0.357   lengthscale: 19.823   noise: 0.001
Iter 257/500 - Loss: 0.357   lengthscale: 19.859   noise: 0.001
Iter 258/500 - Loss: 0.357   lengthscale: 19.896   noise: 0.001
Iter 259/500 - Loss: 0.357   lengthscale: 19.933   noise: 0.001
Iter 260/500 - Loss: 0.356   lengthscale: 19.972   noise: 0.001
Iter 261/500 - Loss: 0.356   lengthscale: 20.012   noise: 0.001
Iter 262/500 - Loss: 0.356   lengthscale: 20.052   noise: 0.001
Iter 263/500 - Loss: 0.356   lengthscale: 20.094   noise: 0.002
Iter 264/500 - Loss: 0.356   lengthscale: 20.136   noise: 0.002
Iter 265/500 - Loss: 0.355   lengthscale: 20.179   noise: 0.002
Iter 266/500 - Loss: 0.355   lengthscale: 20.224   noise: 0.002
Iter 267/500 - Loss: 0.355   lengthscale: 20.269   noise: 0.002
Iter 268/500 - Loss: 0.355   lengthscale: 20.315   noise: 0.002
Iter 269/500 - Loss: 0.355   lengthscale: 20.362   noise: 0.002
Iter 270/500 - Loss: 0.354   lengthscale: 20.409   noise: 0.002
Iter 271/500 - Loss: 0.354   lengthscale: 20.458   noise: 0.002
Iter 272/500 - Loss: 0.354   lengthscale: 20.507   noise: 0.002
Iter 273/500 - Loss: 0.354   lengthscale: 20.557   noise: 0.002
Iter 274/500 - Loss: 0.354   lengthscale: 20.607   noise: 0.002
Iter 275/500 - Loss: 0.353   lengthscale: 20.658   noise: 0.002
Iter 276/500 - Loss: 0.353   lengthscale: 20.709   noise: 0.002
Iter 277/500 - Loss: 0.353   lengthscale: 20.761   noise: 0.002
Iter 278/500 - Loss: 0.353   lengthscale: 20.813   noise: 0.002
Iter 279/500 - Loss: 0.353   lengthscale: 20.866   noise: 0.002
Iter 280/500 - Loss: 0.352   lengthscale: 20.919   noise: 0.002
Iter 281/500 - Loss: 0.352   lengthscale: 20.972   noise: 0.002
Iter 282/500 - Loss: 0.352   lengthscale: 21.025   noise: 0.002
Iter 283/500 - Loss: 0.352   lengthscale: 21.078   noise: 0.002
Iter 284/500 - Loss: 0.352   lengthscale: 21.132   noise: 0.002
Iter 285/500 - Loss: 0.352   lengthscale: 21.185   noise: 0.002
Iter 286/500 - Loss: 0.351   lengthscale: 21.239   noise: 0.003
Iter 287/500 - Loss: 0.351   lengthscale: 21.292   noise: 0.003
Iter 288/500 - Loss: 0.351   lengthscale: 21.345   noise: 0.003
Iter 289/500 - Loss: 0.351   lengthscale: 21.398   noise: 0.003
Iter 290/500 - Loss: 0.351   lengthscale: 21.451   noise: 0.003
Iter 291/500 - Loss: 0.350   lengthscale: 21.504   noise: 0.003
Iter 292/500 - Loss: 0.350   lengthscale: 21.556   noise: 0.003
Iter 293/500 - Loss: 0.350   lengthscale: 21.608   noise: 0.003
Iter 294/500 - Loss: 0.350   lengthscale: 21.661   noise: 0.003
Iter 295/500 - Loss: 0.350   lengthscale: 21.712   noise: 0.003
Iter 296/500 - Loss: 0.350   lengthscale: 21.764   noise: 0.003
Iter 297/500 - Loss: 0.349   lengthscale: 21.815   noise: 0.003
Iter 298/500 - Loss: 0.349   lengthscale: 21.867   noise: 0.003
Iter 299/500 - Loss: 0.349   lengthscale: 21.918   noise: 0.003
Iter 300/500 - Loss: 0.349   lengthscale: 21.968   noise: 0.003
Iter 301/500 - Loss: 0.349   lengthscale: 22.019   noise: 0.003
Iter 302/500 - Loss: 0.349   lengthscale: 22.070   noise: 0.003
Iter 303/500 - Loss: 0.348   lengthscale: 22.120   noise: 0.003
Iter 304/500 - Loss: 0.348   lengthscale: 22.170   noise: 0.003
Iter 305/500 - Loss: 0.348   lengthscale: 22.220   noise: 0.003
Iter 306/500 - Loss: 0.348   lengthscale: 22.270   noise: 0.003
Iter 307/500 - Loss: 0.348   lengthscale: 22.320   noise: 0.003
Iter 308/500 - Loss: 0.348   lengthscale: 22.370   noise: 0.003
Iter 309/500 - Loss: 0.347   lengthscale: 22.419   noise: 0.004
Iter 310/500 - Loss: 0.347   lengthscale: 22.469   noise: 0.004
Iter 311/500 - Loss: 0.347   lengthscale: 22.519   noise: 0.004
Iter 312/500 - Loss: 0.347   lengthscale: 22.568   noise: 0.004
Iter 313/500 - Loss: 0.347   lengthscale: 22.618   noise: 0.004
Iter 314/500 - Loss: 0.347   lengthscale: 22.668   noise: 0.004
Iter 315/500 - Loss: 0.347   lengthscale: 22.717   noise: 0.004
Iter 316/500 - Loss: 0.346   lengthscale: 22.767   noise: 0.004
Iter 317/500 - Loss: 0.346   lengthscale: 22.817   noise: 0.004
Iter 318/500 - Loss: 0.346   lengthscale: 22.867   noise: 0.004
Iter 319/500 - Loss: 0.346   lengthscale: 22.916   noise: 0.004
Iter 320/500 - Loss: 0.346   lengthscale: 22.966   noise: 0.004
Iter 321/500 - Loss: 0.346   lengthscale: 23.016   noise: 0.004
Iter 322/500 - Loss: 0.346   lengthscale: 23.066   noise: 0.004
Iter 323/500 - Loss: 0.345   lengthscale: 23.116   noise: 0.004
Iter 324/500 - Loss: 0.345   lengthscale: 23.166   noise: 0.004
Iter 325/500 - Loss: 0.345   lengthscale: 23.217   noise: 0.004
Iter 326/500 - Loss: 0.345   lengthscale: 23.267   noise: 0.004
Iter 327/500 - Loss: 0.345   lengthscale: 23.318   noise: 0.004
Iter 328/500 - Loss: 0.345   lengthscale: 23.369   noise: 0.004
Iter 329/500 - Loss: 0.344   lengthscale: 23.420   noise: 0.005
Iter 330/500 - Loss: 0.344   lengthscale: 23.471   noise: 0.005
Iter 331/500 - Loss: 0.344   lengthscale: 23.523   noise: 0.005
Iter 332/500 - Loss: 0.344   lengthscale: 23.574   noise: 0.005
Iter 333/500 - Loss: 0.344   lengthscale: 23.626   noise: 0.005
Iter 334/500 - Loss: 0.344   lengthscale: 23.679   noise: 0.005
Iter 335/500 - Loss: 0.343   lengthscale: 23.731   noise: 0.005
Iter 336/500 - Loss: 0.343   lengthscale: 23.784   noise: 0.005
Iter 337/500 - Loss: 0.343   lengthscale: 23.837   noise: 0.005
Iter 338/500 - Loss: 0.343   lengthscale: 23.891   noise: 0.005
Iter 339/500 - Loss: 0.343   lengthscale: 23.945   noise: 0.005
Iter 340/500 - Loss: 0.343   lengthscale: 23.999   noise: 0.005
Iter 341/500 - Loss: 0.342   lengthscale: 24.053   noise: 0.005
Iter 342/500 - Loss: 0.342   lengthscale: 24.108   noise: 0.005
Iter 343/500 - Loss: 0.342   lengthscale: 24.164   noise: 0.005
Iter 344/500 - Loss: 0.342   lengthscale: 24.219   noise: 0.005
Iter 345/500 - Loss: 0.342   lengthscale: 24.276   noise: 0.005
Iter 346/500 - Loss: 0.341   lengthscale: 24.332   noise: 0.005
Iter 347/500 - Loss: 0.341   lengthscale: 24.389   noise: 0.006
Iter 348/500 - Loss: 0.341   lengthscale: 24.447   noise: 0.006
Iter 349/500 - Loss: 0.341   lengthscale: 24.505   noise: 0.006
Iter 350/500 - Loss: 0.341   lengthscale: 24.563   noise: 0.006
Iter 351/500 - Loss: 0.340   lengthscale: 24.622   noise: 0.006
Iter 352/500 - Loss: 0.340   lengthscale: 24.681   noise: 0.006
Iter 353/500 - Loss: 0.340   lengthscale: 24.741   noise: 0.006
Iter 354/500 - Loss: 0.340   lengthscale: 24.801   noise: 0.006
Iter 355/500 - Loss: 0.340   lengthscale: 24.862   noise: 0.006
Iter 356/500 - Loss: 0.339   lengthscale: 24.924   noise: 0.006
Iter 357/500 - Loss: 0.339   lengthscale: 24.985   noise: 0.006
Iter 358/500 - Loss: 0.339   lengthscale: 25.048   noise: 0.006
Iter 359/500 - Loss: 0.339   lengthscale: 25.111   noise: 0.006
Iter 360/500 - Loss: 0.338   lengthscale: 25.174   noise: 0.006
Iter 361/500 - Loss: 0.338   lengthscale: 25.239   noise: 0.006
Iter 362/500 - Loss: 0.338   lengthscale: 25.303   noise: 0.006
Iter 363/500 - Loss: 0.338   lengthscale: 25.369   noise: 0.007
Iter 364/500 - Loss: 0.337   lengthscale: 25.435   noise: 0.007
Iter 365/500 - Loss: 0.337   lengthscale: 25.501   noise: 0.007
Iter 366/500 - Loss: 0.337   lengthscale: 25.568   noise: 0.007
Iter 367/500 - Loss: 0.337   lengthscale: 25.636   noise: 0.007
Iter 368/500 - Loss: 0.336   lengthscale: 25.704   noise: 0.007
Iter 369/500 - Loss: 0.336   lengthscale: 25.773   noise: 0.007
Iter 370/500 - Loss: 0.336   lengthscale: 25.842   noise: 0.007
Iter 371/500 - Loss: 0.335   lengthscale: 25.913   noise: 0.007
Iter 372/500 - Loss: 0.335   lengthscale: 25.983   noise: 0.007
Iter 373/500 - Loss: 0.335   lengthscale: 26.055   noise: 0.007
Iter 374/500 - Loss: 0.335   lengthscale: 26.127   noise: 0.007
Iter 375/500 - Loss: 0.334   lengthscale: 26.199   noise: 0.007
Iter 376/500 - Loss: 0.334   lengthscale: 26.273   noise: 0.007
Iter 377/500 - Loss: 0.334   lengthscale: 26.347   noise: 0.007
Iter 378/500 - Loss: 0.333   lengthscale: 26.421   noise: 0.008
Iter 379/500 - Loss: 0.333   lengthscale: 26.496   noise: 0.008
Iter 380/500 - Loss: 0.333   lengthscale: 26.572   noise: 0.008
Iter 381/500 - Loss: 0.332   lengthscale: 26.649   noise: 0.008
Iter 382/500 - Loss: 0.332   lengthscale: 26.726   noise: 0.008
Iter 383/500 - Loss: 0.332   lengthscale: 26.804   noise: 0.008
Iter 384/500 - Loss: 0.331   lengthscale: 26.882   noise: 0.008
Iter 385/500 - Loss: 0.331   lengthscale: 26.961   noise: 0.008
Iter 386/500 - Loss: 0.331   lengthscale: 27.041   noise: 0.008
Iter 387/500 - Loss: 0.330   lengthscale: 27.121   noise: 0.008
Iter 388/500 - Loss: 0.330   lengthscale: 27.202   noise: 0.008
Iter 389/500 - Loss: 0.330   lengthscale: 27.284   noise: 0.008
Iter 390/500 - Loss: 0.329   lengthscale: 27.366   noise: 0.008
Iter 391/500 - Loss: 0.329   lengthscale: 27.449   noise: 0.008
Iter 392/500 - Loss: 0.328   lengthscale: 27.532   noise: 0.008
Iter 393/500 - Loss: 0.328   lengthscale: 27.616   noise: 0.009
Iter 394/500 - Loss: 0.328   lengthscale: 27.701   noise: 0.009
Iter 395/500 - Loss: 0.327   lengthscale: 27.786   noise: 0.009
Iter 396/500 - Loss: 0.327   lengthscale: 27.872   noise: 0.009
Iter 397/500 - Loss: 0.326   lengthscale: 27.959   noise: 0.009
Iter 398/500 - Loss: 0.326   lengthscale: 28.046   noise: 0.009
Iter 399/500 - Loss: 0.326   lengthscale: 28.133   noise: 0.009
Iter 400/500 - Loss: 0.325   lengthscale: 28.222   noise: 0.009
Iter 401/500 - Loss: 0.325   lengthscale: 28.311   noise: 0.009
Iter 402/500 - Loss: 0.324   lengthscale: 28.400   noise: 0.009
Iter 403/500 - Loss: 0.324   lengthscale: 28.490   noise: 0.009
Iter 404/500 - Loss: 0.323   lengthscale: 28.580   noise: 0.009
Iter 405/500 - Loss: 0.323   lengthscale: 28.672   noise: 0.009
Iter 406/500 - Loss: 0.323   lengthscale: 28.763   noise: 0.009
Iter 407/500 - Loss: 0.322   lengthscale: 28.855   noise: 0.009
Iter 408/500 - Loss: 0.322   lengthscale: 28.948   noise: 0.010
Iter 409/500 - Loss: 0.321   lengthscale: 29.041   noise: 0.010
Iter 410/500 - Loss: 0.321   lengthscale: 29.135   noise: 0.010
Iter 411/500 - Loss: 0.320   lengthscale: 29.229   noise: 0.010
Iter 412/500 - Loss: 0.320   lengthscale: 29.324   noise: 0.010
Iter 413/500 - Loss: 0.319   lengthscale: 29.419   noise: 0.010
Iter 414/500 - Loss: 0.319   lengthscale: 29.515   noise: 0.010
Iter 415/500 - Loss: 0.318   lengthscale: 29.611   noise: 0.010
Iter 416/500 - Loss: 0.318   lengthscale: 29.708   noise: 0.010
Iter 417/500 - Loss: 0.317   lengthscale: 29.805   noise: 0.010
Iter 418/500 - Loss: 0.317   lengthscale: 29.903   noise: 0.010
Iter 419/500 - Loss: 0.316   lengthscale: 30.001   noise: 0.010
Iter 420/500 - Loss: 0.316   lengthscale: 30.099   noise: 0.010
Iter 421/500 - Loss: 0.315   lengthscale: 30.198   noise: 0.010
Iter 422/500 - Loss: 0.315   lengthscale: 30.297   noise: 0.010
Iter 423/500 - Loss: 0.314   lengthscale: 30.397   noise: 0.010
Iter 424/500 - Loss: 0.314   lengthscale: 30.497   noise: 0.010
Iter 425/500 - Loss: 0.313   lengthscale: 30.598   noise: 0.011
Iter 426/500 - Loss: 0.313   lengthscale: 30.699   noise: 0.011
Iter 427/500 - Loss: 0.312   lengthscale: 30.800   noise: 0.011
Iter 428/500 - Loss: 0.312   lengthscale: 30.901   noise: 0.011
Iter 429/500 - Loss: 0.311   lengthscale: 31.003   noise: 0.011
Iter 430/500 - Loss: 0.311   lengthscale: 31.106   noise: 0.011
Iter 431/500 - Loss: 0.310   lengthscale: 31.209   noise: 0.011
Iter 432/500 - Loss: 0.310   lengthscale: 31.312   noise: 0.011
Iter 433/500 - Loss: 0.309   lengthscale: 31.415   noise: 0.011
Iter 434/500 - Loss: 0.309   lengthscale: 31.519   noise: 0.011
Iter 435/500 - Loss: 0.308   lengthscale: 31.623   noise: 0.011
Iter 436/500 - Loss: 0.308   lengthscale: 31.727   noise: 0.011
Iter 437/500 - Loss: 0.307   lengthscale: 31.832   noise: 0.011
Iter 438/500 - Loss: 0.306   lengthscale: 31.936   noise: 0.011
Iter 439/500 - Loss: 0.306   lengthscale: 32.042   noise: 0.011
Iter 440/500 - Loss: 0.305   lengthscale: 32.147   noise: 0.011
Iter 441/500 - Loss: 0.305   lengthscale: 32.253   noise: 0.011
Iter 442/500 - Loss: 0.304   lengthscale: 32.359   noise: 0.011
Iter 443/500 - Loss: 0.304   lengthscale: 32.465   noise: 0.011
Iter 444/500 - Loss: 0.303   lengthscale: 32.571   noise: 0.012
Iter 445/500 - Loss: 0.303   lengthscale: 32.678   noise: 0.012
Iter 446/500 - Loss: 0.302   lengthscale: 32.785   noise: 0.012
Iter 447/500 - Loss: 0.301   lengthscale: 32.892   noise: 0.012
Iter 448/500 - Loss: 0.301   lengthscale: 32.999   noise: 0.012
Iter 449/500 - Loss: 0.300   lengthscale: 33.106   noise: 0.012
Iter 450/500 - Loss: 0.300   lengthscale: 33.214   noise: 0.012
Iter 451/500 - Loss: 0.299   lengthscale: 33.322   noise: 0.012
Iter 452/500 - Loss: 0.299   lengthscale: 33.430   noise: 0.012
Iter 453/500 - Loss: 0.298   lengthscale: 33.538   noise: 0.012
Iter 454/500 - Loss: 0.298   lengthscale: 33.646   noise: 0.012
Iter 455/500 - Loss: 0.297   lengthscale: 33.755   noise: 0.012
Iter 456/500 - Loss: 0.296   lengthscale: 33.863   noise: 0.012
Iter 457/500 - Loss: 0.296   lengthscale: 33.972   noise: 0.012
Iter 458/500 - Loss: 0.295   lengthscale: 34.081   noise: 0.012
Iter 459/500 - Loss: 0.295   lengthscale: 34.190   noise: 0.012
Iter 460/500 - Loss: 0.294   lengthscale: 34.299   noise: 0.012
Iter 461/500 - Loss: 0.294   lengthscale: 34.408   noise: 0.012
Iter 462/500 - Loss: 0.293   lengthscale: 34.517   noise: 0.012
Iter 463/500 - Loss: 0.292   lengthscale: 34.626   noise: 0.012
Iter 464/500 - Loss: 0.292   lengthscale: 34.736   noise: 0.012
Iter 465/500 - Loss: 0.291   lengthscale: 34.845   noise: 0.012
Iter 466/500 - Loss: 0.291   lengthscale: 34.955   noise: 0.012
Iter 467/500 - Loss: 0.290   lengthscale: 35.064   noise: 0.012
Iter 468/500 - Loss: 0.290   lengthscale: 35.174   noise: 0.012
Iter 469/500 - Loss: 0.289   lengthscale: 35.284   noise: 0.013
Iter 470/500 - Loss: 0.289   lengthscale: 35.394   noise: 0.013
Iter 471/500 - Loss: 0.288   lengthscale: 35.503   noise: 0.013
Iter 472/500 - Loss: 0.287   lengthscale: 35.613   noise: 0.013
Iter 473/500 - Loss: 0.287   lengthscale: 35.723   noise: 0.013
Iter 474/500 - Loss: 0.286   lengthscale: 35.833   noise: 0.013
Iter 475/500 - Loss: 0.286   lengthscale: 35.943   noise: 0.013
Iter 476/500 - Loss: 0.285   lengthscale: 36.052   noise: 0.013
Iter 477/500 - Loss: 0.285   lengthscale: 36.162   noise: 0.013
Iter 478/500 - Loss: 0.284   lengthscale: 36.272   noise: 0.013
Iter 479/500 - Loss: 0.284   lengthscale: 36.382   noise: 0.013
Iter 480/500 - Loss: 0.283   lengthscale: 36.491   noise: 0.013
Iter 481/500 - Loss: 0.282   lengthscale: 36.601   noise: 0.013
Iter 482/500 - Loss: 0.282   lengthscale: 36.711   noise: 0.013
Iter 483/500 - Loss: 0.281   lengthscale: 36.821   noise: 0.013
Iter 484/500 - Loss: 0.281   lengthscale: 36.930   noise: 0.013
Iter 485/500 - Loss: 0.280   lengthscale: 37.040   noise: 0.013
Iter 486/500 - Loss: 0.280   lengthscale: 37.149   noise: 0.013
Iter 487/500 - Loss: 0.279   lengthscale: 37.259   noise: 0.013
Iter 488/500 - Loss: 0.279   lengthscale: 37.368   noise: 0.013
Iter 489/500 - Loss: 0.278   lengthscale: 37.477   noise: 0.013
Iter 490/500 - Loss: 0.278   lengthscale: 37.586   noise: 0.013
Iter 491/500 - Loss: 0.277   lengthscale: 37.695   noise: 0.013
Iter 492/500 - Loss: 0.276   lengthscale: 37.804   noise: 0.013
Iter 493/500 - Loss: 0.276   lengthscale: 37.913   noise: 0.013
Iter 494/500 - Loss: 0.275   lengthscale: 38.022   noise: 0.013
Iter 495/500 - Loss: 0.275   lengthscale: 38.131   noise: 0.013
Iter 496/500 - Loss: 0.274   lengthscale: 38.239   noise: 0.013
Iter 497/500 - Loss: 0.274   lengthscale: 38.348   noise: 0.013
Iter 498/500 - Loss: 0.273   lengthscale: 38.456   noise: 0.013
Iter 499/500 - Loss: 0.273   lengthscale: 38.564   noise: 0.013
Iter 500/500 - Loss: 0.272   lengthscale: 38.672   noise: 0.013

Now, we test predictions. For simplicity, we will assume a fixed variance of \(0.01.\)

[8]:
# Get into evaluation (predictive posterior) mode
model.eval()
likelihood.eval()

# Test points are regularly spaced along [0,1]
# Make predictions by feeding model through likelihood
with torch.no_grad(), gpytorch.settings.fast_pred_var():
    test_x = torch.linspace(0, 1, 51)
    test_x_distributional = torch.stack((test_x, (1e-2 * torch.ones_like(test_x)).log()), dim=1)
    observed_pred = likelihood(model(test_x_distributional))

with torch.no_grad():
    # Initialize plot
    f, ax = plt.subplots(1, 1, figsize=(8, 3))

    # Get upper and lower confidence bounds
    lower, upper = observed_pred.confidence_region()
    # Plot training data as black stars
    ax.errorbar(train_x_mean.numpy(), train_y.numpy(), xerr=train_x_stdv, fmt='k*')
    # Plot predictive means as blue line
    ax.plot(test_x.numpy(), observed_pred.mean.numpy(), 'b')
    # Shade between the lower and upper confidence bounds
    ax.fill_between(test_x.numpy(), lower.numpy(), upper.numpy(), alpha=0.5)
    ax.set_ylim([-3, 3])
    ax.legend(['Observed Data', 'Mean', 'Confidence'])
../../_images/examples_01_Exact_GPs_GP_Regression_DistributionalKernel_11_0.png

As a final note, we’ve made it very easy to extend the distributional kernel class by exposing a generic DistributionalInputKernel class that takes as input any distance function over probability distributions.

[ ]: