{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# GP Regression with Uncertain Inputs\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this notebook, we're going to demonstrate one way of dealing with uncertainty in our training data. Let's say that we're collecting training data that models the following function.\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "y &= \\sin(2\\pi x) + \\epsilon \\\\\n",
    "  \\epsilon &\\sim \\mathcal{N}(0, 0.2) \n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "However, now assume that we're a bit uncertain about our features. In particular, we're going to assume that every `x_i` value is not a point but a distribution instead. E.g.\n",
    "\n",
    "$$ x_i \\sim \\mathcal{N}(\\mu_i, \\sigma_i). $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Using a distributional kernel to deal with uncertain inputs\n",
    "\n",
    "Rather than using a variational method (see the GP Regression with Uncertian Inputs tutorial in the variational examples), if we explicitly know the type of uncertainty in our inputs we can pass that into our kernel.\n",
    "\n",
    "More specifically, assuming Gaussian inputs, we will compute the symmetrized KL divergence between the Gaussian inputs.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import math\n",
    "import torch\n",
    "import tqdm\n",
    "import gpytorch\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Training data is 100 points in [0,1] inclusive regularly spaced\n",
    "train_x_mean = torch.linspace(0, 1, 20)\n",
    "# We'll assume the variance shrinks the closer we get to 1\n",
    "train_x_stdv = torch.linspace(0.03, 0.01, 20)\n",
    "\n",
    "# True function is sin(2*pi*x) with Gaussian noise\n",
    "train_y = torch.sin(train_x_mean * (2 * math.pi)) + torch.randn(train_x_mean.size()) * 0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "To effectively pass in the training distributional data, we will need to stack the mean and log variances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "train_x_distributional = torch.stack((train_x_mean, (train_x_stdv**2).log()), dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<matplotlib.legend.Legend at 0x7f0646c7ceb0>"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": "<Figure size 800x300 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqsAAAESCAYAAADaNpzRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApS0lEQVR4nO3df3RU9Z3/8dcNyQz5bQ0kE0zCj275qRKdHCAIFbM1COsv6jZ03UWwFKQct2KOG8H+EGrbrKiU4oJRN0oBG9wWcfsDWThrQAIc6rCkVaHUU8FESSRkS0JiSSC53z/4ZjYhIWSSuTN3Zp6Pc+aEudwf75yPwVc+87nva5imaQoAAACwoahgFwAAAABcCWEVAAAAtkVYBQAAgG0RVgEAAGBbhFUAAADYFmEVAAAAtkVYBQAAgG1FB7sAf2tvb9epU6eUmJgowzCCXQ4AAAAuY5qmzp07p2HDhikqqve507ALq6dOnVJmZmawywAAAMBVVFdXKyMjo9d9wi6sJiYmSrr0zSclJQW5GgAAAFyusbFRmZmZ3tzWm7ALqx0f/SclJRFWAQAAbKwvSza5wQoAAAC2RVgFAACAbRFWAQAAYFtht2YVAACElra2Nl24cCHYZcDPYmJiNGjQoAGfh7AKhDmPx6OioiKtXr1aOTk5wS4HALxM01Rtba3Onj0b7FJgkWuuuUYul2tAve8Jq0CY27Rpk8rLy7V582bCKgBb6QiqqampiouL42E+YcQ0TX3++ec6ffq0JCk9Pb3f5yKsAkHU3NxsyXmrqqpUX18vwzC0detWSVJZWZkKCgpkmqZSUlKUlZXl9+vGx8f7/ZwAwlNbW5s3qKakpAS7HFggNjZWknT69Gmlpqb2e0kAYRUIooSEhIBdq66uTtOmTbP0GqZpWnp+AOGjY41qXFxckCuBlTrG98KFC/0Oq3QDAAAAQcNH/+HNH+PLzCoQRE1NTZadu7KysseZ1IqKCmVnZ1t2XQAA/ImwCgSRlWs8O9YKRUVFqb293fs1NjaWtaUAgJDBMgAgTKWmpsrlcsntdqukpERut1sul0upqanBLg0A0MmMGTO0bNmyYJdhW4RVIExlZGTo5MmTOnTokB566CEdOnRIJ0+eVEZGRrBLAwC/8ng8ysvLk8fjsfQ6hmH0+lqwYEG/zvvGG2/oqaeeGlBtCxYs8NYRExOjtLQ03X777XrllVfU3t7u07k2btyoa665ZkD1+BNhFQhjTqfTu7jdMAw5nc4gVwQA/te5n7SVampqvK+1a9cqKSmpy7af/vSnXfbv61O5rr32WiUmJg64vjvuuEM1NTU6efKk3nrrLd1222165JFHdOedd+rixYsDPn+wEFYBAEDIaG5uVnNzs44dO6aKigrt37+/Sz/p/fv3q6KiQseOHfPu6y8ul8v7Sk5OlmEY3vfnz5/XNddco//4j//QjBkzNHjwYG3ZskX19fX6h3/4B2VkZCguLk433HCDysrKupz38mUAI0aM0I9//GN94xvfUGJiorKysvTSSy9dtT6n0ymXy6XrrrtON998s5544gn953/+p9566y1t3LjRu9+aNWt0ww03KD4+XpmZmVq6dKn3ht89e/bowQcfVENDg3emduXKlZKkLVu2KCcnR4mJiXK5XLr//vu9Tf+tRFgFAAAhIyEhQQkJCRo/frymT5+uadOmqa6uTtL/9ZOePn26xo8f7903kB5//HF9+9vf1rFjxzRz5kydP39ebrdbv/nNb/T+++9r8eLFmjdvng4dOtTreZ577jnl5OToyJEjWrp0qb71rW/pj3/8o8/15OXlaeLEiXrjjTe826KiorRu3Tq9//77+tnPfqa3335bRUVFkqSpU6d2mzV+7LHHJEmtra166qmn9Pvf/15vvvmmTpw40e+lD76gGwAAAICfLFu2TF/96le7bOsIe5L0z//8z9q5c6d+8YtfaPLkyVc8z+zZs7V06VJJlwLwT37yE+3Zs0djx471uaaxY8fqD3/4Q5caO4wcOVJPPfWUvvWtb2nDhg1yOBxdZo07+8Y3vuH986hRo7Ru3TpNmjRJTU1Nlv5SwMwqAAAIGU1NTd5XRUVFj/tUVFR02S+QcnJyurxva2vTj370I914441KSUlRQkKCdu3apaqqql7Pc+ONN3r/3BEc+/uRu2maXZrzl5eX6/bbb9d1112nxMREPfDAA6qvr7/qkokjR47onnvu0fDhw5WYmKgZM2ZI0lW/l4EirAIAgJARHx/vfXXuJ935a0c/6Y5XoOvr7LnnntNPfvITFRUV6e2331ZlZaVmzpyp1tbWXs8TExPT5b1hGD7f1d/h2LFjGjlypCTp448/1uzZs3X99ddr27ZtOnz4sNavXy+p9xvCmpublZ+fr4SEBG3ZskXvvvuutm/fLklX/V4GirAKwK8C1UIGAEKhn/S+fft0zz336J/+6Z80ceJEjRo1Sh9++GHArv/222/rvffe03333Sfp0r/RFy9e1HPPPacpU6Zo9OjROnXqVJdjHA6H2traumz74x//qDNnzuhf//VfNX36dI0dOzYgN1dJrFkF4GedW8hc/nEYAPhTRz9ph8MhwzC0ePFitba22qpN39/8zd9o27ZtOnDggL7whS9ozZo1qq2t1bhx4/x+rZaWFtXW1qqtrU2fffaZdu7cqeLiYt1555164IEHJElf/OIXdfHiRT3//PO66667tH//fpWUlHQ5z4gRI9TU1KT//u//1sSJExUXF6esrCw5HA49//zzWrJkid5///0B94btK8IqEGH82calQ1VVlerr62UYRpcWMgUFBTJNUykpKcrKyvLb9XhcLIAOnYOpHftJf+9739OJEyc0c+ZMxcXFafHixbr33nvV0NDg92vt3LlT6enpio6O1he+8AVNnDhR69at0/z5871LJLKzs7VmzRo9/fTTWrFihb785S+ruLjYG2alSx0BlixZorlz56q+vl5PPvmkVq5cqY0bN+qJJ57QunXrdPPNN+vZZ5/V3Xff7ffv43KGaZqm5VcJoMbGRiUnJ6uhoUFJSUnBLgewnc6L7ENVmP2zBUSk8+fP68SJExo5cqQGDx4c7HJgkSuNsy95jTWrAAAAsC2WAQARxqo2LpWVlZo2bVq37RUVFcrOzrbkmgCA8EdYBSKMVes9O7eQaW9v937taCEDAEB/sAwAEO2W/CEUWsgAAEIPM6uAaLfkD6HQQgaA/fS30T1Cgz/Gl7AK27KixVJngW63dLlw/Gjc7i1kANiHw+FQVFSUTp06paFDh3p/0UV4ME1Tra2tqqurU1RUlBwOR7/PResq2Fa4/6MVZj96AOCz1tZW1dTU6PPPPw92KbBIXFyc0tPTu4VVX/IaM6sAACAoHA6HsrKydPHixW6P90ToGzRokKKjowc8+URYhW1Z1WKpM9otAUBwGYahmJgYxcTEBLsU2BRhFbYViDWdtFsCAMDeLG1d9c477+iuu+7SsGHDZBiG3nzzzases3fvXrndbg0ePFijRo1SSUmJlSUiwtFuCQAAe7N0ZrW5uVkTJ07Ugw8+qPvuu++q+584cUKzZ8/WokWLtGXLFu3fv19Lly7V0KFD+3Q84CvaLQEAYG+WhtVZs2Zp1qxZfd6/pKREWVlZWrt2rSRp3Lhx8ng8evbZZ68YVltaWtTS0uJ939jYOKCaEXlotwQAgH3Z6glWBw8eVH5+fpdtM2fOlMfj0YULF3o8pri4WMnJyd5XZmZmIEoFAABAANgqrNbW1iotLa3LtrS0NF28eFFnzpzp8ZgVK1aooaHB+6qurg5EqQAAAAgA23UDuLwXV0fj9Cv16HI6nXxsCwAAEKZsNbPqcrlUW1vbZdvp06cVHR2tlJSUIFUFAACAYLFVWM3NzdXu3bu7bNu1a5dycnJoFgwAABCBLA2rTU1NqqysVGVlpaRLrakqKytVVVUl6dJ60wceeMC7/5IlS/Txxx+rsLBQx44d0yuvvKLS0lI99thjVpYJAAAAm7J0zarH49Ftt93mfV9YWChJmj9/vjZu3KiamhpvcJWkkSNHaseOHXr00Ue1fv16DRs2TOvWraPHKgAAQIQyzI47mMJEY2OjkpOT1dDQoKSkpGCXAwAAgMv4ktdstWYVAAAA6IywCgAAANsirAIAAMC2CKsAAACwLcIqAAAAbIuwCgAAANsirAIAAMC2CKsAAACwLcIqAAAAbIuwCgAAANsirAIAAMC2CKsAAACwLcIqAAAAbIuwCgAAANsirAIAAMC2CKsAAACwLcIqgJDl8XiUl5cnj8cT7FIAABYhrAIIWZs2bVJ5ebk2b94c7FIAABaJDnYBAMJfc3Oz385VVVWl+vp6GYahrVu3SpLKyspUUFAg0zSVkpKirKwsv10vPj7eb+cCAPiOsArAcgkJCZaev66uTtOmTbPk3KZpWnJeAEDfsAwAAAAAtsXMKgDLNTU1+fV8lZWVPc6kVlRUKDs726/XAgAEF2EVgOX8ve4zNjZWkhQVFaX29nbv19jYWEvXmHo8HhUVFWn16tXKycmx7DoAgP/DMgAAISc1NVUul0tut1slJSVyu91yuVxKTU219Lp0HwCAwDPMMLt7oLGxUcnJyWpoaFBSUlKwywF6xAzdwLW0tMjhcMgwDJmmqdbWVjmdTknWdR+YM2eO6urqNHToUG3fvt2v3QfoOgAgkviS11gGAARB5xk6wmr/dARTSTIMo8v7UOw+EGbzBgDgN4RVoBN/zshdLtD9QTtj1g4AEKoIq0AnVs/IXc7K/qCdRdqsHd0HACB8EFYBhJ1w6T4AAAhAWN2wYYOeeeYZ1dTUaMKECVq7dq2mT5/e47579uzRbbfd1m37sWPHNHbsWKtLBfw+I3c5ZuhCU0f3gczMTC1cuFClpaWqrq62vPsAAMDisPr6669r2bJl2rBhg2655Ra9+OKLmjVrlo4ePdrr2rzjx493uTNs6NChVpYJeFk9S8YMXWjKyMjQyZMnvd0HFi9e3KX7AADAOpb2WV2zZo0WLlyob37zmxo3bpzWrl2rzMxMvfDCC70e1zGL0fEaNGiQlWUCAROs/qAYOKfTKcMwJHXvPgAAsI5lM6utra06fPiwli9f3mV7fn6+Dhw40OuxN910k86fP6/x48fru9/9bo9LAzq0tLSopaXF+76xsXFghQMWYoYOAADfWDazeubMGbW1tSktLa3L9rS0NNXW1vZ4THp6ul566SVt27ZNb7zxhsaMGaO//du/1TvvvHPF6xQXFys5Odn7yszM9Ov3AfgbM3QAAPSd5TdYdfxPuYNpmt22dRgzZozGjBnjfZ+bm6vq6mo9++yz+vKXv9zjMStWrFBhYaH3fWNjI4EVAAAgTFg2szpkyBANGjSo2yzq6dOnu8229mbKlCn68MMPr/j3TqdTSUlJXV4AAAAID5aFVYfDIbfbrd27d3fZvnv3bk2dOrXP5zly5IjS09P9XR4AAABCgKXLAAoLCzVv3jzl5OQoNzdXL730kqqqqrRkyRJJlz7C//TTT7Vp0yZJ0tq1azVixAhNmDBBra2t2rJli7Zt26Zt27ZZWSYAAABsytKwOnfuXNXX1+sHP/iBampqdP3112vHjh0aPny4JKmmpkZVVVXe/VtbW/XYY4/p008/VWxsrCZMmKDf/va3mj17tpVlAgAAwKYMM8weGt7Y2Kjk5GQ1NDSwfhUAAMCGfMlrlj4UAAAAABgIwioAAABsi7AKAAAA2yKsAgAAwLYIqwAAALAtwioAAABsi7AKAAAA2yKswrY8Ho/y8vLk8XiCXQoAAAgSwipsa9OmTSovL9fmzZuDXQoAAAgSSx+3ivDS3Nxs+TWqqqpUX18vwzC0detWSVJZWZkKCgpkmqZSUlKUlZVlaQ3x8fGWnh8AAPQdYRV9lpCQEJTr1tXVadq0aQG7Xpg9gRgAgJDGMgAAAADYFjOr6LOmpqaAXKeysrLHmdSKigplZ2cHpAYAAGAPhFX0WaDWcsbGxkqSoqKi1N7e7v0aGxvLelIAACIMywBgO6mpqXK5XHK73SopKZHb7ZbL5VJqamqwSwMAAAFmmGF2N0ljY6OSk5PV0NCgpKSkYJeDfmppaZHD4ZBhGDJNU62trXI6ncEuCwAA+IEveY1lALClzsHUMAyCKgAAEYplAAAAALAtwioAAABsi7AKAAAA2yKsAgAAwLYIqwAAALAtwir6zOPxKC8vTx6PJ9ilAACACEFYRZ9t2rRJ5eXl2rx5c7BLAQAAEYI+qyGqubk5INepqqpSfX29DMPQ1q1bJUllZWUqKCiQaZpKSUlRVlZWQGrhUasAAEQewmqISkhICNq16+rqNG3atIBfN8wetgYAAPqAZQAAAACwLWZWQ1RTU1PArlVZWdnjTGpFRYWys7MDVgcAAIg8hNUQFcj1m7GxsZKkqKgotbe3e7/GxsayjhSwiMfjUVFRkVavXq2cnJxglwMAQcMyAFxVamqqXC6X3G63SkpK5Ha75XK5lJqaGuzSgLBF9w0AuMQwLb5rZcOGDXrmmWdUU1OjCRMmaO3atZo+ffoV99+7d68KCwv1wQcfaNiwYSoqKtKSJUv6fL3GxkYlJyeroaFBSUlJ/vgWIKmlpUUOh0OGYcg0TbW2tsrpdAa7LMA2/NGho3P3jTlz5qiurk5Dhw7V9u3b/dp9g09EAASbL3nN0mUAr7/+upYtW6YNGzbolltu0YsvvqhZs2bp6NGjPf6De+LECc2ePVuLFi3Sli1btH//fi1dulRDhw7VfffdZ2WpuIrOwdQwDIIqcBmrOnRY0X2DzhoAQomlM6uTJ0/WzTffrBdeeMG7bdy4cbr33ntVXFzcbf/HH39cv/rVr3Ts2DHvtiVLluj3v/+9Dh482KdrMrMKIBgMwwh2CX1GWAUQbLaYWW1tbdXhw4e1fPnyLtvz8/N14MCBHo85ePCg8vPzu2ybOXOmSktLdeHCBcXExHQ7pqWlRS0tLd73jY2NfqgeAHzjrw4ddN8AgK4sC6tnzpxRW1ub0tLSumxPS0tTbW1tj8fU1tb2uP/Fixd15swZpaendzumuLhYq1at8l/hANAP/loHSvcNAOjK8m4Al380Zppmrx+X9bR/T9s7rFixQg0NDd5XdXX1ACsGgOCh+wYQ3jwej/Ly8uTxeIJdSsiwbGZ1yJAhGjRoULdZ1NOnT3ebPe3gcrl63D86OlopKSk9HuN0OrnZB0DYyMjI0MmTJ73dNxYvXkz3DSCMdG5LRw/lvrEsrDocDrndbu3evVtz5szxbt+9e7fuueeeHo/Jzc3Vr3/96y7bdu3apZycnB7XqwJAOKL7BhB6emtf17kt3datWyVJZWVlKigo6FNbukhfAmRp66rCwkLNmzdPOTk5ys3N1UsvvaSqqipv39QVK1bo008/1aZNmyRduvP/3/7t31RYWKhFixbp4MGDKi0tVVlZmZVlAgAADIiv7et8aUsX6R08LA2rc+fOVX19vX7wgx+opqZG119/vXbs2KHhw4dLkmpqalRVVeXdf+TIkdqxY4ceffRRrV+/XsOGDdO6devosQoAABChLH+CVaDRZxUAAATa1Z5iN5C2dOG4DMCXvGZ5NwAAAIBwFx8f3+urc1u6zl872tL19hqIcOg+QFgFAACwWLDa0nXuPhCqWAYQ4jwej4qKirR69WpaYAAAYGMtLS3etnSmaQ6oLV1fuw/MmTNHdXV1Gjp0qLZv395r94FALjewxeNWERj0awMAIDT4sy2dFd0H7Dp/SVj1k6strPangfZr86dwXPQNAADsg7DqJ77+huNvvvRr8ye7/hYGAEA4a2pq6vXvB9J9wG4IqwAAACHmap9sdu4+0N7e7v3a0X0glBBW/eRqv+H4Wzj9xgQAAPyro/tAZmamFi5cqNLSUlVXV1vefcAKhFU/CfRvKeH0GxMAAPCvjIwMnTx50tt9YPHixQPqPhBM9FkNUcHq1wYAAEKD0+mUYRiSBt59IJjosxrC/NmvDQAAIFDosxoh/NmvDQAAwI5YBgAAAADbIqwCAADAtgirAAAAsC3CKgAAAGyLsAoAAADbIqwCAADAtgirAAAAsC3CKgBAHo9HeXl58ng8wS4FALogrAIAtGnTJpWXl2vz5s3BLgUAuuAJVgAQgpqbmwd8jqqqKtXX18swDG3dulWSVFZWpoKCApmmqZSUFGVlZfX7/PHx8QOuEQAIqwAQghISEiw5b11dnaZNm+aXc5mm6ZfzAIhsLAMAAACAbTGzCgAhqKmpyS/nqays7HEmtaKiQtnZ2X65BgAMBGEVAEKQv9aDxsbGSpKioqLU3t7u/RobG8uaUwC2wDIAAIhgqampcrlccrvdKikpkdvtlsvlUmpqarBLAwBJkmGG2Qr4xsZGJScnq6GhQUlJScEuBwBsr6WlRQ6HQ4ZhyDRNtba2yul0BrssAGHMl7zGMgAAiHCdg6lhGARVALbCMgAAAADYlqVh9S9/+YvmzZun5ORkJScna968eTp79myvxyxYsECGYXR5TZkyxcoyAQAAYFOWLgO4//779cknn2jnzp2SpMWLF2vevHn69a9/3etxd9xxh1599VXve4fDYWWZAAAAsCnLZlaPHTumnTt36t///d+Vm5ur3Nxcvfzyy/rNb36j48eP93qs0+mUy+Xyvq699lqrygQAwG88Ho/y8vLk8XiCXQoQNiwLqwcPHlRycrImT57s3TZlyhQlJyfrwIEDvR67Z88epaamavTo0Vq0aJFOnz59xX1bWlrU2NjY5QUAQDBs2rRJ5eXl2rx5c7BLAcKGZcsAamtre+zTl5qaqtra2iseN2vWLH3ta1/T8OHDdeLECX3ve99TXl6eDh8+3OMdqsXFxVq1apVfawcARKbm5mafj6mqqlJ9fb0Mw9DWrVslSWVlZSooKJBpmkpJSVFWVpZP5+SBDMD/8Tmsrly58qrh8N1335V0qQXK5UzT7HF7h7lz53r/fP311ysnJ0fDhw/Xb3/7W331q1/ttv+KFStUWFjofd/Y2KjMzMyrfh8AAFwuISHBL+epq6vr8TG2fRVmLdCBAfE5rD788MP6+te/3us+I0aM0B/+8Ad99tln3f6urq5OaWlpfb5eenq6hg8frg8//LDHv3c6nfQEBAAACFM+h9UhQ4ZoyJAhV90vNzdXDQ0N+t3vfqdJkyZJkg4dOqSGhgZNnTq1z9err69XdXW10tPTfS0VAACfNDU19eu4ysrKHmdSKyoqlJ2dPcCqgMhm2ZrVcePG6Y477tCiRYv04osvSrrUuurOO+/UmDFjvPuNHTtWxcXFmjNnjpqamrRy5Urdd999Sk9P18mTJ/XEE09oyJAhmjNnjlWlAgAgqf9rRWNjYyVJUVFRam9v936NjY1l/SkwQJY+FOC1117TDTfcoPz8fOXn5+vGG2/sdofk8ePH1dDQIEkaNGiQ3nvvPd1zzz0aPXq05s+fr9GjR+vgwYNKTEy0slQAAPotNTVVLpdLbrdbJSUlcrvdcrlcPd5oDMA3hhlmq7gbGxuVnJyshoYGJSUlBbscAECEaGlpkcPhkGEYMk1Tra2t3FMBXIEvec3SJ1gBABApOgdTwzAIqoCfWLoMAAAAABgIwioAAABsi7AKAAAimsfjUV5enjweT7BLQQ8IqwAAIKJt2rRJ5eXl3ToWwR64wQoAAISV5ubmq+5TVVWl+vp6GYahrVu3SpLKyspUUFAg0zSVkpKirKysXs9BD93AIKwCAICwkpCQ0K/j6urqenwS2ZWEWfdP22IZAAAAAGyLmVUAABBWmpqa+rRfZWVljzOpFRUVys7O9nNV6C/CKgAACCt9XUsaGxsrSYqKilJ7e7v3a2xsLOtRbYRlAAAAICKlpqbK5XLJ7XarpKREbrdbLpdLqampwS4NnRhmmK0O9uVZswAAILK1tLTI4XDIMAyZpqnW1lYelRsAvuQ1ZlYBAGGNhu/ojdPplGEYkiTDMAiqNkRYBQCENRq+A6GNG6wAALbUl8buV+KPhu894aYbIPAIqwAAW+pvY/cr8bXhe0/C7DYPICSwDAAAAAC2xcwqAMCW+trY/Upo+O5fHo9HRUVFWr16tXJycoJdDiIIYRUAYEsDXR9Kw3f/6nyjGmEVgURYBQCEpY6G75mZmVq4cKFKS0tVXV0d8Q3ffblxzZ83qvELAvqLhwIAAMIWDd+76+gpGmhhFjcwQL7kNWZWAQBhq3MwpeE7EJoIqwAARBBfb1zjRjUEG2EVAIAI4uvaUW5UQ7DRZxUAAFxRx41qbrdbJSUlcrvdcrlcEX+jGgKHG6wAAECvuFEN/sYNVgAAwG+4UQ3BxDIAAAAA2BZhFQAAALZFWAUABJTH41FeXp48Hk+wSwEQAiwNqz/60Y80depUxcXF6ZprrunTMaZpauXKlRo2bJhiY2M1Y8YMffDBB1aWCQAIoM7PmAeAq7H0BqvW1lZ97WtfU25urkpLS/t0zOrVq7VmzRpt3LhRo0eP1g9/+EPdfvvtOn78uBITE60sFwDQC1+eKX85fz5j/nL0+gTCW0BaV23cuFHLli3T2bNne93PNE0NGzZMy5Yt0+OPPy7pUruMtLQ0Pf3003rooYe6HdPS0qKWlhbv+8bGRmVmZtK6CgD8LFjPlL+aMOvACEQEX1pX2WrN6okTJ1RbW6v8/HzvNqfTqVtvvVUHDhzo8Zji4mIlJyd7X5mZmYEqFwAAABazVZ/V2tpaSVJaWlqX7Wlpafr44497PGbFihUqLCz0vu+YWQUA+Jevz5S/HM+YB9AfPofVlStXatWqVb3u8+677yonJ6ffRV3+UZNpmlf8+MnpdNKcGAACYKBrQ3nGPID+8DmsPvzww/r617/e6z4jRozoVzEul0vSpRnW9PR07/bTp093m20FAISWjmfMZ2ZmauHChSotLVV1dTXPmAfQK5/D6pAhQzRkyBAratHIkSPlcrm0e/du3XTTTZIudRTYu3evnn76aUuuCQAIjIyMDJ08edL7jPnFixfzjHn0yOPxqKioSKtXrx7QJ7UID5beYFVVVaXKykpVVVWpra1NlZWVqqys7LLuaezYsdq+fbukSx//L1u2TD/+8Y+1fft2vf/++1qwYIHi4uJ0//33W1kqACAAnE6nd1kXz5jHldCLF51ZeoPV97//ff3sZz/zvu+YLS0vL9eMGTMkScePH1dDQ4N3n6KiIv31r3/V0qVL9Ze//EWTJ0/Wrl276LEKAECI6kuPXn/14mX9c/gJSJ/VQPKlbxcAALBeIHv0hlmsCVsh22cVAAAA6MxWfVYBAED46WuPXnrxoieEVQAAYKm+riOlFy96wjIAAABgCx29eN1ut0pKSuR2u+VyuejFG+G4wQoAANhGS0uLtxevaZr04g1TvuQ1lgEAAADb6BxM6cULiWUAAAAAsDHCKgAAIcrj8SgvL08ejyfYpQCWIawCABCieCwpIgFrVgEACKK+PIq0M389llTi0aQIDYRVAACCKCEhYcDnqKur67GZ/tWEWUMghCmWAQAAAMC2mFkFACCI+voo0s54LCkiCWEVAIAg6s+6UR5LikjCMgAAAEIMjyVFJOFxqwAAhCAeS4pQxuNWAQAIczyWFJGCZQAAAACwLcIqAAAAbIuwCgAAANsKuzWrHfeLNTY2BrkSAAAA9KQjp/XlPv+wC6vnzp2TJGVmZga5EgAAAPTm3LlzSk5O7nWfsGtd1d7erlOnTikxMVGGYQS7HMs1NjYqMzNT1dXVtOqyIcbH/hgje2N87I3xsT+7jpFpmjp37pyGDRumqKjeV6WG3cxqVFSUMjIygl1GwCUlJdnqP0J0xfjYH2Nkb4yPvTE+9mfHMbrajGoHbrACAACAbRFWAQAAYFuE1RDndDr15JNP8uQSm2J87I8xsjfGx94YH/sLhzEKuxusAAAAED6YWQUAAIBtEVYBAABgW4RVAAAA2BZhFQAAALZFWAUAAIBtEVZDwIYNGzRy5EgNHjxYbrdb+/bt63X/vXv3yu12a/DgwRo1apRKSkoCVGlk8mV83njjDd1+++0aOnSokpKSlJubq//6r/8KYLWRydefoQ779+9XdHS0srOzrS0wwvk6Pi0tLfrOd76j4cOHy+l06otf/KJeeeWVAFUbeXwdn9dee00TJ05UXFyc0tPT9eCDD6q+vj5A1UaWd955R3fddZeGDRsmwzD05ptvXvWYkMwIJmxt69atZkxMjPnyyy+bR48eNR955BEzPj7e/Pjjj3vc/6OPPjLj4uLMRx55xDx69Kj58ssvmzExMeYvf/nLAFceGXwdn0ceecR8+umnzd/97nfmn/70J3PFihVmTEyM+T//8z8Brjxy+DpGHc6ePWuOGjXKzM/PNydOnBiYYiNQf8bn7rvvNidPnmzu3r3bPHHihHno0CFz//79Aaw6cvg6Pvv27TOjoqLMn/70p+ZHH31k7tu3z5wwYYJ57733BrjyyLBjxw7zO9/5jrlt2zZTkrl9+/Ze9w/VjEBYtblJkyaZS5Ys6bJt7Nix5vLly3vcv6ioyBw7dmyXbQ899JA5ZcoUy2qMZL6OT0/Gjx9vrlq1yt+l4f/r7xjNnTvX/O53v2s++eSThFUL+To+b731lpmcnGzW19cHoryI5+v4PPPMM+aoUaO6bFu3bp2ZkZFhWY24pC9hNVQzAssAbKy1tVWHDx9Wfn5+l+35+fk6cOBAj8ccPHiw2/4zZ86Ux+PRhQsXLKs1EvVnfC7X3t6uc+fO6dprr7WixIjX3zF69dVX9ec//1lPPvmk1SVGtP6Mz69+9Svl5ORo9erVuu666zR69Gg99thj+utf/xqIkiNKf8Zn6tSp+uSTT7Rjxw6ZpqnPPvtMv/zlL/V3f/d3gSgZVxGqGSE62AXgys6cOaO2tjalpaV12Z6Wlqba2toej6mtre1x/4sXL+rMmTNKT0+3rN5I05/xudxzzz2n5uZmFRQUWFFixOvPGH344Ydavny59u3bp+ho/om0Un/G56OPPlJFRYUGDx6s7du368yZM1q6dKn+93//l3Wrftaf8Zk6dapee+01zZ07V+fPn9fFixd199136/nnnw9EybiKUM0IzKyGAMMwurw3TbPbtqvt39N2+Iev49OhrKxMK1eu1Ouvv67U1FSryoP6PkZtbW26//77tWrVKo0ePTpQ5UU8X36G2tvbZRiGXnvtNU2aNEmzZ8/WmjVrtHHjRmZXLeLL+Bw9elTf/va39f3vf1+HDx/Wzp07deLECS1ZsiQQpaIPQjEjMG1gY0OGDNGgQYO6/QZ7+vTpbr8ZdXC5XD3uHx0drZSUFMtqjUT9GZ8Or7/+uhYuXKhf/OIX+spXvmJlmRHN1zE6d+6cPB6Pjhw5oocffljSpXBkmqaio6O1a9cu5eXlBaT2SNCfn6H09HRdd911Sk5O9m4bN26cTNPUJ598oi996UuW1hxJ+jM+xcXFuuWWW/Qv//IvkqQbb7xR8fHxmj59un74wx/aduYuUoRqRmBm1cYcDofcbrd2797dZfvu3bs1derUHo/Jzc3ttv+uXbuUk5OjmJgYy2qNRP0ZH+nSjOqCBQv085//nHVcFvN1jJKSkvTee++psrLS+1qyZInGjBmjyspKTZ48OVClR4T+/AzdcsstOnXqlJqamrzb/vSnPykqKkoZGRmW1htp+jM+n3/+uaKiukaLQYMGSfq/GTwET8hmhCDd2IU+6mgbUlpaah49etRctmyZGR8fb548edI0TdNcvny5OW/ePO/+HW0pHn30UfPo0aNmaWlpSLSlCFW+js/Pf/5zMzo62ly/fr1ZU1PjfZ09ezZY30LY83WMLkc3AGv5Oj7nzp0zMzIyzL//+783P/jgA3Pv3r3ml770JfOb3/xmsL6FsObr+Lz66qtmdHS0uWHDBvPPf/6zWVFRYebk5JiTJk0K1rcQ1s6dO2ceOXLEPHLkiCnJXLNmjXnkyBFva7FwyQiE1RCwfv16c/jw4abD4TBvvvlmc+/evd6/mz9/vnnrrbd22X/Pnj3mTTfdZDocDnPEiBHmCy+8EOCKI4sv43Prrbeakrq95s+fH/jCI4ivP0OdEVat5+v4HDt2zPzKV75ixsbGmhkZGWZhYaH5+eefB7jqyOHr+Kxbt84cP368GRsba6anp5v/+I//aH7yyScBrjoylJeX9/r/lHDJCIZpMi8PAAAAe2LNKgAAAGyLsAoAAADbIqwCAADAtgirAAAAsC3CKgAAAGyLsAoAAADbIqwCAADAtgirAAAAsC3CKgAAAGyLsAoAAADbIqwCAADAtv4fR8zZ42TnHRIAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "f, ax = plt.subplots(1, 1, figsize=(8, 3))\n",
    "ax.errorbar(train_x_mean, train_y, xerr=(train_x_stdv * 2), fmt=\"k*\", label=\"Train Data\")\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We train the hyperparameters of the resulting distributional GP via type-II gradient descent, as is standard in many settings. We could also do fully Bayesian inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from gpytorch.models import ExactGP\n",
    "from gpytorch.kernels import GaussianSymmetrizedKLKernel, ScaleKernel\n",
    "from gpytorch.means import ConstantMean\n",
    "\n",
    "class ExactGPModel(ExactGP):\n",
    "    def __init__(self, train_x, train_y, likelihood):\n",
    "        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)\n",
    "        self.mean_module = ConstantMean()\n",
    "        self.covar_module = ScaleKernel(GaussianSymmetrizedKLKernel())\n",
    "        \n",
    "    def forward(self, x):\n",
    "        mean_x = self.mean_module(x)\n",
    "        covar_x = self.covar_module(x)\n",
    "        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)\n",
    "\n",
    "# initialize likelihood and model\n",
    "likelihood = gpytorch.likelihoods.GaussianLikelihood()\n",
    "model = ExactGPModel(train_x_distributional, train_y, likelihood)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 1/500 - Loss: 1.290   lengthscale: 0.693   noise: 0.693\n",
      "Iter 2/500 - Loss: 1.253   lengthscale: 0.826   noise: 0.576\n",
      "Iter 3/500 - Loss: 1.194   lengthscale: 0.953   noise: 0.475\n",
      "Iter 4/500 - Loss: 1.165   lengthscale: 1.098   noise: 0.389\n",
      "Iter 5/500 - Loss: 1.148   lengthscale: 1.260   noise: 0.318\n",
      "Iter 6/500 - Loss: 1.133   lengthscale: 1.431   noise: 0.264\n",
      "Iter 7/500 - Loss: 1.141   lengthscale: 1.605   noise: 0.225\n",
      "Iter 8/500 - Loss: 1.154   lengthscale: 1.783   noise: 0.203\n",
      "Iter 9/500 - Loss: 1.147   lengthscale: 1.970   noise: 0.193\n",
      "Iter 10/500 - Loss: 1.134   lengthscale: 2.172   noise: 0.193\n",
      "Iter 11/500 - Loss: 1.122   lengthscale: 2.386   noise: 0.198\n",
      "Iter 12/500 - Loss: 1.105   lengthscale: 2.611   noise: 0.207\n",
      "Iter 13/500 - Loss: 1.089   lengthscale: 2.843   noise: 0.217\n",
      "Iter 14/500 - Loss: 1.079   lengthscale: 3.083   noise: 0.225\n",
      "Iter 15/500 - Loss: 1.074   lengthscale: 3.329   noise: 0.229\n",
      "Iter 16/500 - Loss: 1.069   lengthscale: 3.579   noise: 0.227\n",
      "Iter 17/500 - Loss: 1.061   lengthscale: 3.832   noise: 0.220\n",
      "Iter 18/500 - Loss: 1.052   lengthscale: 4.087   noise: 0.208\n",
      "Iter 19/500 - Loss: 1.041   lengthscale: 4.343   noise: 0.193\n",
      "Iter 20/500 - Loss: 1.029   lengthscale: 4.600   noise: 0.175\n",
      "Iter 21/500 - Loss: 1.015   lengthscale: 4.858   noise: 0.157\n",
      "Iter 22/500 - Loss: 1.000   lengthscale: 5.115   noise: 0.138\n",
      "Iter 23/500 - Loss: 0.984   lengthscale: 5.374   noise: 0.121\n",
      "Iter 24/500 - Loss: 0.970   lengthscale: 5.633   noise: 0.105\n",
      "Iter 25/500 - Loss: 0.958   lengthscale: 5.894   noise: 0.090\n",
      "Iter 26/500 - Loss: 0.947   lengthscale: 6.155   noise: 0.078\n",
      "Iter 27/500 - Loss: 0.937   lengthscale: 6.417   noise: 0.067\n",
      "Iter 28/500 - Loss: 0.928   lengthscale: 6.679   noise: 0.058\n",
      "Iter 29/500 - Loss: 0.920   lengthscale: 6.940   noise: 0.051\n",
      "Iter 30/500 - Loss: 0.913   lengthscale: 7.199   noise: 0.045\n",
      "Iter 31/500 - Loss: 0.907   lengthscale: 7.455   noise: 0.040\n",
      "Iter 32/500 - Loss: 0.902   lengthscale: 7.706   noise: 0.036\n",
      "Iter 33/500 - Loss: 0.899   lengthscale: 7.952   noise: 0.033\n",
      "Iter 34/500 - Loss: 0.897   lengthscale: 8.191   noise: 0.031\n",
      "Iter 35/500 - Loss: 0.896   lengthscale: 8.421   noise: 0.029\n",
      "Iter 36/500 - Loss: 0.894   lengthscale: 8.643   noise: 0.028\n",
      "Iter 37/500 - Loss: 0.893   lengthscale: 8.855   noise: 0.028\n",
      "Iter 38/500 - Loss: 0.891   lengthscale: 9.058   noise: 0.028\n",
      "Iter 39/500 - Loss: 0.888   lengthscale: 9.252   noise: 0.028\n",
      "Iter 40/500 - Loss: 0.884   lengthscale: 9.438   noise: 0.029\n",
      "Iter 41/500 - Loss: 0.879   lengthscale: 9.618   noise: 0.031\n",
      "Iter 42/500 - Loss: 0.873   lengthscale: 9.793   noise: 0.034\n",
      "Iter 43/500 - Loss: 0.867   lengthscale: 9.965   noise: 0.037\n",
      "Iter 44/500 - Loss: 0.862   lengthscale: 10.136   noise: 0.041\n",
      "Iter 45/500 - Loss: 0.856   lengthscale: 10.306   noise: 0.046\n",
      "Iter 46/500 - Loss: 0.851   lengthscale: 10.478   noise: 0.051\n",
      "Iter 47/500 - Loss: 0.848   lengthscale: 10.651   noise: 0.057\n",
      "Iter 48/500 - Loss: 0.845   lengthscale: 10.826   noise: 0.063\n",
      "Iter 49/500 - Loss: 0.843   lengthscale: 11.003   noise: 0.069\n",
      "Iter 50/500 - Loss: 0.841   lengthscale: 11.181   noise: 0.075\n",
      "Iter 51/500 - Loss: 0.840   lengthscale: 11.361   noise: 0.079\n",
      "Iter 52/500 - Loss: 0.838   lengthscale: 11.542   noise: 0.082\n",
      "Iter 53/500 - Loss: 0.836   lengthscale: 11.723   noise: 0.083\n",
      "Iter 54/500 - Loss: 0.833   lengthscale: 11.905   noise: 0.083\n",
      "Iter 55/500 - Loss: 0.830   lengthscale: 12.086   noise: 0.081\n",
      "Iter 56/500 - Loss: 0.826   lengthscale: 12.268   noise: 0.078\n",
      "Iter 57/500 - Loss: 0.822   lengthscale: 12.449   noise: 0.074\n",
      "Iter 58/500 - Loss: 0.819   lengthscale: 12.630   noise: 0.070\n",
      "Iter 59/500 - Loss: 0.816   lengthscale: 12.811   noise: 0.066\n",
      "Iter 60/500 - Loss: 0.813   lengthscale: 12.991   noise: 0.063\n",
      "Iter 61/500 - Loss: 0.811   lengthscale: 13.170   noise: 0.060\n",
      "Iter 62/500 - Loss: 0.809   lengthscale: 13.347   noise: 0.057\n",
      "Iter 63/500 - Loss: 0.807   lengthscale: 13.524   noise: 0.055\n",
      "Iter 64/500 - Loss: 0.805   lengthscale: 13.700   noise: 0.055\n",
      "Iter 65/500 - Loss: 0.803   lengthscale: 13.874   noise: 0.055\n",
      "Iter 66/500 - Loss: 0.801   lengthscale: 14.046   noise: 0.055\n",
      "Iter 67/500 - Loss: 0.798   lengthscale: 14.217   noise: 0.057\n",
      "Iter 68/500 - Loss: 0.796   lengthscale: 14.387   noise: 0.058\n",
      "Iter 69/500 - Loss: 0.793   lengthscale: 14.556   noise: 0.061\n",
      "Iter 70/500 - Loss: 0.791   lengthscale: 14.724   noise: 0.063\n",
      "Iter 71/500 - Loss: 0.789   lengthscale: 14.890   noise: 0.065\n",
      "Iter 72/500 - Loss: 0.788   lengthscale: 15.056   noise: 0.068\n",
      "Iter 73/500 - Loss: 0.786   lengthscale: 15.221   noise: 0.069\n",
      "Iter 74/500 - Loss: 0.784   lengthscale: 15.385   noise: 0.070\n",
      "Iter 75/500 - Loss: 0.782   lengthscale: 15.548   noise: 0.071\n",
      "Iter 76/500 - Loss: 0.781   lengthscale: 15.710   noise: 0.071\n",
      "Iter 77/500 - Loss: 0.779   lengthscale: 15.871   noise: 0.070\n",
      "Iter 78/500 - Loss: 0.777   lengthscale: 16.031   noise: 0.069\n",
      "Iter 79/500 - Loss: 0.775   lengthscale: 16.190   noise: 0.067\n",
      "Iter 80/500 - Loss: 0.773   lengthscale: 16.348   noise: 0.066\n",
      "Iter 81/500 - Loss: 0.771   lengthscale: 16.505   noise: 0.064\n",
      "Iter 82/500 - Loss: 0.770   lengthscale: 16.662   noise: 0.063\n",
      "Iter 83/500 - Loss: 0.768   lengthscale: 16.817   noise: 0.062\n",
      "Iter 84/500 - Loss: 0.767   lengthscale: 16.971   noise: 0.062\n",
      "Iter 85/500 - Loss: 0.765   lengthscale: 17.124   noise: 0.062\n",
      "Iter 86/500 - Loss: 0.764   lengthscale: 17.276   noise: 0.062\n",
      "Iter 87/500 - Loss: 0.762   lengthscale: 17.427   noise: 0.063\n",
      "Iter 88/500 - Loss: 0.761   lengthscale: 17.577   noise: 0.064\n",
      "Iter 89/500 - Loss: 0.759   lengthscale: 17.726   noise: 0.065\n",
      "Iter 90/500 - Loss: 0.758   lengthscale: 17.874   noise: 0.066\n",
      "Iter 91/500 - Loss: 0.756   lengthscale: 18.021   noise: 0.066\n",
      "Iter 92/500 - Loss: 0.755   lengthscale: 18.168   noise: 0.067\n",
      "Iter 93/500 - Loss: 0.754   lengthscale: 18.313   noise: 0.067\n",
      "Iter 94/500 - Loss: 0.752   lengthscale: 18.458   noise: 0.068\n",
      "Iter 95/500 - Loss: 0.751   lengthscale: 18.602   noise: 0.067\n",
      "Iter 96/500 - Loss: 0.750   lengthscale: 18.745   noise: 0.067\n",
      "Iter 97/500 - Loss: 0.748   lengthscale: 18.888   noise: 0.066\n",
      "Iter 98/500 - Loss: 0.747   lengthscale: 19.029   noise: 0.066\n",
      "Iter 99/500 - Loss: 0.746   lengthscale: 19.170   noise: 0.065\n",
      "Iter 100/500 - Loss: 0.745   lengthscale: 19.310   noise: 0.065\n",
      "Iter 101/500 - Loss: 0.744   lengthscale: 19.449   noise: 0.064\n",
      "Iter 102/500 - Loss: 0.742   lengthscale: 19.587   noise: 0.064\n",
      "Iter 103/500 - Loss: 0.741   lengthscale: 19.725   noise: 0.064\n",
      "Iter 104/500 - Loss: 0.740   lengthscale: 19.862   noise: 0.064\n",
      "Iter 105/500 - Loss: 0.739   lengthscale: 19.998   noise: 0.065\n",
      "Iter 106/500 - Loss: 0.738   lengthscale: 20.133   noise: 0.065\n",
      "Iter 107/500 - Loss: 0.737   lengthscale: 20.268   noise: 0.066\n",
      "Iter 108/500 - Loss: 0.736   lengthscale: 20.402   noise: 0.066\n",
      "Iter 109/500 - Loss: 0.734   lengthscale: 20.535   noise: 0.066\n",
      "Iter 110/500 - Loss: 0.733   lengthscale: 20.668   noise: 0.066\n",
      "Iter 111/500 - Loss: 0.732   lengthscale: 20.800   noise: 0.067\n",
      "Iter 112/500 - Loss: 0.731   lengthscale: 20.931   noise: 0.066\n",
      "Iter 113/500 - Loss: 0.730   lengthscale: 21.062   noise: 0.066\n",
      "Iter 114/500 - Loss: 0.729   lengthscale: 21.192   noise: 0.066\n",
      "Iter 115/500 - Loss: 0.728   lengthscale: 21.321   noise: 0.066\n",
      "Iter 116/500 - Loss: 0.727   lengthscale: 21.450   noise: 0.065\n",
      "Iter 117/500 - Loss: 0.726   lengthscale: 21.578   noise: 0.065\n",
      "Iter 118/500 - Loss: 0.725   lengthscale: 21.706   noise: 0.065\n",
      "Iter 119/500 - Loss: 0.724   lengthscale: 21.833   noise: 0.065\n",
      "Iter 120/500 - Loss: 0.723   lengthscale: 21.959   noise: 0.065\n",
      "Iter 121/500 - Loss: 0.722   lengthscale: 22.085   noise: 0.065\n",
      "Iter 122/500 - Loss: 0.722   lengthscale: 22.210   noise: 0.066\n",
      "Iter 123/500 - Loss: 0.721   lengthscale: 22.335   noise: 0.066\n",
      "Iter 124/500 - Loss: 0.720   lengthscale: 22.459   noise: 0.066\n",
      "Iter 125/500 - Loss: 0.719   lengthscale: 22.582   noise: 0.066\n",
      "Iter 126/500 - Loss: 0.718   lengthscale: 22.705   noise: 0.066\n",
      "Iter 127/500 - Loss: 0.717   lengthscale: 22.828   noise: 0.066\n",
      "Iter 128/500 - Loss: 0.716   lengthscale: 22.950   noise: 0.066\n",
      "Iter 129/500 - Loss: 0.715   lengthscale: 23.071   noise: 0.066\n",
      "Iter 130/500 - Loss: 0.715   lengthscale: 23.192   noise: 0.066\n",
      "Iter 131/500 - Loss: 0.714   lengthscale: 23.312   noise: 0.066\n",
      "Iter 132/500 - Loss: 0.713   lengthscale: 23.432   noise: 0.066\n",
      "Iter 133/500 - Loss: 0.712   lengthscale: 23.552   noise: 0.066\n",
      "Iter 134/500 - Loss: 0.711   lengthscale: 23.671   noise: 0.066\n",
      "Iter 135/500 - Loss: 0.710   lengthscale: 23.789   noise: 0.066\n",
      "Iter 136/500 - Loss: 0.710   lengthscale: 23.907   noise: 0.066\n",
      "Iter 137/500 - Loss: 0.709   lengthscale: 24.024   noise: 0.066\n",
      "Iter 138/500 - Loss: 0.708   lengthscale: 24.141   noise: 0.066\n",
      "Iter 139/500 - Loss: 0.707   lengthscale: 24.258   noise: 0.066\n",
      "Iter 140/500 - Loss: 0.707   lengthscale: 24.374   noise: 0.066\n",
      "Iter 141/500 - Loss: 0.706   lengthscale: 24.490   noise: 0.066\n",
      "Iter 142/500 - Loss: 0.705   lengthscale: 24.605   noise: 0.066\n",
      "Iter 143/500 - Loss: 0.704   lengthscale: 24.720   noise: 0.066\n",
      "Iter 144/500 - Loss: 0.704   lengthscale: 24.834   noise: 0.066\n",
      "Iter 145/500 - Loss: 0.703   lengthscale: 24.948   noise: 0.066\n",
      "Iter 146/500 - Loss: 0.702   lengthscale: 25.062   noise: 0.066\n",
      "Iter 147/500 - Loss: 0.702   lengthscale: 25.175   noise: 0.066\n",
      "Iter 148/500 - Loss: 0.701   lengthscale: 25.288   noise: 0.066\n",
      "Iter 149/500 - Loss: 0.700   lengthscale: 25.400   noise: 0.066\n",
      "Iter 150/500 - Loss: 0.700   lengthscale: 25.512   noise: 0.066\n",
      "Iter 151/500 - Loss: 0.699   lengthscale: 25.624   noise: 0.066\n",
      "Iter 152/500 - Loss: 0.698   lengthscale: 25.735   noise: 0.066\n",
      "Iter 153/500 - Loss: 0.698   lengthscale: 25.846   noise: 0.066\n",
      "Iter 154/500 - Loss: 0.697   lengthscale: 25.956   noise: 0.066\n",
      "Iter 155/500 - Loss: 0.696   lengthscale: 26.067   noise: 0.066\n",
      "Iter 156/500 - Loss: 0.696   lengthscale: 26.176   noise: 0.066\n",
      "Iter 157/500 - Loss: 0.695   lengthscale: 26.286   noise: 0.066\n",
      "Iter 158/500 - Loss: 0.694   lengthscale: 26.395   noise: 0.066\n",
      "Iter 159/500 - Loss: 0.694   lengthscale: 26.503   noise: 0.066\n",
      "Iter 160/500 - Loss: 0.693   lengthscale: 26.612   noise: 0.066\n",
      "Iter 161/500 - Loss: 0.692   lengthscale: 26.720   noise: 0.066\n",
      "Iter 162/500 - Loss: 0.692   lengthscale: 26.827   noise: 0.066\n",
      "Iter 163/500 - Loss: 0.691   lengthscale: 26.935   noise: 0.066\n",
      "Iter 164/500 - Loss: 0.691   lengthscale: 27.042   noise: 0.066\n",
      "Iter 165/500 - Loss: 0.690   lengthscale: 27.149   noise: 0.066\n",
      "Iter 166/500 - Loss: 0.689   lengthscale: 27.255   noise: 0.066\n",
      "Iter 167/500 - Loss: 0.689   lengthscale: 27.361   noise: 0.066\n",
      "Iter 168/500 - Loss: 0.688   lengthscale: 27.467   noise: 0.066\n",
      "Iter 169/500 - Loss: 0.688   lengthscale: 27.572   noise: 0.066\n",
      "Iter 170/500 - Loss: 0.687   lengthscale: 27.677   noise: 0.066\n",
      "Iter 171/500 - Loss: 0.686   lengthscale: 27.782   noise: 0.066\n",
      "Iter 172/500 - Loss: 0.686   lengthscale: 27.887   noise: 0.066\n",
      "Iter 173/500 - Loss: 0.685   lengthscale: 27.991   noise: 0.066\n",
      "Iter 174/500 - Loss: 0.685   lengthscale: 28.095   noise: 0.066\n",
      "Iter 175/500 - Loss: 0.684   lengthscale: 28.199   noise: 0.066\n",
      "Iter 176/500 - Loss: 0.684   lengthscale: 28.302   noise: 0.066\n",
      "Iter 177/500 - Loss: 0.683   lengthscale: 28.405   noise: 0.066\n",
      "Iter 178/500 - Loss: 0.683   lengthscale: 28.508   noise: 0.066\n",
      "Iter 179/500 - Loss: 0.682   lengthscale: 28.611   noise: 0.066\n",
      "Iter 180/500 - Loss: 0.682   lengthscale: 28.713   noise: 0.066\n",
      "Iter 181/500 - Loss: 0.681   lengthscale: 28.816   noise: 0.066\n",
      "Iter 182/500 - Loss: 0.680   lengthscale: 28.917   noise: 0.066\n",
      "Iter 183/500 - Loss: 0.680   lengthscale: 29.019   noise: 0.066\n",
      "Iter 184/500 - Loss: 0.679   lengthscale: 29.120   noise: 0.066\n",
      "Iter 185/500 - Loss: 0.679   lengthscale: 29.221   noise: 0.066\n",
      "Iter 186/500 - Loss: 0.678   lengthscale: 29.322   noise: 0.066\n",
      "Iter 187/500 - Loss: 0.678   lengthscale: 29.423   noise: 0.067\n",
      "Iter 188/500 - Loss: 0.677   lengthscale: 29.523   noise: 0.067\n",
      "Iter 189/500 - Loss: 0.677   lengthscale: 29.624   noise: 0.067\n",
      "Iter 190/500 - Loss: 0.676   lengthscale: 29.723   noise: 0.067\n",
      "Iter 191/500 - Loss: 0.676   lengthscale: 29.823   noise: 0.067\n",
      "Iter 192/500 - Loss: 0.675   lengthscale: 29.923   noise: 0.067\n",
      "Iter 193/500 - Loss: 0.675   lengthscale: 30.022   noise: 0.067\n",
      "Iter 194/500 - Loss: 0.674   lengthscale: 30.121   noise: 0.067\n",
      "Iter 195/500 - Loss: 0.674   lengthscale: 30.220   noise: 0.067\n",
      "Iter 196/500 - Loss: 0.674   lengthscale: 30.318   noise: 0.067\n",
      "Iter 197/500 - Loss: 0.673   lengthscale: 30.417   noise: 0.067\n",
      "Iter 198/500 - Loss: 0.673   lengthscale: 30.515   noise: 0.067\n",
      "Iter 199/500 - Loss: 0.672   lengthscale: 30.613   noise: 0.067\n",
      "Iter 200/500 - Loss: 0.672   lengthscale: 30.711   noise: 0.067\n",
      "Iter 201/500 - Loss: 0.671   lengthscale: 30.808   noise: 0.067\n",
      "Iter 202/500 - Loss: 0.671   lengthscale: 30.906   noise: 0.067\n",
      "Iter 203/500 - Loss: 0.670   lengthscale: 31.003   noise: 0.067\n",
      "Iter 204/500 - Loss: 0.670   lengthscale: 31.100   noise: 0.067\n",
      "Iter 205/500 - Loss: 0.669   lengthscale: 31.196   noise: 0.067\n",
      "Iter 206/500 - Loss: 0.669   lengthscale: 31.293   noise: 0.067\n",
      "Iter 207/500 - Loss: 0.668   lengthscale: 31.389   noise: 0.067\n",
      "Iter 208/500 - Loss: 0.668   lengthscale: 31.486   noise: 0.067\n",
      "Iter 209/500 - Loss: 0.668   lengthscale: 31.582   noise: 0.067\n",
      "Iter 210/500 - Loss: 0.667   lengthscale: 31.678   noise: 0.067\n",
      "Iter 211/500 - Loss: 0.667   lengthscale: 31.773   noise: 0.067\n",
      "Iter 212/500 - Loss: 0.666   lengthscale: 31.869   noise: 0.067\n",
      "Iter 213/500 - Loss: 0.666   lengthscale: 31.964   noise: 0.067\n",
      "Iter 214/500 - Loss: 0.665   lengthscale: 32.059   noise: 0.067\n",
      "Iter 215/500 - Loss: 0.665   lengthscale: 32.154   noise: 0.067\n",
      "Iter 216/500 - Loss: 0.665   lengthscale: 32.249   noise: 0.067\n",
      "Iter 217/500 - Loss: 0.664   lengthscale: 32.344   noise: 0.067\n",
      "Iter 218/500 - Loss: 0.664   lengthscale: 32.438   noise: 0.067\n",
      "Iter 219/500 - Loss: 0.663   lengthscale: 32.532   noise: 0.067\n",
      "Iter 220/500 - Loss: 0.663   lengthscale: 32.627   noise: 0.067\n",
      "Iter 221/500 - Loss: 0.662   lengthscale: 32.721   noise: 0.067\n",
      "Iter 222/500 - Loss: 0.662   lengthscale: 32.814   noise: 0.067\n",
      "Iter 223/500 - Loss: 0.662   lengthscale: 32.908   noise: 0.067\n",
      "Iter 224/500 - Loss: 0.661   lengthscale: 33.002   noise: 0.067\n",
      "Iter 225/500 - Loss: 0.661   lengthscale: 33.095   noise: 0.067\n",
      "Iter 226/500 - Loss: 0.660   lengthscale: 33.188   noise: 0.067\n",
      "Iter 227/500 - Loss: 0.660   lengthscale: 33.281   noise: 0.067\n",
      "Iter 228/500 - Loss: 0.660   lengthscale: 33.374   noise: 0.067\n",
      "Iter 229/500 - Loss: 0.659   lengthscale: 33.467   noise: 0.067\n",
      "Iter 230/500 - Loss: 0.659   lengthscale: 33.560   noise: 0.067\n",
      "Iter 231/500 - Loss: 0.658   lengthscale: 33.652   noise: 0.067\n",
      "Iter 232/500 - Loss: 0.658   lengthscale: 33.744   noise: 0.067\n",
      "Iter 233/500 - Loss: 0.658   lengthscale: 33.837   noise: 0.067\n",
      "Iter 234/500 - Loss: 0.657   lengthscale: 33.929   noise: 0.067\n",
      "Iter 235/500 - Loss: 0.657   lengthscale: 34.021   noise: 0.067\n",
      "Iter 236/500 - Loss: 0.657   lengthscale: 34.112   noise: 0.067\n",
      "Iter 237/500 - Loss: 0.656   lengthscale: 34.204   noise: 0.067\n",
      "Iter 238/500 - Loss: 0.656   lengthscale: 34.296   noise: 0.067\n",
      "Iter 239/500 - Loss: 0.655   lengthscale: 34.387   noise: 0.067\n",
      "Iter 240/500 - Loss: 0.655   lengthscale: 34.478   noise: 0.067\n",
      "Iter 241/500 - Loss: 0.655   lengthscale: 34.569   noise: 0.067\n",
      "Iter 242/500 - Loss: 0.654   lengthscale: 34.660   noise: 0.067\n",
      "Iter 243/500 - Loss: 0.654   lengthscale: 34.751   noise: 0.067\n",
      "Iter 244/500 - Loss: 0.654   lengthscale: 34.842   noise: 0.067\n",
      "Iter 245/500 - Loss: 0.653   lengthscale: 34.933   noise: 0.067\n",
      "Iter 246/500 - Loss: 0.653   lengthscale: 35.023   noise: 0.067\n",
      "Iter 247/500 - Loss: 0.652   lengthscale: 35.113   noise: 0.067\n",
      "Iter 248/500 - Loss: 0.652   lengthscale: 35.204   noise: 0.067\n",
      "Iter 249/500 - Loss: 0.652   lengthscale: 35.294   noise: 0.067\n",
      "Iter 250/500 - Loss: 0.651   lengthscale: 35.384   noise: 0.067\n",
      "Iter 251/500 - Loss: 0.651   lengthscale: 35.474   noise: 0.067\n",
      "Iter 252/500 - Loss: 0.651   lengthscale: 35.564   noise: 0.067\n",
      "Iter 253/500 - Loss: 0.650   lengthscale: 35.653   noise: 0.067\n",
      "Iter 254/500 - Loss: 0.650   lengthscale: 35.743   noise: 0.067\n",
      "Iter 255/500 - Loss: 0.650   lengthscale: 35.832   noise: 0.067\n",
      "Iter 256/500 - Loss: 0.649   lengthscale: 35.922   noise: 0.067\n",
      "Iter 257/500 - Loss: 0.649   lengthscale: 36.011   noise: 0.067\n",
      "Iter 258/500 - Loss: 0.649   lengthscale: 36.100   noise: 0.067\n",
      "Iter 259/500 - Loss: 0.648   lengthscale: 36.189   noise: 0.067\n",
      "Iter 260/500 - Loss: 0.648   lengthscale: 36.278   noise: 0.067\n",
      "Iter 261/500 - Loss: 0.648   lengthscale: 36.366   noise: 0.067\n",
      "Iter 262/500 - Loss: 0.647   lengthscale: 36.455   noise: 0.067\n",
      "Iter 263/500 - Loss: 0.647   lengthscale: 36.544   noise: 0.067\n",
      "Iter 264/500 - Loss: 0.647   lengthscale: 36.632   noise: 0.067\n",
      "Iter 265/500 - Loss: 0.646   lengthscale: 36.720   noise: 0.067\n",
      "Iter 266/500 - Loss: 0.646   lengthscale: 36.809   noise: 0.067\n",
      "Iter 267/500 - Loss: 0.646   lengthscale: 36.897   noise: 0.067\n",
      "Iter 268/500 - Loss: 0.645   lengthscale: 36.985   noise: 0.067\n",
      "Iter 269/500 - Loss: 0.645   lengthscale: 37.073   noise: 0.067\n",
      "Iter 270/500 - Loss: 0.645   lengthscale: 37.161   noise: 0.067\n",
      "Iter 271/500 - Loss: 0.644   lengthscale: 37.248   noise: 0.067\n",
      "Iter 272/500 - Loss: 0.644   lengthscale: 37.336   noise: 0.067\n",
      "Iter 273/500 - Loss: 0.644   lengthscale: 37.424   noise: 0.067\n",
      "Iter 274/500 - Loss: 0.643   lengthscale: 37.511   noise: 0.067\n",
      "Iter 275/500 - Loss: 0.643   lengthscale: 37.598   noise: 0.067\n",
      "Iter 276/500 - Loss: 0.643   lengthscale: 37.686   noise: 0.067\n",
      "Iter 277/500 - Loss: 0.642   lengthscale: 37.773   noise: 0.067\n",
      "Iter 278/500 - Loss: 0.642   lengthscale: 37.860   noise: 0.067\n",
      "Iter 279/500 - Loss: 0.642   lengthscale: 37.947   noise: 0.067\n",
      "Iter 280/500 - Loss: 0.641   lengthscale: 38.034   noise: 0.067\n",
      "Iter 281/500 - Loss: 0.641   lengthscale: 38.120   noise: 0.067\n",
      "Iter 282/500 - Loss: 0.641   lengthscale: 38.207   noise: 0.067\n",
      "Iter 283/500 - Loss: 0.640   lengthscale: 38.294   noise: 0.067\n",
      "Iter 284/500 - Loss: 0.640   lengthscale: 38.380   noise: 0.067\n",
      "Iter 285/500 - Loss: 0.640   lengthscale: 38.467   noise: 0.067\n",
      "Iter 286/500 - Loss: 0.639   lengthscale: 38.553   noise: 0.067\n",
      "Iter 287/500 - Loss: 0.639   lengthscale: 38.639   noise: 0.067\n",
      "Iter 288/500 - Loss: 0.639   lengthscale: 38.725   noise: 0.067\n",
      "Iter 289/500 - Loss: 0.639   lengthscale: 38.811   noise: 0.067\n",
      "Iter 290/500 - Loss: 0.638   lengthscale: 38.897   noise: 0.067\n",
      "Iter 291/500 - Loss: 0.638   lengthscale: 38.983   noise: 0.067\n",
      "Iter 292/500 - Loss: 0.638   lengthscale: 39.069   noise: 0.067\n",
      "Iter 293/500 - Loss: 0.637   lengthscale: 39.155   noise: 0.067\n",
      "Iter 294/500 - Loss: 0.637   lengthscale: 39.240   noise: 0.067\n",
      "Iter 295/500 - Loss: 0.637   lengthscale: 39.326   noise: 0.067\n",
      "Iter 296/500 - Loss: 0.636   lengthscale: 39.411   noise: 0.067\n",
      "Iter 297/500 - Loss: 0.636   lengthscale: 39.497   noise: 0.067\n",
      "Iter 298/500 - Loss: 0.636   lengthscale: 39.582   noise: 0.067\n",
      "Iter 299/500 - Loss: 0.636   lengthscale: 39.667   noise: 0.067\n",
      "Iter 300/500 - Loss: 0.635   lengthscale: 39.752   noise: 0.067\n",
      "Iter 301/500 - Loss: 0.635   lengthscale: 39.837   noise: 0.067\n",
      "Iter 302/500 - Loss: 0.635   lengthscale: 39.922   noise: 0.067\n",
      "Iter 303/500 - Loss: 0.634   lengthscale: 40.007   noise: 0.067\n",
      "Iter 304/500 - Loss: 0.634   lengthscale: 40.092   noise: 0.067\n",
      "Iter 305/500 - Loss: 0.634   lengthscale: 40.177   noise: 0.067\n",
      "Iter 306/500 - Loss: 0.634   lengthscale: 40.261   noise: 0.067\n",
      "Iter 307/500 - Loss: 0.633   lengthscale: 40.346   noise: 0.067\n",
      "Iter 308/500 - Loss: 0.633   lengthscale: 40.430   noise: 0.067\n",
      "Iter 309/500 - Loss: 0.633   lengthscale: 40.515   noise: 0.067\n",
      "Iter 310/500 - Loss: 0.632   lengthscale: 40.599   noise: 0.067\n",
      "Iter 311/500 - Loss: 0.632   lengthscale: 40.683   noise: 0.067\n",
      "Iter 312/500 - Loss: 0.632   lengthscale: 40.767   noise: 0.067\n",
      "Iter 313/500 - Loss: 0.632   lengthscale: 40.851   noise: 0.067\n",
      "Iter 314/500 - Loss: 0.631   lengthscale: 40.935   noise: 0.067\n",
      "Iter 315/500 - Loss: 0.631   lengthscale: 41.019   noise: 0.067\n",
      "Iter 316/500 - Loss: 0.631   lengthscale: 41.103   noise: 0.067\n",
      "Iter 317/500 - Loss: 0.630   lengthscale: 41.187   noise: 0.067\n",
      "Iter 318/500 - Loss: 0.630   lengthscale: 41.271   noise: 0.067\n",
      "Iter 319/500 - Loss: 0.630   lengthscale: 41.354   noise: 0.067\n",
      "Iter 320/500 - Loss: 0.630   lengthscale: 41.438   noise: 0.067\n",
      "Iter 321/500 - Loss: 0.629   lengthscale: 41.521   noise: 0.068\n",
      "Iter 322/500 - Loss: 0.629   lengthscale: 41.605   noise: 0.068\n",
      "Iter 323/500 - Loss: 0.629   lengthscale: 41.688   noise: 0.068\n",
      "Iter 324/500 - Loss: 0.629   lengthscale: 41.771   noise: 0.068\n",
      "Iter 325/500 - Loss: 0.628   lengthscale: 41.855   noise: 0.068\n",
      "Iter 326/500 - Loss: 0.628   lengthscale: 41.938   noise: 0.068\n",
      "Iter 327/500 - Loss: 0.628   lengthscale: 42.021   noise: 0.068\n",
      "Iter 328/500 - Loss: 0.627   lengthscale: 42.104   noise: 0.068\n",
      "Iter 329/500 - Loss: 0.627   lengthscale: 42.187   noise: 0.068\n",
      "Iter 330/500 - Loss: 0.627   lengthscale: 42.269   noise: 0.068\n",
      "Iter 331/500 - Loss: 0.627   lengthscale: 42.352   noise: 0.068\n",
      "Iter 332/500 - Loss: 0.626   lengthscale: 42.435   noise: 0.068\n",
      "Iter 333/500 - Loss: 0.626   lengthscale: 42.517   noise: 0.068\n",
      "Iter 334/500 - Loss: 0.626   lengthscale: 42.600   noise: 0.068\n",
      "Iter 335/500 - Loss: 0.626   lengthscale: 42.682   noise: 0.068\n",
      "Iter 336/500 - Loss: 0.625   lengthscale: 42.765   noise: 0.068\n",
      "Iter 337/500 - Loss: 0.625   lengthscale: 42.847   noise: 0.068\n",
      "Iter 338/500 - Loss: 0.625   lengthscale: 42.929   noise: 0.068\n",
      "Iter 339/500 - Loss: 0.625   lengthscale: 43.012   noise: 0.068\n",
      "Iter 340/500 - Loss: 0.624   lengthscale: 43.094   noise: 0.068\n",
      "Iter 341/500 - Loss: 0.624   lengthscale: 43.176   noise: 0.068\n",
      "Iter 342/500 - Loss: 0.624   lengthscale: 43.258   noise: 0.068\n",
      "Iter 343/500 - Loss: 0.624   lengthscale: 43.340   noise: 0.068\n",
      "Iter 344/500 - Loss: 0.623   lengthscale: 43.422   noise: 0.068\n",
      "Iter 345/500 - Loss: 0.623   lengthscale: 43.503   noise: 0.068\n",
      "Iter 346/500 - Loss: 0.623   lengthscale: 43.585   noise: 0.068\n",
      "Iter 347/500 - Loss: 0.623   lengthscale: 43.667   noise: 0.068\n",
      "Iter 348/500 - Loss: 0.622   lengthscale: 43.748   noise: 0.068\n",
      "Iter 349/500 - Loss: 0.622   lengthscale: 43.830   noise: 0.068\n",
      "Iter 350/500 - Loss: 0.622   lengthscale: 43.911   noise: 0.068\n",
      "Iter 351/500 - Loss: 0.622   lengthscale: 43.993   noise: 0.068\n",
      "Iter 352/500 - Loss: 0.621   lengthscale: 44.074   noise: 0.068\n",
      "Iter 353/500 - Loss: 0.621   lengthscale: 44.155   noise: 0.068\n",
      "Iter 354/500 - Loss: 0.621   lengthscale: 44.237   noise: 0.068\n",
      "Iter 355/500 - Loss: 0.621   lengthscale: 44.318   noise: 0.068\n",
      "Iter 356/500 - Loss: 0.620   lengthscale: 44.399   noise: 0.068\n",
      "Iter 357/500 - Loss: 0.620   lengthscale: 44.480   noise: 0.068\n",
      "Iter 358/500 - Loss: 0.620   lengthscale: 44.561   noise: 0.068\n",
      "Iter 359/500 - Loss: 0.620   lengthscale: 44.642   noise: 0.068\n",
      "Iter 360/500 - Loss: 0.619   lengthscale: 44.722   noise: 0.068\n",
      "Iter 361/500 - Loss: 0.619   lengthscale: 44.803   noise: 0.068\n",
      "Iter 362/500 - Loss: 0.619   lengthscale: 44.884   noise: 0.068\n",
      "Iter 363/500 - Loss: 0.619   lengthscale: 44.964   noise: 0.068\n",
      "Iter 364/500 - Loss: 0.618   lengthscale: 45.045   noise: 0.068\n",
      "Iter 365/500 - Loss: 0.618   lengthscale: 45.125   noise: 0.068\n",
      "Iter 366/500 - Loss: 0.618   lengthscale: 45.206   noise: 0.067\n",
      "Iter 367/500 - Loss: 0.618   lengthscale: 45.286   noise: 0.067\n",
      "Iter 368/500 - Loss: 0.617   lengthscale: 45.367   noise: 0.067\n",
      "Iter 369/500 - Loss: 0.617   lengthscale: 45.447   noise: 0.067\n",
      "Iter 370/500 - Loss: 0.617   lengthscale: 45.527   noise: 0.067\n",
      "Iter 371/500 - Loss: 0.617   lengthscale: 45.607   noise: 0.067\n",
      "Iter 372/500 - Loss: 0.617   lengthscale: 45.687   noise: 0.067\n",
      "Iter 373/500 - Loss: 0.616   lengthscale: 45.767   noise: 0.067\n",
      "Iter 374/500 - Loss: 0.616   lengthscale: 45.847   noise: 0.067\n",
      "Iter 375/500 - Loss: 0.616   lengthscale: 45.927   noise: 0.067\n",
      "Iter 376/500 - Loss: 0.616   lengthscale: 46.007   noise: 0.067\n",
      "Iter 377/500 - Loss: 0.615   lengthscale: 46.087   noise: 0.067\n",
      "Iter 378/500 - Loss: 0.615   lengthscale: 46.166   noise: 0.067\n",
      "Iter 379/500 - Loss: 0.615   lengthscale: 46.246   noise: 0.067\n",
      "Iter 380/500 - Loss: 0.615   lengthscale: 46.325   noise: 0.067\n",
      "Iter 381/500 - Loss: 0.614   lengthscale: 46.405   noise: 0.067\n",
      "Iter 382/500 - Loss: 0.614   lengthscale: 46.484   noise: 0.067\n",
      "Iter 383/500 - Loss: 0.614   lengthscale: 46.564   noise: 0.067\n",
      "Iter 384/500 - Loss: 0.614   lengthscale: 46.643   noise: 0.067\n",
      "Iter 385/500 - Loss: 0.614   lengthscale: 46.722   noise: 0.067\n",
      "Iter 386/500 - Loss: 0.613   lengthscale: 46.802   noise: 0.067\n",
      "Iter 387/500 - Loss: 0.613   lengthscale: 46.881   noise: 0.067\n",
      "Iter 388/500 - Loss: 0.613   lengthscale: 46.960   noise: 0.067\n",
      "Iter 389/500 - Loss: 0.613   lengthscale: 47.039   noise: 0.067\n",
      "Iter 390/500 - Loss: 0.612   lengthscale: 47.118   noise: 0.067\n",
      "Iter 391/500 - Loss: 0.612   lengthscale: 47.197   noise: 0.067\n",
      "Iter 392/500 - Loss: 0.612   lengthscale: 47.276   noise: 0.067\n",
      "Iter 393/500 - Loss: 0.612   lengthscale: 47.354   noise: 0.067\n",
      "Iter 394/500 - Loss: 0.612   lengthscale: 47.433   noise: 0.067\n",
      "Iter 395/500 - Loss: 0.611   lengthscale: 47.512   noise: 0.067\n",
      "Iter 396/500 - Loss: 0.611   lengthscale: 47.590   noise: 0.067\n",
      "Iter 397/500 - Loss: 0.611   lengthscale: 47.669   noise: 0.067\n",
      "Iter 398/500 - Loss: 0.611   lengthscale: 47.747   noise: 0.067\n",
      "Iter 399/500 - Loss: 0.610   lengthscale: 47.826   noise: 0.067\n",
      "Iter 400/500 - Loss: 0.610   lengthscale: 47.904   noise: 0.067\n",
      "Iter 401/500 - Loss: 0.610   lengthscale: 47.983   noise: 0.067\n",
      "Iter 402/500 - Loss: 0.610   lengthscale: 48.061   noise: 0.067\n",
      "Iter 403/500 - Loss: 0.610   lengthscale: 48.139   noise: 0.067\n",
      "Iter 404/500 - Loss: 0.609   lengthscale: 48.217   noise: 0.067\n",
      "Iter 405/500 - Loss: 0.609   lengthscale: 48.295   noise: 0.067\n",
      "Iter 406/500 - Loss: 0.609   lengthscale: 48.373   noise: 0.067\n",
      "Iter 407/500 - Loss: 0.609   lengthscale: 48.451   noise: 0.067\n",
      "Iter 408/500 - Loss: 0.609   lengthscale: 48.529   noise: 0.067\n",
      "Iter 409/500 - Loss: 0.608   lengthscale: 48.607   noise: 0.067\n",
      "Iter 410/500 - Loss: 0.608   lengthscale: 48.685   noise: 0.067\n",
      "Iter 411/500 - Loss: 0.608   lengthscale: 48.763   noise: 0.067\n",
      "Iter 412/500 - Loss: 0.608   lengthscale: 48.840   noise: 0.067\n",
      "Iter 413/500 - Loss: 0.608   lengthscale: 48.918   noise: 0.067\n",
      "Iter 414/500 - Loss: 0.607   lengthscale: 48.996   noise: 0.067\n",
      "Iter 415/500 - Loss: 0.607   lengthscale: 49.073   noise: 0.067\n",
      "Iter 416/500 - Loss: 0.607   lengthscale: 49.151   noise: 0.067\n",
      "Iter 417/500 - Loss: 0.607   lengthscale: 49.228   noise: 0.067\n",
      "Iter 418/500 - Loss: 0.607   lengthscale: 49.305   noise: 0.067\n",
      "Iter 419/500 - Loss: 0.606   lengthscale: 49.383   noise: 0.067\n",
      "Iter 420/500 - Loss: 0.606   lengthscale: 49.460   noise: 0.067\n",
      "Iter 421/500 - Loss: 0.606   lengthscale: 49.537   noise: 0.067\n",
      "Iter 422/500 - Loss: 0.606   lengthscale: 49.614   noise: 0.067\n",
      "Iter 423/500 - Loss: 0.606   lengthscale: 49.691   noise: 0.067\n",
      "Iter 424/500 - Loss: 0.605   lengthscale: 49.768   noise: 0.067\n",
      "Iter 425/500 - Loss: 0.605   lengthscale: 49.845   noise: 0.067\n",
      "Iter 426/500 - Loss: 0.605   lengthscale: 49.922   noise: 0.067\n",
      "Iter 427/500 - Loss: 0.605   lengthscale: 49.999   noise: 0.067\n",
      "Iter 428/500 - Loss: 0.605   lengthscale: 50.076   noise: 0.067\n",
      "Iter 429/500 - Loss: 0.604   lengthscale: 50.153   noise: 0.067\n",
      "Iter 430/500 - Loss: 0.604   lengthscale: 50.230   noise: 0.067\n",
      "Iter 431/500 - Loss: 0.604   lengthscale: 50.306   noise: 0.067\n",
      "Iter 432/500 - Loss: 0.604   lengthscale: 50.383   noise: 0.067\n",
      "Iter 433/500 - Loss: 0.604   lengthscale: 50.459   noise: 0.067\n",
      "Iter 434/500 - Loss: 0.603   lengthscale: 50.536   noise: 0.067\n",
      "Iter 435/500 - Loss: 0.603   lengthscale: 50.612   noise: 0.067\n",
      "Iter 436/500 - Loss: 0.603   lengthscale: 50.689   noise: 0.067\n",
      "Iter 437/500 - Loss: 0.603   lengthscale: 50.765   noise: 0.067\n",
      "Iter 438/500 - Loss: 0.603   lengthscale: 50.841   noise: 0.067\n",
      "Iter 439/500 - Loss: 0.602   lengthscale: 50.917   noise: 0.067\n",
      "Iter 440/500 - Loss: 0.602   lengthscale: 50.994   noise: 0.067\n",
      "Iter 441/500 - Loss: 0.602   lengthscale: 51.070   noise: 0.067\n",
      "Iter 442/500 - Loss: 0.602   lengthscale: 51.146   noise: 0.067\n",
      "Iter 443/500 - Loss: 0.602   lengthscale: 51.222   noise: 0.067\n",
      "Iter 444/500 - Loss: 0.601   lengthscale: 51.298   noise: 0.067\n",
      "Iter 445/500 - Loss: 0.601   lengthscale: 51.374   noise: 0.067\n",
      "Iter 446/500 - Loss: 0.601   lengthscale: 51.449   noise: 0.067\n",
      "Iter 447/500 - Loss: 0.601   lengthscale: 51.525   noise: 0.067\n",
      "Iter 448/500 - Loss: 0.601   lengthscale: 51.601   noise: 0.067\n",
      "Iter 449/500 - Loss: 0.600   lengthscale: 51.677   noise: 0.067\n",
      "Iter 450/500 - Loss: 0.600   lengthscale: 51.752   noise: 0.067\n",
      "Iter 451/500 - Loss: 0.600   lengthscale: 51.828   noise: 0.067\n",
      "Iter 452/500 - Loss: 0.600   lengthscale: 51.903   noise: 0.067\n",
      "Iter 453/500 - Loss: 0.600   lengthscale: 51.979   noise: 0.067\n",
      "Iter 454/500 - Loss: 0.600   lengthscale: 52.054   noise: 0.067\n",
      "Iter 455/500 - Loss: 0.599   lengthscale: 52.130   noise: 0.067\n",
      "Iter 456/500 - Loss: 0.599   lengthscale: 52.205   noise: 0.067\n",
      "Iter 457/500 - Loss: 0.599   lengthscale: 52.280   noise: 0.067\n",
      "Iter 458/500 - Loss: 0.599   lengthscale: 52.355   noise: 0.067\n",
      "Iter 459/500 - Loss: 0.599   lengthscale: 52.431   noise: 0.067\n",
      "Iter 460/500 - Loss: 0.598   lengthscale: 52.506   noise: 0.067\n",
      "Iter 461/500 - Loss: 0.598   lengthscale: 52.581   noise: 0.067\n",
      "Iter 462/500 - Loss: 0.598   lengthscale: 52.656   noise: 0.067\n",
      "Iter 463/500 - Loss: 0.598   lengthscale: 52.731   noise: 0.067\n",
      "Iter 464/500 - Loss: 0.598   lengthscale: 52.806   noise: 0.067\n",
      "Iter 465/500 - Loss: 0.598   lengthscale: 52.880   noise: 0.067\n",
      "Iter 466/500 - Loss: 0.597   lengthscale: 52.955   noise: 0.067\n",
      "Iter 467/500 - Loss: 0.597   lengthscale: 53.030   noise: 0.067\n",
      "Iter 468/500 - Loss: 0.597   lengthscale: 53.105   noise: 0.067\n",
      "Iter 469/500 - Loss: 0.597   lengthscale: 53.179   noise: 0.067\n",
      "Iter 470/500 - Loss: 0.597   lengthscale: 53.254   noise: 0.067\n",
      "Iter 471/500 - Loss: 0.596   lengthscale: 53.328   noise: 0.067\n",
      "Iter 472/500 - Loss: 0.596   lengthscale: 53.403   noise: 0.067\n",
      "Iter 473/500 - Loss: 0.596   lengthscale: 53.477   noise: 0.067\n",
      "Iter 474/500 - Loss: 0.596   lengthscale: 53.552   noise: 0.067\n",
      "Iter 475/500 - Loss: 0.596   lengthscale: 53.626   noise: 0.067\n",
      "Iter 476/500 - Loss: 0.596   lengthscale: 53.700   noise: 0.067\n",
      "Iter 477/500 - Loss: 0.595   lengthscale: 53.774   noise: 0.067\n",
      "Iter 478/500 - Loss: 0.595   lengthscale: 53.849   noise: 0.067\n",
      "Iter 479/500 - Loss: 0.595   lengthscale: 53.923   noise: 0.067\n",
      "Iter 480/500 - Loss: 0.595   lengthscale: 53.997   noise: 0.067\n",
      "Iter 481/500 - Loss: 0.595   lengthscale: 54.071   noise: 0.067\n",
      "Iter 482/500 - Loss: 0.595   lengthscale: 54.145   noise: 0.067\n",
      "Iter 483/500 - Loss: 0.594   lengthscale: 54.219   noise: 0.067\n",
      "Iter 484/500 - Loss: 0.594   lengthscale: 54.293   noise: 0.067\n",
      "Iter 485/500 - Loss: 0.594   lengthscale: 54.366   noise: 0.067\n",
      "Iter 486/500 - Loss: 0.594   lengthscale: 54.440   noise: 0.067\n",
      "Iter 487/500 - Loss: 0.594   lengthscale: 54.514   noise: 0.067\n",
      "Iter 488/500 - Loss: 0.594   lengthscale: 54.587   noise: 0.067\n",
      "Iter 489/500 - Loss: 0.593   lengthscale: 54.661   noise: 0.067\n",
      "Iter 490/500 - Loss: 0.593   lengthscale: 54.735   noise: 0.067\n",
      "Iter 491/500 - Loss: 0.593   lengthscale: 54.808   noise: 0.067\n",
      "Iter 492/500 - Loss: 0.593   lengthscale: 54.882   noise: 0.067\n",
      "Iter 493/500 - Loss: 0.593   lengthscale: 54.955   noise: 0.067\n",
      "Iter 494/500 - Loss: 0.592   lengthscale: 55.028   noise: 0.067\n",
      "Iter 495/500 - Loss: 0.592   lengthscale: 55.102   noise: 0.067\n",
      "Iter 496/500 - Loss: 0.592   lengthscale: 55.175   noise: 0.067\n",
      "Iter 497/500 - Loss: 0.592   lengthscale: 55.248   noise: 0.067\n",
      "Iter 498/500 - Loss: 0.592   lengthscale: 55.321   noise: 0.067\n",
      "Iter 499/500 - Loss: 0.592   lengthscale: 55.394   noise: 0.067\n",
      "Iter 500/500 - Loss: 0.592   lengthscale: 55.467   noise: 0.067\n"
     ]
    }
   ],
   "source": [
    "# this is for running the notebook in our testing framework\n",
    "import os\n",
    "smoke_test = ('CI' in os.environ)\n",
    "training_iter = 2 if smoke_test else 500\n",
    "\n",
    "\n",
    "# Find optimal model hyperparameters\n",
    "model.train()\n",
    "likelihood.train()\n",
    "\n",
    "# Use the adam optimizer\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.25)  # Includes GaussianLikelihood parameters\n",
    "\n",
    "# \"Loss\" for GPs - the marginal log likelihood\n",
    "mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)\n",
    "\n",
    "for i in range(training_iter):\n",
    "    # Zero gradients from previous iteration\n",
    "    optimizer.zero_grad()\n",
    "    # Output from model\n",
    "    output = model(train_x_distributional)\n",
    "    # Calc loss and backprop gradients\n",
    "    loss = -mll(output, train_y)\n",
    "    loss.backward()\n",
    "    print('Iter %d/%d - Loss: %.3f   lengthscale: %.3f   noise: %.3f' % (\n",
    "        i + 1, training_iter, loss.item(),\n",
    "        model.covar_module.base_kernel.lengthscale.item(),\n",
    "        model.likelihood.noise.item()\n",
    "    ))\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now, we test predictions. For simplicity, we will assume a fixed variance of $0.01.$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 800x300 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp4AAAEYCAYAAAD1d/bLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABgYElEQVR4nO3deXiU5d0+/POefZ/se0LCvoOAYBBFq6Jo3VqVPlorVm193PXX2vp0Udr6UqW1tu61Fqp1qwpqFVFQNkF2kC0QspGQfZ19n+v9Y8hAIEASMltyfo5jjsnM3DNzhTsJ51zL95KEEAJERERERBEmi3UDiIiIiGhwYPAkIiIioqhg8CQiIiKiqGDwJCIiIqKoYPAkIiIioqhg8CQiIiKiqGDwJCIiIqKoYPAkIiIioqhg8CQiIiKiqGDwJCIiIqKoiGjwfOmllzBx4kSYTCaYTCYUFxfjs88+i+RbEhEREVGckiK5V/t///tfyOVyDB8+HADwr3/9C4sWLcLOnTsxbty4SL0tEREREcWhiAbP7qSkpGDRokW44447ovm2RERERBRjimi9USAQwHvvvQeHw4Hi4uJuj/F4PPB4POHbwWAQbW1tSE1NhSRJ0WoqEREREfWQEAI2mw05OTmQyU4/izPiwXPPnj0oLi6G2+2GwWDAsmXLMHbs2G6PXbhwIRYsWBDpJhERERFRP6upqUFeXt5pj4n4ULvX60V1dTU6OjrwwQcf4B//+AfWrl3bbfg8scfTYrGgoKAANTU1MJlMkWwmEREREfWB1WpFfn4+Ojo6YDabT3ts1Od4XnrppRg2bBheeeWVMx5rtVphNpthsVgYPImIiIjiUG/yWtTreAohuvRqEhEREdHgENE5nv/3f/+HuXPnIj8/HzabDe+88w7WrFmDFStWRPJtiYiIiCgORTR4NjY24tZbb0V9fT3MZjMmTpyIFStW4LLLLovk2xIRERFRHIpo8Hzttdci+fJERER0VDAYhNfrjXUzaIBSqVRnLJXUE1Gr40lERESR4fV6UVlZiWAwGOum0AAlk8lQVFQElUp1Vq/D4ElERJTAhBCor6+HXC5Hfn5+v/RKER0vGAyirq4O9fX1KCgoOKtNfRg8iYiIEpjf74fT6UROTg50Ol2sm0MDVHp6Ourq6uD3+6FUKvv8OvxYRERElMACgQAAnPUQKNHpdP58df689RWDJxER0QBwNsOfRGfSXz9fDJ5EREREFBUMnkRERBTXCgsL8eyzz8a6Gf1moH0/vcHgSURERDFRU1ODO+64Azk5OVCpVBgyZAgefPBBtLa2xrppMfXEE09AkiRIkgSFQoG0tDRceOGFePbZZ3u97fiaNWsgSRI6Ojoi09heYvAkIiKiqKuoqMC0adNQWlqKt99+G2VlZXj55Zfx5Zdfori4GG1tbTFrWyAQiHlN1HHjxqG+vh7V1dVYvXo1brzxRixcuBAzZ86EzWaLadvOBoMnERERRd29994LlUqFL774ArNnz0ZBQQHmzp2LVatWoba2Fr/61a+6HG+z2XDzzTfDYDAgJycHzz33XJfHn3jiCRQUFECtViMnJwcPPPBA+DGv14tHH30Uubm50Ov1mDFjBtasWRN+fMmSJUhKSsInn3yCsWPHQq1W49VXX4VGozmpp/CBBx7A7Nmzw7c3btyICy+8EFqtFvn5+XjggQfgcDjCjzc1NeHqq6+GVqtFUVER3nzzzR79+ygUCmRlZSEnJwcTJkzA/fffj7Vr12Lv3r146qmnwsf9+9//xrRp02A0GpGVlYWbb74ZTU1NAICqqipcfPHFAIDk5GRIkoT58+cDAFasWIFZs2YhKSkJqamp+O53v4vy8vIete1sMHgSERENIEIADkdsLkL0rI1tbW34/PPPcc8990Cr1XZ5LCsrC7fccgveffddiONecNGiRZg4cSJ27NiBxx57DA8//DBWrlwJAHj//ffxl7/8Ba+88goOHTqEDz/8EBMmTAg/9/bbb8eGDRvwzjvvYPfu3bjxxhtxxRVX4NChQ+FjnE4nFi5ciH/84x/Yt28ffvjDHyIpKQkffPBB+JhAIID//Oc/uOWWWwAAe/bsweWXX47vfe972L17N9599118/fXXuO+++8LPmT9/PqqqqvDVV1/h/fffx4svvhgOhr01evRozJ07F0uXLg3f5/V68fvf/x7ffvstPvzwQ1RWVobDZX5+frj9Bw8eRH19Pf76178CABwOBx555BFs3boVX375JWQyGa6//vrI9/SKOGaxWAQAYbFYYt0UIiKiuORyucT+/fuFy+USQghhtwsRioDRv9jtPWvzpk2bBACxbNmybh9/5plnBADR2NgohBBiyJAh4oorruhyzLx588TcuXOFEEL8+c9/FiNHjhRer/ek1yorKxOSJIna2tou919yySXiscceE0IIsXjxYgFA7Nq1q8sxDzzwgPjOd74Tvv35558LlUol2trahBBC3HrrreInP/lJl+esX79eyGQy4XK5xMGDBwUAsWnTpvDjJSUlAoD4y1/+cqp/HvH444+LSZMmdfvYL37xC6HVak/53C1btggAwmazCSGEWL16tQAg2tvbT/kcIYRoamoSAMSePXu6ffzEn7Pj9SavsceTiIiI4oo42tN5fO3I4uLiLscUFxejpKQEAHDjjTfC5XJh6NChuOuuu7Bs2TL4/X4AwI4dOyCEwMiRI2EwGMKXtWvXdhlaVqlUmDhxYpf3uOWWW7BmzRrU1dUBAN58801ceeWVSE5OBgBs374dS5Ys6fK6l19+OYLBICorK1FSUgKFQoFp06aFX3P06NFISko6q3+b4/9ddu7ciWuvvRZDhgyB0WjERRddBACorq4+7euUl5fj5ptvxtChQ2EymVBUVNSj550tbplJREQ0gOh0gN0eu/fuieHDh0OSJOzfvx/XXXfdSY8fOHAAycnJSEtLO+3rdAaw/Px8HDx4ECtXrsSqVatwzz33YNGiRVi7di2CwSDkcjm2b98OuVze5fkGgyH8tVarPalI+vTp0zFs2DC88847+N///V8sW7YMixcvDj8eDAbx05/+tMt80k4FBQU4ePBgl3b2h5KSknBIdDgcmDNnDubMmYN///vfSE9PR3V1NS6//HJ4vd7Tvs7VV1+N/Px8vPrqq8jJyUEwGMT48ePP+LyzxeBJREQ0gEgSoNfHuhWnl5qaissuuwwvvvgiHn744S7zPBsaGvDmm2/iRz/6UZfAtmnTpi6vsWnTJowePTp8W6vV4pprrsE111yDe++9F6NHj8aePXtwzjnnIBAIoKmpCRdccEGv23rzzTfjzTffRF5eHmQyGa666qrwY1OmTMG+ffswfPjwbp87ZswY+P1+bNu2DdOnTwcQmmvZ19JGBw4cwIoVK/DYY4+Fb7e0tOCPf/wj8vPzAQDbtm3r8pzutrpsbW1FSUkJXnnllfC/yddff92nNvUWh9qJiIgo6p5//nl4PB5cfvnlWLduHWpqarBixQpcdtllyM3NxZNPPtnl+A0bNuDpp59GaWkpXnjhBbz33nt48MEHAYRWpb/22mvYu3cvKioq8MYbb0Cr1WLIkCEYOXIkbrnlFvzoRz/C0qVLUVlZia1bt+Kpp57C8uXLz9jOW265BTt27MCTTz6JG264ARqNJvzYL37xC3zzzTe49957sWvXLhw6dAgff/wx7r//fgDAqFGjcMUVV+Cuu+7C5s2bsX37dtx5550nLajqjt/vR0NDA+rq6rBnzx4899xzmD17NiZPnoyf//znAEK9qiqVCs899xwqKirw8ccf4/e//32X1xkyZAgkScInn3yC5uZm2O12JCcnIzU1FX//+99RVlaGr776Co888sgZ29QvzjgLNIa4uIiIiOj0TrfoI95VVVWJ+fPni6ysLKFUKkV+fr64//77RUtLS5fjhgwZIhYsWCBuuukmodPpRGZmpnj22WfDjy9btkzMmDFDmEwmodfrxXnnnSdWrVoVftzr9Yrf/va3orCwUCiVSpGVlSWuv/56sXv3biFEaHGR2Ww+ZTvPPfdcAUB89dVXJz22ZcsWcdlllwmDwSD0er2YOHGiePLJJ8OP19fXi6uuukqo1WpRUFAgXn/9dTFkyJAzLi4CIAAIuVwuUlJSxKxZs8Rf/vIX4Xa7uxz71ltvicLCQqFWq0VxcbH4+OOPBQCxc+fO8DG/+93vRFZWlpAkSdx2221CCCFWrlwpxowZI9RqtZg4caJYs2bNaRd89dfiIkmInhY/iD6r1Qqz2QyLxQKTyRTr5hAREcUdt9uNyspKFBUVdemNI+pPp/s5601e41A7EREREUUFgycRERERRQWDJxERERFFBYMnEREREUUFgycRERERRQWDJxERERFFBYMnEREREUUFgycRERERRQWDJxERERFFBYMnEREREUWFIpIvvnDhQixduhQHDhyAVqvFzJkz8dRTT2HUqFGRfFsiIqJB7y8rS6P6fg9fNrJXx8+fPx//+te/8NOf/hQvv/xyl8fuuecevPTSS7jtttuwZMmSfmwlxVpEezzXrl2Le++9F5s2bcLKlSvh9/sxZ84cOByOSL4tERERJYD8/Hy88847cLlc4fvcbjfefvttFBQUxLBlFCkRDZ4rVqzA/PnzMW7cOEyaNAmLFy9GdXU1tm/fHsm3JSIiogQwZcoUFBQUYOnSpeH7li5divz8fJxzzjnh+4QQePrppzF06FBotVpMmjQJ77//fvjxQCCAO+64A0VFRdBqtRg1ahT++te/dnmv+fPn47rrrsOf/vQnZGdnIzU1Fffeey98Pl/kv1EKi+hQ+4ksFgsAICUlpdvHPR4PPB5P+LbVao1Ku4iIiCg2br/9dixevBi33HILAOCf//wnfvzjH2PNmjXhY379619j6dKleOmllzBixAisW7cOP/zhD5Geno7Zs2cjGAwiLy8P//nPf5CWloaNGzfiJz/5CbKzs3HTTTeFX2f16tXIzs7G6tWrUVZWhnnz5mHy5Mm46667ov1tD1pRC55CCDzyyCOYNWsWxo8f3+0xCxcuxIIFC6LVJCIiIoqxW2+9FY899hiqqqogSRI2bNiAd955Jxw8HQ4HnnnmGXz11VcoLi4GAAwdOhRff/01XnnlFcyePRtKpbJLfigqKsLGjRvxn//8p0vwTE5OxvPPPw+5XI7Ro0fjqquuwpdffsngGUVRC5733Xcfdu/eja+//vqUxzz22GN45JFHwretVivy8/Oj0TwiIiKKgbS0NFx11VX417/+BSEErrrqKqSlpYUf379/P9xuNy677LIuz/N6vV2G419++WX84x//wOHDh+FyueD1ejF58uQuzxk3bhzkcnn4dnZ2Nvbs2ROZb4y6FZXgef/99+Pjjz/GunXrkJeXd8rj1Go11Gp1NJpEREREceLHP/4x7rvvPgDACy+80OWxYDAIAPj000+Rm5vb5bHOzPCf//wHDz/8MP785z+juLgYRqMRixYtwubNm7scr1Qqu9yWJCn8+hQdEQ2eQgjcf//9WLZsGdasWYOioqJIvh0REREloCuuuAJerxcAcPnll3d5bOzYsVCr1aiursbs2bO7ff769esxc+ZM3HPPPeH7ysvLI9dg6rOIBs97770Xb731Fj766CMYjUY0NDQAAMxmM7RabSTfmoiIiBKEXC5HSUlJ+OvjGY1G/OxnP8PDDz+MYDCIWbNmwWq1YuPGjTAYDLjtttswfPhwvP766/j8889RVFSEN954A1u3bmWHVxyKaPB86aWXAAAXXXRRl/sXL16M+fPnR/KtiYiIKIGYTKZTPvb73/8eGRkZWLhwISoqKpCUlIQpU6bg//7v/wAAd999N3bt2oV58+ZBkiT8z//8D+655x589tln0Wo+9ZAkhBCxbsSpWK1WmM1mWCyW0/5AEhERDVZutxuVlZUoKiqCRqOJdXNogDrdz1lv8hr3aiciIiKiqGDwJCIiIqKoYPAkIiIioqhg8CQiIiKiqGDwJCIiIqKoYPAkIiIioqiI2l7tRHRmwaCANxCENxCEzx+ELyDg67wdCMIfEJDLJChkEuQyCUq5rMtthUwGuTx0WyWXQSaTYv0tERERhTF4EkVBIChgd/thdftgcflgdftgc/thdYWuPf5QsAwE+6+sriQBepUCRo0CRo0SRo0CBo0CpuNu61T8E0BERNHD/3WI+pHV7UOT1YNmmwcWlxdWVyhs2j1+RHurBiEAu8cPu8ePeou722MUMgkGjQIpehXSjWpkmjTINGlgUPNPAxER9T/+70LUB0IItDm8aLZ7wkGzyeaB2xeIddN6xR8U6HD60OH0oaLZEb7foFYgw6RGhlGDTFMokOoZRokoQQkh8NOf/hTvv/8+2tvbsXPnTjz00EOYPHkynn322VM+r7CwEA899BAeeuihqLV1oOP/JEQ9YHX7cKTNhQarC01WD1rsHvgCcbvb7Fmze/ywN/u7DaN5yToUpemRolfFsIVENFA0NDTgySefxKeffora2lpkZGRg8uTJeOihh3DJJZf0y3usWLECS5YswZo1azB06FCkpaVh6dKlUCqV/fL61HMMnkTdsLl9ONLuwpF2F2ranLC4fLFuUswdH0bXlTbDpFWiKE2HIal65CfroFKwSAZRotu2bRseffRRPP3005g2bVrE36+qqgrnn38+kpKS8PTTT2PixInw+Xz4/PPPce+99+LAgQP98j7l5eXIzs7GzJkzw/elpKT0y2tT7/B/CiIADo8fBxqsWLW/EUs2VOIf6yuxYm8D9tZaGDpPwery4dsaCz7eVYdX1pbjg+1HsP1wO1rtnlg3jYj66PXXX8fq1avxxhtvROX97rnnHkiShC1btuCGG27AyJEjMW7cODzyyCPYtGkTAKC6uhrXXnstDAYDTCYTbrrpJjQ2NoZf44knnsDkyZPxxhtvoLCwEGazGT/4wQ9gs9kAAPPnz8f999+P6upqSJKEwsJCAMBFF13UZQi9qakJV199NbRaLYqKivDmm2+e1F6LxYKf/OQnyMjIgMlkwne+8x18++23PW4LAASDQTz11FMYPnw41Go1CgoK8OSTT4Yfr62txbx585CcnIzU1FRce+21qKqq6o9/7rjAHk8alIJBgdoOF8qb7Tjc6kSbwxvrJiU0f1Cgus2J6jYn1gHh3tARGUbkJWshSSzrRBRPHI5j02iqq6vR2toKSZLwzjvvAADefvtt3HTTTRBCIDU1FQUFBQAAvV7fb21oa2vDihUr8OSTT3b7uklJSRBC4LrrroNer8fatWvh9/txzz33YN68eVizZk342PLycnz44Yf45JNP0N7ejptuugl//OMf8eSTT+Kvf/0rhg0bhr///e/YunUr5HJ5t+2ZP38+ampq8NVXX0GlUuGBBx5AU1NT+HEhBK666iqkpKRg+fLlMJvNeOWVV3DJJZegtLQ03IN6urYAwGOPPYZXX30Vf/nLXzBr1izU19eHe3adTicuvvhiXHDBBVi3bh0UCgX+8Ic/4IorrsDu3buhUiX+FCcGTxo03L4ADrc6UdFsR1WrM64WAtWU7sF/X12Eq+/6OfJHToh1c85aZ2/otzUWmLVKjM42Ymy2CUm6xP+jSTQQGAyG0z7e3NyMWbNmnXS/6MfyHGVlZRBCYPTo0ac8ZtWqVdi9ezcqKyuRn58PAHjjjTcwbtw4bN26Feeeey6AUC/ikiVLYDQaAQC33norvvzySzz55JMwm80wGo2Qy+XIysrq9n1KS0vx2WefYdOmTZgxYwYA4LXXXsOYMWPCx6xevRp79uxBU1MT1Go1AOBPf/oTPvzwQ7z//vv4yU9+csa22Gw2/PWvf8Xzzz+P2267DQAwbNiw8L/1O++8A5lMhn/84x/hD+yLFy9GUlIS1qxZgzlz5vThXzq+MHjSgGZx+VDRbEdFswO1Ha5+rZPZn7au/Ahl327GtlUfDYjgeTyLy4fNFW3YXNGGnCQNxmSbMDLTCI2y+14HIhocOkPs6UZESkpKkJ+fHw6dADB27FgkJSWhpKQkHDwLCwvDQQ8AsrOzu/RWnklJSQkUCkWXea2jR49GUlJS+Pb27dtht9uRmpra5bkulwvl5eXh26drS0lJCTwezykXTW3fvh1lZWVdng8Abre7y3skMgZPGnCabR4carShvMWBFlts5ht6XM4zHtPeVAentQOQJOxc8ykAYMfqTzHpwrmAENCZkpCckXPG11FrdWfb3Kip63CjrsONtQebUZSux5hsE4pS9dxhiSjK7HZ7l9u7du3qtofz66+/xuTJkyPShhEjRkCSJJSUlOC6667r9hghRLfB9MT7T1ydLkkSgsFgj9vSkxAcDAaRnZ3dZYi/0/EB9XRt0Wq1p21HMBjE1KlTu51fmp6eftrnJgoGTxoQ2h1eHGy0obTRhlZ77OdrPnbtOX16nsPShucfublXz3nmi4N9eq9Y8gcFDjXacajRDp1KjlFZRkzOT+JQPFGUnDinsjMQyWQyBIPB8LVWq+3XeZ3HS0lJweWXX44XXngBDzzwwEnv09HRgbFjx6K6uho1NTXhXs/9+/fDYrF0GQY/W2PGjIHf78e2bdswffp0AMDBgwfR0dERPmbKlCloaGiAQqEIL1DqrREjRkCr1eLLL7/EnXfeedLjU6ZMwbvvvhtevDQQcVU7JSyb24fth9vw1uZqLNlYhW/KW+MidFLvOL0B7KzuwJKNVfj42zrUtJ25t5iI+ldGRgaysrIwdepUvPzyy5g6dSqysrKQkZER0fd98cUXEQgEMH36dHzwwQc4dOgQSkpK8Le//Q3FxcW49NJLMXHiRNxyyy3YsWMHtmzZgh/96EeYPXt2v5Z7GjVqFK644grcdddd2Lx5M7Zv344777yzSw/lpZdeiuLiYlx33XX4/PPPUVVVhY0bN+LXv/41tm3b1qP30Wg0+MUvfoFHH30Ur7/+OsrLy7Fp0ya89tprAIBbbrkFaWlpuPbaa7F+/XpUVlZi7dq1ePDBB3HkyJF++35jiT2elFCcXj8ONdpxsMGGOosr6ttQ9tTCj3b26Lja8pJuezjve+Yt5A7rv0/ziUAIoLzJjvImOzJMapyTn4xRWUbIOQxPFHF5eXmoqqqCSqWCJEn4yU9+Aq/XG15EEylFRUXYsWMHnnzySfy///f/UF9fj/T0dEydOhUvvfQSJEnChx9+iPvvvx8XXnghZDIZrrjiCjz33HP93pbFixfjzjvvxOzZs5GZmYk//OEP+M1vfhN+XJIkLF++HL/61a/w4x//GM3NzcjKysKFF16IzMzMHr/Pb37zGygUCvz2t79FXV0dsrOzcffddwMAdDod1q1bh1/84hf43ve+B5vNhtzcXFxyySUDpgdUEv25RK2fWa1WmM1mWCyWAfMPTr3nCwRR3mxHSb0V1a0uBOP3R7bXjhzah2fu/R4kSQrPWRJC4JEXliJvxLhYNy/mDGoFJuUnYWKemYuRiE7B7XajsrISRUVF0Gg0sW4ODVCn+znrTV5jjyfFrdoOF/bXWVHaaIPX3/NJ4onEkJQKY3IaktKzMeOKG7B5xfvoaK6HISn1zE8+S4lQwsnu8WNDWQu2VLZiTLYJ5xQkc6tOIqIExuBJccXm9qGk3ob9dRa0Owf+jkFJ6Vn4zRurIVcqIUkSiq+ah4DPB0UUigQnUgknX0Bg9xEL9tRaUJSmx3lDU5FpYs8OEVGiYfCkmOscSt9fZ0V1mzNu521GyvEhU5KkPofOaJVwimX5JiGAimYHKpodGJquR/GwVGQYGUCJiBIFgyfFTIPFjb21FpQ22eDxnXooPRGGhONBtEo4xUv5popmBypbHBiWbkDxsFSkGSK7CIKIiM4egydFldsXQEm9FXvrrD0u7p5IQ8IUXUIAZU12lDfbMSLDiPOGpiCVAZQGqTheK0wDQH/9fDF4UlTUtDmxr86CvVVN8Pdg28r+2tUHSKydfc7GYC7hJARQ2mjDoSYbRmUaMWNoKhch0aAhl4cqPni93jPujEPUV15vqE52589bX0U0eK5btw6LFi3C9u3bUV9fj2XLlp1yWywaeBweP/bXW7Gv9thCoZ9fPbnvr9eHXX2A+BkaPp4QgL1DjrYGJdqbFXDZ5fA4JXhcMridMnicR69PuO3zyiCXCyiUx11UAnJF1/vkSgG1NoikND/M6X4kpftCX6f5oVKH5kSeWMJJpdYkfEgXAjjQYENpox2jsgw4b2gqd0OiAU+hUECn06G5uRlKpRIyGfeGof4VDAbR3NwMnU4HheLsomNEg6fD4cCkSZNw++234/vf/34k34rihBACVa1O7Km1oLLZMaBqbvaWyy5DW6MSrfVKtDUo0NaoRFvDsYvXE5v/HDR6FWTyTKjUucge+kO01r8Or7sWLkcWggFANgDKZQaFQEl9KIBOzDNjRlEqtKoB8I0RdUOSJGRnZ6OyshKHDx+OdXNogJLJZCgoKDjtfvY9EbUC8pIk9brHkwXkE4fV7cO+Wiv21Vlgc/tPeVxPVl536q8h4Uj34gkBtNQpUVumxpEyDY6UqlFbroHDevqgI0kCplQ/kjP80JsCUGuD0OiDoWtdEGrdcdfaINT6IJSqIIIBCX6fBL/36HWXiwx+n4SAT4LLIYOlRYGOZsXRayU8rs6w6wGgAiABEAC8ANRQaYLIHeZB7nA38kZ4kDfcjcwhXpzlyErMqZUyzChKwaS8JCjk7A2igSkYDIaHQ4n6m0qlOmVvesIWkPd4PPB4ji04sVqtMWwNnUkgKFDZYseeWgsOt/asDFJvQmA8DgkHg0BLrRI1hzSoLVOHr92O7pOZIcmPlEwfUrJ8SMkKfZ2a5UNKtg/J6X4oVNHtEXY5ZLA0K9BxNJB2NCvR0axAU40KdeVBeN0yVO7TonLfsXliClUQOUUe5I3wIHe4B4VjXMgq9OIsP/RGlccXxLrSFnxbY8H5w9MwKssY6yYR9TuZTMadiyjuxVXwXLhwIRYsWBDrZtAZdDi92Ftrxf56CxyeQMTeJ5a7+nTy+4Dqgxoc2qVD2bc6HCnVHNdreIxCGUTO0FA4yxvhRu4wDzLyvVBr42uqgVYfhFbvRVbhyb0iwQDQXKtCTakatWUaHClTh0K1U47qg1pUHzwWRs1pPoye5sToaQ6MnOKE1pAYO0tZXD4s31OPndXtuGBkOnKTuBCDiCia4mqovbsez/z8fA61xwF/IIiyZjv2HLGgtsMVtSLvfq83vKuPECLiu/oEA0BtuToUNHfpULFXC6+7a9BUqkPD0Xkj3MgbHrrOLPBCHlcf4/pHMAi01h+bRlBTqkblPi383mP/JjKZwJAxboye5sDocx3IHe5BoqxtGJ5hwKzhaUjmCngioj5L2KF2tVoNtZo1+OJJo9WNfXUWHGyww+2LXO/mqfTXrj6nIgTQeFiFQ7t0OLRLh/LdWrjsXYfN9WY/Rkx2YfgkJ4aOdyEj3zsgFuD0hEwGpOf6kJ7rw+TZdgCA1yOhco8WB7bpcWCbDo3V6vDw/Gf/SoPe7MeoKU6MPteBMdMd0Jvitze0rMmOyhYHJuSZcR4XIBERRVxcBU+KDy5vACUNVuzrRZH3ROL3AWXf6rB3owF7vzHA2tr110CjC2DYJBdGTHZi+GQnsoZ4E6YHLxpUaoFR05wYNc2JawG0NSpw8GgILd2pg8OiwI7VJuxYbYJcITBmugNTL7Fi3AxH1Oe09kQgKLCrugMH6m0oHpaKiblmyGQJNIGViCiBRDR42u12lJWVhW9XVlZi165dSElJQUFBQSTfmnopGBSoanVgX50VlS0OBHpQ5D2RuOwylGzVY+9GPUq26uFxHuvZUqqDKBrXGTRdyBvhTvhV3NGUkulH8VUWFF9lQcAPVO3X4uB2HfZtNqC+Qh0K+BsN0BoCmHShDdMutaJonDvuFie5fQGsPtCEPbUWXDQyHfkpiV3TlIgoHkV0jueaNWtw8cUXn3T/bbfdhiVLlpzx+SynFHntDi/21VlRUm+F3XPqMkiJqL1JgX2b9Ni70YCyb3UIBo4lHWOKH+POs2PCTDuGT3ZBGYc9cQNBfaUK2740YcdXRlhalOH7UzJ9mHqJFVMvtSIjzxfDFp7aiEwDLhiRDrNWeeaDiYgGsd7ktagtLuoLBs/IcHj8KG204WCDDfUWd6yb06/aGhTYsdqI3V8bceRQ17IimQUejCt2YMJMO/JHuTl8HkXBAFC+W4ttX5qwe72xS2WAglEuTL3UhqnfsUJnjK/5oEq5hKlDUjCtMBlK1v8kIuoWgyedxOMPoLzJgQMNVtS0uQbUjkK2djm+XW/Ajq9MqNp/rDyOJAkUjnVjXLEd42fa47ZnbbDxuiXs/caAbatMKN2uQzAY6olWqYM4d44VF1zXjoz8+DpXRo0CF45Mx8hM1v8kIjoRgycB6Czw7sDBBhsqW+zwBeL2VPea2yHDno167FhtwqEdx8KLJAkMn+TC5ItsGF9shzE5+ivxqeds7XLsXGPE5hVm1Fceq2gxZrodF17fgZFTnHE1FzQvWYuLRmUg3cjqG0REnRg8BzEhBI60u3CwwYZDTbEpgRQpfq+Ekq067Fhtwr5N+i61JPNHujHlYismX2SDOXXgfM8DSU3pHvz31UW4+q6fI3/khC6PCQGUfavFuqXJ2L9ZDyFCaTNriAcXXN+BqZdYoVLHx58qmSRhYp4ZxcNSoVFyFRoRUcLW8aS+8QWCqG5zovxoTUKnd+AELyGA6gMabPnChF1rjV1qbKbneTHlO1ZMudiG9Nz4Gpqlk21d+RHKvt2Mbas+Oil4ShIwYrILIya70FyrxPoPk7DlczMaDqvx3rOZWP7PNBRf1YHzr+mI+QeLoBDYVdOBg402zByWigm5Zkjx1C1LRBTH2OOZoFzeACpa7ChvdqC61TGghtEBwNomx/YvTdjyuQmN1ceGNc1pPpxzkQ1TLrYhd7gnroZhByOPy3nax9ub6uC0dgCShMUL7oPD0ga9OQW3P/48IAR0piQkZ+R0+1yXXY5tX6bgm08z0N4YWlkukwtMnm3DJfPakF108rafsZBhUuPiURnI4fabRDRIcah9gLI4fShrtqO82Y76DveAWiAEAAE/sH+zHls+N6Nkiz48b1OpDmLiLDumz7Fg2ETXoNk1KBE8MmdUxN9j0WcHse8bA9YtTULF3lBtTUkSmHShHXNuae123/lOpxve70+SBIzOMuGCEWnQqzmQRESDC4faBwi3L4Aj7S4caXeips2JFnt89PD0t7pKFbZ+bsb2L42wW479SA4Z48L0y62YPNsGrT66ZXaiFVjozORyYOIsOybOsqOmVI2v3k3Bt+uN2LXWiG/XGTDpQhvm/LANWUNO/v043fB+fxICKKm3orzZjhlFKTinIBly7n5ERHQS9njGEbcvgNoOVzhsNts8iN+zc3bcDhl2rA6tZq4pPVZv05jix7RLrZg+x4rMgtgF7aUv/AFff/QGLrjuVlx/z69j1o54d6ahdgCoLS/B84/cfNL99z3zFnKHjTnj89XarjsIeVxO1Fdp8NU7Wdj7TRKAUA/ohFkduGReA5Tqyj4P73f3fn2Roldh9sh0FKbpz/q1iIjiHXs8E4THH0Bte2fQdKHJ5h6wQRM4ulDooAabPjNj52ojvO7QqnSZXGBcsR3T51gx+lxHv29X2ZNwBHSdj7hzzacAgB2rP8WkC+f2KLB06o/gkih68r2q1KEPFpIkQQgRvlapNX36t3rs2nOOuzURwG8hxPexe30ydq83Axh70nMclrZuw293nvniYK/bdKI2hxfLdtZiWIYBF45IQ5JOddavSUQ0EDB4RonV7UOzzYNmmwct9tC1xeUb0EGzk8shw44vjfhmuRl1Fcd6NzPyPThvrgXTLrXBkBS5lcpdg0rv9CawdOqP4DKQGJJSYUxOQ1J6NmZccQM2r3gfHc31MCSl9sOr7wZwA4BJAB4HcD2AfwOYDyD2W8CWN9lR1eLA5PwkzBiaArWCE5SJaHDjUHs/CwQFWu0eNB0XMFvs3gFVT7MnhAAOl2jwzXIzdq01wucJ9W4qlEFMutCO4istKBrvisqq9GgsgDkeg+fJ/F4v5EpluLcz4PNBoepbL+DperDrKrT48p0s7N9cDmDqSY/3ZHg/Uj3WOpUcxcNSMT7HDBnnfxLRAMKh9hgqb7bj0931sW5GzDhtMmxbZcKm5aEajJ0yh3hQfKUFUy+xQm+K7kKhhR/t7PGxZzsfsb9IEqBSyKBWyI9ehy4KmQxBIcKXQBAIBgUCQiAQPHp/UCAgQvVdXXFS0/X4kClJUp9DJ3D6YFg0Drjz943Y/mU93nwKAGQAgsdda2M2FcLpDeDLkiZ8e8SCi0amIz9l8EzJICLqxOBJZ00IoGq/BpuWm7FzrTG8o5BCFcQ5s20470oLCse6Y1ZzszdBo7/nI570+goZTFolzMddTBoFNEo51AoZVJ0XuaxfipL7A0E4PAHYvX7Y3X7YPaGLw3PstsPjhz8YtwMffTJsohHG5DRoDTnwee9Ee+MbAGqw+HeTceV8LaZfbun3ucQ91WLz4P3tRzA0XY/ZI9M5/5OIBhUGT+ozl+NY7+bx+2xnF3lw3pUWTLvECq0hur2bZ6s/5iMq5RLSjWok6VRdAqZZq4x6jUeFXAazTgazTnna4zrnIDdZPWi2e9BkdcPmjv0cyb5KSs/Cb95YDblSCUDCrnXfw6evmdDWYMR7zwLrliXh6jtbMGa6I2YfiCqaHTjc6sSk/CTMKErh9ptE1CuBoEBlix0efxDjcsyxbk6PcY5nPytttA3oofbOLSy/WW7GzjXHzd082rtZfJUFQ8bErnezP/RmPqJCJiHNqEamSY0MowaZJg1S9aoBMYfP7QuEwqjNffTag3aHL2E3LvD7gI2fJOGLf6fCaQuFvOGTnLjmJ83IG+GJadt0KjnOGxrafnMg/OwQUeTUdrhwoN6K0kY73L4AxuaYcPm4rJi2iTsXxdBADZ5uhwzbvzLim0+7rkzPGuJB8VWhuZs6Y2L1bvaWXCYhzRAKmZkmDTJMaqTq1YOqULgvEER9hxuH2xyoanWixRbbwNYXLrsMq95OwfoPk+D3hT44TfmOFVfe3oKUzNj28ibrlCgeloaRmQbu/05EYe0OL0rqrTjQYIPF5evyGINnP2LwjC0hgMMHQnM3d609VndToQxi8mw7iq/qiOnczUiTJCDNoEZ+ig4FKTrkJmmhUshi3ay4Yvf4Ud3qxOFWB6rbnHDGyWKmnmhrVGD54jTs+Cr0t0WhDOLiG9vxnXltUGtj+2cxw6TG+cPSWICeaBBzev042GDDgQYbGizuUx7H4NmPGDxjI7wy/TMzGqqOzd3MyA/1bmYXbsCqt58akNtJmrVKFKTokJ+iQ36KFjoVp0H3lBACzTYPqo4G0XqLG4EEWLRUU6rGx6+mo/zb0OIxc5oP372zBVMutsX8Q1VeshbnD09DTpI2tg0hoqhw+wIoa7KjrMmOw63OHk1tSrTgyf9VCUCod7NijxabPjPj2/WGLivTJ18YWpleNC7Uu7n0hWVR2f86GjRKOYak6pCfHOrVPNMiHDo1SZKQYdIgw6TB9KIUePwBVDQ7cKDBiupWV9zODc0f6cE9Tx/Bng0GfPxKOtoalXjzj9nY8HESrr+nCfkjYzed4Ei7C+9urcHQdD3OH56GNIP6zE8iooTi9gVQ3mzHoUY7qtucCfGB/WwweA5iHpcTdoscO75KwbaVqWiuPW7uZqEL0+e0YvLsdmgNAbQ31aFyb8dZbycJxH5LSaNGgWHpBgxLNyAvWcvFHBGiVsgxJtuEMdkmODx+HGy04UC9DY3WUw8ZxYokARNn2TFmugNrP0jGqrdSULVfi2fvL8D0y0PzP43JsZtGUNHsQGWLA6OzTCgelgqzlh+QiBKZ2xf6YH6oyYbDrQM/bB6PQ+39LBGG2gMBoHS7Dq/+eiOA6wB0rti2A3gbwKsAtkbs/WOxs0+KXoXhGaGwmWlSc+FGDLU5vDjQYMWB+pMnyceLjhYFPvnHsfmfGl0Ac37YhlnXtkMR48wnl0mYkGvG1MJkmDQMoESJwusPorzZjtJGG6pbnf1WPznRhtoZPPtZPAfPxmoltq40Y9sqE6ytx3d2b0EobL6DUPiMrGgET0kCMk0aDEs3YHiGASl6FumOR3UdLhxoCJUFiZddlo5XuU+DD1/KQE1paDQgPdeLa+9uxtgZjhi3LBRAR2UZcW5hCn++ieLYkXYn9tVZUdZkh9ff/9VfGDz7EYPn2XM5ZNi1xogtX5hwuOTYAgW92Y9JF7Zj2iWtyC7q2dBnf20nGcmh9gyTGqOzTBiZaYCRvUEJIxAUONhgw66ajrgbig8GgW0rTfj0n2mwtYc+sI0+14Hr/rcJGXmx77GVJGBYugHTi1KQadKc+QlEFHE2tw8l9Tbsr7Og3RnZvxOJFjw5x3MACgaBsl06bPnchN0bji0UkskExkx34Nw5VoydYT86ZCgD0LMgGOntJPvKrFVidJYRo7NN7PlJUHKZhLE5JozNMaG2w4Vd1R0oa7LHxYIkmQyYfrkVE2fZsfKtFKxblowDW/VYtLMQF17fjstuboNGH7satkIgvAq2IEWH6UUp3AeeKAYCQYGKZjv21Vl7vCJ9MGLwTHA1pXvw31cX4eq7fg6laip2rDZi2yoTOpqP9fZlDvFg+hwrpl5ihSml78OZ/bGdZH/RquQYmWnA6CwTS80MMLlJWuQmaWF1+7C7xoI9tRa4fbEfhtfog7j6rhacd6UFH72cjv2bDVj9Xgq2fWnCd+9oxtRLbJDFuMxrdZsT1W1OZJs1mFaYgmHpes5nJoqwZpsH++osONBgi8spQ/GGQ+39LNpD7W8vehJbV74OneleOK3Ph+/XGgI45yIbpl9uQf5IT7/VI+zNdpL9TSmXMCzdgFFZRgxJ1Q+qHYMGM18giAP1NuysaUer3Rvr5oTt36zHRy+no7k29PM/ZIwL19/ThIJR8bObU5pBhSlDkjEq0wiFnJsfEPUXIQQqWxzYdrgdte2umLYl0YbaGTz7WV+Cp8fl7NXxtRWN2PO1H6U7zKiv/D6AJgAZkMk/xZAxDkyYKce0y4xQqnp2amNd3uhMcpI0GJdjxohMA9QKeaybQzFU3erEjup2VLbEfnEPAPi9EtZ9mISVb6bC45JBkkRclF86kUYpx9gcEybmmpHM6ShEfRYIChxosGLH4Xa0xMkHYQbPbrz44otYtGgR6uvrMW7cODz77LO44IILzvi8wRI8H5kzqgdHGQBcC+B/AHz3uPslAGd3CmNR3uhMdKpQDchxOSaksmg2naDB4sbmylZUNMdHALW0yvHpa+nYtqoUwKNQqRdi7u1FmHVNB+RxNqEpP0WHiXlmDEs3cNSAqIc8/gD21lqws7oDNrc/1s3pItGCZ8T/JL777rt46KGH8OKLL+L888/HK6+8grlz52L//v0oKCiI9NsnODNCIfN6AHNxbBHQvwHcBiCAsw2d8UQmSShM02FcjglFafxPkU4ty6zBtZNz0Wh1Y1NF7AOoOTWAmx9tgNfzAnavXw2v5y189PJfsWm5Gdfd3YxR03o3qhFJNW1O1LQ5oVfLMT7HjPF5ZtYDJToFh8ePndUd2F3bAY8vdosIB5KI93jOmDEDU6ZMwUsvvRS+b8yYMbjuuuuwcOHC0z53sPR4Hj/Ubm1ToGSzGfs2mVG+x4hg4Fj4SstxY9Lsdky6oANe965+KW0ExH6oPUmnxLgcM8bmmGBQx1n3ECWESAfQ002HaW+qg9PaAUgSFi+4Dw5LG9TaVEiy5XA75ADSMGpqEq78cR0y8k6e/xnr37/OD3wT85JQmKrjYiQiAO0OL7YdbseBemu/FXqPFPZ4Hsfr9WL79u345S9/2eX+OXPmYOPGjZF864RibTNjzwYD9m404HCJBkIc+8OfNcSDCefbMeF8O3KHdy4SkuPIofgsbdRTcpmEERkGjM81s/QLnbVMU6gHtMnqxjcRCKCPXXtOr473uFoBzAjfPrhd4OB2PYAXASwA0BZ+LNZTXYJCoKLZgYpmBwxqBUZkhhbwZZtZLYIGn3aHF5sqWnGw0Ya+dMsdX2kmf+SE/m/gABDR4NnS0oJAIIDMzMwu92dmZqKhoeGk4z0eDzyeYz0CVqs1ks2LmWAAqCnVoGSrHns2GFBf2XUOY8FoVzhsnqpAdTyVNuqNZJ0SE/LMGJtthlbFhULUvzIiHED7JhPAxwCuAfAAgB8C+B1CITT2BeiPZz86rLizugMmrRKjMo0YmWlABgvT0wBncfqwqbIVB+ptZ1V/c+vKj1D27WZsW/URg+cpRGVc88Shm84euhMtXLgQCxYsiEaTos7SKsfBbXoc2KZD6Q49nLZjoUsmExg+yYkJ59sxbqYDSWlnnriclJ6F37yxOlzaqPiqeVEtbdQbclmoDNLEPDPykrUcyqOI6wygjVY31pU248hZljtZ+NHO0z7ek129yr4tw6ev5aLhcAqAZ5GW80dceXsdhPD3W7mz/mR1+bC1qg1bq9qQoleFekIzjVzsRwOK1e3D5oo27K+z9jpwdk7BOX66zc41nwIAdqz+FJMunAsIAZ0pCckZOQkxGhkNEZ3j6fV6odPp8N577+H6668P3//ggw9i165dWLt2bZfju+vxzM/PT8g5nn6vhIq9WhzYpsPB7fqTejU1+gBGnuPEuGI7xs5wQG8aeJOWzdrO3k0T9Jy7STFU0WzHhrKWiJU/OXJoH56593snTX155IWlyBsxLnxcMABs/tyMz5akwt4R+p0YeY4D19zdjJyi+CjNciZpRjVGZRoxLF3PEEoJy+YOfbDaW2tFoI9zOHtWkeaYSE2rUbRVYNkrT+Ppp5/GtGnTIvIeZxI3czxVKhWmTp2KlStXdgmeK1euxLXXXnvS8Wq1Gmp14v4hq6oC/vj0brz77/+D170Ift+I8GOSJJA/0o1R05wYPc2BgtFuyAfgSLNMkjA0XY+JeWYUpHChAsWHoekGFKbqsb/eim/KW2H39G85lJ5OfZHJgeIrLThntg2r3knB2qVJKN2px5//V4cZV1gw97bWuKr/2Z0WmwctNg82lLXApFViSIoOhWl65KdoWWeX4p7D48eWqjbsPWKJ+0VDPbXq4/exevVqvPHGGzELnr0R8VXt7777Lm699Va8/PLLKC4uxt///ne8+uqr2LdvH4YMGXLa5ybaqvY//Qn4+c8fAPAcgAdgSvkzRk1zYPRUJ0ZOHZi9mp0MagXG55oxPtcEI0uzUBzzBYLYcbgd2w63w+vvv9/Jvuzq1VqvxCf/SMO3640AAJUmiAuvb8fFN7VDG8P93/tCLpOQbdagME2PwlQ90o2J24lAA4/T68e2qnbsPtIBX6B/Ys/x1S56Mt2mP4bauxvef+P398PS3or09HQsW7YMQgikpqZizJjeVbg5G3HT4wkA8+bNQ2trK373u9+hvr4e48ePx/Lly88YOmPF4ej9YoTq6mq0trYiM1OCQvkO/D5Aa/g3bv3VeZAQmt+hUObA08NpZokyD0SSgPxkHSblmzE0zQAZ625SAlDKZZgxNBUT8szYXNGGPbWWPg+1He/4kClJUo/mW6dm+3Dbb+pRsacDH/89DdUHtVj1dio2fJKES+a1YdY1HVBpEqNXJhAUONLuwpF2F74+1AKDWoEhqToUpemRm6yFTsXpNhR9Tq8f2w+3Y/cRS79+0AS6/l+tUken0szpKmw0Nzdj1qxZ4dvxujElt8w8QTwMDce6vMqZcPs9GkjaHV5sKG/BoUZ7TNshBLBngwGfLUlFY3Wot9CU6secW1ox4wpL3O2A1FspehVykrTISdIgL0kHs44jIxQ5Lm8A2w63RSRwdqejuQF/ue/7J023efj5D5CU3n81NnszrzSa8S7utszsKwbP+JJt1mBCnhmjMo1QyGWxbg5Rv6rtcGHtwWY0Wt0xbUcwAGz/0oQVr6eivSkUztJyvLjitlZMnm2DbID86hnUinAQzU3SIt2ojou/v5TYXN4Ath9ux7dHOqISOI/Xl+k2vdWT4f2vv/4akydPhl6v79f3Ph0Gz7PQl6F2ANi1a1eXLu5OibiT0PFUChlGZRoxMc/MWn404AkhsK/Oio3lLXB4YrvIx++VsPFTM1a9lQK7JdTdmTPUjStvb8WY6Y64LMF0NlQKGXKSNMg0apBh0iDTpOZ8ceqxWAbOWKnavx9/e+j6cNCVyWQIBoPYvn07pkyZEtW2xNUcz0TT108IWm1olw9JkkGIYMLtJHSiNIMKE/KSMCbbyJWqNGhIkoTxuWaMyDRgS2UbdlZ39Mv8z75QqAQuvL4DM66wYN3SZKx+Lxl1FRr84ze5KBrvxJxb2jByinPABFCvP4iqFieqWo716BjUCmSY1MgwhoJopknD0mzUhdsXCpy7agZm4HTaZGitV6KlTonWeiVa61VHr5Vob9ICyIIQ+Zg+/Q4I8RpqamqQkZER62afFns8+8mRI0dw7rnnIj0rB6MuvDZi8zsiSSGTMDzDgIn5SchN4nZ5RB1OL9YdakF5U2znfwKAwyrDV++mYP1HSfB7Q+Pt+SPd+M68Nkw43z5ghuDPxKhRhHpEjWqkGdVIN6phYs/ooNPm8GJndTtK6q39tko91mztclTu06JynwZV+7VoqlHBZT99x49S5UJOgRz/c4MK/9//J+D1emNSlpJD7THi8XhQ1e7B8j0NEZvfEQlmrRIT88wYm2PiylOiblS3OrG2tCliBeh7o6NFga/eTcbmFWb4PDIA26BU/wwX3/BLXHrzUCgGYQZTK2VIM4RCaLpBjTSDGqkGFZSciz7gHG51YGd1B6paHX3aSz1eCAG01ClRuVeLir1aVO7TovlI93nBmOJHapYPaTk+pGZ7kZrtC1+MyQGMyzXh8nGx7eBi8Iyhzp2L4p1MklCUrsfEXDOGpLLQO9GZBIMCu2st2FTRCpc39kXe7R1yrP8wCV+99wQCvucBPICk9D/hohvaMeMKC9TauP3THhUySUKSThkOpCl6FdIMKpi1Sv69SzD+QBAl9TbsqmmPiw9/fSEEUFehRtkuLSr2aVG5Vxvevex4WYUeDB3vQuFYF3KHeZCS5Tvj7/LYnMQKnuzeGmSMms5C72YYOFeKqMdkMgmT85MwOsuIbypasbvG0uu9nXvj+NWrJ+osHj1qmoSNn7wFhw+QpLfR0XwbPnxJYMXrSZh1rQ4zr2qB1nAsJCfifPO+CgqBNocXbQ4vShtt4fuVcgkp+lCPaJpBhdSjX3MhU/yxe/z4tqYDe2otcfFhry/qK1XYucaIXWuNaKnr2qOpUAZRMMqNovEuFI13o3CMCzrjwJuneiImj0FAJkkoTNNhYl4SCtm7SXRWNEo5Lh6VgQm5Zqw92IzqtlMHxLNxukLR3RGiGcBUAIDbAax6S2DVWwYArwB4AUBV3JZqiyZfQKDR6j6pbJZaKUPa0RCaog8F0hSDih/Qo0wIgTqLG3uOdKC00R6zxX1no6lGiZ1rQ2Gz8fCx+ZYKVRAjz3Fi6AQXisa5kD/CA4Uq8b6/s8XfqAHMqFFgXA63sSSKhDSDGt+fmoeyJhvWlrbA6vLFuknHmQFgF4DJAH4G4BEAn2PvRj3GzHBAzkIVJ/H4gqjtcKG2o+sWcxqlHKn6UBhNMaiQxkDa74QQqLe4UdpoQ1mTHTa3P9ZN6rXWeiV2rTVg5xoj6iqOlR6UK4MYPc2Jcy6yYdx59kE/BQZg8BxwOns3J+SaUZiq5zaWRBE2PMOIwlQ9th9ux9aqtn5bYbvwo52nffxMe0MLARzcXo6Nn6Tj0E4TgLn45xNAUpoPM660YMYVViSlJd5/8NHm9gVOGUhT9Eqk6NXHrnUqmLQKjir1UL3FhdJGOw412hIybLodMmxdacK2L02oOXgsbMrkAiOnOHHObBvGz7RDaxj4w+e9weA5QJi0SozLMWFcDns3iaJNcXT/97E5Jnx9qAUHGmxnftIZnGk+Zk/2hp50QQCTLmhAS10rvlluxpbPTehoUeLz19Ow8t+pGFdsx8zvWjDiHOegKcfUEzWle/DfVxfh6rt+jvyRE7o9xu0LoK4jgLqOrkP2SrmEZL0KKbqjvaRHL0k6FeTsCEDD0Z7NQ032OBsl6LnmWiW+/igJW74wweMMDR9IMoHhk0I9mxPOt0NvYtg8FQbPBCaXSRiWbsD4XBMKUjh3kyjWjBol5k7IxsT8JKw52IQmqydi72VISoUxOe2kvaENSaknHZuW48PVd7Zg7o9asXuDARv/a0bFXh32bDBizwYjUnO8KL7SgulzrDAkJeYijv60deVHKPt2M7at+uiUwfNUfAGBJqvnpHMvkyQYNQok65VI0qmQrFMhWRf62qQZmL2kQgi0O32ot7jQaHWjqsUJS4KGzWAQKN2hw/oPk1CyxRC+PyPfg/OvtmDybBuMyfzd6QmWU+pn0SinlGpQYVyOCWOyWXeTKF4JIbC3NrT9pjNCK3LPZm/ohioVNn5qxraVJriP9trI5AKjpjpwzkU2jC92QKMfuL02J1YN6KwUAEnC4gX3wWFpg96cgtsffx4QAjpTEpIzcro8p7+qBChkodJPnYHUrFXCqFHAdPQ6UeqROr1+NFjcoYs1dPH4EvtnyO2UsG2VCV9/mIymo3U2JUlgzHQHLriuIy52D0u0ckoMnv0sUsFTKZcwItOI8blm7ipElEA8/gC2VrZjZ3U7/HG4QtfjkrBzjRHffJqEmtJj89QUyiDGTHdg8mwbxs5wDLhFEY/MGXXWrxGtKgE6lRwmrRImzbFAatIoYNQooVXJoVXKozaMHwwKOH0BODx+ODx+tDt9aLS6UW9xJ+zQeXeaa5XY8HEStnx+7IOZRhfA9MutOP+aDqTnxs/3mmjBk91lcS7brMG4HDNGZhm4ZzpRAlIr5Jg1Ig0T8sz4+lBLl5qS8UCtFThvrhXnzbWisVqJXWuN2LnGiKYadXgoXqUOYlyxHZNn2zD6XCeUg7AETCw5vQE4vQE0WNynPEalkEGrlEOjlEOrOva1RhkKpmqlDBKkLr1znV927bGTEBTiaLAMwOH1w+n1w+4JwOnxw+ULJPSOQWdSfVCNVW+nYt83eggR+odJz/Ni1rUdOPcyCzS6AfzNRwl7PPtZf/R46lRyjMkOLRRKNUR/z1Uiipy6DhfWljafNkTEWmiXFVU4hLY1HBu+1+gCGD/TjkkX2DF8sjNhe0K7K9B/pkoBJxpMBfkHMiGA8j1arHorBaU79OH7R5/rwAXXtWPU1PhefMceT+qTzjJI43LMGJrGMkhEA1VOkhY/ODcfBxps2FDWEpdlZCQJyB3mRe6wVlx5eytqStXYucaIb9ca0dGixLZVZmxbZYZcGcSw8S6MPteB0ec6kVngjfl8t57qLjT2pFIAJZbTVSgQAjiwVYdVb6eicl9oCptMJjDlO1Zc8oN2ZBYk5vac8Y7BM8ZSDSqMzQ4tFNKzIDHRoCBJEsZkmzA8w4Adh9ux7XA7vP74XIQhSUDBKA8KRnlw9V0tqNqvwa61RuzfZEBboxKlO/Uo3anHx38HkjN8oRA6zYER5zgTbliyN5UCKDF0V6EgGAT2fG3AqndSUFsW+rAhVwYx43IrLr6xDanZ8fdhcCDhUHs/68lQu1opw6hMI8bmmJBt5kIhosHO7vHjm/JW7K+zRnT/9/4kRGhrwAPb9DiwVY/y3Vr4fcfGI2VygaLxLow514ERk53IGeZJiB2TzqZSAMXO8VMnTlWh4LZfP4/S7UbsWDMCbQ2hxWUqTRAzv9uB2d9vhzk1McshJdpQO4NnPztV8JRJEoak6jA2x4ShaXooEqQ8BhFFT6vdg28qWlHWZE+4BRxet4Sy3Toc2KpDyVY9Wuu6hjW1LoDCMW4MHe9C0XgXCka7oVIn2DdJcav3VQracNktQVx4fXvCF3tPtODJsd0ISzOoMDbHhFFZJu7tS0SnlWpQ47sTc9BodWNDWQsOt568ACZeqTQCY6c7MHa6A0AzmmuVOLBVjwPbdKjcq4XbKcfB7Xoc3B5avCFXCOSPdKNovAtDx7lQOM51ygDQk52EiHpGAeByAEMw97ZtsW7MoMQkFAFalRyjsowYm21Cpklz5icQER0n06TB96bkoabNiY3lLSdty5gI0nN9SM/twAXXdSAYAOqr1KjYq0XlXi0q9mhhbVOgar8WVfu1WH30OVmFHgwZ40b+CDcKRrmRVeiBQnl2OwnR4LDwo53hrx1WOVa8Xo9tKy8/6bif/vENFI4ZHc2m0QkYPPtZQYoOd10wlHvyEtFZy0/RYV5KAcqb7dhY3ooWW+S24IwkmRzIHeZB7jAPLri2A0IAbQ1KVOzRomKvFhV71Giu1aChSo2GKjU2f2YGcBgyuQVpOV60NX4GANi26lNMOH8uZLLI7iQ0WAyknmS1Vof2JgXWfJCMzcvN8Ho6C7zLAATDc3b1RjV/TmKMwbOfaZQJMHueiBLKsHQDhqbpcbDRhm/KW9HhjJ9dU/pCkoDUbB9Ss304d4716Py8dADnAzgXwDQAlyMYAJpqgM5S5y57O178+fF1Nn8EYBeAEgC+qO0kNFAMlJ7kpiNKfPVuCrZ/aULAH/pZySo0oaM5Hem5WaxQEGcYPImIEoAkSRidZcLIDCP21Vmx7XBbwgfQrpoBfHj0ciJxwrUCwBIAtxy97QWwH289nYmcoaGe1ZxhnoRfNNIfTrcn/c41nwIAdqz+FJMunHvSnvTx3DMoBFC1X4N1y5Kxe70hvMvQsElOXDqvDSOnBhHwfRWuUFB81TxWKIgTXNVORJSAgkGBQ012bK1qQ3OCDsF36m4XIeDUOwld+j+fwGGdjoYqLeqrtHA7uh9pSkrzIWeYBzlDPcgu9CK7yIP0PC/kg6jL5Wz2pI/HHmSPS8KOr0zY8F8z6iqOraEYd54dl/ygDYVjE28+9NniqnYiIoo4mUzCqCwjRmUZUdniwNaqNtS2u2LdrD45Vc/aqXYSmjjLi7wR7QDaIQTQ3qhAXYUatRVq1JWrUVehRmu9Ch0tSnS0KLF/syH8mnJlEJn5XmQXhYJodqEH2UVemNP8CbPr0mDUWK3Exk+SsPULE9zO0AcNhSqIKRfbcOH17cgZyl2GEgWDJxFRgitK06MoTY+6Dhe2VrWhssWRcHVAu9OTnYQkCUjJ8iMly4/xMx3h+90OGeoqVag7Gkbrjy5c8rhkqKvQdOktAwCtIRAOoVmFoR7SzCGJP1x//GrvTr3dkz5WAgFg3zcGbPivGYd2HttDPS3Hi5nf7cC5c6wJf34Go4gGzyeffBKffvopdu3aBZVKhY6Ojki+HRHRoJaTpMW1k3PRYvdgW1U7DjbY+n0npGiuhE5Kz8Jv3ljdp3l6Gn0QQ8e7MXT8saHXYDDUO1pfpUZ9pRr1lSrUV6nRXKOCyy5HxV4dKvZ27X01pfqRXegJh9GsQg8yC7xQaxMj2SfinvTWNjk2LTfjm+VmWFqUAABJJjB2hgPnX92BkVOckHEPloQV0eDp9Xpx4403ori4GK+99lok34qIiI5KM6hxxfgsFA9Lxc7qdpTU2+D29c92gNFeCX18yJQk6awWh8hkQGq2H6nZfowvPtY76vdKaKxRhsJolRqNh0OBtL1RCWurAtZWRbjwfaeULO/RXlEvsoZ4kDXEi4wCb0LsxhSPe9K3NSqwd6MBezYaULFHCxEMzXswmP2YMdeC4qssSMkcmHuoqxQymLRKmE+4BIICTq8fTm/g2LUnAMfRr73+xOztjcrioiVLluChhx7qdY8nFxcREZ09XyCI0kYb9hyxoN5y6sUXp1rkc6q9r29//PmTVkIfLx56z86G2yFDQ7UKDVWqo0P1KjRUqWFr777PRpIEUrJ9yDoaRjMLvMgq9CIjP/4Caaz3pBcCaKhSYc8GA/ZuNOBIWdepD4VjXTj/mg5MmmWHQhVf/3Z9lWZUI9OoDgVL3bGAqVP1rQ/QFwjC6Q19oDRrlf3Z1F6Lu73aexo8PR4PPJ5jqzOtVivy8/MZPImI+kmTzY09Ryw40GA7qcfkbFZAdyceV0X3B3uHHPVVKjRWh4Jow2EVGg+r4bB2v7pekgmkZPmQmR8KocdfDObE7LXqi2AQOFyiCYfNlrrjerNlAkXjXJhwvh3ji+1IzU783k2FTEJ+ii40BztdD5MmtuEwkhJ2VfvChQuxYMGCWDeDiGjAyjBqcMkYDWaNSMPBBht2H7EkfDmmaDMkBTBisgsjJh+rIiBEKJA2HFah4XBouL6hKvS10yZHa50KrXUq7N/c9bX0Zv/REHosmKbleJGS6U/4nj6vW0JdhRpHytSoKdVg3zclcNr+D8DTAKZBoQxi5BQnxs+0Y3yxA4ak/pkOEksGtQKFRxf7FaTooFJwMuqJet3j+cQTT5wxHG7duhXTpk0L32aPJxFR/Kq3uLD7iAV7qxrhC3T/X0JfVkIn+lB7f+gMpI01KjRVq0LXRy/tjafuAZMkAXOqHylHd3hKzfIhJcsX3vHJmByIq/JPbqeEunINag6pUVumwZFDajTWqMJzNUMeAPAc0vPuxtz5v8HoaQ5odIkdrgEgw6TG0DQDhqbrkWFUQ4qnExMlEe3xvO+++/CDH/zgtMcUFhb29mUBAGq1Gmq1uk/PJSKivsk2a5Ft1uLiURmoaLHjYIMNh1udCASPhYJ4XwkdryQJMCYHYEx2YfjErnVWPS4JLXWhIfvOMNpYo0JrnRJetyxch7Riz8mvq1QHkZLlgynFD0NSAMakAAxJfhiTA6HbyUfvTw5AeZY9p36vBLtVDodFBodFAYdFDrtFDodVjuYjShwp06ClVhnePeh4OlMZ0nNqkZ7vwd6Nb8HtAJy2/8CUciHqyhNnp6QTaZRyjMk2YkKuGakG5pbe6HXwTEtLQ1paWiTaQkREMaRSyDA6y4TRWSa4fQGUN9tR2mhDdasrLldCJzq1ViB3WGiLz+MJATgscrTWK9HaoAxdH/26rV6JjmYFfB4ZGg+r0Xj4zKFHrQtAbwpAoQBkcgG5QkAmP/q1PPR16L7Q1yIY6qV1WEMB0+Psfu7qiZLSfcgb7sHeb54GsAPADjit9ThsBQ4fOHacw9LWbe95IswJzk3WYkKuGSMyDFDIOYzeFxGd41ldXY22tjZUV1cjEAhg165dAIDhw4fDYDCc/slERBQzGqUc43LMGJdjhssbQFlTBkZ/ugkNdj8A7n0dSZIUmkdqSApgyJiTqxD4fUB7UyiE2jrksHcoQtftctg6FLB3fm2RI+CTwePseXg8FZlMQG8OBVh9UgAGUwB6cyAUNkeEwrMxOTRH85E5vz+r94o37N3sXxFd1T5//nz861//Oun+1atX46KLLjrj81lOiYgovjg8fpQ22lDe7EBdh6vLcDz1XSQK8wsRKgll65DDaZUj4JcQDACBgIRgQELADwQ7vw7g6H0SJJmAwRwKlp1hU2sI9rho+4lluXo6PzjehtrZu9lzcbOqfcmSJViyZEkk34KIiKJIr1bgnIJknFOQDK8/iJp2J6paHKhqdcLq8sW6eQkrEoX5JQnQGoLQGoIAonduTgyQiTQ/WCmXMCbbhMn5SezdjJC4KqdERESJQ6WQYVi6AcPSQ1On2hxeVLY4cLjVgdp2F/zsDe3ixJ7A4wvz71zzKQBgx+pPMenCud0W5o+3kNZTiTA/WKeSY1J+EiblJUGrOrtpCXR6USkg31ccaiciSky+QBA1bU4cbnWipt2JNocX8fu/TXScbYH+RFh8cyqx3inpVFINKkwpSMboLCOH089C3Ay1ExHR4KSUyzA03YChR3tDXd4AajtcqDt6abR6EBzsSXQQOT5kSpIU89CZn6LD1CHJKEzVDcq6m7HEHk8iIoo6XyCIBosbR9pdqO1wocHiOmXx+oHixKF2oHeF+RN1qD1eyGUSRmYaMWVIEjKMmjM/gXqMPZ5ERBTXlHIZ8lN0yE8JhalgUKDJ5kG9JdQb2mh1o905sIbnuwuOibTwJlFplHJMyDVjUr4ZxgG8X3qiYPAkIqKYk8kkZJk1yDIf64ny+ANoOhpCG60eNFjdA27lfCIsvElUZq0S5xQkYVyOmXumxxEOtRMRUcJwev3hHtEmmwfNNk/Ch9F4XXiTqHKTtJgyJAnD0g2cvxklHGonIqIBSadSoChNgaI0ffg+ty+AZpsHzfZQEG22edDm8CZMcft4W3iTiGSShOEZBkwZkoRsszbWzaHTYPAkIqKEplHKu8wXBYBAUKDV7gn1ito9aLF50GL3wu0LxLCl1N9UChnG55oxOT8JZi3nbyYCBk8iIhpw5DIJGSYNMkxdVy/b3D602L1oCYdRD9qdvoTpHaWQbLMG43PNGJlp5PzNBMPgSUREg4ZRo4RRo+wyVB8ICrQ6PGixhQJp59d2jz+GLaUTaZRyjM42YnyOGelGbmeZqBg8iYhoUJPLJGQYNSfVdnR5A6GeUbsn3Eva5vDC6w/GqKWDjyQBeck6jM81YXi6gbsLDQAMnkRERN3Qqk6eOyqEgMXl6xJGW+1edDh93ImpH+nVcozNNmN8rglJOi62GkgYPImIiHpIkiQk6VRI0qkwPOPY/f5AEG1OL1psXrQ6QmG0xe6Bzc3h+p4yaZUoStOhKM2AISk6yGQshTQQMXgSERGdJYVc1u1wvdsXQKvDi1a7B60OL9rsXrQ5OH8UCJVAyjZrUJSuR1GaHmkGztscDBg8iYiIIkSjlCM3SYvcpK61Jd2+ANocXrTaQz2kbY5QIB3oPaQapRyFqToUpoXCpkYpj3WTKMoYPImIiKJMo5QjJ0mLnBMCqccfQIfTd/TiRYfr6LXTB6c3sWqQyiQJSTol0gxqpBpUyEvWIses5RD6IMfgSUREFCfUCjkyTXJknlB/FAiFUovTh/ajodTi8sHu8cPh8cPhDcAVw2CqU8mRZlAjzahGql6FdKMaKXoVlFyFTidg8CQiIkoAaoUcGSb5SUXxO/kDQTi8ATg8fji9ftg9oa/tHj9c3gD8QYFAMHj0WsAfOHp93P2dC/OVcglqhRxqpQyao9dqhSx8n1ohh1ohg1GjQJpBDb2acYJ6hj8pREREA4BCLoNZKzurrSM7d3CSczicIoTBk4iIiAAwcFLkcfIFEREREUUFgycRERERRQWDJxERERFFBYMnEREREUUFgycRERERRQWDJxERERFFBYMnEREREUVFxIJnVVUV7rjjDhQVFUGr1WLYsGF4/PHH4fV6I/WWRERERBTHIlZA/sCBAwgGg3jllVcwfPhw7N27F3fddRccDgf+9Kc/ReptiYiIiChOSUJ07swaeYsWLcJLL72EioqKHh1vtVphNpthsVhgMpki3DoiIiIi6q3e5LWobplpsViQkpJyysc9Hg88Hk/4ttVqjUaziIiIiCgKora4qLy8HM899xzuvvvuUx6zcOFCmM3m8CU/Pz9azSMiIiKiCOt18HziiScgSdJpL9u2bevynLq6OlxxxRW48cYbceedd57ytR977DFYLJbwpaampvffERERERHFpV7P8WxpaUFLS8tpjyksLIRGowEQCp0XX3wxZsyYgSVLlkAm63nW5RxPIiIiovgW0TmeaWlpSEtL69GxtbW1uPjiizF16lQsXry4V6GTiIiIiAaWiC0uqqurw0UXXYSCggL86U9/QnNzc/ixrKysSL0tEREREcWpiAXPL774AmVlZSgrK0NeXl6Xx6JYwYmIiIiI4kTExr7nz58PIUS3FyIiIiIafDjpkoiIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIoiKiwfOaa65BQUEBNBoNsrOzceutt6Kuri6Sb0lEREREcSqiwfPiiy/Gf/7zHxw8eBAffPABysvLccMNN0TyLYmIiIgoTklCCBGtN/v4449x3XXXwePxQKlUnvF4q9UKs9kMi8UCk8kUhRYSERERUW/0Jq8potQmtLW14c0338TMmTNPGTo9Hg88Hk/4tsViARD6hoiIiIgo/nTmtB71ZYoIe/TRR4VOpxMAxHnnnSdaWlpOeezjjz8uAPDCCy+88MILL7zwkmCXmpqaM+bCXg+1P/HEE1iwYMFpj9m6dSumTZsGAGhpaUFbWxsOHz6MBQsWwGw245NPPoEkSSc978Qez2AwiLa2NqSmpnZ7fDyyWq3Iz89HTU0NpwckAJ6vxMLzlXh4zhILz1fiiYdzJoSAzWZDTk4OZLLTLx/qdfBsaWlBS0vLaY8pLCyERqM56f4jR44gPz8fGzduRHFxcW/eNmFwXmpi4flKLDxfiYfnLLHwfCWeRDtnvZ7jmZaWhrS0tD69WWfGPb5Xk4iIiIgGh4gtLtqyZQu2bNmCWbNmITk5GRUVFfjtb3+LYcOGDdjeTiIiIiI6tYjV8dRqtVi6dCkuueQSjBo1Cj/+8Y8xfvx4rF27Fmq1OlJvG3NqtRqPP/74gP4eBxKer8TC85V4eM4SC89X4km0cxbVOp5ERERENHhxr3YiIiIiigoGTyIiIiKKCgZPIiIiIooKBk8iIiIiigoGz1568cUXUVRUBI1Gg6lTp2L9+vWnPX7t2rWYOnUqNBoNhg4dipdffjlKLaVOvTlnS5cuxWWXXYb09HSYTCYUFxfj888/j2Jrqbe/Y502bNgAhUKByZMnR7aBdJLenjOPx4Nf/epXGDJkCNRqNYYNG4Z//vOfUWot9fZ8vfnmm5g0aRJ0Oh2ys7Nx++23o7W1NUqtHdzWrVuHq6++Gjk5OZAkCR9++OEZnxP3uaM/9mMfLN555x2hVCrFq6++Kvbv3y8efPBBodfrxeHDh7s9vqKiQuh0OvHggw+K/fv3i1dffVUolUrx/vvvR7nlg1dvz9mDDz4onnrqKbFlyxZRWloqHnvsMaFUKsWOHTui3PLBqbfnq1NHR4cYOnSomDNnjpg0aVJ0GktCiL6ds2uuuUbMmDFDrFy5UlRWVorNmzeLDRs2RLHVg1dvz9f69euFTCYTf/3rX0VFRYVYv369GDdunLjuuuui3PLBafny5eJXv/qV+OCDDwQAsWzZstMenwi5g8GzF6ZPny7uvvvuLveNHj1a/PKXv+z2+EcffVSMHj26y30//elPxXnnnRexNlJXvT1n3Rk7dqxYsGBBfzeNutHX8zVv3jzx61//Wjz++OMMnlHW23P22WefCbPZLFpbW6PRPDpBb8/XokWLxNChQ7vc97e//U3k5eVFrI3UvZ4Ez0TIHRxq7yGv14vt27djzpw5Xe6fM2cONm7c2O1zvvnmm5OOv/zyy7Ft2zb4fL6ItZVC+nLOThQMBmGz2ZCSkhKJJtJx+nq+Fi9ejPLycjz++OORbiKdoC/n7OOPP8a0adPw9NNPIzc3FyNHjsTPfvYzuFyuaDR5UOvL+Zo5cyaOHDmC5cuXQwiBxsZGvP/++7jqqqui0WTqpUTIHRHbMnOgaWlpQSAQQGZmZpf7MzMz0dDQ0O1zGhoauj3e7/ejpaUF2dnZEWsv9e2cnejPf/4zHA4Hbrrppkg0kY7Tl/N16NAh/PKXv8T69euhUPDPWbT15ZxVVFTg66+/hkajwbJly9DS0oJ77rkHbW1tnOcZYX05XzNnzsSbb76JefPmwe12w+/345prrsFzzz0XjSZTLyVC7mCPZy9JktTlthDipPvOdHx391Pk9PacdXr77bfxxBNP4N1330VGRkakmkcn6On5CgQCuPnmm7FgwQKMHDkyWs2jbvTmdywYDEKSJLz55puYPn06rrzySjzzzDNYsmQJez2jpDfna//+/XjggQfw29/+Ftu3b8eKFStQWVmJu+++OxpNpT6I99zBLoIeSktLg1wuP+lTYVNT00mfLjplZWV1e7xCoUBqamrE2kohfTlnnd59913ccccdeO+993DppZdGspl0VG/Pl81mw7Zt27Bz507cd999AEKhRggBhUKBL774At/5znei0vbBqi+/Y9nZ2cjNzYXZbA7fN2bMGAghcOTIEYwYMSKibR7M+nK+Fi5ciPPPPx8///nPAQATJ06EXq/HBRdcgD/84Q9x0YNGxyRC7mCPZw+pVCpMnToVK1eu7HL/ypUrMXPmzG6fU1xcfNLxX3zxBaZNmwalUhmxtlJIX84ZEOrpnD9/Pt566y3OY4qi3p4vk8mEPXv2YNeuXeHL3XffjVGjRmHXrl2YMWNGtJo+aPXld+z8889HXV0d7HZ7+L7S0lLIZDLk5eVFtL2DXV/Ol9PphEzWNSrI5XIAx3rSKH4kRO6I0aKmhNRZhuK1114T+/fvFw899JDQ6/WiqqpKCCHEL3/5S3HrrbeGj+8sa/Dwww+L/fv3i9deey3uyhoMdL09Z2+99ZZQKBTihRdeEPX19eFLR0dHrL6FQaW35+tEXNUefb09ZzabTeTl5YkbbrhB7Nu3T6xdu1aMGDFC3HnnnbH6FgaV3p6vxYsXC4VCIV588UVRXl4uvv76azFt2jQxffr0WH0Lg4rNZhM7d+4UO3fuFADEM888I3bu3Bkuf5WIuYPBs5deeOEFMWTIEKFSqcSUKVPE2rVrw4/ddtttYvbs2V2OX7NmjTjnnHOESqUShYWF4qWXXopyi6k352z27NkCwEmX2267LfoNH6R6+zt2PAbP2OjtOSspKRGXXnqp0Gq1Ii8vTzzyyCPC6XRGudWDV2/P19/+9jcxduxYodVqRXZ2trjlllvEkSNHotzqwWn16tWn/T8pEXOHJAT7yomIiIgo8jjHk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIooLBk4iIiIiigsGTiIiIiKKCwZOIiIiIouL/B0RcN+rdLzIMAAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get into evaluation (predictive posterior) mode\n",
    "model.eval()\n",
    "likelihood.eval()\n",
    "\n",
    "# Test points are regularly spaced along [0,1]\n",
    "# Make predictions by feeding model through likelihood\n",
    "with torch.no_grad(), gpytorch.settings.fast_pred_var():\n",
    "    test_x = torch.linspace(0, 1, 51)\n",
    "    test_x_distributional = torch.stack((test_x, (1e-3 * torch.ones_like(test_x)).log()), dim=1)\n",
    "    observed_pred = likelihood(model(test_x_distributional))\n",
    "\n",
    "with torch.no_grad():\n",
    "    # Initialize plot\n",
    "    f, ax = plt.subplots(1, 1, figsize=(8, 3))\n",
    "\n",
    "    # Get upper and lower confidence bounds\n",
    "    lower, upper = observed_pred.confidence_region()\n",
    "    # Plot training data as black stars\n",
    "    ax.errorbar(train_x_mean.numpy(), train_y.numpy(), xerr=train_x_stdv, fmt='k*')\n",
    "    # Plot predictive means as blue line\n",
    "    ax.plot(test_x.numpy(), observed_pred.mean.numpy(), 'b')\n",
    "    # Shade between the lower and upper confidence bounds\n",
    "    ax.fill_between(test_x.numpy(), lower.numpy(), upper.numpy(), alpha=0.5)\n",
    "    ax.set_ylim([-3, 3])\n",
    "    ax.legend(['Observed Data', 'Mean', 'Confidence'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "As a final note, we've made it very easy to extend the distributional kernel class by exposing a generic `DistributionalInputKernel` class that takes as input any distance function over probability distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}