gpytorch.constraints

Parameter Constraints

Constraints keep a parameter’s value within a valid domain (for example, strictly positive or bounded in an interval) while the optimizer works on an unconstrained tensor under the hood. GPyTorch stores the unconstrained value of a parameter; the constraint is applied on every read so the model only ever sees a valid value.

The base class is Interval (a closed interval [lower_bound, upper_bound]), and the most common derived classes are:

• Positive – x > 0 (the most common constraint, used for lengthscales, noise scales, and outputscales).

• GreaterThan – x >= lower_bound (a one-sided lower bound).

• LessThan – x <= upper_bound (a one-sided upper bound).

A constraint is attached to a parameter via register_constraint(), which adds a property of the same name to the module that returns the constrained value of the parameter.

Example

The example below constrains a custom parameter to be strictly positive and shows the round-trip behavior of the underlying transform.

import torch
import gpytorch

class MyModule(gpytorch.Module):
    def __init__(self):
        super().__init__()
        self.register_parameter(
            "raw_scale", torch.nn.Parameter(torch.tensor(0.0)),
        )
        # Expose ``self.scale`` as a strictly positive quantity.
        self.register_constraint("raw_scale", gpytorch.constraints.Positive())

    @property
    def scale(self):
        # Auto-generated property: returns the constrained value.
        return self._constraints["raw_scale_constraint"].transform(self.raw_scale)

m = MyModule()
m.scale  # -> tensor constrained to (0, infinity)

Constraint Reference

Interval

class gpytorch.constraints.Interval(lower_bound, upper_bound, transform=<built-in method sigmoid of type object>, inv_transform=<function inv_sigmoid>, initial_value=None)

property initial_value: Tensor | None

The initial value assigned to the constrained parameter at registration time (if one was provided to __init__(), otherwise None).

intersect(other)

Returns a new Interval constraint that is the intersection of this one and another specified one.

Parameters:

other (Interval) – Interval constraint to intersect with

Returns:

intersection if this interval with the other one.

Return type:

Interval

inverse_transform(transformed_tensor)

Applies the inverse transformation, mapping a constrained tensor back to its unconstrained representation.

Parameters:

transformed_tensor (torch.Tensor) – A tensor whose values lie in [lower_bound, upper_bound].

Returns:

The unconstrained tensor.

Return type:

torch.Tensor

transform(tensor)

Transforms a tensor to satisfy the specified bounds.

If upper_bound is finite, we assume that self._transform saturates at 1 as tensor -> infinity. Similarly, if lower_bound is finite, we assume that self._transform saturates at 0 as tensor -> -infinity.

Example transforms for one of the bounds being finite include torch.exp and torch.nn.functional.softplus. An example transform for the case where both are finite is torch.nn.functional.sigmoid.

Parameters:

tensor (torch.Tensor) – The unconstrained tensor to be transformed.

Returns:

A tensor whose values lie in [lower_bound, upper_bound].

Return type:

torch.Tensor

GreaterThan

class gpytorch.constraints.GreaterThan(lower_bound, transform=Softplus(beta=1, threshold=20), inv_transform=<function inv_softplus>, initial_value=None)

Positive

class gpytorch.constraints.Positive(transform=Softplus(beta=1, threshold=20), inv_transform=<function inv_softplus>, initial_value=None)

LessThan

class gpytorch.constraints.LessThan(upper_bound, transform=Softplus(beta=1, threshold=20), inv_transform=<function inv_softplus>, initial_value=None)