Machine Learning

hilo_mpc.ANN

alias of ArtificialNeuralNetwork

class hilo_mpc.ArtificialNeuralNetwork(features, labels, id=None, name=None, **kwargs)[source]

Artificial neural network class

add_data_set(data_set)[source]
Parameters:

data_set

Returns:

add_layers(layers)[source]
Parameters:

layers

Returns:

build_graph(weights=None, bias=None)[source]
Parameters:

  • weights

  • bias

Returns:

close_tensorboard()[source]
Returns:

is_setup()[source]
Returns:

is_trained()[source]
Returns:

predict(X_query)[source]
Parameters:

X_query

Returns:

prepare_data_set(train_split=1.0, validation_split=0.0, scale_data=False, scaler=None, scaler_backend=None, shuffle=True)[source]
Parameters:

  • train_split

  • validation_split

  • scale_data

  • scaler

  • scaler_backend

  • shuffle

Returns:

set_input_scaling(scaler, backend=None)[source]
Parameters:

  • scaler

  • backend

Returns:

set_output_scaling(scaler, backend=None)[source]
Parameters:

  • scaler

  • backend

Returns:

set_scaling(scaler, backend=None)[source]
Parameters:

  • scaler

  • backend

Returns:

setup(**kwargs)[source]
Parameters:

kwargs

Returns:

show_tensorboard(browser=None)[source]
Parameters:

browser

Returns:

train(batch_size, epochs, verbose=1, validation_split=0.0, test_split=0.0, scale_data=False, scaler=None, scaler_backend=None, shuffle=True, patience=None)[source]
Parameters:

  • batch_size

  • epochs

  • verbose

  • validation_split

  • test_split

  • scale_data

  • scaler

  • scaler_backend

  • shuffle

  • patience

Returns:

property backend

return:

property depth

return:

property features

return:

property labels

return:

property learning_rate

return:

property loss

return:

property metric

return:

property optimizer

return:

property seed

return:

property shape

return:

class hilo_mpc.ConstantKernel(bias=1.0, bounds=None)[source]

Constant covariance function

Parameters:

  • bias

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]
Covariance function of constant as follows:

bias^2

Parameters:

  • x

  • x_bar

  • active_dims

Returns:

class hilo_mpc.ConstantMean(bias=1.0, hyperprior=None, **kwargs)[source]

Constant mean function

Parameters:

  • bias

  • hyperprior

  • kwargs

get_mean_function(x, active_dims)[source]
Parameters:

  • x

  • active_dims

Returns:

class hilo_mpc.Dense(nodes, activation='linear', initializer=None, parent=None, **kwargs)[source]

Dense layer

property nodes

return:

class hilo_mpc.DotProductKernel(active_dims=None, signal_variance=1.0, offset=1.0, bounds=None)[source]

Base for all dot product kernels

Parameters:

  • active_dims

  • signal_variance

  • offset

  • bounds

class hilo_mpc.Dropout(rate, parent=None)[source]

Dropout layer

Parameters:

rate (float) – Dropout rate

set_initializer(initializer, **kwargs)[source]
Parameters:

  • initializer

  • kwargs

Returns:

property rate

return:

class hilo_mpc.ExponentialKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Exponential covariance function

Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

hilo_mpc.GP

alias of GaussianProcess

class hilo_mpc.GaussianProcess(features, labels, inference=None, likelihood=None, mean=None, kernel=None, noise_variance=1.0, hyperprior=None, id=None, name=None, solver=None, solver_options=None, **kwargs)[source]

Gaussian Process Regression

Note:

The Gaussian process regressor currently does not use the Cholesky factorization for training and prediction. This will be implemented at a later time.

Parameters:

  • features (list of strings) – names of the features

  • labels (list of strings) – names of the labels

  • inference

  • likelihood

  • kernel (kernel) – Specifying the covariance function of the GP. The kernel hyperparameters are optimized during training, defaults to hilo_mpc.core.learning.kernels.SquaredExponential.

  • noise_variance

  • hyperprior

  • id (str, optional)

  • name (str, optional) – name of the GP

  • solver (str, optional)

  • solver_options (dict) – Options to the solver, e.g. ipopt.

  • kwargs

fit_model()[source]

Optimizes the hyperparameters by minimizing the negative log marginal likelihood.

The model fit is the training phase in the GP regression. Given the design data (training observations and their targets) and suitable start values and bounds for the hyperparameters, the hyperparameters will be adapted by minimizing the negative log marginal likelihood.

Note:

Instead of boundary values the string ‘fixed’ can be passed which flags the hyperparameter, keeping it constant during the optimization routine.

Returns:

get_hyperparameter_by_name(name)[source]
Parameters:

name

Returns:

initialize(features, labels, inference=None, likelihood=None, mean=None, kernel=None, noise_variance=1.0, hyperprior=None, id=None, name=None, solver=None, solver_options=None, **kwargs)[source]
Parameters:

  • features

  • labels

  • inference

  • likelihood

  • mean

  • kernel

  • noise_variance

  • hyperprior

  • id

  • name

  • solver

  • solver_options

  • kwargs

Returns:

is_setup()[source]
Returns:

log_marginal_likelihood()[source]
Returns:

plot(X_query, backend=None, **kwargs)[source]
Parameters:

  • X_query

  • backend

  • kwargs

Returns:

plot_1d(quantiles=None, resolution=200, backend=None, **kwargs)[source]
Parameters:

  • quantiles

  • resolution

  • backend

  • kwargs

Returns:

plot_prediction_error(X_query, y_query, noise_free=False, backend=None, **kwargs)[source]
Parameters:

  • X_query

  • y_query

  • noise_free

  • backend

  • kwargs

Returns:

predict(X_query, noise_free=False)[source]
Parameters:

  • X_query

  • noise_free

Returns:

predict_quantiles(quantiles=None, X_query=None, mean=None, var=None)[source]
Parameters:

  • quantiles

  • X_query

  • mean

  • var

Returns:

set_training_data(X, y)[source]

Sets the training matrix and its target vector

Parameters:

  • X

  • y

Returns:

setup(**kwargs)[source]
Parameters:

kwargs

Returns:

update(*args)[source]
Parameters:

args

Returns:

update_hyperparameters(names, values=None, bounds=None)[source]

Updates hyperparameter values and boundaries

If names of hyperparameters are found in the given dictionaries, their respective value and/or boundaries will be updated to the value and/or boundaries defined in the dictionary.

Note:

Instead of boundary values the string ‘fixed’ can be passed which flags the hyperparameter, keeping him constant during any optimization routine. This flag will be automatically set back if a list of two floats is passed.

Parameters:

  • names

  • values

  • bounds

Returns:

property X_train

The tuple contains the training data matrix as ndarray of shape (dimensions_input, number_observations) and its respective CasADi SX and MX symbols of same dimensions that are used to define the CasADi functions during fitting and prediction. It can be changed by just assigned an ndarray and its setter method will automatically adjust the symbols.

Returns:

property hyperparameter_names

return:

property hyperparameters

Includes all hyperparameters of the kernel as well as the sigma_noise hyperparameter

Returns:

property y_train

The tuple contains the training target matrix as ndarray of shape (dimensions_input, number_observations) and its respective CasADi SX and MX symbols of same dimensions that are used to define the CasADi functions during fitting and prediction. It can be changed by just assigned an ndarray and its setter method will automatically adjust the symbols.

Returns:

class hilo_mpc.Kernel(active_dims=None)[source]

Kernel base class

This base class implements the basic methods that are used for all basic kernels, e.g. dot product kernels and radial basis function kernels like the squared exponential. It also implements the methods for the composition of kernels with the addition, multiplication and exponentiation operation.

Note:

The Kernel classes return a CasADi function for the respective inputs when called. Each basic Kernel class implements its own scalar covariance function. This base class implements the call method where the respective scalar covariance functions are generalized to the form of covariance matrices which will be returned as CasADi SX function. The SX symbol was chosen in accordance with the CasADi docs which recommend that low-level expressions are faster computed for SX symbols.

Parameters:

active_dims (int, list of int, optional)

static constant(bias=1.0, bounds=None)[source]
Parameters:

  • bias

  • bounds

Returns:

static exponential(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

Returns:

abstract get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x

  • x_bar

  • active_dims

Returns:

get_covariance_matrix(covariance_function, X, X_bar)[source]
Parameters:

  • covariance_function

  • X

  • X_bar

Returns:

is_bounded()[source]
Returns:

static linear(active_dims=None, signal_variance=1.0, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • bounds

Returns:

static matern_32(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

Returns:

static matern_52(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

Returns:

static neural_network(active_dims=None, signal_variance=1.0, weight_variance=1.0, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • weight_variance

  • bounds

Returns:

static periodic(active_dims=None, signal_variance=1.0, length_scales=1.0, period=1.0, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • period

  • bounds

Returns:

static piecewise_polynomial(degree, active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • degree

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

Returns:

static polynomial(degree, active_dims=None, signal_variance=1.0, offset=1.0, bounds=None)[source]
Parameters:

  • degree

  • active_dims

  • signal_variance

  • offset

  • bounds

Returns:

static rational_quadratic(active_dims=None, signal_variance=1.0, length_scales=1.0, alpha=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • alpha

  • ard

  • bounds

Returns:

static squared_exponential(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

Returns:

update(*args)[source]
Parameters:

args

Returns:

property hyperparameter_names

return:

property hyperparameters

List of all hyperparameters in the kernel.

This attribute can be easily used to access specific attributes of all hyperparameters in generator expressions or list comprehensions.

class hilo_mpc.Layer(nodes, activation=None, initializer=None, parent=None, **kwargs)[source]

Base layer

static dense(nodes, activation='linear', initializer=None, dropout=None, where=None, parent=None, **kwargs)[source]
Parameters:

  • nodes

  • activation

  • initializer

  • dropout

  • where

  • parent

  • kwargs

Returns:

static dropout(rate, parent=None)[source]
Parameters:

  • rate

  • parent

Returns:

set_initializer(initializer, **kwargs)[source]
Parameters:

  • initializer

  • kwargs

Returns:

property activation

return:

property initializer

return:

property parent

return:

property type

return:

class hilo_mpc.LinearKernel(active_dims=None, signal_variance=1.0, bounds=None)[source]

Linear covariance function

Parameters:

  • active_dims

  • signal_variance

  • bounds

class hilo_mpc.LinearMean(active_dims=None, coefficient=1.0, hyperprior=None, **kwargs)[source]

Linear mean function

Parameters:

  • active_dims

  • coefficient

  • hyperprior

  • kwargs

class hilo_mpc.Matern32Kernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Matérn covariance function where \({\nu}\) = 3/2

Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

class hilo_mpc.Matern52Kernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Matérn covariance function where \({\nu}\) = 5/2

Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

class hilo_mpc.MaternKernel(p, active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Base for Matérn class of covariance functions

Parameters:

  • p

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x

  • x_bar

  • active_dims

Returns:

class hilo_mpc.Mean(active_dims=None)[source]

Mean function base class

Parameters:

active_dims

Type:

active_dims: int, list of int, optional

static constant(bias=1.0, hyperprior=None, **kwargs)[source]
Parameters:

  • bias

  • hyperprior

  • kwargs

Returns:

abstract get_mean_function(x, active_dims)[source]
Parameters:

  • x

  • active_dims

Returns:

is_bounded()[source]
Returns:

static linear(active_dims=None, coefficient=1.0, hyperprior=None, **kwargs)[source]
Parameters:

  • active_dims

  • coefficient

  • hyperprior

  • kwargs

Returns:

static one()[source]
Returns:

static polynomial(degree, active_dims=None, coefficient=1.0, offset=1.0, hyperprior=None, **kwargs)[source]
Parameters:

  • degree

  • active_dims

  • coefficient

  • offset

  • hyperprior

  • kwargs

Returns:

update(*args)[source]
Parameters:

args

Returns:

static zero()[source]
Returns:

property hyperparameter_names

return:

property hyperparameters

return:

class hilo_mpc.NeuralNetworkKernel(active_dims=None, signal_variance=1.0, weight_variance=1.0, bounds=None)[source]

Neural network covariance function

Parameters:

  • active_dims

  • signal_variance

  • weight_variance

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x

  • x_bar

  • active_dims

Returns:

class hilo_mpc.OneMean[source]

One mean function

class hilo_mpc.PeriodicKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, period=1, bounds=None)[source]

Periodic covariance function

Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • period

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]

Covariance function of exponential sine as follows:

\[\sigma^2 \cdot \exp\left(-1/2 \cdot \frac{\sin(x-x_{bar})^T}{p} M \frac{\sin(x-x_{bar})}{p}\right)\]

where \(M\) is the parameterized matrix of length scales

Parameters:

  • x

  • x_bar

  • active_dims

Returns:

class hilo_mpc.PiecewisePolynomialKernel(degree, active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Piecewise polynomial covariance function with compact support

Parameters:

  • degree

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x

  • x_bar

  • active_dims

Returns:

property degree

return:

class hilo_mpc.PolynomialKernel(degree, active_dims=None, signal_variance=1.0, offset=1.0, bounds=None)[source]

Polynomial covariance function

Parameters:

  • degree

  • active_dims

  • signal_variance

  • offset

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]
Covariance function of polynomial as follows:

(sigma_bias^2 + sigma_signal^2 * (x dot x_bar))^p

Parameters:

  • x

  • x_bar

  • active_dims

Returns:

property degree

return:

class hilo_mpc.PolynomialMean(degree, active_dims=None, coefficient=1.0, offset=1.0, hyperprior=None, **kwargs)[source]

Polynomial mean function

Parameters:

  • degree

  • active_dims

  • coefficient

  • offset

  • hyperprior

  • kwargs

get_mean_function(x, active_dims)[source]
Parameters:

  • x

  • active_dims

Returns:

static get_parameterized_coefficients(dimension_input_space, coefficient)[source]
Parameters:

  • dimension_input_space

  • coefficient

Returns:

property degree

return:

class hilo_mpc.RationalQuadraticKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, alpha=1.0, ard=False, bounds=None)[source]

Rational quadratic covariance function

Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • alpha

  • ard

  • bounds

get_covariance_function(x, x_bar, active_dims)[source]

Covariance function of rational quadratic as follows:

\[\sigma \cdot [1+1/(2\cdot\alpha)\cdot(x-x_{bar})^T M (x-x_{bar})]^{-\alpha}\]

where \(M\) is the parameterized matrix of length scales

Parameters:

  • x

  • x_bar

  • active_dims

Returns:

class hilo_mpc.SquaredExponentialKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Squared exponential covariance function

Parameters:

  • active_dims

  • signal_variance

  • length_scales

  • ard

  • bounds

class hilo_mpc.ZeroMean[source]

Zero mean function