Machine Learning

hilo_mpc.ANN

alias of ArtificialNeuralNetwork

class hilo_mpc.ArtificialNeuralNetwork(features, labels, id=None, name=None, **kwargs)[source]

Artificial neural network class

add_data_set(data_set)[source]
Parameters:

data_set –

Returns:

add_layers(layers)[source]
Parameters:

layers –

Returns:

build_graph(weights=None, bias=None)[source]
Parameters:

  • weights –

  • bias –

Returns:

close_tensorboard()[source]
Returns:

is_setup()[source]
Returns:

is_trained()[source]
Returns:

predict(X_query)[source]
Parameters:

X_query –

Returns:

prepare_data_set(train_split=1.0, validation_split=0.0, scale_data=False, scaler=None, scaler_backend=None, shuffle=True)[source]
Parameters:

  • train_split –

  • validation_split –

  • scale_data –

  • scaler –

  • scaler_backend –

  • shuffle –

Returns:

set_input_scaling(scaler, backend=None)[source]
Parameters:

  • scaler –

  • backend –

Returns:

set_output_scaling(scaler, backend=None)[source]
Parameters:

  • scaler –

  • backend –

Returns:

set_scaling(scaler, backend=None)[source]
Parameters:

  • scaler –

  • backend –

Returns:

setup(**kwargs)[source]
Parameters:

kwargs –

Returns:

show_tensorboard(browser=None)[source]
Parameters:

browser –

Returns:

train(batch_size, epochs, verbose=1, validation_split=0.0, test_split=0.0, scale_data=False, scaler=None, scaler_backend=None, shuffle=True, patience=None)[source]
Parameters:

  • batch_size –

  • epochs –

  • verbose –

  • validation_split –

  • test_split –

  • scale_data –

  • scaler –

  • scaler_backend –

  • shuffle –

  • patience –

Returns:

property backend
Returns:

property depth
Returns:

property features
Returns:

property labels
Returns:

property learning_rate
Returns:

property loss
Returns:

property metric
Returns:

property optimizer
Returns:

property seed
Returns:

property shape
Returns:

class hilo_mpc.ConstantKernel(bias=1.0, bounds=None)[source]

Constant covariance function

Parameters:

  • bias –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]
Covariance function of constant as follows:

bias^2

Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

class hilo_mpc.ConstantMean(bias=1.0, hyperprior=None, **kwargs)[source]

Constant mean function

Parameters:

  • bias –

  • hyperprior –

  • kwargs –

get_mean_function(x, active_dims)[source]
Parameters:

  • x –

  • active_dims –

Returns:

class hilo_mpc.Dense(nodes, activation='linear', initializer=None, parent=None, **kwargs)[source]

Dense layer

property nodes
Returns:

class hilo_mpc.DotProductKernel(active_dims=None, signal_variance=1.0, offset=1.0, bounds=None)[source]

Base for all dot product kernels

Parameters:

  • active_dims –

  • signal_variance –

  • offset –

  • bounds –

class hilo_mpc.Dropout(rate, parent=None)[source]

Dropout layer

Parameters:

rate (float) – Dropout rate

set_initializer(initializer, **kwargs)[source]
Parameters:

  • initializer –

  • kwargs –

Returns:

property rate
Returns:

class hilo_mpc.ExponentialKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Exponential covariance function

Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

hilo_mpc.GP

alias of GaussianProcess

class hilo_mpc.GaussianProcess(features, labels, inference=None, likelihood=None, mean=None, kernel=None, noise_variance=1.0, hyperprior=None, id=None, name=None, solver=None, solver_options=None, **kwargs)[source]

Gaussian Process Regression

Note:

The Gaussian process regressor currently does not use the Cholesky factorization for training and prediction. This will be implemented at a later time.

Parameters:

  • features (list of strings) – names of the features

  • labels (list of strings) – names of the labels

  • inference –

  • likelihood –

  • kernel (kernel) – Specifying the covariance function of the GP. The kernel hyperparameters are optimized during training, defaults to hilo_mpc.core.learning.kernels.SquaredExponential.

  • noise_variance –

  • hyperprior –

  • id (str, optional) –

  • name (str, optional) – name of the GP

  • solver (str, optional) –

  • solver_options (dict) – Options to the solver, e.g. ipopt.

  • kwargs –

fit_model()[source]

Optimizes the hyperparameters by minimizing the negative log marginal likelihood.

The model fit is the training phase in the GP regression. Given the design data (training observations and their targets) and suitable start values and bounds for the hyperparameters, the hyperparameters will be adapted by minimizing the negative log marginal likelihood.

Note:

Instead of boundary values the string ‘fixed’ can be passed which flags the hyperparameter, keeping it constant during the optimization routine.

Returns:

get_hyperparameter_by_name(name)[source]
Parameters:

name –

Returns:

initialize(features, labels, inference=None, likelihood=None, mean=None, kernel=None, noise_variance=1.0, hyperprior=None, id=None, name=None, solver=None, solver_options=None, **kwargs)[source]
Parameters:

  • features –

  • labels –

  • inference –

  • likelihood –

  • mean –

  • kernel –

  • noise_variance –

  • hyperprior –

  • id –

  • name –

  • solver –

  • solver_options –

  • kwargs –

Returns:

is_setup()[source]
Returns:

log_marginal_likelihood()[source]
Returns:

plot(X_query, backend=None, **kwargs)[source]
Parameters:

  • X_query –

  • backend –

  • kwargs –

Returns:

plot_1d(quantiles=None, resolution=200, backend=None, **kwargs)[source]
Parameters:

  • quantiles –

  • resolution –

  • backend –

  • kwargs –

Returns:

plot_prediction_error(X_query, y_query, noise_free=False, backend=None, **kwargs)[source]
Parameters:

  • X_query –

  • y_query –

  • noise_free –

  • backend –

  • kwargs –

Returns:

predict(X_query, noise_free=False)[source]
Parameters:

  • X_query –

  • noise_free –

Returns:

predict_quantiles(quantiles=None, X_query=None, mean=None, var=None)[source]
Parameters:

  • quantiles –

  • X_query –

  • mean –

  • var –

Returns:

set_training_data(X, y)[source]

Sets the training matrix and its target vector

Parameters:

  • X –

  • y –

Returns:

setup(**kwargs)[source]
Parameters:

kwargs –

Returns:

update(*args)[source]
Parameters:

args –

Returns:

update_hyperparameters(names, values=None, bounds=None)[source]

Updates hyperparameter values and boundaries

If names of hyperparameters are found in the given dictionaries, their respective value and/or boundaries will be updated to the value and/or boundaries defined in the dictionary.

Note:

Instead of boundary values the string ‘fixed’ can be passed which flags the hyperparameter, keeping him constant during any optimization routine. This flag will be automatically set back if a list of two floats is passed.

Parameters:

  • names –

  • values –

  • bounds –

Returns:

property X_train

The tuple contains the training data matrix as ndarray of shape (dimensions_input, number_observations) and its respective CasADi SX and MX symbols of same dimensions that are used to define the CasADi functions during fitting and prediction. It can be changed by just assigned an ndarray and its setter method will automatically adjust the symbols.

Returns:

property hyperparameter_names
Returns:

property hyperparameters

Includes all hyperparameters of the kernel as well as the sigma_noise hyperparameter

Returns:

property y_train

The tuple contains the training target matrix as ndarray of shape (dimensions_input, number_observations) and its respective CasADi SX and MX symbols of same dimensions that are used to define the CasADi functions during fitting and prediction. It can be changed by just assigned an ndarray and its setter method will automatically adjust the symbols.

Returns:

class hilo_mpc.Kernel(active_dims=None)[source]

Kernel base class

This base class implements the basic methods that are used for all basic kernels, e.g. dot product kernels and radial basis function kernels like the squared exponential. It also implements the methods for the composition of kernels with the addition, multiplication and exponentiation operation.

Note:

The Kernel classes return a CasADi function for the respective inputs when called. Each basic Kernel class implements its own scalar covariance function. This base class implements the call method where the respective scalar covariance functions are generalized to the form of covariance matrices which will be returned as CasADi SX function. The SX symbol was chosen in accordance with the CasADi docs which recommend that low-level expressions are faster computed for SX symbols.

Parameters:

active_dims (int, list of int, optional) –

static constant(bias=1.0, bounds=None)[source]
Parameters:

  • bias –

  • bounds –

Returns:

static exponential(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

Returns:

abstract get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

get_covariance_matrix(covariance_function, X, X_bar)[source]
Parameters:

  • covariance_function –

  • X –

  • X_bar –

Returns:

is_bounded()[source]
Returns:

static linear(active_dims=None, signal_variance=1.0, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • bounds –

Returns:

static matern_32(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

Returns:

static matern_52(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

Returns:

static neural_network(active_dims=None, signal_variance=1.0, weight_variance=1.0, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • weight_variance –

  • bounds –

Returns:

static periodic(active_dims=None, signal_variance=1.0, length_scales=1.0, period=1.0, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • period –

  • bounds –

Returns:

static piecewise_polynomial(degree, active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • degree –

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

Returns:

static polynomial(degree, active_dims=None, signal_variance=1.0, offset=1.0, bounds=None)[source]
Parameters:

  • degree –

  • active_dims –

  • signal_variance –

  • offset –

  • bounds –

Returns:

static rational_quadratic(active_dims=None, signal_variance=1.0, length_scales=1.0, alpha=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • alpha –

  • ard –

  • bounds –

Returns:

static squared_exponential(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]
Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

Returns:

update(*args)[source]
Parameters:

args –

Returns:

property hyperparameter_names
Returns:

property hyperparameters

List of all hyperparameters in the kernel.

This attribute can be easily used to access specific attributes of all hyperparameters in generator expressions or list comprehensions.

class hilo_mpc.Layer(nodes, activation=None, initializer=None, parent=None, **kwargs)[source]

Base layer

static dense(nodes, activation='linear', initializer=None, dropout=None, where=None, parent=None, **kwargs)[source]
Parameters:

  • nodes –

  • activation –

  • initializer –

  • dropout –

  • where –

  • parent –

  • kwargs –

Returns:

static dropout(rate, parent=None)[source]
Parameters:

  • rate –

  • parent –

Returns:

set_initializer(initializer, **kwargs)[source]
Parameters:

  • initializer –

  • kwargs –

Returns:

property activation
Returns:

property initializer
Returns:

property parent
Returns:

property type
Returns:

class hilo_mpc.LinearKernel(active_dims=None, signal_variance=1.0, bounds=None)[source]

Linear covariance function

Parameters:

  • active_dims –

  • signal_variance –

  • bounds –

class hilo_mpc.LinearMean(active_dims=None, coefficient=1.0, hyperprior=None, **kwargs)[source]

Linear mean function

Parameters:

  • active_dims –

  • coefficient –

  • hyperprior –

  • kwargs –

class hilo_mpc.Matern32Kernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Matérn covariance function where \({\nu}\) = 3/2

Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

class hilo_mpc.Matern52Kernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Matérn covariance function where \({\nu}\) = 5/2

Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

class hilo_mpc.MaternKernel(p, active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Base for Matérn class of covariance functions

Parameters:

  • p –

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

class hilo_mpc.Mean(active_dims=None)[source]

Mean function base class

Parameters:

active_dims –

Type:

active_dims: int, list of int, optional

static constant(bias=1.0, hyperprior=None, **kwargs)[source]
Parameters:

  • bias –

  • hyperprior –

  • kwargs –

Returns:

abstract get_mean_function(x, active_dims)[source]
Parameters:

  • x –

  • active_dims –

Returns:

is_bounded()[source]
Returns:

static linear(active_dims=None, coefficient=1.0, hyperprior=None, **kwargs)[source]
Parameters:

  • active_dims –

  • coefficient –

  • hyperprior –

  • kwargs –

Returns:

static one()[source]
Returns:

static polynomial(degree, active_dims=None, coefficient=1.0, offset=1.0, hyperprior=None, **kwargs)[source]
Parameters:

  • degree –

  • active_dims –

  • coefficient –

  • offset –

  • hyperprior –

  • kwargs –

Returns:

update(*args)[source]
Parameters:

args –

Returns:

static zero()[source]
Returns:

property hyperparameter_names
Returns:

property hyperparameters
Returns:

class hilo_mpc.NeuralNetworkKernel(active_dims=None, signal_variance=1.0, weight_variance=1.0, bounds=None)[source]

Neural network covariance function

Parameters:

  • active_dims –

  • signal_variance –

  • weight_variance –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

class hilo_mpc.OneMean[source]

One mean function

class hilo_mpc.PeriodicKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, period=1, bounds=None)[source]

Periodic covariance function

Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • period –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]

Covariance function of exponential sine as follows:

\[\sigma^2 \cdot \exp\left(-1/2 \cdot \frac{\sin(x-x_{bar})^T}{p} M \frac{\sin(x-x_{bar})}{p}\right)\]

where \(M\) is the parameterized matrix of length scales

Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

class hilo_mpc.PiecewisePolynomialKernel(degree, active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Piecewise polynomial covariance function with compact support

Parameters:

  • degree –

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]
Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

property degree
Returns:

class hilo_mpc.PolynomialKernel(degree, active_dims=None, signal_variance=1.0, offset=1.0, bounds=None)[source]

Polynomial covariance function

Parameters:

  • degree –

  • active_dims –

  • signal_variance –

  • offset –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]
Covariance function of polynomial as follows:

(sigma_bias^2 + sigma_signal^2 * (x dot x_bar))^p

Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

property degree
Returns:

class hilo_mpc.PolynomialMean(degree, active_dims=None, coefficient=1.0, offset=1.0, hyperprior=None, **kwargs)[source]

Polynomial mean function

Parameters:

  • degree –

  • active_dims –

  • coefficient –

  • offset –

  • hyperprior –

  • kwargs –

get_mean_function(x, active_dims)[source]
Parameters:

  • x –

  • active_dims –

Returns:

static get_parameterized_coefficients(dimension_input_space, coefficient)[source]
Parameters:

  • dimension_input_space –

  • coefficient –

Returns:

property degree
Returns:

class hilo_mpc.RationalQuadraticKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, alpha=1.0, ard=False, bounds=None)[source]

Rational quadratic covariance function

Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • alpha –

  • ard –

  • bounds –

get_covariance_function(x, x_bar, active_dims)[source]

Covariance function of rational quadratic as follows:

\[\sigma \cdot [1+1/(2\cdot\alpha)\cdot(x-x_{bar})^T M (x-x_{bar})]^{-\alpha}\]

where \(M\) is the parameterized matrix of length scales

Parameters:

  • x –

  • x_bar –

  • active_dims –

Returns:

class hilo_mpc.SquaredExponentialKernel(active_dims=None, signal_variance=1.0, length_scales=1.0, ard=False, bounds=None)[source]

Squared exponential covariance function

Parameters:

  • active_dims –

  • signal_variance –

  • length_scales –

  • ard –

  • bounds –

class hilo_mpc.ZeroMean[source]

Zero mean function