Introduction#
The integration of Machine Learning (ML) techniques and Mathematical Optimization is a topic of growing interest. One particular approach is to be able to use trained ML models in optimization models (Bergman et al. [BHB+22], Maragno et al. [MWB+21], Ceccon et al. [CJH+22]). In this approach, the input and the prediction of the ML model are decision variables of the optimization while its parameters are fixed. We say that the ML model is formulated in the optimization model, and we refer to the input and predictions as input and output variables respectively. They are linked in that, in a feasible solution, the output variables values are the values predicted by the ML model from the input variables values.
Gurobi Machine Learning is an opensource Python package to formulate trained regression models [1] in a gurobipy model to be solved with the Gurobi solver.
The aim of the package is to:
Provide a simple way to formulate regression models in a gurobipy model.
Promote a deeper integration between predictive and prescriptive analytics.
Allow to easily compare the effect of using different regression models in the same mathematical optimization application.
Improve the algorithmic performance of Gurobi on those models.
The package currently supports various scikitlearn objects. It can also formulate gradient boosting regression models from XGboost and LightGBM <https://lightgbm.readthedocs.io/en/stable/>. Finally, it has limited support for the Keras API of TensorFlow and PyTorch. Only neural networks with ReLU activation can be used with these two packages.
The package is actively developed and users are encouraged to contact us if they have applications where they use a regression model that is currently not available.
Below, we give basic installation and usage instructions.
Install#
We encourage to install the package via pip (or add it to your requirements.txt file):
(.venv) pip install gurobimachinelearning
Note
The package has been tested with and is supported for Python 3.9 and Python 3.10.
The following table lists the version of the relevant packages that are tested and supported in the current version (1.5.1).
Package 
Version 

11.0.3 

1.26.4 

1.13.0 

2.2.2 

2.4.0 

1.5.1 

2.16.1 

2.1.1 

4.5.0 
Installing any of the machine learning packages is only required if the predictor you want to insert uses them (i.e. to insert a Keras based predictor you need to have tensorflow installed).
Usage#
The main function provided by the package is
gurobi_ml.add_predictor_constr()
. It takes as arguments: a gurobipy model, a
supported regression model, input Gurobi variables and
output Gurobi variables.
By invoking the function, the gurobipy model is augmented with variables and constraints so that, in a solution, the values of the output variables are predicted by the regression model from the values of the input variables. More formally, if we denote by \(g\) the prediction function of the regression model, by \(x\) the input variables and by \(y\) the output variables, then \(y = g(x)\) in any solution.
The function add_predictor_constr
returns a modeling object derived from the class
AbstractPredictorConstr
. That object keeps track of all
the variables and constraints that have been added to the gurobipy model to
establish the relationship between input and output variables of the regression.
The modeling object can perform a few tasks:
Everything it created (i.e. variables and constraints to establish the relationship between input and output) can be removed with the
remove
method.It can print a summary of what it added with the
print_stats
method.Once Gurobi computed a solution to the optimization problem, it can compute the difference between what the regression model predicts from the input values and the values of the output variables in Gurobi’s solution with the
get_error
method.
The function add_predictor_constr
is
a shorthand that should add the correct model for any supported regression
model, but individual functions for each regression model are also available.
For the list of frameworks and regression models supported, and the corresponding
functions please refer to the Supported Regression models section. We also briefly
outline how the various regression models are expressed in Gurobi in the Mixed Integer Formulations
section.
For some regression models, additional optional parameters can be set to tune the way the predictor is inserted in the Gurobi model. Those are documented in the corresponding function linked from Supported Regression models.
For a simple example on how to use the package please refer to Usage Example. More advanced examples are available in the Examples section.
Footnotes