Ctrl+K

Introduction#

The integration of Machine Learning (ML) techniques and Mathematical Optimization is a topic of growing interest. One particular approach is to be able to use trained ML models in optimization models (Bergman et al. [BHB+22], Maragno et al. [MWB+21], Ceccon et al. [CJH+22]). In this approach, the input and the prediction of the ML model are decision variables of the optimization while its parameters are fixed. We say that the ML model is formulated in the optimization model, and we refer to the input and predictions as input and output variables respectively. They are linked in that, in a feasible solution, the output variables values are the values predicted by the ML model from the input variables values.

Gurobi Machine Learning is an open-source Python package to formulate trained regression models [1] in a gurobipy model to be solved with the Gurobi solver.

The aim of the package is to:

  1. Provide a simple way to formulate regression models in a gurobipy model.

  2. Promote a deeper integration between predictive and prescriptive analytics.

  3. Allow to easily compare the effect of using different regression models in the same mathematical optimization application.

  4. Improve the algorithmic performance of Gurobi on those models.

The package currently supports various scikit-learn objects. It can also formulate gradient boosting regression models from XGboost and LightGBM <https://lightgbm.readthedocs.io/en/stable/>. Finally, it has limited support for the Keras API of TensorFlow and PyTorch. Only neural networks with ReLU activation can be used with these two packages.

The package is actively developed and users are encouraged to contact us if they have applications where they use a regression model that is currently not available.

Below, we give basic installation and usage instructions.

Install#

We encourage to install the package via pip (or add it to your requirements.txt file):

(.venv) pip install gurobi-machinelearning

Note

If not already installed, this should install the gurobipy, numpy and scipy packages.

Note

The package has been tested with and is supported for Python 3.9 and Python 3.10.

The following table lists the version of the relevant packages that are tested and supported in the current version (1.5.0).

Supported packages with version 1.5.0#

Package

Version

gurobipy

11.0.1

numpy

1.26.4

scipy

1.12.0

pandas

2.2.1

torch

2.2.1

scikit-learn

1.4.1.post1

tensorflow

2.15.0.post1

xgboost

2.0.3

lightgbm

4.3.0

Installing any of the machine learning packages is only required if the predictor you want to insert uses them (i.e. to insert a Keras based predictor you need to have tensorflow installed).

Usage#

The main function provided by the package is gurobi_ml.add_predictor_constr(). It takes as arguments: a gurobipy model, a supported regression model, input Gurobi variables and output Gurobi variables.

By invoking the function, the gurobipy model is augmented with variables and constraints so that, in a solution, the values of the output variables are predicted by the regression model from the values of the input variables. More formally, if we denote by \(g\) the prediction function of the regression model, by \(x\) the input variables and by \(y\) the output variables, then \(y = g(x)\) in any solution.

The function add_predictor_constr returns a modeling object derived from the class AbstractPredictorConstr. That object keeps track of all the variables and constraints that have been added to the gurobipy model to establish the relationship between input and output variables of the regression.

The modeling object can perform a few tasks:

  • Everything it created (i.e. variables and constraints to establish the relationship between input and output) can be removed with the remove method.

  • It can print a summary of what it added with the print_stats method.

  • Once Gurobi computed a solution to the optimization problem, it can compute the difference between what the regression model predicts from the input values and the values of the output variables in Gurobi’s solution with the get_error method.

The function add_predictor_constr is a shorthand that should add the correct model for any supported regression model, but individual functions for each regression model are also available. For the list of frameworks and regression models supported, and the corresponding functions please refer to the Supported Regression models section. We also briefly outline how the various regression models are expressed in Gurobi in the Mixed Integer Formulations section.

For some regression models, additional optional parameters can be set to tune the way the predictor is inserted in the Gurobi model. Those are documented in the corresponding function linked from Supported Regression models.

For a simple example on how to use the package please refer to Usage Example. More advanced examples are available in the Examples section.

Footnotes

On this page