Introduction

The integration of Machine Learning (ML) techniques and Mathematical Optimization is a topic of growing interest. One particular approach is to be able to use trained ML models in optimization models (Bergman et al. [BHB+22], Maragno et al. [MWB+21], Ceccon et al. [CJH+22]). In this approach, the input and the prediction of the ML model are decision variables of the optimization while its parameters are fixed. We say that the ML model is formulated in the optimization model, and we refer to the input and predictions as input and output variables respectively. They are linked in that, in a feasible solution, the output variables values are the values predicted by the ML model from the input variables values.

Gurobi Machine Learning is an open-source Python package to formulate trained regression models [1] in a Model to be solved with the Gurobi solver.

The aim of the package is to:

  1. Provide a simple way to formulate regression models in a gurobipy model.

  2. Promote a deeper integration between predictive and prescriptive analytics.

  3. Allow to easily compare the effect of using different regression models in the same mathematical optimization application.

  4. Improve the algorithmic performance of Gurobi on those models.

The package currently supports various scikit-learn objects. It can also formulate gradient boosting regression models from XGboost and LightGBM <https://lightgbm.readthedocs.io/en/stable/>. Finally, it has limited support for Keras. Only neural networks with ReLU activation can be used with these two packages.

The package is actively developed and users are encouraged to contact us if they have applications where they use a regression model that is currently not available.

Below, we give basic installation and usage instructions.

Install

We encourage to install the package via pip (or add it to your requirements.txt file):

(.venv) pip install gurobi-machinelearning

Note

If not already installed, this should install the gurobipy, numpy and scipy packages.

Note

The package is tested with and is supported for Python 3.9, 3.10, 3.11 and 3.12. It is also tested and supported with Gurobi 10, 11 and 12. Note however, that some newer features of Gurobi from later versions are used and some models may perform significantly worse with the older versions.

The following table lists the version of the relevant packages that are tested and supported in the current version (1.5.4).

Supported packages with version 1.5.4

Package

Version

numpy

2.0.2

scipy

1.13.1

pandas

2.2.3

torch

2.7.0

scikit-learn

1.6.1

keras

3.9.2

xgboost

2.1.4

lightgbm

4.6.0

Installing any of the machine learning packages is only required if the predictor you want to insert uses them (i.e. to insert a Keras based predictor you need to have keras installed).

Usage

The main function provided by the package is gurobi_ml.add_predictor_constr(). It takes as arguments: a Model, a supported regression model, input Gurobi variables and output Gurobi variables.

By invoking the function, the Model is augmented with variables and constraints so that, in a solution, the values of the output variables are predicted by the regression model from the values of the input variables. More formally, if we denote by \(g\) the prediction function of the regression model, by \(x\) the input variables and by \(y\) the output variables, then \(y = g(x)\) in any solution.

The function add_predictor_constr returns a modeling object derived from the class AbstractPredictorConstr. That object keeps track of all the variables and constraints that have been added to the Model to establish the relationship between input and output variables of the regression.

The modeling object can perform a few tasks:

  • Everything it created (i.e. variables and constraints to establish the relationship between input and output) can be removed with the remove method.

  • It can print a summary of what it added with the print_stats method.

  • Once Gurobi computed a solution to the optimization problem, it can compute the difference between what the regression model predicts from the input values and the values of the output variables in Gurobi’s solution with the get_error method.

The function add_predictor_constr is a shorthand that should add the correct model for any supported regression model, but individual functions for each regression model are also available. For the list of frameworks and regression models supported, and the corresponding functions please refer to the Supported Regression models section. We also briefly outline how the various regression models are expressed in Gurobi in the Mixed Integer Formulations section.

For some regression models, additional optional parameters can be set to tune the way the predictor is inserted in the Gurobi model. Those are documented in the corresponding function linked from Supported Regression models.

For a simple example on how to use the package please refer to Usage Example. More advanced examples are available in the Examples section.

Footnotes