Further Reading

The approach of formulating machine learning models in optimization models has received sustained attention in recent years with several publications and published research software packages. Here, we make an informal annotated bibliography of the works in which we have been interested when developing the package. We don’t claim to be exhaustive.

The JANOS framework was proposed in Bergman et al. [BHB+22] with an associated Python package. The package works with various Scikit-learn models and solves optimization problems with Gurobi. The Student Enrollment example was proposed in that paper.

Another framework is reluMIP, Lueg et al. [LGMS21]. It is mostly aimed at neural networks with ReLU activation formulated with TensorFlow. The same authors study in particular the use of neural networks in surrogate models, e.g. Grimstad and Andersson [GA19].

The OptiCL framework was proposed in Maragno et al. [MWB+21]. An associated python package, Maragno and Wiberg [MW21], is available. The package can model several Scikit-learn objects. The authors proposed several interesting applications: palatable diet, cancer treatment. They also propose original algorithmic approaches to ensure credible predictions and avoid extrapolations.

Finally, among research software packages, OMLT (Ceccon et al. [CJH+22]) is a Python package that supports a variety of neural network structures (dense layers, convolutional layers, pooling layers) and gradient boosting trees. It is hooked with the ONNX open format. It is actively developed and evolving. It is in particular aimed at studying alternative formulations for the neural network structures.

There is a growing literature on efficient MIP formulation for neural networks. Anderson et al. [AHM+20], Tjandraatmadja et al. [TAH+20] and Kronqvist et al. [KMT21] are good starting points.