Machine Learning Robustness and Monitoring

About this talk

Mercury is a python library used in BBVA in data science projects. Recently, we open-sourced some parts of the library to the python and data science community.

In this workshop, we will introduce mercury.robust and mercury.monitoring. With mercury.robust we can perform dataset and model testing to detect undesirable conditions such as high level of noise in the labels, label leaking in the features of the dataset, or model excessive sensitivity to drift in certain features. Mercury.monitoring includes components that can help us detect data drift in our inference data and estimate the performance of our model when real labels are not available yet.

During the workshop we will use components of these libraries during different phases of a machine learning project: dataset generation, model training, and model inference.

The workshop is intended for data scientists and machine learning engineers interested in tools that can help improve the robustness and performance degradation detection of their machine learning models.