[Examples| Tutorials| Docs]
Utilities and Models to perform Uncertainty Quantification on Keras.Keras-Uncertainty is a high-level API to perform uncertainty quantification of machine learning models built with Keras. Many real-world applications not only require a model to make a prediction, but also to provide a confidence value that indicates how trustworthy is the prediction, which can be integrated into the decision making progress.
Typical research in machine learning applications (Computer Vision, NLP, etc) usually does not consider ways to produce well behaved uncertainty estimates, and machine learning methods can be used to extract or include uncertainty information into the model.
Code example available here
Code example available here
You can easily install with pip, using the following command:
pip install --user git+https://github.com/mvaldenegro/keras-uncertainty.git
- Entropy and Negative Log-Likelihood metrics.
- Calibration plots for classification.
- Accuracy vs Confidence plot for classification.
- Statistical metrics like Interval Score and others.
| Method | Classification | Regression |
|---|---|---|
| Ensembles | ✔️ | ✔️ |
| MC-Dropout | ✔️ | ✔️ |
| MC-DropConnect | ✔️ | ✔️ |
| Direct UQ | ✔️ | ❌ |
| Bayes by Backprop | ✔️ | ✔️ |
| Flipout | ✔️ | ✔️ |
| Gradient | ✔️ | ❌ |
Please note that some UQ methods do not support some tasks like regression. Most methods support classification.
