Better multi-class confusion matrix plots for Scikit-Learn, incorporating per-class and overall evaluation measures.
The sklearn.metrics module allows for the plotting of a confusion matrix from a
classifier (with plot_confusion_matrix), or directly from a pre-computed confusion matrix (with the internal ConfusionMatrixDisplay class).
A confusion matrix shows the discrepancy between the true labels of a dataset and the labels predicted by a classifier.
While the confusion matrix plots generated by Scikit-Learn are very informative, they omit important evaluation measures that can summarize classification performance. True positives, precision, F1 score and accuracy are example of such measures – all of which can be derived from the confusion matrix. The classification_report function in the same module provides these measures.
Daze adjusts plot_confusion_matrix to incorporate these evaluation measures directly in the confusion matrix plot, while still maintaining a very similar API
to the original Scikit-Learn function.
- Similar API to Scikit-Learn's
plot_confusion_matrix. - All common confusion matrix measures:
Accuracy, TP, FP, TN, FN, TPR, FPR, TNR, FNR, Precision, Recall, F1. - Macro & micro averaging for overall evaluation measures:
TPR, FPR, TNR, FNR, Precision, Recall, F1. - Supports both classifiers and pre-computed confusion matrices.
pip install dazeThe package API remains largely the same as that of sklearn.metrics.plot_confusion_matrix with a few additions and changes to the function arguments:
Click here to view the changes.
estimator(changed): Supports the usual fitted Scikit-Learn classifier (orsklearn.pipeline.Pipeline), but also now accepts a pre-computed confusion matrix.X(changed): Ifestimatoris a classifier, thenXare input values as usual. Ifestimatoris a confusion matrix, thenXshould be set toNone.y_true(changed): Ifestimatoris a classifier, theny_trueare target values as usual. Ifestimatoris a confusion matrix, theny_trueshould be set toNone.normalize(added): Whether or not to normalize the plotted confusion matrix (True/False). Note that if a confusion matrix is provided, it should always be un-normalized.include_measures(added): Whether or not to include evaluation measures in the confusion matrix plot (True/False).measures(added): Collection of labels for evaluation measures to display in the plot (see documentation)measures_format(added): Format string for the evaluation measure values.include_summary(added): Whether or not to include summary measures (True/False). Note thatinclude_measures=Falseoverrides this setting.summary_type(added): The type of averaging ('micro'/'macro') used for summary measures.
Documentation for the package is available on Read The Docs.
# Load the 'iris' dataset
from sklearn import datasets
from sklearn.model_selection import train_test_split
iris = datasets.load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=1)
# Train a SVM classifier on a subset of the data
from sklearn.svm import SVC
clf = SVC(kernel='linear').fit(X_train[:10], y_train[:10])
# Plot the confusion matrix
import matplotlib.pyplot as plt
from daze import plot_confusion_matrix
plt.figure(figsize=(5.5, 5.5))
plot_confusion_matrix(clf, X_test, y_test, display_labels=iris.target_names, measures=...)
plt.show()
measures=
|
a, c, p, r, f1
|
a, tp, fp, fpr, tnr, p
|
a, tn, fn, tpr, fnr, r
|
|---|---|---|---|
| Plot |
|
|
|
# Use the previous classifier to make predictions and create a confusion matrix
from sklearn.metrics import confusion_matrix
y_pred = clf.predict(X_test)
cm = confusion_matrix(y_test, y_pred)
# Make a plot from a pre-computed confusion matrix
plt.figure(figsize=(5.5, 5.5))
plot_confusion_matrix(cm, display_labels=iris.target_names)
plt.show()Daze uses Scikit-Learn source code for the majority of the ConfusionMatrixDisplay class and plot_confusion_matrix function re-implemetations, under the terms of the BSD-3-Clause license.
Click here to view the redistribution license.
BSD 3-Clause License Copyright (c) 2007-2020 The scikit-learn developers. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Daze © 2021-2022, Edwin Onuonga - Released under the MIT License.
Authored and maintained by Edwin Onuonga.