Confusion Matrix: Principles and Metrics for Classification Evaluation

A confusion matrix is a classification model evaluation table that compares true classes with predicted classes. It counts correct and incorrect samples and supports the calculation of accuracy, precision, and recall, making it suitable for diagnosing binary classification and multiclass models.

What Is a Confusion Matrix?

A confusion matrix, also called an error matrix, is a tabular evaluation method used to show the prediction results of a classifier or classification model. It places true classes and predicted classes in the same matrix for comparison, using sample counts to show which judgments the model got right and which it got wrong.

Unlike providing only a single overall score, a confusion matrix focuses on the distribution of classification results. It not only shows how many samples the model predicted correctly, but also helps observe whether there is obvious confusion between different classes. Therefore, it is commonly used for model evaluation in machine learning and statistical classification problems.

Key point: A confusion matrix itself is a statistical table or matrix, not a single evaluation score; metrics such as accuracy, precision, and recall are usually calculated further based on these statistical results.

How a Confusion Matrix Organizes True Classes and Predicted Classes

The core of a confusion matrix is cross-tabulation: one side represents the true classes, the other side represents the classes predicted by the model, and each cell in the matrix records the number of samples that fall into a certain “true class × predicted class” combination.

When reading a specific matrix, you need to first confirm what the rows and columns represent. Different tools or materials may use different orientations, but the basic logic is the same: the intersection of true class and predicted class is used to record classification results.

ComponentMeaningReading focus
True classThe class to which the sample actually belongsUsed as the evaluation reference
Predicted classThe class output by the modelUsed to observe the model’s judgment results
Cross cellThe number of samples in which a certain true class is predicted as a certain classUsed to judge the distribution of correct or incorrect predictions
Diagonal positionPositions where the true class and predicted class are the sameUsually indicates correctly predicted samples
Off-diagonal positionPositions where the true class and predicted class are differentUsually indicates incorrect predictions or class confusion

For binary classification tasks, the matrix usually revolves around two classes; for multiclass tasks, the matrix can be extended to n rows and n columns, with the number of classes determining the matrix size. The more classes there are, the more the matrix can show misclassification relationships among different classes, but when reading it, it is also more necessary to focus on key class combinations.

Understanding a Confusion Matrix with a Simple Analogy

A confusion matrix can be understood as a wrong-answer statistics table after an exam. The standard answer for each question is equivalent to the true class, and the answer filled in by the student is equivalent to the predicted class; the statistics table not only records how many questions were answered correctly, but also records which question types are most likely to be answered incorrectly and which wrong answers occur most often.

Corresponding to a classification model, a confusion matrix not only tells you the number of samples the model predicted correctly, but also shows which class a certain class is often misclassified as. Compared with looking only at accuracy, this distribution information is more suitable for analyzing the sources of model errors.

For example, in a multiclass task, if two classes frequently appear in the corresponding off-diagonal cells, it indicates that the model may have difficulty distinguishing these two classes. At this point, the evaluator can further inspect data labeling, class definitions, or model training results, rather than drawing a conclusion based only on one overall score.

Which Evaluation Metrics Can a Confusion Matrix Support?

A confusion matrix stores classification statistics and is commonly used to further calculate evaluation metrics such as accuracy, precision, and recall. These metrics summarize the sample distribution in the matrix from different perspectives, helping evaluators turn detailed statistical results into values that are easier to compare.

MetricQuestion it focuses onRelationship with the confusion matrix
AccuracyHow many of the model’s overall predictions are correctCalculated based on the statistical relationship between correctly predicted samples and total samples
PrecisionWhen the model predicts a certain class, how reliable are these predictionsDepends on the relevant statistical results under a certain predicted class
RecallHow many samples in a certain true class are identified by the modelDepends on the relevant statistical results under a certain true class

It should be noted that metrics and the matrix are not things at the same level. A confusion matrix provides raw classification counts, while metrics are further summaries based on the counting results. Before reading metrics, you should first confirm which true-class and predicted-class positions the samples are counted into in the matrix, to avoid misreading the results due to misunderstanding the row and column orientation.

Which Classification Tasks Are Confusion Matrices Suitable For?

A confusion matrix is mainly used for evaluating classification models where true classes are available for comparison, and is common in supervised learning scenarios. It can be used for both binary classification evaluation and multiclass model evaluation.

User needSolutionEffect
Want to quickly summarize classification model resultsUse a matrix to count the intersections of true classes and predicted classesIntuitively see the distribution of correct and incorrect predictions
Want to locate the classes where the model is prone to making mistakesCheck where samples are concentrated in off-diagonal positionsDiscover which classes are more easily confused with each other
Want to further calculate evaluation metricsCalculate accuracy, precision, and recall based on the statistical results in the matrixConvert classification results into comparable metrics
Evaluate a binary classification taskUse two classes to form the matrixObserve correct classifications and misclassifications between the two classes
Evaluate a multiclass taskExtend the matrix to n rows and n columnsAnalyze prediction relationships among multiple classes

In practice, a confusion matrix is especially suitable as a diagnostic tool for classification models. It helps evaluators move from “what is the model’s score” to further asking “where is the model wrong.”

What Is the Difference Between a Confusion Matrix and a Matching Matrix?

A confusion matrix usually discusses classification tasks in supervised learning, because it requires true classes as a reference. Without true classes or labels, it is not possible to judge in the same way whether a predicted class is consistent with the true class.

In unsupervised learning scenarios, a similar matrix is generally called a matching matrix. It can be used to compare the correspondence between clustering results and some external reference, but not all statistics of clustering or unsupervised results should be directly called confusion matrices.

Comparison itemConfusion matrixMatching matrix
Common scenarioSupervised learning classification tasksSimilar result comparison in unsupervised learning
Whether it depends on true classesUsually requires true classes as the evaluation benchmarkUsually used for matching relationships between results and references
Main purposeSummarize the distribution of correct and incorrect classification predictionsDescribe the correspondence between different result sets

Checklist for Reading a Confusion Matrix

When reading a confusion matrix, you can check in the following order to avoid looking only at local numbers or misreading the row and column orientation.

  • Confirm the task type: First determine whether it is a binary classification task or a multiclass task. A binary classification matrix is more compact, while a multiclass matrix usually expands to n rows and n columns as the number of classes increases.
  • Confirm the meaning of rows and columns: Clarify which dimension represents true classes and which dimension represents predicted classes. Reversing the direction will affect the understanding of subsequent metrics.
  • Check correctly predicted samples: Observe the positions where true classes and predicted classes are the same to understand on which classes the model makes relatively stable judgments.
  • Check incorrectly predicted samples: Focus on the positions where true classes and predicted classes are different, especially cells where the number of samples is relatively concentrated.
  • Evaluate performance together with metrics: After understanding the matrix distribution, evaluate it together with metrics such as accuracy, precision, and recall.
  • For multiclass tasks, focus on class combinations: If it is a multiclass task, do not look only at the total count; also pay attention to which classes are most easily confused with each other.

Key point: A confusion matrix is suitable for looking at the distribution first and then the metrics. Only by first understanding the correspondence between true classes and predicted classes can you interpret the evaluation results of a classification model more accurately.