How to Choose the F1 Score: A Guide to Trade-offs Between Accuracy, Precision, and Recall
The F1 score is used to measure the balance between precision and recall. This explains how accuracy, precision, recall, and F1 are used together under a selected threshold, and how to reduce misjudgment through imbalanced class evaluation and a confusion matrix. Suitable for classification model evaluation reviews.
What Is the F1 Score?
The F1 score is the harmonic mean of precision and recall, using one value to reflect the balance between the two. A common calculation form is: F1 = 2 × precision × recall ÷ (precision + recall).
In classification model evaluation, the F1 score is usually used to answer one core question: whether the model can both predict the positive class relatively reliably and recover as many actual positive samples as possible. Its value is usually between 0 and 1; under the same evaluation criteria, a higher value indicates better overall performance of precision and recall.
You can think of the F1 score as a “balance score”: if precision is very high but recall is very low, or if recall is very high but precision is very low, the F1 score will be affected. Therefore, compared with looking at either metric alone, it is better suited for observing whether both have reached an acceptable level at the same time.
Key point: The F1 score is not a universal score that replaces all metrics. Rather, when both precision and recall are important, it is a comprehensive metric used to observe the balance between them.
What Questions Do Accuracy, Precision, Recall, and the F1 Score Each Answer?
Accuracy, precision, recall, and the F1 score can all be used to evaluate the prediction results of a classification model under a selected threshold, but they focus on different questions. Understanding these differences is the first step in choosing metrics.
| Metric | Main question answered | Situations where it is more suitable to focus on | Where it is easily misread |
|---|---|---|---|
| Accuracy | Whether the overall predictions are correct | The class distribution is relatively balanced, and the cost of errors across classes is similar | When classes are imbalanced, it may hide recognition problems for minority classes or key classes |
| Precision | How reliable the results predicted as positive by the model are | Tasks where the cost of false positives matters more | Looking only at precision may overlook how many actual positives the model missed |
| Recall | How many actual positive samples are recovered | Tasks where the cost of false negatives matters more | Looking only at recall may overlook whether the results predicted as positive are reliable enough |
| F1 score | Whether precision and recall are relatively balanced | Both precision and recall are important, or a comprehensive observation metric is needed | A single F1 score may still hide performance differences between different classes |
These metrics are not substitutes for one another. Accuracy is more like an overview of overall correctness; precision and recall observe two different types of errors respectively; and the F1 score combines precision and recall into a balance metric.
Why Pay More Attention to the F1 Score When Classes Are Imbalanced?
Class imbalance means there are obvious differences in the number of samples across different classes. In this type of data, accuracy may appear high, but the model’s recognition performance on minority classes or key classes may not be good.
For example, if a model mainly predicts the majority class, the overall proportion of correct predictions may still not be low. But if the business truly cares about the minority class, looking only at accuracy can easily lead to an overly optimistic judgment. In this case, because the F1 score considers both precision and recall, it is better suited for observing the overall performance of key classes.
In imbalanced data evaluation, you should also avoid relying only on accuracy or micro-averaged values. A more reliable approach is to combine macro results, per-class results, and the confusion matrix to check whether the performance of different classes is hidden by overall metrics.
| Data situation | Risk of looking only at accuracy | More recommended to focus on |
|---|---|---|
| Class distribution is relatively balanced | The risk is relatively low, but task goals still need to be considered | Accuracy, precision, recall, F1 score |
| Minority classes are very important | Majority-class performance may hide minority-class problems | F1 score, per-class results, confusion matrix |
| The costs of false positives and false negatives are both high | A single metric is difficult to use to reflect trade-offs | Precision, recall, F1 score |
| Class differences are obvious | The overall score may be overly optimistic | Macro results, per-class results, confusion matrix |
How to Use These Metrics Together Under a Selected Threshold
Classification models usually divide prediction results into specific classes under a selected threshold. The readings of accuracy, precision, recall, and the F1 score all depend on this prediction result, so changes in the threshold will affect these metrics.
First Clarify the Positive Class and Evaluation Goal
First determine which class is the positive class in the evaluation, or which classes are key classes. Different tasks have different tolerance levels for errors: some tasks care more about false positives, some care more about false negatives, and some require the two to be balanced as much as possible.
Then Choose an Evaluation Threshold
After the threshold is selected, the model output will be converted into specific prediction results. Only then can accuracy, precision, recall, and the F1 score be calculated with a clear basis for comparison.
Review Multiple Metrics at the Same Time
Do not use only one overall score to judge whether a model is good or bad. You can first use accuracy to understand overall correctness, then use precision and recall to check error trade-offs, and finally use the F1 score to observe whether the two are balanced.
Adjust Trade-offs Based on the Costs of False Positives and False Negatives
If the cost of false positives is higher, you can focus more on precision; if the cost of false negatives is higher, you can focus more on recall; if both are important, you can prioritize the F1 score and continue checking per-class results.
Key point: Metric selection does not have a fixed answer. It is jointly determined by the definition of the positive class, class distribution, threshold settings, and error costs.
Checklist for Imbalanced Data
With imbalanced data, a single overall metric can easily hide problems in key classes. During evaluation, you can check each item in the following checklist.
- Do not look only at accuracy, especially when majority-class samples clearly dominate.
- Review the F1 score to judge whether precision and recall are in a relatively balanced state.
- Review macro results to avoid overall performance being dominated by classes with larger sample sizes.
- Review per-class results to confirm whether key classes or minority classes perform stably.
- Analyze the error distribution together with the confusion matrix to locate which classes are more likely to be misclassified.
The confusion matrix is suitable for helping observe where errors occur: which classes are often predicted as other classes, and which classes are most easily confused with each other. Looking at the F1 score together with per-class results and the confusion matrix can reduce the risk of misjudgment caused by a single overall metric.
| Check item | Purpose | What to do after discovering a problem |
|---|---|---|
| Accuracy | Understand overall prediction correctness | If accuracy is high but key classes perform poorly, continue reviewing per-class results |
| F1 score | Observe the balance between precision and recall | If F1 is low, further break it down into precision and recall |
| Macro results | Reduce the impact of majority classes dominating the overall judgment | If macro results are obviously low, check minority-class performance |
| Per-class results | Discover performance differences for specific classes | Analyze false positives and false negatives for key classes |
| Confusion matrix | Locate misclassification relationships between classes | Identify the most common confused classes and further analyze the reasons |
Common Selection Approaches and Misunderstandings
When making trade-offs between accuracy, precision, recall, and the F1 score, you can start from the task goal rather than first choosing a “default optimal” metric.
| Evaluation goal | More suitable to review first | Reason |
|---|---|---|
| Want to understand whether overall predictions are correct | Accuracy | It reflects overall prediction correctness, but is not suitable for use alone with imbalanced data |
| Want to reduce false positives | Precision | It focuses on whether results predicted as positive are reliable |
| Want to reduce false negatives | Recall | It focuses on whether actual positive samples are recovered |
| Want to balance precision and recall | F1 score | It combines the two into a balance metric |
| Want to explain the sources of errors | Per-class results, confusion matrix | They can help locate the error distribution of specific classes |
There are mainly three common misunderstandings.
First, judging whether a model is good or bad by looking only at accuracy. For imbalanced class data, this approach may overlook low-quality predictions for minority classes or key classes.
Second, treating the F1 score as the only conclusion. The F1 score helps observe the balance between precision and recall, but it is still a summary metric and cannot replace per-class results and the confusion matrix.
Third, ignoring the impact of the threshold on metrics. Accuracy, precision, recall, and the F1 score all evaluate prediction results under a selected threshold; after the threshold changes, the prediction results and each metric may also change accordingly.
A more reliable selection method is: first clarify the evaluation goal and key classes, then review multiple metrics at the same time under a selected threshold; if the data is imbalanced, include the F1 score, macro results, per-class results, and the confusion matrix in the judgment together.