How to Choose the F1 Score: A Guide to Trade-offs Between Accuracy, Precision, and Recall

The F1 score is used to measure the balance between precision and recall. This explains how accuracy, precision, recall, and F1 are used together under a selected threshold, and how to reduce misjudgment through imbalanced class evaluation and a confusion matrix. Suitable for classification model evaluation reviews.

What Is the F1 Score?

The F1 score is the harmonic mean of precision and recall, using one value to reflect the balance between the two. A common calculation form is: F1 = 2 × precision × recall ÷ (precision + recall).

In classification model evaluation, the F1 score is usually used to answer one core question: whether the model can both predict the positive class relatively reliably and recover as many actual positive samples as possible. Its value is usually between 0 and 1; under the same evaluation criteria, a higher value indicates better overall performance of precision and recall.

You can think of the F1 score as a “balance score”: if precision is very high but recall is very low, or if recall is very high but precision is very low, the F1 score will be affected. Therefore, compared with looking at either metric alone, it is better suited for observing whether both have reached an acceptable level at the same time.

Key point: The F1 score is not a universal score that replaces all metrics. Rather, when both precision and recall are important, it is a comprehensive metric used to observe the balance between them.

What Questions Do Accuracy, Precision, Recall, and the F1 Score Each Answer?

Accuracy, precision, recall, and the F1 score can all be used to evaluate the prediction results of a classification model under a selected threshold, but they focus on different questions. Understanding these differences is the first step in choosing metrics.

MetricMain question answeredSituations where it is more suitable to focus onWhere it is easily misread
AccuracyWhether the overall predictions are correctThe class distribution is relatively balanced, and the cost of errors across classes is similarWhen classes are imbalanced, it may hide recognition problems for minority classes or key classes
PrecisionHow reliable the results predicted as positive by the model areTasks where the cost of false positives matters moreLooking only at precision may overlook how many actual positives the model missed
RecallHow many actual positive samples are recoveredTasks where the cost of false negatives matters moreLooking only at recall may overlook whether the results predicted as positive are reliable enough
F1 scoreWhether precision and recall are relatively balancedBoth precision and recall are important, or a comprehensive observation metric is neededA single F1 score may still hide performance differences between different classes

These metrics are not substitutes for one another. Accuracy is more like an overview of overall correctness; precision and recall observe two different types of errors respectively; and the F1 score combines precision and recall into a balance metric.

Why Pay More Attention to the F1 Score When Classes Are Imbalanced?

Class imbalance means there are obvious differences in the number of samples across different classes. In this type of data, accuracy may appear high, but the model’s recognition performance on minority classes or key classes may not be good.

For example, if a model mainly predicts the majority class, the overall proportion of correct predictions may still not be low. But if the business truly cares about the minority class, looking only at accuracy can easily lead to an overly optimistic judgment. In this case, because the F1 score considers both precision and recall, it is better suited for observing the overall performance of key classes.

In imbalanced data evaluation, you should also avoid relying only on accuracy or micro-averaged values. A more reliable approach is to combine macro results, per-class results, and the confusion matrix to check whether the performance of different classes is hidden by overall metrics.

Data situationRisk of looking only at accuracyMore recommended to focus on
Class distribution is relatively balancedThe risk is relatively low, but task goals still need to be consideredAccuracy, precision, recall, F1 score
Minority classes are very importantMajority-class performance may hide minority-class problemsF1 score, per-class results, confusion matrix
The costs of false positives and false negatives are both highA single metric is difficult to use to reflect trade-offsPrecision, recall, F1 score
Class differences are obviousThe overall score may be overly optimisticMacro results, per-class results, confusion matrix

How to Use These Metrics Together Under a Selected Threshold

Classification models usually divide prediction results into specific classes under a selected threshold. The readings of accuracy, precision, recall, and the F1 score all depend on this prediction result, so changes in the threshold will affect these metrics.

First Clarify the Positive Class and Evaluation Goal

First determine which class is the positive class in the evaluation, or which classes are key classes. Different tasks have different tolerance levels for errors: some tasks care more about false positives, some care more about false negatives, and some require the two to be balanced as much as possible.

Then Choose an Evaluation Threshold

After the threshold is selected, the model output will be converted into specific prediction results. Only then can accuracy, precision, recall, and the F1 score be calculated with a clear basis for comparison.

Review Multiple Metrics at the Same Time

Do not use only one overall score to judge whether a model is good or bad. You can first use accuracy to understand overall correctness, then use precision and recall to check error trade-offs, and finally use the F1 score to observe whether the two are balanced.

Adjust Trade-offs Based on the Costs of False Positives and False Negatives

If the cost of false positives is higher, you can focus more on precision; if the cost of false negatives is higher, you can focus more on recall; if both are important, you can prioritize the F1 score and continue checking per-class results.

Key point: Metric selection does not have a fixed answer. It is jointly determined by the definition of the positive class, class distribution, threshold settings, and error costs.

Checklist for Imbalanced Data

With imbalanced data, a single overall metric can easily hide problems in key classes. During evaluation, you can check each item in the following checklist.

  • Do not look only at accuracy, especially when majority-class samples clearly dominate.
  • Review the F1 score to judge whether precision and recall are in a relatively balanced state.
  • Review macro results to avoid overall performance being dominated by classes with larger sample sizes.
  • Review per-class results to confirm whether key classes or minority classes perform stably.
  • Analyze the error distribution together with the confusion matrix to locate which classes are more likely to be misclassified.

The confusion matrix is suitable for helping observe where errors occur: which classes are often predicted as other classes, and which classes are most easily confused with each other. Looking at the F1 score together with per-class results and the confusion matrix can reduce the risk of misjudgment caused by a single overall metric.

Check itemPurposeWhat to do after discovering a problem
AccuracyUnderstand overall prediction correctnessIf accuracy is high but key classes perform poorly, continue reviewing per-class results
F1 scoreObserve the balance between precision and recallIf F1 is low, further break it down into precision and recall
Macro resultsReduce the impact of majority classes dominating the overall judgmentIf macro results are obviously low, check minority-class performance
Per-class resultsDiscover performance differences for specific classesAnalyze false positives and false negatives for key classes
Confusion matrixLocate misclassification relationships between classesIdentify the most common confused classes and further analyze the reasons

Common Selection Approaches and Misunderstandings

When making trade-offs between accuracy, precision, recall, and the F1 score, you can start from the task goal rather than first choosing a “default optimal” metric.

Evaluation goalMore suitable to review firstReason
Want to understand whether overall predictions are correctAccuracyIt reflects overall prediction correctness, but is not suitable for use alone with imbalanced data
Want to reduce false positivesPrecisionIt focuses on whether results predicted as positive are reliable
Want to reduce false negativesRecallIt focuses on whether actual positive samples are recovered
Want to balance precision and recallF1 scoreIt combines the two into a balance metric
Want to explain the sources of errorsPer-class results, confusion matrixThey can help locate the error distribution of specific classes

There are mainly three common misunderstandings.

First, judging whether a model is good or bad by looking only at accuracy. For imbalanced class data, this approach may overlook low-quality predictions for minority classes or key classes.

Second, treating the F1 score as the only conclusion. The F1 score helps observe the balance between precision and recall, but it is still a summary metric and cannot replace per-class results and the confusion matrix.

Third, ignoring the impact of the threshold on metrics. Accuracy, precision, recall, and the F1 score all evaluate prediction results under a selected threshold; after the threshold changes, the prediction results and each metric may also change accordingly.

A more reliable selection method is: first clarify the evaluation goal and key classes, then review multiple metrics at the same time under a selected threshold; if the data is imbalanced, include the F1 score, macro results, per-class results, and the confusion matrix in the judgment together.