What Is Logit: From Log-Odds to Logistic Regression Applications

Logit is a function that converts the probability of an event into log-odds. It is the inverse function of Sigmoid and helps explain how logistic regression estimates the likelihood of occurrence in binary classification tasks.

What Is Logit?

Logit is a function that converts the probability of an event occurring into log-odds. In Chinese, it is often translated as “分对数” or “洛吉”. Its core formula is: logit(p)=ln(p/(1-p)). Here, p represents the probability that the event occurs, and 1-p represents the probability that the event does not occur.

In this formula, p/(1-p) is the odds of the event occurring relative to not occurring, and Logit is the natural logarithm of those odds. In other words, Logit does not directly represent the probability itself; instead, it rewrites probability onto another numerical scale that is more convenient for modeling and comparison.

Key point: Probability answers “how likely is it to occur,” odds answer “how strong is occurrence relative to non-occurrence,” and Logit takes the natural logarithm of the odds.

The naming of Logit is usually traced back to Joseph Berkson’s use and naming of the function in 1944. In the context of modern statistical learning and machine learning, understanding Logit is usually helpful for further understanding logistic regression, Sigmoid, and the outputs of binary classification models.

From Probability to Odds to Logit

Probability, odds, and Logit describe the same thing in different forms. Probability focuses on the proportion of an event occurring among all possible outcomes; odds focus on the relative relationship between an event occurring and not occurring; Logit further takes the natural logarithm of the odds.

Using the event occurrence probability p as an example:

ConceptExpressionMeaning
ProbabilitypThe likelihood that the event occurs, usually between 0 and 1
Probability of non-occurrence1-pThe likelihood that the event does not occur
Oddsp/(1-p)The relative ratio of occurrence to non-occurrence
Logitln(p/(1-p))The natural logarithm of the odds

If p=0.5, then the odds are 1, and the Logit value is 0, indicating that occurrence and non-occurrence are equal in terms of odds. If p=0.8, then the odds are 4, and the Logit value is approximately 1.39, indicating that occurrence has an advantage over non-occurrence. If p=0.2, then the odds are 0.25, and the Logit value is approximately -1.39, indicating that non-occurrence has an advantage.

Probability pOdds p/(1-p)Logit valueIntuitive understanding
0.20.25approximately -1.39The event is less likely to occur than not occur
0.510The odds of occurrence and non-occurrence are equal
0.84approximately 1.39The event is more likely to occur than not occur

The role of Logit can be understood as follows: it converts probability, which is originally compressed between 0 and 1, into a numerical scale that can be either positive or negative. This helps describe the relationship between input factors and the likelihood of an event occurring in a model.

What Is the Relationship Between Logit and Sigmoid?

The Logit function and the Sigmoid function are inverse functions of each other. If a value is the output of the Sigmoid function, then its Logit value is the input of the Sigmoid function; conversely, if a value is passed through Sigmoid to obtain a probability, that probability can then be passed through Logit to return to the original input scale.

Their relationship can be summarized in one sentence: Logit pulls probability back into real-number space, while Sigmoid maps real numbers into probabilities.

DirectionInputOutputRole
LogitProbability pLog-oddsConverts from the probability scale to a numerical scale more suitable for modeling
SigmoidReal-valued numberProbability between 0 and 1Converts model values into probability form

In binary classification models, this inverse relationship is important. Internally, a model can handle a numerical relationship that may not directly appear as a probability, while the final decision still needs the likelihood that a certain type of event will occur. Logit and Sigmoid help explain the conversion between these two scales.

What Role Does Logit Play in Logistic Regression?

Logistic regression is a model used for classification and is often used to solve binary classification problems, such as tasks where the outcome is 0 or 1. Although its name contains “regression,” it is usually used to estimate the likelihood that a certain type of event will occur and to make classification decisions accordingly, rather than to output continuous numerical predictions.

From a machine learning perspective, logistic regression is a classification algorithm in supervised machine learning and can be used to predict discrete or categorical outcomes. It connects input factors with target outcomes through mathematical relationships and predicts classes based on those relationships.

The value of Logit here is mainly reflected in probability interpretation:

  • Logistic regression focuses on the likelihood that a certain event will occur;
  • Binary classification problems need to connect input features with “belonging to a certain class” or “not belonging to a certain class”;
  • The inverse relationship between Logit and Sigmoid can help explain the conversion process between model values and probability outputs.

Therefore, Logit is frequently encountered when learning logistic regression. It is not logistic regression itself, but an important mathematical tool for understanding the relationship among probability, odds, and classification decisions.

What Are the Differences Among Logit, Probability, and Logistic Regression?

Logit, probability, and logistic regression often appear together, but they are not concepts at the same level.

ConceptEssenceProblem solvedRelationship with binary classification
ProbabilityA representation of the likelihood that an event occursIndicates how likely an event is to occurCan serve as one basis for classification decisions
OddsThe ratio of the probability of occurrence to the probability of non-occurrenceIndicates the relative strength of occurrence versus non-occurrenceHelps compare the relative tendency of two outcomes
LogitA function or transformation that takes the natural logarithm of the oddsConverts probability into the log-odds scaleHelps explain the relationship between model values and probability
SigmoidA function that is the inverse of LogitConverts real-valued numbers into probability formOften used to understand probability outputs of classification models
Logistic regressionA classification algorithm in supervised machine learningPredicts discrete or categorical outcomes based on data relationshipsOften used for 0/1 binary classification problems

You can understand them in a layered relationship: probability is the expression of the result, Logit is a probability transformation, and logistic regression is a model that uses related mathematical relationships to make classification predictions. For beginners, it is easier to connect these concepts by first understanding probability and odds, then understanding Logit and Sigmoid, and finally looking at logistic regression.

When Do You Need to Understand Logit?

The value of understanding Logit lies not only in the formula itself, but also in its ability to help readers understand probability transformations and classification decisions in binary classification models.

User needSolutionEffect
Getting started with machine learningUse Logit to understand the relationship among probability, odds, and SigmoidMakes it easier to understand how binary classification models convert numerical relationships into classification results
Learning logistic regressionView logistic regression as a classification model that estimates the likelihood of an event occurringAvoids misunderstanding “logistic regression” as an ordinary regression task
Explaining modelsDistinguish probability, odds, Logit, and the final classification decisionMakes it clearer to explain why a certain type of event is more likely to occur
Data analysis modelingUnderstand how models predict discrete outcomes based on relationships among factorsHelps better select and explain classification analysis methods

In an actual learning path, you can use the following sequence to check whether you have understood it properly:

  • When you see p, first confirm that it represents the probability that the event occurs;
  • When you see p/(1-p), understand that it represents the odds of occurrence versus non-occurrence;
  • When you see ln(p/(1-p)), know that it is Logit;
  • When you see Sigmoid, know that it is the inverse function of Logit;
  • When you see logistic regression, understand that it is a model used for classification prediction and is often used for binary classification problems.