Logistic Regression

What Is Logistic Regression?

Logistic Regression is mainly used for classification problems.

Instead of predicting a continuous number such as a house price, it helps answer questions like:

A Simple Example

Imagine that we want to predict whether a student will Pass or Fail an exam.

We can use the student's study hours as a feature.

The model learns the relationship between study hours and the known results.

Why Does It Predict a Probability?

Logistic Regression does not immediately say only "Pass" or "Fail."

It first produces a probability for the class we are interested in.

The probability is between 0 and 1, or equivalently between 0% and 100%.

From Probability to Class

We need a rule to convert the predicted probability into a class.

A common default threshold is 0.5.

For our Pass/Fail example, we can define:

Understanding the Classification Decision

Think of Logistic Regression as a two-step process.

This is the key idea behind using Logistic Regression for binary classification.

The Logistic Function

Logistic Regression uses a mathematical function called the logistic function, also called the sigmoid function.

Its job is to convert the model's calculated score into a value between 0 and 1.

You do not need to memorize the formula yet. The important idea is what the function does:

Logistic Regression With Multiple Features

A real model usually uses more than one feature.

For example, to predict whether a customer will leave an e-commerce service, we might use:

Logistic Regression With Python

Scikit-learn provides Logistic Regression through LogisticRegression.

from sklearn.linear_model import LogisticRegression

X = [
    [1],
    [2],
    [3],
    [5],
    [6],
    [7]
]

y = [
    0,
    0,
    0,
    1,
    1,
    1
]

model = LogisticRegression()

model.fit(X, y)

prediction = model.predict([[6]])

print(prediction)

Here:

In this example, the classes are represented as:

Prediction vs Probability

Logistic Regression can provide both the predicted class and the probability of the classes.

prediction = model.predict([[6]])

probability = model.predict_proba([[6]])

print(prediction)
print(probability)

The difference is important.

Check Your Understanding