What Is a Loss Function?

Why Do We Need a Loss Function?

Remember what happened during forward propagation. We gave the neural network some input and it produced a prediction.

Input
  ↓
Neural Network
  ↓
Prediction

But there is a problem:

How do we know whether the prediction is good or bad?

We compare the prediction with the actual answer.

Prediction
    ↓
Compare with
    ↓
Actual Answer
    ↓
Loss

A Simple Example

Suppose we are building a model that predicts whether a student will pass an exam.

The correct answer is:

Actual Answer = 1

Here, we can use:

1 = Pass
0 = Fail

Our model predicts:

Prediction = 0.9

The prediction is quite close to the correct answer:

Actual      = 1.0
Prediction  = 0.9

Therefore, we expect a relatively small loss.

What If the Prediction Is Bad?

Now suppose the actual answer is still:

Actual Answer = 1

But the model predicts:

Prediction = 0.1

Now the model is far away from the correct answer.

Actual      = 1.0
Prediction  = 0.1

Therefore, the loss should be much larger.

Compare Two Predictions

Suppose the actual answer is:

Actual = 1

We have two models:

Model A → 0.9
Model B → 0.2

Model A is much closer to the correct answer.

Actual = 1

Model A
Prediction = 0.9
       ↓
   Small Error


Model B
Prediction = 0.2
       ↓
   Large Error

A suitable loss function will therefore give Model A a smaller loss than Model B.

What Does the Loss Value Mean?

The loss function produces a numerical value.

Prediction
    ↓
Loss Function
    ↓
Loss Value

For example:

Loss = 0.05

or:

Loss = 2.50

The exact meaning of the number depends on which loss function we are using. So you should not assume that every loss function uses the same scale.

But within the same loss function and problem, a lower loss generally means the prediction is better.

Loss Is Not the Same as Accuracy

This is an important distinction.

Loss measures how far the model's predictions are from the target according to a chosen mathematical function.

Accuracy measures how many predictions were classified correctly.

Loss
↓
How wrong are the predictions?


Accuracy
↓
How many predictions are correct?

A model can have high accuracy while still having different loss values because loss can capture the confidence or magnitude of prediction errors.

Real-World Example — Spam Detection

Imagine a model that predicts whether an email is spam.

1 = Spam
0 = Not Spam

Suppose the email is actually spam:

Actual = 1

Model A predicts:

Prediction = 0.95

Model B predicts:

Prediction = 0.10

Model A is much closer to the correct target, so a suitable loss function should assign it a smaller loss.

Actual = 1

Model A → 0.95 → smaller loss
Model B → 0.10 → larger loss

How Does Loss Help the Neural Network Learn?

This is where loss becomes extremely important.

During training, the neural network repeatedly makes predictions and measures its errors.

Input
  ↓
Forward Propagation
  ↓
Prediction
  ↓
Calculate Loss
  ↓
Backpropagation
  ↓
Update Weights
  ↓
Repeat

The loss tells the training process how well the current model parameters are performing.

A Simple Training Example

Imagine the model starts with a poor prediction:

Actual = 1
Prediction = 0.20

After training updates the weights, the model might produce:

Prediction = 0.60

After more training:

Prediction = 0.85

Eventually:

Prediction = 0.95

As the predictions become better, the appropriate loss should generally decrease.

Prediction
    ↓
Better Prediction
    ↓
Lower Loss
    ↓
Better Model

Think of Loss Like a Score

Imagine a student taking a test.

The teacher gives feedback after every test. The loss function plays a similar role for the model: it gives the training process a numerical measure of error.

Good Prediction
      ↓
Low Loss


Bad Prediction
      ↓
High Loss

The training process tries to find model parameters that produce a lower loss.

Why Are There Different Loss Functions?

Different machine-learning problems need different ways of measuring error.

For example, predicting a house price is different from predicting whether an email is spam.

House Price
    ↓
Regression
    ↓
Mean Squared Error
    or other regression losses


Spam Detection
    ↓
Binary Classification
    ↓
Binary Cross-Entropy

Later in this lesson, we will study these loss functions individually.

The Basic Idea Behind a Loss Function

Every loss function takes information about the model's prediction and the correct target, then produces a numerical loss.

Loss = Loss Function(
    Actual Answer,
    Model Prediction
)

For example:

Actual = 1
Prediction = 0.9

        ↓

Loss Function

        ↓

Loss Value

The exact calculation depends on the selected loss function.

Where Loss Fits Into Deep Learning

Training Data
      ↓
Neural Network
      ↓
Forward Propagation
      ↓
Prediction
      ↓
Loss Function
      ↓
Loss
      ↓
Backpropagation
      ↓
Gradients
      ↓
Optimizer
      ↓
Updated Weights
      ↓
Repeat

This is the bigger picture of neural-network training. Loss is the measurement that tells us how well the current model is performing.