Calculating the Error

What Is Error?

Error is the difference between what the neural network predicted and what the correct answer actually was.

Actual Answer
      ↓
Compare
      ↑
Prediction
      ↓
Error

If the prediction is very close to the actual answer, the error is small.

If the prediction is far from the actual answer, the error is large.

A Simple Example

Imagine a neural network predicting the price of a house.

The actual price is:

Actual Price = $300,000

The neural network predicts:

Predicted Price = $280,000

The prediction is $20,000 lower than the actual price.

Actual     = $300,000
Prediction = $280,000

Difference = $20,000

The prediction is therefore wrong by $20,000.

Basic Error Formula

A simple way to calculate the error is:

Error = Actual - Prediction

Using our house-price example:

Actual     = 300,000
Prediction = 280,000

Error = 300,000 - 280,000

Error = 20,000

This tells us that the prediction was $20,000 below the actual value.

Classification Example

Error is not only used for numerical predictions such as house prices. Neural networks also make classification predictions.

Imagine a model predicting whether an email is spam.

Actual = 1
1 = Spam

The model predicts:

Prediction = 0.20

The model is saying there is only a 20% probability that the email is spam.

But the correct answer is 1.

Actual     = 1.00
Prediction = 0.20

Difference = 1.00 - 0.20

Difference = 0.80

The prediction is far from the correct answer.

What Does a Negative Error Mean?

Sometimes the prediction can be higher than the actual answer.

For example:

Actual     = 300
Prediction = 350

Error = 300 - 350

Error = -50

The negative sign tells us that the prediction was higher than the actual value.

Positive Error
→ Prediction was too low


Negative Error
→ Prediction was too high

The sign can provide useful information about the direction of the mistake. However, when measuring overall prediction quality, we often use loss functions that handle positive and negative errors appropriately.

Error vs Loss

These two terms are related, but they are not always exactly the same thing.

Error
 ↓
Difference between prediction
and actual answer


Loss
 ↓
A mathematical measure used
to quantify prediction error
for training the model

For a simple example, we might calculate:

Actual     = 10
Prediction = 8

Error = 10 - 8

Error = 2

A loss function can then transform the prediction error into a value that is useful for training.

For example, with squared error:

Error = 2

Squared Error = 2²

Squared Error = 4

So don't treat every occurrence of "error" and "loss" as interchangeable. The exact loss calculation depends on the chosen loss function.

Why Do We Need to Calculate the Error?

A neural network needs feedback to learn.

If the network makes a prediction but never measures the mistake, it has no useful signal telling it how well it performed.

Input
  ↓
Prediction
  ↓
Calculate Error
  ↓
Measure the Mistake
  ↓
Learn From the Mistake

The error and loss provide the signal that is later used during backpropagation to calculate gradients.

From Prediction to Error

Let's connect this with the forward pass from the previous topic.

Input
  ↓
Forward Pass
  ↓
Prediction
  ↓
Compare with Actual Answer
  ↓
Calculate Error / Loss

This is the point where the network discovers how well or poorly its prediction performed.

Error Leads to Backpropagation

Calculating the error is not the end of the learning process. It is the beginning of the correction process.

Prediction
     ↓
Calculate Error / Loss
     ↓
Backpropagation
     ↓
Calculate Gradients
     ↓
Update Weights

Backpropagation uses the loss to determine how changes in the network's weights affect that loss.

Complete Example

Let's follow a very simple prediction from beginning to error calculation.

Suppose a neural network predicts whether a customer will buy a product.

Actual Answer = 1

1 = Customer bought the product

The network produces:

Prediction = 0.30

Now compare the two values:

Actual     = 1.00
Prediction = 0.30

Difference = 1.00 - 0.30

Difference = 0.70

The prediction is significantly different from the actual answer.

A suitable loss function can then convert this prediction error into the loss value used for training.

Prediction
     ↓
Compare with Actual
     ↓
Error
     ↓
Loss
     ↓
Backpropagation
     ↓
Gradients
     ↓
Weight Update

What If There Are Many Predictions?

A neural network normally processes many training examples, not just one.

Example 1
Actual = 1
Prediction = 0.9


Example 2
Actual = 0
Prediction = 0.2


Example 3
Actual = 1
Prediction = 0.4

Each prediction has its own error or loss contribution. A loss function can combine these into an overall loss value for the training step.

Example 1 → Loss
Example 2 → Loss
Example 3 → Loss
        ↓
Overall Loss

The overall loss provides a training signal for backpropagation.

What Happens When the Model Learns?

During training, the goal is generally to reduce the loss.

Imagine the model starts with:

Prediction = 0.20
Loss       = 0.80

After training updates the weights:

Prediction = 0.60
Loss       = 0.40

After more training:

Prediction = 0.90
Loss       = 0.10

This is the general idea behind learning: the model adjusts its parameters so that its predictions become better according to the chosen loss function.

Real-Life Example

Imagine you are trying to hit a target with a ball.

Target
  ●

Your Throw
       ●

       ↓

Measure the distance
from the target

If your throw lands far from the target, the error is large.

If it lands close to the target, the error is small.

Large Error
     ↓
Need a bigger correction


Small Error
     ↓
Need a smaller correction

A neural network works with the same basic idea, but it measures errors mathematically and uses gradients to determine how its parameters should change.

Remember This

Prediction
     ↓
Compare with Actual Answer
     ↓
Calculate Error / Loss
     ↓
Backpropagation
     ↓
Calculate Gradients
     ↓
Update Weights
     ↓
Make a Better Prediction

The important point is that a neural network cannot improve simply because it knows its prediction. It needs a measurable training signal that tells it how far the prediction is from the desired result.