Backpropagation Step by Step

Step 1 — Start With Input

Every training example starts with input data.

To make the mathematics easy to understand, we will use a very small neural network with one input and one weight.

Input  = 2
Weight = 3
Target = 10

The target is the correct answer that the model is trying to predict.

Step 2 — Make a Prediction

The model uses its current weight to calculate a prediction.

Prediction = Input × Weight

Substitute our values:

Prediction = 2 × 3

Prediction = 6

So the model predicts 6, but the correct answer is 10.

Prediction = 6
Target     = 10

The model is wrong, so we need to measure how wrong it is.

Step 3 — Calculate the Loss

The loss function measures the difference between the prediction and the target.

For this simple example, we will use squared error:

Loss = (Target - Prediction)²

Substitute the values:

Loss = (10 - 6)²

Loss = 4²

Loss = 16

The loss is 16.

A lower loss generally means the prediction is closer to the target.

Step 4 — Start the Backward Pass

Now the important part begins.

The model knows its prediction was wrong, but it needs to know:

"How should I change my weight
to reduce this loss?"

This is where backpropagation is used.

Forward Pass
      ↓
Prediction
      ↓
Loss

        ↓
        ↓
        ↓

Backward Pass
      ↓
Calculate Gradients

Backpropagation works backward through the calculations that produced the loss.

Step 5 — Calculate the Gradient

A gradient tells us how the loss changes when a weight changes.

Gradient
=
How much does the Loss change
when the Weight changes?

For our simple model:

Prediction = Input × Weight

Loss = (Target - Prediction)²

Using the chain rule, the gradient of the loss with respect to the weight is:

dLoss / dWeight
=
2 × (Prediction - Target) × Input

Insert our values:

Prediction = 6
Target     = 10
Input      = 2

Gradient
= 2 × (6 - 10) × 2

= 2 × (-4) × 2

= -16

Therefore:

Gradient = -16

The negative gradient tells us that the weight needs to move upward to reduce the loss in this example.

Step 6 — Choose the Learning Rate

We should not change the weight by the entire gradient. That could make the update far too large.

Instead, we use a learning rate.

Learning Rate = 0.1

The learning rate controls the size of the weight update.

Large Learning Rate
→ Larger Updates


Small Learning Rate
→ Smaller Updates

Step 7 — Update the Weight

Now we can update the weight using gradient descent.

New Weight
=
Old Weight
-
Learning Rate × Gradient

Our values are:

Old Weight    = 3
Learning Rate = 0.1
Gradient      = -16

Calculate:

New Weight
= 3 - (0.1 × -16)

= 3 - (-1.6)

= 4.6

So the new weight becomes:

Old Weight = 3
New Weight = 4.6

Step 8 — Make a New Prediction

The weight has changed, so we run the forward pass again.

Input = 2
New Weight = 4.6

Calculate the prediction:

Prediction
= 2 × 4.6

= 9.2

Compare this with the original prediction:

Before Weight Update

Prediction = 6


After Weight Update

Prediction = 9.2


Target = 10

The prediction moved much closer to the target.

Step 9 — Calculate the New Loss

We should check whether the loss actually improved.

The new prediction is 9.2.

New Loss
= (Target - Prediction)²

= (10 - 9.2)²

= 0.8²

= 0.64

Compare the losses:

Before Update
Loss = 16


After Update
Loss = 0.64

The loss became much smaller.

That means this weight update moved the model in a better direction.

One Complete Backpropagation Step

Input = 2
Weight = 3
Target = 10

        ↓

1. Forward Pass

Prediction = 2 × 3
Prediction = 6

        ↓

2. Calculate Loss

Loss = (10 - 6)²
Loss = 16

        ↓

3. Backpropagation

Calculate Gradient

Gradient = -16

        ↓

4. Update Weight

Learning Rate = 0.1

New Weight
= 3 - (0.1 × -16)

New Weight = 4.6

        ↓

5. Forward Pass Again

Prediction = 2 × 4.6
Prediction = 9.2

        ↓

6. New Loss

Loss = (10 - 9.2)²
Loss = 0.64

Step 10 — Repeat the Process

Training does not stop after one update.

The model repeats the same process many times.

Forward Pass
      ↓
Calculate Loss
      ↓
Backpropagation
      ↓
Calculate Gradients
      ↓
Update Weights
      ↓
Forward Pass Again
      ↓
Calculate Loss Again
      ↓
Repeat...

Each update attempts to move the weights toward values that produce lower loss.

What Happens Over Many Updates?

Imagine that the model starts with a poor weight.

Iteration 1
Weight = 3
Prediction = 6
Loss = 16


Iteration 2
Weight = 4.6
Prediction = 9.2
Loss = 0.64

In a real training process, the model continues making smaller adjustments as it learns.

Training

High Loss
    ↓
Update Weights
    ↓
Lower Loss
    ↓
Update Weights
    ↓
Even Lower Loss
    ↓
Repeat

What Happens in a Real Neural Network?

The example above uses only one weight so that the mathematics is easy to follow.

A real neural network may have thousands or millions of weights.

Input Layer
      ↓
Hidden Layer
      ↓
Hidden Layer
      ↓
Output Layer
      ↓
Prediction
      ↓
Loss

        ↑
        │
 Backpropagation
        │
        ↑

Gradients for many weights
        ↓
Update many weights

Backpropagation efficiently calculates how each weight contributes to the final loss.

Where Does the Chain Rule Come In?

In a deep network, a weight may be several layers away from the final loss.

Weight
  ↓
Neuron
  ↓
Hidden Layer
  ↓
Another Hidden Layer
  ↓
Prediction
  ↓
Loss

Backpropagation works backward through these connected calculations.

The chain rule lets us combine the individual effects to determine how the original weight affected the final loss.

Weight
   ↓
Effect on next calculation
   ↓
Effect on next calculation
   ↓
Effect on prediction
   ↓
Effect on loss

        ↓

Gradient for the Weight

Real-Life Example

Imagine you are trying to reach the lowest point of a mountain.

Current Position
      ↓
Check the slope
      ↓
Find the downhill direction
      ↓
Take a step
      ↓
Check the slope again
      ↓
Take another step

The gradient is like the slope. It tells you the direction in which the loss increases.

Gradient descent moves in the opposite direction to try to reduce the loss.

Gradient
→ Direction of increasing loss


Gradient Descent
→ Move in the opposite direction


Goal
→ Lower Loss

The Complete Learning Cycle

              TRAINING
                 │
                 ▼
             Input Data
                 │
                 ▼
          Forward Propagation
                 │
                 ▼
             Prediction
                 │
                 ▼
           Calculate Loss
                 │
                 ▼
          Backpropagation
                 │
                 ▼
        Calculate Gradients
                 │
                 ▼
        Update the Weights
                 │
                 ▼
          Make Prediction
             Again
                 │
                 ▼
        Is Loss Lower?
                 │
                 ▼
              Repeat

Remember This

1. Forward Pass
   → Make a prediction

2. Loss
   → Measure the error

3. Backpropagation
   → Work backward through the network

4. Gradients
   → Find how weights affect the loss

5. Weight Update
   → Change the weights

6. Repeat
   → Keep learning

That is the basic backpropagation training loop. The network repeatedly makes predictions, measures its error, calculates gradients, and updates its weights.