Forward Pass vs Backward Pass

What Is a Forward Pass?

A forward pass is the process of sending input data through the neural network from the first layer to the final layer.

Each layer performs calculations and passes its result to the next layer.

Input
  ↓
Hidden Layer 1
  ↓
Hidden Layer 2
  ↓
Output Layer
  ↓
Prediction

The information always moves forward during this process. That is why it is called a forward pass.

Simple Forward Pass Example

Imagine a neural network that predicts whether a student will pass an exam based on the number of hours they studied.

Input:

Hours Studied = 5

The input enters the neural network.

Hours Studied
      ↓
Input Layer
      ↓
Hidden Layer
      ↓
Output Layer
      ↓
Prediction

Suppose the network produces:

Prediction = 0.80

The model is saying there is an 80% probability that the student will pass.

What Happens During the Forward Pass?

Each neuron receives values, multiplies them by weights, adds a bias, and usually applies an activation function.

Input
  ↓
Multiply by weights
  ↓
Add bias
  ↓
Activation function
  ↓
Next layer

This process continues until the network reaches the output layer.

Input
  ↓
Calculate Layer 1
  ↓
Calculate Layer 2
  ↓
Calculate Layer 3
  ↓
Calculate Output
  ↓
Prediction

What Happens After the Forward Pass?

After the network produces a prediction, we compare that prediction with the correct answer.

Prediction = 0.80
Actual     = 1

        ↓

Calculate Loss

The loss tells us how far the prediction is from the expected answer.

Forward Pass
      ↓
Prediction
      ↓
Loss

At this point, the network knows how wrong its prediction was. It now needs to determine how the weights should change.

What Is a Backward Pass?

The backward pass is the process of sending gradient information backward through the neural network.

It starts at the output layer and works backward toward the earlier layers.

Loss
  ↓
Output Layer
  ↓
Hidden Layer 2
  ↓
Hidden Layer 1
  ↓
Input Side

This is why it is called a backward pass.

Why Does the Network Go Backward?

The final prediction depends on weights throughout the network.

If the prediction is wrong, the network needs to determine how those weights contributed to the error.

Wrong Prediction
       ↓
Calculate Loss
       ↓
Backward Pass
       ↓
Calculate Gradients
       ↓
Determine Weight Changes

The backward pass gives the network the information it needs to learn from its mistake.

What Happens During the Backward Pass?

The backward pass calculates gradients for the network's parameters.

A gradient tells us how changing a particular weight affects the loss.

Loss
 ↓
Gradient for Weight 3
 ↓
Gradient for Weight 2
 ↓
Gradient for Weight 1

These gradients are then used by an optimizer to update the weights.

A Simple Example

Imagine a very small network:

Input
  ↓
Weight 1
  ↓
Hidden Layer
  ↓
Weight 2
  ↓
Output

Suppose the network receives:

Input = 5

The forward pass calculates a prediction:

Input
  ↓
Weight 1
  ↓
Hidden Layer
  ↓
Weight 2
  ↓
Prediction = 0.30

Suppose the correct answer is:

Actual = 1

The network calculates the loss.

Prediction = 0.30
Actual     = 1

       ↓

Loss

Now the backward pass begins.

Loss
 ↓
Gradient for Weight 2
 ↓
Gradient for Weight 1

The optimizer can then use those gradients to update the weights.

Two Different Directions

FORWARD PASS

Input
  ↓
Hidden Layer
  ↓
Output
  ↓
Prediction
  ↓
Loss

BACKWARD PASS

Loss
  ↓
Output Layer
  ↓
Hidden Layer
  ↓
Gradients
  ↓
Earlier Layers

The forward pass produces the prediction. The backward pass uses the error from that prediction to calculate gradients.

Real-Life Example

Think about learning to throw a basketball into a basket.

First, you throw the ball.

You decide:

Angle
Force
Direction

       ↓

Throw the ball
       ↓
Ball reaches the basket
       ↓
Result

This is similar to a forward pass. You start with inputs and produce an outcome.

Now suppose you miss the basket.

Missed Basket
     ↓
Analyze the mistake
     ↓
Too much force?
Wrong angle?
Wrong direction?
     ↓
Adjust technique

This is similar to the idea behind the backward pass: use the error to determine what needs to change.

Forward and Backward Pass Together

During training, both processes work together.

Input Data
    ↓
FORWARD PASS
    ↓
Prediction
    ↓
Calculate Loss
    ↓
BACKWARD PASS
    ↓
Calculate Gradients
    ↓
Optimizer Updates Weights
    ↓
Repeat

The network repeats this process many times while learning from the training data.

Forward Pass vs Backward Pass

FORWARD PASS

Purpose:
Make a prediction

Direction:
Input → Output

Uses:
Inputs, weights, biases,
activation functions

Result:
Prediction

BACKWARD PASS

Purpose:
Learn from the error

Direction:
Output → Earlier Layers

Uses:
Loss and chain rule

Result:
Gradients

Easy Way to Remember

FORWARD
"What will I predict?"

        ↓

PREDICTION
        ↓

LOSS
"How wrong was I?"

        ↓

BACKWARD
"How should I change?"

        ↓

GRADIENTS
        ↓

OPTIMIZER
        ↓

UPDATED WEIGHTS

Remember This

Forward Pass
Input → Output
Makes a prediction


Backward Pass
Output → Earlier Layers
Calculates gradients


Optimizer
Uses gradients
Updates weights

So the complete learning process is:

Input
 ↓
Forward Pass
 ↓
Prediction
 ↓
Loss
 ↓
Backward Pass
 ↓
Gradients
 ↓
Optimizer
 ↓
Updated Weights
 ↓
Forward Pass Again