Forward Propagation With Python

What We Will Build

We will create a very small neural network:

2 Inputs
   ↓
2 Hidden Neurons
   ↓
1 Output Neuron
   ↓
Prediction

We will use the same example from the previous topic:

Input:

x1 = 5
x2 = 8

Step 1 — Import Python's Math Module

We need the exponential function to calculate the Sigmoid activation.

import math

Python's math module contains mathematical functions that we can use in our program.

Step 2 — Create the ReLU Function

Our hidden layer uses ReLU.

ReLU is very simple:

ReLU(x) = max(0, x)

In Python:

def relu(x):
    return max(0, x)

For example:

relu(5)
→ 5

relu(-3)
→ 0

Step 3 — Create the Sigmoid Function

Our output layer uses Sigmoid because this example is a binary classification problem.

The Sigmoid formula is:

Sigmoid(x) = 1 / (1 + e⁻ˣ)

In Python:

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

For example:

sigmoid(0)
→ 0.5

sigmoid(3.68)
→ approximately 0.976

Step 4 — Define the Input

Our neural network receives two input values:

x1 = 5
x2 = 8

Think of them as:

x1 = Study Hours
x2 = Attendance

So the input going into our network is:

[5, 8]

Step 5 — Calculate Hidden Neuron 1

The first hidden neuron has:

weight 1 = 0.4
weight 2 = 0.2
bias = 0.5

The mathematical calculation is:

z1 = (x1 × 0.4) + (x2 × 0.2) + 0.5

In Python:

z1 = (x1 * 0.4) + (x2 * 0.2) + 0.5

With our values:

z1 = (5 * 0.4) + (8 * 0.2) + 0.5

z1 = 4.1

Now apply ReLU:

h1 = relu(z1)

Therefore:

h1 = 4.1

Step 6 — Calculate Hidden Neuron 2

The second hidden neuron has different weights and bias:

weight 1 = 0.1
weight 2 = 0.5
bias = -0.2

In Python:

z2 = (x1 * 0.1) + (x2 * 0.5) - 0.2

Calculate:

z2 = (5 * 0.1) + (8 * 0.5) - 0.2

z2 = 4.3

Apply ReLU:

h2 = relu(z2)

Therefore:

h2 = 4.3

Step 7 — Get the Hidden Layer Output

We now have two hidden-neuron outputs:

h1 = 4.1
h2 = 4.3

Together:

hidden_output = [h1, h2]

print(hidden_output)

Output:

[4.1, 4.3]

Step 8 — Calculate the Output Neuron

The output neuron receives the hidden-layer values:

h1 = 4.1
h2 = 4.3

It uses:

weight 1 = 0.6
weight 2 = 0.4
bias = -0.5

In Python:

z_output = (
    (h1 * 0.6)
    + (h2 * 0.4)
    - 0.5
)

The result is:

z_output = 3.68

Step 9 — Apply Sigmoid

Now we pass the output neuron's value through Sigmoid:

prediction = sigmoid(z_output)

Since:

z_output = 3.68

the result is approximately:

prediction ≈ 0.976

Step 10 — Convert the Output Into a Prediction

Our model produces:

prediction = 0.976

Suppose we use 0.5 as the classification threshold:

if prediction >= 0.5:
    result = "Pass"
else:
    result = "Fail"

Because:

0.976 >= 0.5

The final result is:

Pass

Complete Python Code

Now let's combine everything into one program:

import math


def relu(x):
    return max(0, x)


def sigmoid(x):
    return 1 / (1 + math.exp(-x))


# -------------------------
# Input
# -------------------------

x1 = 5
x2 = 8


# -------------------------
# Hidden Neuron 1
# -------------------------

z1 = (x1 * 0.4) + (x2 * 0.2) + 0.5
h1 = relu(z1)


# -------------------------
# Hidden Neuron 2
# -------------------------

z2 = (x1 * 0.1) + (x2 * 0.5) - 0.2
h2 = relu(z2)


# -------------------------
# Hidden Layer Output
# -------------------------

hidden_output = [h1, h2]


# -------------------------
# Output Neuron
# -------------------------

z_output = (
    (h1 * 0.6)
    + (h2 * 0.4)
    - 0.5
)


# -------------------------
# Output Activation
# -------------------------

prediction = sigmoid(z_output)


# -------------------------
# Final Prediction
# -------------------------

if prediction >= 0.5:
    result = "Pass"
else:
    result = "Fail"


print("Hidden Output:", hidden_output)
print("Raw Output:", z_output)
print("Probability:", prediction)
print("Prediction:", result)

Expected Output

Hidden Output: [4.1, 4.3]
Raw Output: 3.68
Probability: 0.9759...
Prediction: Pass

Your exact decimal output may contain more digits because Python calculates the Sigmoid value more precisely.

Understand the Code Flow

Don't try to memorize the entire program. Understand what each part is doing.

Input
 ↓
x1, x2
 ↓
Hidden Neuron 1
 ↓
h1
 ↓
Hidden Neuron 2
 ↓
h2
 ↓
Hidden Output
 ↓
Output Neuron
 ↓
z_output
 ↓
Sigmoid
 ↓
prediction
 ↓
Pass / Fail

Why Did We Create Functions?

We created:

def relu(x):
    return max(0, x)


def sigmoid(x):
    return 1 / (1 + math.exp(-x))

Instead of writing the mathematical formula every time, we can simply call:

relu(z1)

sigmoid(z_output)

This makes the program easier to read and reuse.

Why Does Every Neuron Have Different Weights?

Notice that our two hidden neurons use different weights:

Neuron 1:
0.4, 0.2

Neuron 2:
0.1, 0.5

This allows different neurons to respond differently to the same input.

During real neural-network training, these weights are learned from data rather than manually chosen like in this small teaching example.

Another Simple Example

Suppose the input changes:

x1 = 2
x2 = 3

The same network can process these new values:

Input
[2, 3]
   ↓
Hidden Layer
   ↓
New Hidden Outputs
   ↓
Output Layer
   ↓
New Prediction

The code does not need to change. Only the input values change.

Real-World Example

Imagine a neural network predicting whether an email is spam.

Input Features

Number of links
Number of suspicious words
Sender reputation
Email length

        ↓

Neural Network

        ↓

Output

0.92

        ↓

Prediction

Spam

The same basic forward-propagation process happens, although a real model may contain thousands or millions of parameters and many more neurons.

The Complete Process

1. Receive input
        ↓
2. Calculate hidden neurons
        ↓
3. Apply activation functions
        ↓
4. Get hidden-layer output
        ↓
5. Calculate output neuron
        ↓
6. Apply output activation
        ↓
7. Get model output
        ↓
8. Interpret the output
        ↓
9. Make prediction

Does This Code Train the Model?

No.

This code only performs a forward pass using fixed weights and biases.

Fixed Weights
      ↓
Forward Pass
      ↓
Prediction

There is no loss calculation, gradient calculation, or weight update in this code.

Training requires additional steps:

Forward Pass
      ↓
Calculate Loss
      ↓
Backpropagation
      ↓
Update Weights
      ↓
Repeat