Hidden Layer Calculation

What Is a Hidden Layer?

A hidden layer is a layer between the input layer and the output layer.

Input Layer
     ↓
Hidden Layer
     ↓
Output Layer

A hidden layer can contain many neurons.

             ┌── Neuron 1
             │
Input ───────┼── Neuron 2
             │
             └── Neuron 3

Each neuron performs its own calculation.

All Neurons Receive the Same Inputs

Suppose our input contains two features:

x1 = 2
x2 = 3

If the hidden layer has three neurons, all three neurons receive these same two inputs.

                 ┌── Neuron 1
                 │
x1 = 2 ──────────┼── Neuron 2
                 │
x2 = 3 ──────────└── Neuron 3

But the calculations will be different because the neurons have different weights and biases.

The Calculation of One Neuron

A neuron first calculates a weighted sum:

z = (x1 × w1) + (x2 × w2) + b

Then an activation function is applied:

output = activation(z)

So the complete process is:

Example — Neuron 1

Let's use:

Inputs:
x1 = 2
x2 = 3

Weights:
w1 = 0.8
w2 = 0.3

Bias:
b = 0.5

Calculate the weighted sum:

z = (2 × 0.8) + (3 × 0.3) + 0.5

z = 1.6 + 0.9 + 0.5

z = 3.0

Now apply ReLU:

ReLU(3.0) = 3.0

So Neuron 1 produces:

Neuron 1 Output = 3.0

Example — Neuron 2

Neuron 2 receives the same inputs:

x1 = 2
x2 = 3

But it has different weights and bias:

w1 = 0.2
w2 = 0.7
b  = -0.5

Calculate:

z = (2 × 0.2) + (3 × 0.7) - 0.5

z = 0.4 + 2.1 - 0.5

z = 2.0

Apply ReLU:

ReLU(2.0) = 2.0

Therefore:

Neuron 2 Output = 2.0

Example — Neuron 3

Neuron 3 again receives the same inputs:

x1 = 2
x2 = 3

But it has its own parameters:

w1 = -0.4
w2 = 0.5
b  = 0.2

Calculate:

z = (2 × -0.4) + (3 × 0.5) + 0.2

z = -0.8 + 1.5 + 0.2

z = 0.9

Apply ReLU:

ReLU(0.9) = 0.9

Therefore:

Neuron 3 Output = 0.9

Complete Hidden Layer Calculation

We now have three neurons:

Therefore, the hidden layer produces:

[3.0, 2.0, 0.9]

This entire collection of values becomes the output of the hidden layer.

Why Have Multiple Neurons?

This is one of the most important ideas in neural networks.

Different neurons can learn different patterns from the same input data.

For example, in an image-recognition network, different neurons might respond to different types of patterns, such as edges, shapes, or textures.

Same Input
     │
     ├── Neuron 1 → learns Pattern A
     │
     ├── Neuron 2 → learns Pattern B
     │
     └── Neuron 3 → learns Pattern C

The network combines these different learned representations as information moves through deeper layers.

Real-World Example — Student Prediction

Imagine that our inputs are:

x1 = Study Hours
x2 = Attendance

Suppose the actual values are:

Study Hours = 5
Attendance = 90

One hidden neuron might learn a pattern related mostly to study time.

Another neuron might learn a pattern related more to attendance.

Another neuron might learn a combination of both.

Inputs
   │
   ├── Neuron 1 → Study-related pattern
   │
   ├── Neuron 2 → Attendance-related pattern
   │
   └── Neuron 3 → Combined pattern

These outputs can then be passed to the next layer.

What Happens to the Hidden Layer Output?

The hidden layer's outputs become inputs to the next layer.

Input Layer
     ↓
Hidden Layer
     ↓
[3.0, 2.0, 0.9]
     ↓
Next Layer

The next layer can then perform another weighted calculation using these values.

This is how information moves forward through a deep neural network.

The Formula for a Hidden Neuron

For a neuron with two inputs:

z = (x1 × w1) + (x2 × w2) + b

Then:

output = activation(z)

For many inputs:

z = x1w1 + x2w2 + x3w3 + ... + xnw_n + b

The same basic calculation is repeated for every neuron in the hidden layer.

Hidden Layer Calculation With Python

We can implement our three-neuron example directly in Python:

def relu(x):
    return max(0, x)


x1 = 2
x2 = 3


# Neuron 1
z1 = (x1 * 0.8) + (x2 * 0.3) + 0.5
output1 = relu(z1)


# Neuron 2
z2 = (x1 * 0.2) + (x2 * 0.7) - 0.5
output2 = relu(z2)


# Neuron 3
z3 = (x1 * -0.4) + (x2 * 0.5) + 0.2
output3 = relu(z3)


hidden_output = [
    output1,
    output2,
    output3
]


print(hidden_output)

Output:

[3.0, 2.0, 0.9]

What Is the Python Code Doing?

First, we create the ReLU function:

def relu(x):
    return max(0, x)

Then we define our inputs:

x1 = 2
x2 = 3

For each neuron, we calculate its own weighted sum and bias.

z1 = (x1 * 0.8) + (x2 * 0.3) + 0.5

Then we apply ReLU:

output1 = relu(z1)

We repeat the process for the other neurons.

Finally, we collect the neuron outputs:

hidden_output = [
    output1,
    output2,
    output3
]

That list represents the output of our hidden layer.

The Big Picture

Input
│
├── x1 = 2
└── x2 = 3
        │
        ▼
   Hidden Layer
        │
        ├── Neuron 1
        │     ↓
        │    3.0
        │
        ├── Neuron 2
        │     ↓
        │    2.0
        │
        └── Neuron 3
              ↓
             0.9
        │
        ▼
Hidden Layer Output

[3.0, 2.0, 0.9]

This output can now be passed to the next layer.