Hidden Layer to Output

From Hidden Layer to Output

In the previous topic, our hidden layer produced:

[3.0, 2.0, 0.9]

These values now become the inputs to the output layer.

The Output Neuron

The output neuron works in a similar way to a hidden neuron.

It takes the hidden-layer outputs, multiplies them by weights, adds a bias, and then applies an activation function.

z =
(hidden1 × weight1)
+
(hidden2 × weight2)
+
(hidden3 × weight3)
+
bias

Then:

output = activation(z)

Let's Use Our Previous Example

Our hidden layer produced three values:

h1 = 3.0
h2 = 2.0
h3 = 0.9

Suppose the output neuron has these weights:

w1 = 0.4
w2 = 0.3
w3 = 0.2

And the output neuron has a bias:

bias = 0.1

Step 1 — Multiply Hidden Outputs by Weights

First, multiply each hidden-layer output by its corresponding output-layer weight.

h1 × w1
= 3.0 × 0.4
= 1.2

h2 × w2
= 2.0 × 0.3
= 0.6

h3 × w3
= 0.9 × 0.2
= 0.18

Step 2 — Add the Weighted Values

Now add the three weighted values:

1.2 + 0.6 + 0.18

= 1.98

Step 3 — Add the Bias

Now add the output neuron's bias:

z = 1.98 + 0.1

z = 2.08

So the output neuron's raw value is:

z = 2.08

Step 4 — Apply the Activation Function

The activation function depends on what the output needs to represent.

Suppose this is a binary classification problem and we use Sigmoid.

Sigmoid(2.08) ≈ 0.889

So the final output is approximately:

0.889

Complete Calculation

Everything we just calculated can be written as one equation:

z = (3.0 × 0.4)
  + (2.0 × 0.3)
  + (0.9 × 0.2)
  + 0.1

z = 1.2 + 0.6 + 0.18 + 0.1

z = 2.08

Apply Sigmoid:

Sigmoid(2.08) ≈ 0.889

What Does 0.889 Mean?

If the output neuron is designed to represent the probability of the positive class, then:

Output = 0.889

can be interpreted as approximately an 88.9% predicted probability for that class, assuming the model is trained for that purpose.

If the classification rule uses 0.5 as the decision threshold:

0.889 > 0.5

Prediction = Positive Class

The threshold is a separate decision rule; the neural network itself produces the output value.

Real-World Example — Student Prediction

Suppose our neural network predicts whether a student will pass.

The original inputs were:

Study Hours
Attendance

The hidden layer processed these inputs and produced:

[3.0, 2.0, 0.9]

The output layer then processes those values:

[3.0, 2.0, 0.9]
        ↓
Output Neuron
        ↓
0.889

The model can therefore produce a high probability for the positive class.

Why Does the Output Layer Use Hidden Outputs?

The output layer does not normally work directly with the original raw inputs.

The hidden layers have already transformed the original data into representations that the later layers can use.

Raw Input
   ↓
Hidden Layer 1
   ↓
Hidden Layer 2
   ↓
Useful Representations
   ↓
Output Layer
   ↓
Prediction

This is one reason deep neural networks can learn complicated relationships.

One Output Neuron or Multiple?

The number of output neurons depends on the problem.

Binary Classification

For a binary classification problem, a common design is one output neuron with a Sigmoid activation.

Hidden Layer
     ↓
Output Neuron
     ↓
Sigmoid
     ↓
0.889

Multi-Class Classification

If the model needs to choose between several classes, the output layer can contain multiple outputs.

Hidden Layer
     ↓
Output Layer
     ↓
┌─────────────┐
│ Cat = 0.80  │
│ Dog = 0.15  │
│ Bird = 0.05 │
└─────────────┘

A common approach for mutually exclusive multi-class classification is to use Softmax in the output layer.

What About Regression?

The output activation also depends on the type of prediction.

For many regression problems, the output layer uses a linear activation, meaning the raw output is used directly.

Hidden Layer
     ↓
Output Neuron
     ↓
Linear Output
     ↓
Predicted Price

For example:

Predicted House Price
= $250,000

You would not normally use Sigmoid here because a house price is not naturally restricted to a 0-to-1 range.

Hidden Layer to Output With Python

We can implement the example directly in Python:

import math


def sigmoid(x):
    return 1 / (1 + math.exp(-x))


# Hidden layer output
h1 = 3.0
h2 = 2.0
h3 = 0.9


# Output layer weights
w1 = 0.4
w2 = 0.3
w3 = 0.2


# Output bias
bias = 0.1


# Weighted sum
z = (
    (h1 * w1)
    + (h2 * w2)
    + (h3 * w3)
    + bias
)


# Activation
prediction = sigmoid(z)


print("Raw output:", z)
print("Prediction:", prediction)

Output:

Raw output: 2.08
Prediction: 0.889...

Where We Are in Forward Propagation

We can now connect the previous topics together.

Input
  ↓
Input → Hidden Layer
  ↓
Hidden Layer Calculation
  ↓
Hidden Layer Output
  ↓
Hidden Layer → Output
  ↓
Output Layer Calculation
  ↓
Activation Function
  ↓
Prediction

We have now reached the final stage of the forward pass.

The Big Picture

Input Data
    ↓
Hidden Layer
    ↓
[3.0, 2.0, 0.9]
    ↓
Output Layer
    ↓
Weighted Sum
    ↓
Bias
    ↓
Activation Function
    ↓
0.889
    ↓
Prediction

This is how information moves from the hidden layer to the final output during forward propagation.