Adding Layers

What Is a Layer?

A layer is a group of neurons that processes input data. Neural networks are built by connecting multiple layers together.

Input
  ↓
Layer 1
  ↓
Layer 2
  ↓
Layer 3
  ↓
Output

Each layer transforms the data before sending it to the next layer.

Think of each layer as a processing stage.

Raw Input
   ↓
First Processing
   ↓
Second Processing
   ↓
Final Processing
   ↓
Prediction

Adding a Dense Layer

One of the most commonly used layers in basic neural networks is the Dense layer.

We create one using:

tf.keras.layers.Dense(4)

The number 4 means the layer contains four neurons.

Dense(4)

Neuron 1
Neuron 2
Neuron 3
Neuron 4

Every neuron in a Dense layer is connected to the outputs of the previous layer.

Adding a Layer to a Model

We can add a layer inside a Sequential model.

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(4)
])

The structure is:

2 Input Features
       ↓
4 Neurons

The input contains two values, and those values are sent to all four neurons.

Adding an Activation Function

A Dense layer can also use an activation function.

tf.keras.layers.Dense(
    4,
    activation="relu"
)

This means:

4
↓
Number of neurons


relu
↓
Activation function

The neurons first calculate their weighted sum and then apply ReLU.

Input
  ↓
Weighted Sum
  ↓
ReLU
  ↓
Neuron Output

Adding Multiple Layers

We can add more than one layer.

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        3,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

The network now looks like this:

Input
2 features
   ↓
Dense
4 neurons
ReLU
   ↓
Dense
3 neurons
ReLU
   ↓
Dense
1 neuron
Sigmoid
   ↓
Output

Data flows through the layers in the order they are defined.

How Data Moves Between Layers

Suppose the input is:

[5, 3]

The first layer receives both values.

[5, 3]
  ↓
4-neuron layer
  ↓
[output1, output2, output3, output4]

The second layer receives those four outputs.

[output1, output2, output3, output4]
                 ↓
            3-neuron layer
                 ↓
        [output1, output2, output3]

Finally, the output layer receives those three values.

[output1, output2, output3]
             ↓
       1-neuron layer
             ↓
          0.92

This is how information moves forward through a neural network.

Example 1 — Simple Network

Let's create a small network for binary classification.

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

It contains:

2 input features
        ↓
4 hidden neurons
        ↓
1 output neuron

This is enough for a simple binary classification problem.

Example 2 — Deeper Network

We can add another hidden layer.

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(5,)),

    tf.keras.layers.Dense(
        16,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        8,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

The structure is:

5 Inputs
   ↓
16 neurons
   ↓
8 neurons
   ↓
4 neurons
   ↓
1 output

Adding layers allows the network to build representations in stages.

Why Use Multiple Layers?

A neural network can learn increasingly complex representations as information moves through layers.

Input Data
    ↓
Simple Patterns
    ↓
More Complex Patterns
    ↓
Higher-Level Patterns
    ↓
Prediction

For example, in image recognition, earlier layers may learn simple visual patterns while deeper layers can combine those patterns into more meaningful structures.

But don't make the mistake of assuming that more layers always means a better model. A larger network can overfit, train more slowly, and use more memory.

Input Layer

The input layer defines the shape of the data entering the network.

tf.keras.layers.Input(shape=(3,))

This means each input example contains three features.

[feature1, feature2, feature3]

For example:

[25, 50000, 1]

could represent three numerical features such as age, income, and membership status.

Hidden Layers

Layers between the input and output are called hidden layers.

Input
  ↓
Hidden Layer
  ↓
Hidden Layer
  ↓
Output

A hidden layer might contain:

Dense(8, activation="relu")

This means the hidden layer has eight neurons using ReLU.

Output Layer

The output layer produces the final result.

Its size and activation function depend on the problem.

Binary Classification

Dense(1, activation="sigmoid")

For multiple classes, the output layer is commonly different:

Multi-Class Classification

Dense(number_of_classes, activation="softmax")

For example, if there are three classes:

Dense(3, activation="softmax")

produces three output probabilities.

Adding Layers With add()

There are two common ways to create a Sequential model.

The first is to provide all layers inside a list.

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),
    tf.keras.layers.Dense(4, activation="relu"),
    tf.keras.layers.Dense(1, activation="sigmoid")
])

Another approach is to create the model first and add layers individually.

model = tf.keras.Sequential()

model.add(
    tf.keras.layers.Input(shape=(2,))
)

model.add(
    tf.keras.layers.Dense(
        4,
        activation="relu"
    )
)

model.add(
    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
)

Both approaches create the same basic sequential structure.

Layer Order Matters

In a Sequential model, layers are processed in the order you define them.

Dense(16)
   ↓
Dense(8)
   ↓
Dense(1)

is different from:

Dense(8)
   ↓
Dense(16)
   ↓
Dense(1)

The architecture changes because the number of neurons in each stage changes.

Therefore, don't treat the layer order as cosmetic. It is part of the model design.

Complete Example

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(4,)),

    tf.keras.layers.Dense(
        16,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        8,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

model.summary()

Read the model from top to bottom:

4 input features
        ↓
16 neurons + ReLU
        ↓
8 neurons + ReLU
        ↓
4 neurons + ReLU
        ↓
1 neuron + Sigmoid
        ↓
Final prediction

What Does Each Number Mean?

Consider:

tf.keras.layers.Dense(
    16,
    activation="relu"
)

The number 16 is the number of neurons.

Now consider:

tf.keras.layers.Input(shape=(4,))

The 4 means the input contains four features.

Finally:

tf.keras.layers.Dense(
    1,
    activation="sigmoid"
)

The 1 means there is one output neuron.

Input shape
→ Number of input features


Dense(16)
→ 16 neurons


Dense(1)
→ 1 neuron

Don't Confuse Neurons With Layers

This is a common beginner mistake.

Dense(8)

means:

1 layer
8 neurons

It does not mean eight layers.

For example:

Dense(16)
Dense(8)
Dense(4)

means:

3 layers

Layer 1 → 16 neurons
Layer 2 → 8 neurons
Layer 3 → 4 neurons

Understand the Python Code

model = tf.keras.Sequential([

Creates a Sequential model. Layers will process data in order.

tf.keras.layers.Input(shape=(4,))

Defines four input features.

tf.keras.layers.Dense(
    16,
    activation="relu"
)

Adds a dense layer with 16 neurons and ReLU activation.

tf.keras.layers.Dense(
    8,
    activation="relu"
)

Adds another dense layer with 8 neurons.

tf.keras.layers.Dense(
    1,
    activation="sigmoid"
)

Adds the final output layer with one sigmoid neuron.

model.summary()

Displays the structure and number of trainable parameters in the model.

The Main Idea

Adding layers means defining how information
flows through the neural network.

Input
 ↓
Layer
 ↓
Layer
 ↓
Layer
 ↓
Output

Every layer transforms the information it receives and passes the result forward.

In Keras, adding layers is easy. The difficult part is choosing an architecture that actually fits the problem. More layers and more neurons are not automatically better.

Remember This

Input Layer
→ Defines input features


Dense Layer
→ Contains fully connected neurons


Dense(8)
→ One layer containing 8 neurons


activation="relu"
→ Applies ReLU


activation="sigmoid"
→ Produces a value between 0 and 1


Sequential
→ Processes layers in order


Multiple Layers
→ Allow the network to learn increasingly complex patterns

The basic pattern is:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(features,)),

    tf.keras.layers.Dense(
        neurons,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        output_neurons,
        activation="sigmoid"
    )
])

In the next topic, we will learn about Compiling the Model — where we tell Keras which optimizer, loss function, and metrics should be used during training.