Compiling the Model

What Does Compile Mean?

When we create a neural network, we define its structure.

Input
  ↓
Hidden Layer
  ↓
Output Layer

But the model does not yet know exactly how it should improve its predictions.

We use compile() to configure the learning process.

Create Model
     ↓
Compile Model
     ↓
Train Model
     ↓
Make Predictions

So compiling happens before training.

Basic Syntax

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)

There are three important things here:

optimizer
    ↓
How the weights are updated

loss
    ↓
How prediction error is measured

metrics
    ↓
How we monitor model performance

1. Optimizer

The optimizer controls how the neural network updates its weights after calculating gradients.

optimizer="adam"

Adam is one of the most commonly used optimizers for neural networks.

During training, the general process is:

Prediction
    ↓
Calculate Loss
    ↓
Calculate Gradients
    ↓
Optimizer
    ↓
Update Weights

The optimizer decides how those weight updates should be performed.

For example:

model.compile(
    optimizer="adam",
    loss="binary_crossentropy"
)

This tells Keras to use the Adam optimization algorithm.

2. Loss Function

The loss function measures how wrong the model's predictions are.

loss="binary_crossentropy"

For binary classification, binary cross-entropy is commonly used.

For example, suppose the correct answer is:

Actual = 1

The model predicts:

Prediction = 0.90

This is a good prediction, so the loss should be relatively small.

But if the model predicts:

Prediction = 0.10

the prediction is very wrong, so the loss will be much larger.

Good Prediction
      ↓
Lower Loss


Bad Prediction
      ↓
Higher Loss

The optimizer then uses this error information to improve the model.

3. Metrics

Metrics are values we use to monitor how well the model is performing.

metrics=["accuracy"]

This tells Keras to calculate accuracy while training.

For example, if the model correctly predicts 90 out of 100 examples:

Accuracy = 90%

Metrics help us understand model performance, but they are different from the loss function.

Complete Example

First, create a neural network:

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

Then compile it:

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)

Now the model is configured and ready for training.

Create
  ↓
Compile
  ↓
Ready for Training

Why Do We Need to Compile?

Creating a model only defines its architecture.

model = tf.keras.Sequential([
    ...
])

At this point, Keras knows:

How many layers
How many neurons
Which activation functions

But we have not specified the learning configuration.

Compilation provides that configuration:

Optimizer
Loss Function
Metrics

Example 1 — Binary Classification

Suppose we want to predict whether a customer will buy a product.

0 = Will Not Buy
1 = Will Buy

We can create:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(5,)),

    tf.keras.layers.Dense(
        16,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])

Then compile:

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)

Why these choices?

Binary output
     ↓
Sigmoid


Binary classification
     ↓
Binary Cross-Entropy


Weight updates
     ↓
Adam


Performance monitoring
     ↓
Accuracy

Example 2 — Regression

Now suppose we want to predict a house price.

Input
 ↓
House Features
 ↓
Neural Network
 ↓
Predicted Price

A regression model can use a single output neuron without a sigmoid activation.

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(5,)),

    tf.keras.layers.Dense(
        16,
        activation="relu"
    ),

    tf.keras.layers.Dense(1)
])

We could compile it using mean squared error:

model.compile(
    optimizer="adam",
    loss="mse",
    metrics=["mae"]
)

Here:

mse
↓
Mean Squared Error


mae
↓
Mean Absolute Error

The important point is that the compile configuration depends on the machine-learning problem.

Common Loss Functions

Different problems commonly use different loss functions.

Binary Classification
→ binary_crossentropy


Regression
→ mse


Multi-Class Classification
→ categorical_crossentropy
or
sparse_categorical_crossentropy

Don't blindly use binary cross-entropy for every problem. The output type and target format determine the appropriate loss.

Common Optimizers

Adam
→ optimizer="adam"


SGD
→ optimizer="sgd"


RMSprop
→ optimizer="rmsprop"

Adam is a strong general starting point for many neural network problems, but it is not automatically the best optimizer for every situation.

Using Multiple Metrics

We can monitor more than one metric.

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=[
        "accuracy"
    ]
)

We can also use other metrics depending on the problem.

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=[
        "accuracy",
        "Precision",
        "Recall"
    ]
)

Metrics are primarily for monitoring and evaluation. The loss is what the optimization process directly minimizes.

Important: Compile Does Not Train the Model

This is one of the most important things to understand.

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)

This does not mean the model has learned from your training data.

It only configures the training process.

Create Model
     ↓
Compile Model
     ↓
Train Model
     ↓
Learn Weights
     ↓
Make Predictions

Actual learning happens when we call:

model.fit(...)

Complete Flow

import tensorflow as tf


# 1. Create model
model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(2,)),

    tf.keras.layers.Dense(
        4,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])


# 2. Compile model
model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)


# 3. Train model
model.fit(
    X_train,
    y_train,
    epochs=10
)

The three important stages are:

model = ...
       ↓
Architecture


model.compile(...)
       ↓
Learning Configuration


model.fit(...)
       ↓
Actual Training

Understand the Python Code

model.compile(

Calls Keras's compile method to configure the model for training.

optimizer="adam"

Selects Adam to update the model's weights.

loss="binary_crossentropy"

Selects binary cross-entropy to measure prediction error for a binary classification problem.

metrics=["accuracy"]

Tells Keras to report accuracy while training and evaluating the model.

model.fit(...)

This is where training actually happens. The model uses the configured loss and optimizer to update its weights.

Simple Way to Remember

Think of building a neural network like setting up a student for an exam.

Create Model
↓
Build the student


Compile
↓
Choose how the student will learn
and how performance will be measured


Train
↓
Student practices and improves

So:

Architecture
= What the model looks like


Compile
= How the model will learn


Fit
= The actual learning

Remember This

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)

Remember the three parts:

Optimizer
→ How weights are updated


Loss
→ How prediction error is measured


Metrics
→ How performance is monitored

The complete workflow is:

Create
  ↓
Compile
  ↓
Train
  ↓
Evaluate
  ↓
Predict

In the next topic, we will learn about Training the Model using model.fit().