Evaluating the Model

What Is Model Evaluation?

Evaluation means checking how well a trained model performs on data that it did not use to update its weights.

The basic workflow is:

Training Data
      ↓
Train Model
      ↓
Learn Patterns
      ↓
Test / Validation Data
      ↓
Evaluate Performance

The purpose is to answer a simple question:

"How well does my trained model
perform on data it needs to predict?"

Training vs Evaluation

Training and evaluation have different purposes.

Training

model.fit()
     ↓
Learn from data
     ↓
Update weights


Evaluation

model.evaluate()
     ↓
Measure performance
     ↓
Do not use the result to
train the model

This distinction is important. During training, the model changes its weights. During evaluation, we are measuring how well the current model performs.

Why Do We Need Evaluation?

Imagine a student memorizes every question from a practice test and gets 100%.

That does not prove the student understands the subject.

Give the student a different test and the result may be much worse.

Training Performance
        ↓
Very Good

New Data Performance
        ↓
Very Bad

The same problem can happen with neural networks.

A model can perform extremely well on training data but poorly on unseen data. This is called overfitting.

Using model.evaluate()

In Keras, we commonly evaluate a trained model using:

model.evaluate(
    X_test,
    y_test
)

Here:

X_test
↓
Test input data


y_test
↓
Correct answers


model.evaluate()
↓
Measures model performance

Simple Example

Suppose we have trained a model to classify whether a student passes an exam.

0 = Fail
1 = Pass

After training, we have test data:

X_test = [
    [2],
    [4],
    [6],
    [8]
]

y_test = [
    0,
    0,
    1,
    1
]

We can evaluate the model:

loss, accuracy = model.evaluate(
    X_test,
    y_test
)

print("Loss:", loss)
print("Accuracy:", accuracy)

Suppose the result is:

Loss: 0.25
Accuracy: 0.90

The accuracy tells us that the model correctly classified about 90% of the examples in the evaluation data.

Evaluation Loss

Loss tells us how far the model's predictions are from the correct answers according to the selected loss function.

Lower Loss
    ↓
Generally better predictions


Higher Loss
    ↓
Generally worse predictions

For example:

Model A
Loss = 0.20


Model B
Loss = 0.80

Assuming the same problem and loss function, Model A has lower evaluation loss.

But don't judge a model from loss alone. The appropriate metrics depend on the problem.

Accuracy

Accuracy measures the proportion of predictions that were correct.

For example, suppose the model predicts 100 test examples and gets 90 correct.

Correct Predictions = 90
Total Predictions   = 100

Accuracy = 90 / 100

Accuracy = 90%

In Keras, accuracy can be calculated automatically if we included it when compiling the model:

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)

Complete Evaluation Example

import tensorflow as tf
import numpy as np


# -----------------------------
# Training data
# -----------------------------

X_train = np.array([
    [1],
    [2],
    [3],
    [4],
    [5],
    [6],
    [7],
    [8]
])

y_train = np.array([
    0,
    0,
    0,
    1,
    1,
    1,
    1,
    1
])


# -----------------------------
# Test data
# -----------------------------

X_test = np.array([
    [2],
    [4],
    [6],
    [8]
])

y_test = np.array([
    0,
    1,
    1,
    1
])


# -----------------------------
# Create model
# -----------------------------

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(1,)),

    tf.keras.layers.Dense(
        8,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        1,
        activation="sigmoid"
    )
])


# -----------------------------
# Compile
# -----------------------------

model.compile(
    optimizer="adam",
    loss="binary_crossentropy",
    metrics=["accuracy"]
)


# -----------------------------
# Train
# -----------------------------

model.fit(
    X_train,
    y_train,
    epochs=20,
    verbose=0
)


# -----------------------------
# Evaluate
# -----------------------------

loss, accuracy = model.evaluate(
    X_test,
    y_test,
    verbose=0
)


print("Test Loss:", loss)
print("Test Accuracy:", accuracy)

Why Use Test Data?

The test data should represent examples that the model did not use during training.

Training Data
     ↓
Used to learn weights


Test Data
     ↓
Used to measure final performance

This gives us a better idea of how the model may perform on new data.

If you train and evaluate on exactly the same examples, the evaluation can give a misleading picture of how well the model generalizes.

Training, Validation, and Test Data

In a typical machine-learning workflow, data can be split into three groups.

Dataset
   |
   +------------------+
   |                  |
Training           Validation
   |                  |
Learn              Tune / Monitor
   |
   +------------------+
            |
          Test
            |
       Final Evaluation

Their roles are different:

Training
→ Learn model parameters


Validation
→ Monitor performance and help
  make development decisions


Test
→ Final check on unseen data

The exact workflow can vary, but the key principle is to keep truly unseen data for an honest final evaluation.

Validation During Training

We can also provide validation data while training:

history = model.fit(
    X_train,
    y_train,
    epochs=20,
    validation_data=(X_val, y_val)
)

Keras then reports training and validation performance during each epoch.

Epoch 1
Training Loss:   0.70
Validation Loss: 0.68

Epoch 2
Training Loss:   0.55
Validation Loss: 0.52

Epoch 3
Training Loss:   0.40
Validation Loss: 0.45

This helps us monitor whether the model is learning patterns that also work on data it did not train on.

Detecting Overfitting

One common warning sign is when training performance keeps improving while validation performance starts getting worse.

Training Loss
0.80 → 0.50 → 0.30 → 0.10
          ↓
Continuously decreasing


Validation Loss
0.75 → 0.55 → 0.60 → 0.90
                 ↓
                 Increasing

This can indicate that the model is becoming too specialized to the training data.

Training Performance
        ↑
      Better


Validation Performance
        ↓
      Worse


Possible Problem
        ↓
    Overfitting

evaluate() vs predict()

These two methods are related but serve different purposes.

model.predict()
       ↓
Produces predictions


model.evaluate()
       ↓
Measures performance
using inputs + correct answers

Example:

prediction = model.predict(X_test)

This gives the model's outputs.

loss, accuracy = model.evaluate(
    X_test,
    y_test
)

This measures how well those outputs match the correct answers according to the configured loss and metrics.

Example 1 — Good Generalization

Training Accuracy = 92%
Test Accuracy     = 90%

The training and test performance are relatively close. That is generally a healthier sign than a huge gap.

Example 2 — Possible Overfitting

Training Accuracy = 99%
Test Accuracy     = 70%

The model performs extremely well on training data but much worse on unseen test data.

That large gap is a warning sign for overfitting.

Accuracy Is Not Always Enough

Accuracy is useful, but it is not the right metric for every problem.

For some classification problems, we may also care about:

Precision
Recall
F1 Score
AUC

For regression problems, common metrics include:

MAE
MSE
RMSE

The metric should match the actual problem you are trying to solve.

Understand the Python Code

loss, accuracy = model.evaluate(
    X_test,
    y_test,
    verbose=0
)

Let's break it down.

X_test

Contains the input examples that we want to evaluate.

y_test

Contains the correct answers for those examples.

model.evaluate()

Runs the model on the evaluation data and calculates the configured loss and metrics.

loss, accuracy

Stores the returned loss and accuracy values.

verbose=0

Suppresses the progress output while evaluation runs.

Complete Neural Network Workflow

1. Create Model
       ↓
2. Add Layers
       ↓
3. Compile Model
       ↓
4. Train Model
       ↓
5. Make Predictions
       ↓
6. Evaluate Model

The important functions are:

model.compile()
        ↓
Configure learning


model.fit()
        ↓
Train


model.predict()
        ↓
Make predictions


model.evaluate()
        ↓
Measure performance

Remember This

model.evaluate(
    X_test,
    y_test
)

Think of evaluation as giving the trained model a test.

Training
→ Learn


Evaluation
→ Check how well it learned


Prediction
→ Use the trained model

The most important lesson is:

High Training Accuracy
does NOT automatically mean
a good model.


Check unseen data too.

A model that performs well on unseen data is much more useful than a model that only performs well on examples it already saw during training.