Train the Model

What Does Training Mean?

Suppose we give the model an image of a handwritten 7.

Image
  ↓
Neural Network
  ↓
Prediction = 3

Correct Answer = 7

The model made a mistake.

During training, the model uses this error to adjust its internal weights.

Prediction
     ↓
Compare with correct answer
     ↓
Calculate loss
     ↓
Update weights
     ↓
Try again

The Training Data

From the previous lesson, we already have:

x_train
y_train

x_train contains the training images.

y_train contains the correct labels.

x_train
   ↓
Images

y_train
   ↓
Correct answers

These two datasets are provided to the model during training.

The Most Important Training Function

model.fit(
    x_train,
    y_train
)

The fit() method starts the learning process.

This is one of the most important lines in the entire project.

model.fit()
    ↓
Start Training
    ↓
Model learns from data

What Happens Inside model.fit()?

Conceptually, training works like this:

1. Take training images
          ↓
2. Make predictions
          ↓
3. Compare predictions
   with correct labels
          ↓
4. Calculate loss
          ↓
5. Calculate gradients
          ↓
6. Update weights
          ↓
7. Repeat

This cycle happens many times during training.

What Is an Epoch?

An epoch means that the model has gone through the entire training dataset once.

60,000 training images

        ↓

Model sees all 60,000 images

        ↓

1 Epoch completed

If we train for 5 epochs:

Epoch 1 → See all training data
Epoch 2 → See all training data
Epoch 3 → See all training data
Epoch 4 → See all training data
Epoch 5 → See all training data

The model gets multiple opportunities to improve its weights.

Train for Multiple Epochs

history = model.fit(
    x_train,
    y_train,
    epochs=5
)

Here, epochs=5 tells the model to process the complete training dataset five times.

Epoch 1
   ↓
Learn

Epoch 2
   ↓
Learn more

Epoch 3
   ↓
Improve

Epoch 4
   ↓
Improve

Epoch 5
   ↓
Final training state

What Is a Batch?

Processing all 60,000 images at once can require a lot of memory. Instead, the training data is normally divided into smaller batches.

60,000 images

        ↓

Batch 1 → 32 images
Batch 2 → 32 images
Batch 3 → 32 images
...
Batch N → remaining images

We can control the batch size using batch_size.

model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=32
)

This means the model processes 32 training examples at a time before updating its weights.

Epoch vs Batch

Dataset
60,000 images
      ↓
Split into batches
      ↓
Batch 1
Batch 2
Batch 3
...
      ↓
All batches completed
      ↓
1 Epoch

So remember the difference:

Batch
→ Small group of training examples

Epoch
→ One complete pass through
  the training dataset

Watch the Training Progress

history = model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=32
)

TensorFlow will display information similar to:

Epoch 1/5
loss: 0.25
accuracy: 0.92

Epoch 2/5
loss: 0.12
accuracy: 0.96

Epoch 3/5
loss: 0.08
accuracy: 0.97

Epoch 4/5
loss: 0.06
accuracy: 0.98

Epoch 5/5
loss: 0.05
accuracy: 0.98

These numbers are only an example. Your actual results will depend on the environment and training setup.

Understand Loss During Training

Loss measures how far the model's predictions are from the correct answers.

High Loss
    ↓
Bad predictions

Low Loss
    ↓
Better predictions

For example:

Epoch 1
Loss = 0.50

Epoch 2
Loss = 0.25

Epoch 3
Loss = 0.12

A decreasing loss generally indicates that the model is fitting the training data better.

Understand Accuracy During Training

Accuracy tells us how many predictions are correct.

100 predictions

95 correct

Accuracy = 95%

During training, we generally hope to see training accuracy improve.

Epoch 1 → 90%
Epoch 2 → 94%
Epoch 3 → 96%
Epoch 4 → 97%
Epoch 5 → 98%

Training the Model With Validation Data

We can also give the model a validation dataset while it trains.

history = model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=32,
    validation_split=0.1
)

validation_split=0.1 means that 10% of the training data is held aside for validation.

Training data
      ↓
90%
      ↓
Used for learning


Training data
      ↓
10%
      ↓
Used for validation

The validation data helps us monitor how the model performs on data that was not directly used to update its weights.

Training vs Validation

Training Accuracy
        ↓
How well the model performs
on data it learned from


Validation Accuracy
        ↓
How well it performs on
held-out validation data

These two values should be looked at together.

If training accuracy continues increasing while validation performance stops improving or gets worse, the model may be starting to overfit.

Complete Training Code

import tensorflow as tf

# Load dataset
(x_train, y_train), (x_test, y_test) = \
    tf.keras.datasets.mnist.load_data()

# Normalize images
x_train = x_train / 255.0
x_test = x_test / 255.0

# Build model
model = tf.keras.Sequential([

    tf.keras.layers.Flatten(
        input_shape=(28, 28)
    ),

    tf.keras.layers.Dense(
        128,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        64,
        activation="relu"
    ),

    tf.keras.layers.Dense(
        10,
        activation="softmax"
    )
])

# Compile model
model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

# Train model
history = model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=32,
    validation_split=0.1
)

What Is history?

history = model.fit(...)

The fit() method returns a History object containing information collected during training.

We can inspect the recorded metrics.

print(history.history.keys())

You will typically see values such as:

loss
accuracy
val_loss
val_accuracy

Understanding One Training Step

Imagine the model receives an image of a handwritten 5.

Input:

Image = 5

        ↓

Model predicts:

5 → 0.30
3 → 0.25
8 → 0.20
2 → 0.10
...
        
        ↓

Correct answer = 5

        ↓

Calculate loss

        ↓

Backpropagation

        ↓

Update weights

        ↓

Next example

This happens again and again for many training examples.

What Is Backpropagation?

Backpropagation is the process used to determine how the model's weights contributed to the error.

Prediction
     ↓
Loss
     ↓
Backpropagation
     ↓
Calculate gradients
     ↓
Optimizer
     ↓
Update weights

You do not normally need to manually calculate these gradients. TensorFlow performs the calculations for us.

What Actually Changes During Training?

The architecture does not change during normal training. The learned parameters change.

Before training:

Weights = Random / Initial values

        ↓

Training

        ↓

After training:

Weights = Learned values

These learned weights are what allow the model to recognize patterns in new images.

A Simple Training Example

First attempt:

Image = 7

Prediction = 2
Correct = 7

        ↓

Large error

        ↓

Update weights


Later:

Image = 7

Prediction = 7
Correct = 7

        ↓

Much smaller error

The model is not memorizing a single answer manually. It is adjusting its parameters so that useful patterns can produce better predictions across many examples.

What Happens After Training?

Training Data
     ↓
Model learns
     ↓
Weights are updated
     ↓
Training finishes
     ↓
Model is ready for evaluation

The next step is not to immediately celebrate a high training accuracy. We need to test the model using data that was not used to train it.

Quick Check