Transfer Learning for Images

What Does Transfer Learning Mean for Images?

Image models need to learn many visual patterns. For example, they can learn edges, colors, textures, shapes, and eventually more complex objects.

A pretrained image model has already learned many of these patterns from a large dataset.

Large Image Dataset
        ↓
Pretrained CNN
        ↓
Learns Visual Features
        ↓
Reuse Model
        ↓
Your Image Dataset
        ↓
New Task

We reuse this existing knowledge instead of starting with random weights.

Simple Example — Plant Disease Detection

Suppose we want to build an application that identifies whether a plant leaf is healthy or diseased.

We may not have millions of plant images. Training a deep image model completely from scratch would therefore be inefficient.

Pretrained Image Model
        ↓
General Visual Features
        ↓
Leaf Images
        ↓
Transfer Learning
        ↓
Healthy / Diseased

The pretrained model already knows general visual patterns. We train it to recognize the specific patterns that matter for our plant images.

How Does an Image Model Learn?

Different parts of a convolutional neural network learn different levels of visual information.

Image
  ↓
Early Layers
  ↓
Edges + Lines
  ↓
Middle Layers
  ↓
Textures + Shapes
  ↓
Deeper Layers
  ↓
Complex Visual Features
  ↓
Classifier
  ↓
Prediction

The early visual features are often useful across many different image tasks. This is one reason pretrained image models are so useful.

Two Ways to Use Transfer Learning

There are two common approaches when adapting a pretrained image model.

                 Transfer Learning
                        │
              ┌─────────┴─────────┐
              ↓                   ↓
      Feature Extraction      Fine-Tuning
              ↓                   ↓
       Freeze Model         Unfreeze Some Layers
              ↓                   ↓
      Train Classifier       Adapt Model

Feature extraction keeps the pretrained model frozen. Fine-tuning allows some pretrained layers to learn from the new dataset.

Using a Pretrained Image Model

TensorFlow and Keras provide several pretrained image models. One example is MobileNetV2.

import tensorflow as tf

base_model = tf.keras.applications.MobileNetV2(
    weights="imagenet",
    include_top=False,
    pooling="avg"
)

base_model.trainable = False

The model uses weights learned from ImageNet. We remove its original classification head by setting include_top=False.

We can then add our own classifier for our image classes.

Add Our Own Image Classifier

Suppose our dataset contains two categories: healthy and diseased.

model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.Dense(
        2,
        activation="softmax"
    )
])

The pretrained model extracts visual features and the Dense layer makes the final classification.

Leaf Image
    ↓
MobileNetV2
    ↓
Visual Features
    ↓
Dense Layer
    ↓
Healthy / Diseased

Preparing Images

Neural networks need images in a consistent format. TensorFlow can load images from folders and prepare them for training.

train_dataset = tf.keras.utils.image_dataset_from_directory(
    "images/",
    image_size=(224, 224),
    batch_size=32
)

Here, every image is resized to 224 × 224 pixels and the images are grouped into batches of 32.

images/
│
├── healthy/
│   ├── leaf1.jpg
│   ├── leaf2.jpg
│   └── leaf3.jpg
│
└── diseased/
    ├── leaf4.jpg
    ├── leaf5.jpg
    └── leaf6.jpg

The folder names can be used as the class labels.

Train the New Classifier

Initially, we can keep the pretrained model frozen and train only the new classification layer.

base_model.trainable = False

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

model.fit(
    train_dataset,
    epochs=5
)

At this stage, the pretrained image features stay unchanged. The new classifier learns how to use those features for our specific image categories.

Fine-Tune the Image Model

If feature extraction is not enough, we can fine-tune some of the pretrained layers.

base_model.trainable = True

for layer in base_model.layers[:-20]:
    layer.trainable = False

for layer in base_model.layers[-20:]:
    layer.trainable = True

Now the final part of the pretrained model can adapt to our image dataset.

We should use a small learning rate during this stage.

model.compile(
    optimizer=tf.keras.optimizers.Adam(
        learning_rate=0.00001
    ),
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

Complete Example

import tensorflow as tf

# Load image dataset
train_dataset = tf.keras.utils.image_dataset_from_directory(
    "images/",
    image_size=(224, 224),
    batch_size=32
)

# Load pretrained model
base_model = tf.keras.applications.MobileNetV2(
    weights="imagenet",
    include_top=False,
    pooling="avg"
)

# Freeze pretrained layers
base_model.trainable = False

# Build model
model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.Dense(
        2,
        activation="softmax"
    )
])

# Compile model
model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"]
)

# Train
model.fit(
    train_dataset,
    epochs=5
)

This is the basic image transfer learning workflow. The pretrained model extracts useful visual features while the new Dense layer learns the new classes.

Complete Workflow

Your Images
     ↓
Resize Images
     ↓
Load Dataset
     ↓
Pretrained Image Model
     ↓
Freeze Model
     ↓
Add New Classifier
     ↓
Train Classifier
     ↓
Evaluate Model
     ↓
Optional Fine-Tuning
     ↓
Final Image Model

This workflow allows us to build useful image classifiers without training a large neural network completely from scratch.

Another Example — Car Classification

Suppose we want to classify images into two categories: electric cars and petrol cars.

Car Image
    ↓
Pretrained Image Model
    ↓
Edges
Shapes
Textures
Car Features
    ↓
New Classifier
    ↓
Electric / Petrol

We do not need to teach the model how to detect basic edges and shapes from zero. We reuse the pretrained visual knowledge and specialize it for our car dataset.

Quick Check