Chain Rule

What Is the Chain Rule?

Sometimes a function contains another function inside it. This is called a composite function.

For example:

y = (x² + 1)³

Here, one function is inside another function.

Inside function:

u = x² + 1

Outside function:

y = u³

The Chain Rule allows us to calculate the derivative step by step.

Why Do We Need the Chain Rule?

For a simple function, finding a derivative is easy.

y = x²

dy/dx = 2x

But real AI models contain many operations connected together.

Input
  ↓
Multiply
  ↓
Add
  ↓
Activation Function
  ↓
Another Layer
  ↓
Output

The output depends on the previous operation, which depends on another operation before it.

The Chain Rule lets us calculate how a change at the beginning affects the final output.

The Basic Chain Rule Formula

Suppose:

y = f(u)

u = g(x)

Then the derivative of y with respect to x is:

dy/dx = dy/du × du/dx

In simple words:

Total Change
     =
Change in outer function
     ×
Change in inner function

This is the main formula you need to remember.

Simple Real-World Example

Imagine a shop where the number of products sold affects revenue, and revenue affects profit.

Products Sold
      ↓
Revenue
      ↓
Profit

A change in the number of products does not directly reach profit. It passes through revenue first.

Products
   ↓
Revenue
   ↓
Profit

The Chain Rule calculates the total effect through this chain.

Change in Profit
      =
Change in Profit / Change in Revenue
      ×
Change in Revenue / Change in Products

This is the same mathematical idea used when calculations are connected inside an AI model.

Step-by-Step Mathematical Example

Let's calculate the derivative of:

y = (x² + 1)³

First, identify the inner function:

u = x² + 1

Therefore:

y = u³

Now calculate the derivative of the outer function:

dy/du = 3u²

Next, calculate the derivative of the inner function:

du/dx = 2x

Apply the Chain Rule:

dy/dx = dy/du × du/dx

      = 3u² × 2x

      = 6xu²

Finally, replace u with x² + 1:

dy/dx = 6x(x² + 1)²

We solved a complicated derivative by breaking it into two simple derivatives.

A Very Simple Numerical Example

Let's use an easier example to understand the idea.

u = 2x

y = u²

First:

dy/du = 2u

Second:

du/dx = 2

Apply the Chain Rule:

dy/dx = dy/du × du/dx

      = 2u × 2

      = 4u

Since:

u = 2x

Therefore:

dy/dx = 4(2x)

      = 8x

Chain Rule in AI

Now let's connect the idea to AI.

A neural network is basically a chain of mathematical operations.

Input
  ↓
Layer 1
  ↓
Activation
  ↓
Layer 2
  ↓
Activation
  ↓
Output

The final output depends on the operations that happened before it.

If we want to know how a particular weight affects the final output or the loss, we need to calculate the effect through all the connected operations.

Weight
  ↓
Layer 1
  ↓
Layer 2
  ↓
Output
  ↓
Loss

The Chain Rule allows us to multiply the derivatives through this chain.

Chain Rule and Backpropagation

During neural network training, we need to know how much each weight contributed to the final error.

Loss
 ↑
Layer 3
 ↑
Layer 2
 ↑
Layer 1
 ↑
Weight

We calculate derivatives backward through the network.

Loss
  ↓
Derivative
  ↓
Layer 3
  ↓
Layer 2
  ↓
Layer 1
  ↓
Weight

This process is called backpropagation.

The Chain Rule is one of the main mathematical ideas that makes backpropagation possible.

Chain Rule With Three Functions

The same idea works when there are more than two connected functions.

x
↓
u
↓
v
↓
y

The total derivative becomes:

dy/dx

= dy/dv
  × dv/du
  × du/dx

Notice what happened: we multiply the derivative of every step in the chain.

Final Effect
     =
Effect of Step 3
×
Effect of Step 2
×
Effect of Step 1

Neural networks can have many such steps, which is why the Chain Rule becomes extremely important in deep learning.

Why Chain Rule Is Important in AI

AI models contain many connected calculations. We need a way to trace how one parameter affects the final result.

Parameter
    ↓
Mathematical Operation
    ↓
Another Operation
    ↓
Another Operation
    ↓
Prediction
    ↓
Loss

The Chain Rule lets us calculate the total effect through this entire chain.

Without this idea, training deep neural networks would be much more difficult.

Chain Rule → Gradients → Learning

Now connect the three concepts from this lesson series.

Chain Rule
     ↓
Calculate derivatives
through connected operations
     ↓
Gradients
     ↓
Understand how weights
affect Loss
     ↓
Gradient Descent
     ↓
Update weights
     ↓
Reduce Loss

This is the mathematical foundation behind how neural networks learn from their errors.