Understand the Python Code

The Basic Pattern

Almost every loss calculation follows this basic idea:

Actual Answer
      +
Model Prediction
      ↓
Loss Formula
      ↓
Python Code
      ↓
Loss Value

For example:

Actual = 1
Prediction = 0.9

       ↓

Binary Cross-Entropy

       ↓

Loss ≈ 0.105

Example 1 — MSE Code

Let's start with the MSE code:

actual = [10, 20, 30]
prediction = [12, 18, 29]

errors = []

for y, p in zip(actual, prediction):

    error = y - p
    squared_error = error ** 2

    errors.append(squared_error)

mse = sum(errors) / len(errors)

print("MSE:", mse)

This code calculates the Mean Squared Error for three predictions.

Step 1 — Store the Actual Values

actual = [10, 20, 30]

This list contains the correct answers.

Actual values:

10
20
30

Think of these as the answers that the model should have predicted.

Step 2 — Store the Predictions

prediction = [12, 18, 29]

These are the values produced by the model.

Actual       Prediction

10           12
20           18
30           29

Now we can compare each prediction with its corresponding actual value.

Step 3 — Create an Empty List

errors = []

We create an empty list to store the squared errors.

errors = []

        ↓

errors = [ ]

Values will be added to this list during the loop.

Step 4 — Understand zip()

for y, p in zip(actual, prediction):

zip() pairs the values from the two lists.

actual      prediction

10          12
20          18
30          29

The loop processes these pairs one at a time.

First loop:
y = 10
p = 12


Second loop:
y = 20
p = 18


Third loop:
y = 30
p = 29

Here:

y = actual value
p = prediction

Step 5 — Calculate the Error

error = y - p

For the first pair:

y = 10
p = 12

error = 10 - 12

error = -2

For the second:

y = 20
p = 18

error = 20 - 18

error = 2

For the third:

y = 30
p = 29

error = 30 - 29

error = 1

Step 6 — Square the Error

squared_error = error ** 2

The ** operator means exponentiation in Python.

error ** 2

means

error × error

Therefore:

(-2)² = 4
( 2)² = 4
( 1)² = 1

Squaring is important because it removes negative signs and gives larger errors a stronger penalty.

Step 7 — Add the Error to the List

errors.append(squared_error)

append() adds a value to the end of a list.

First loop:
errors = [4]


Second loop:
errors = [4, 4]


Third loop:
errors = [4, 4, 1]

At the end:

errors = [4, 4, 1]

Step 8 — Add the Errors

sum(errors)

Python's sum() function adds all values in the list.

errors = [4, 4, 1]

sum(errors)

= 4 + 4 + 1

= 9

Step 9 — Count the Values

len(errors)

len() tells us how many values are inside the list.

errors = [4, 4, 1]

len(errors)

= 3

Step 10 — Calculate the MSE

mse = sum(errors) / len(errors)

Now Python performs:

mse = 9 / 3

mse = 3

Therefore:

MSE = 3.0

Step 11 — Display the Result

print("MSE:", mse)

print() displays the result in the terminal.

MSE: 3.0

Complete MSE Code Flow

actual
  ↓
prediction
  ↓
zip()
  ↓
Calculate error
  ↓
Square error
  ↓
append to list
  ↓
sum()
  ↓
len()
  ↓
Divide
  ↓
MSE

Example 2 — Binary Cross-Entropy

Now let's understand the BCE code:

import math

actual = 1
prediction = 0.9

loss = -(
    actual * math.log(prediction)
    + (1 - actual) * math.log(1 - prediction)
)

print("BCE:", loss)

Step 1 — Import math

import math

Python's math module contains mathematical functions.

We need it because BCE uses the logarithm function:

math.log()

Step 2 — Store the Actual Answer

actual = 1

The actual class is 1.

1 = Positive Class

Step 3 — Store the Prediction

prediction = 0.9

The model predicts a 90% probability for class 1.

Prediction = 0.9

Meaning:

90% probability of class 1

Step 4 — Understand math.log()

math.log(0.9)

This calculates the natural logarithm of 0.9.

math.log(0.9)

≈ -0.105

You don't need to manually calculate logarithms. Python performs the mathematical operation for you.

Step 5 — Read the BCE Formula in Python

loss = -(
    actual * math.log(prediction)
    + (1 - actual) * math.log(1 - prediction)
)

The mathematical formula is:

Loss = -[y log(p) + (1-y) log(1-p)]

Python is simply translating this formula into executable code.

Step 6 — Put the Values Into the Formula

actual = 1
prediction = 0.9

Substitute them:

Loss =
-[
    1 × log(0.9)
    +
    (1 - 1) × log(1 - 0.9)
]

The second part becomes zero:

(1 - 1) = 0

So the calculation becomes:

Loss = -log(0.9)

Loss ≈ 0.105

Example — Wrong Prediction

Change only the prediction:

actual = 1
prediction = 0.1

Now:

Loss = -log(0.1)

Loss ≈ 2.303

The code is exactly the same. Only the prediction changed.

Prediction = 0.9
       ↓
BCE ≈ 0.105


Prediction = 0.1
       ↓
BCE ≈ 2.303

This is why BCE strongly penalizes confident wrong predictions.

Example 3 — Categorical Cross-Entropy

Now let's understand the Categorical Cross-Entropy code:

import math

actual = [0, 1, 0]

prediction = [0.10, 0.80, 0.10]

loss = 0

for y, p in zip(actual, prediction):

    loss += y * math.log(p)

loss = -loss

print("Categorical Cross-Entropy:", loss)

Step 1 — Store the Actual Class

actual = [0, 1, 0]

There are three classes.

Cat   → 0
Dog   → 1
Horse → 0

Therefore, Dog is the correct class.

Step 2 — Store the Predictions

prediction = [0.10, 0.80, 0.10]

The model gives:

Cat   → 10%
Dog   → 80%
Horse → 10%

The correct class, Dog, received 80%.

Step 3 — Start Loss at Zero

loss = 0

We start with no accumulated loss.

loss = 0

The loop will add the contribution from each class.

Step 4 — Pair Actual and Prediction

for y, p in zip(actual, prediction):

The values are processed together:

First:

y = 0
p = 0.10


Second:

y = 1
p = 0.80


Third:

y = 0
p = 0.10

Step 5 — Add Each Class Contribution

loss += y * math.log(p)

Let's process the first class:

y = 0
p = 0.10

0 × log(0.10)

= 0

Second class:

y = 1
p = 0.80

1 × log(0.80)

= log(0.80)

Third class:

y = 0
p = 0.10

0 × log(0.10)

= 0

Therefore only the correct class contributes to this one-hot loss calculation.

Step 6 — Make the Loss Positive

loss = -loss

Logarithms of probabilities between 0 and 1 are negative. The negative sign makes the final loss positive.

Loss = -log(0.80)

Loss ≈ 0.223

All Three Loss Functions Together

import math


# =================================
# MSE
# =================================

actual_values = [10, 20, 30]
predicted_values = [12, 18, 29]

errors = []

for actual, prediction in zip(
    actual_values,
    predicted_values
):
    error = actual - prediction
    errors.append(error ** 2)

mse = sum(errors) / len(errors)

print("MSE:", mse)


# =================================
# Binary Cross-Entropy
# =================================

actual = 1
prediction = 0.9

bce = -(
    actual * math.log(prediction)
    + (1 - actual) * math.log(1 - prediction)
)

print("BCE:", bce)


# =================================
# Categorical Cross-Entropy
# =================================

actual = [0, 1, 0]
prediction = [0.10, 0.80, 0.10]

cce = 0

for y, p in zip(actual, prediction):

    cce += y * math.log(p)

cce = -cce

print("Categorical Cross-Entropy:", cce)

How to Read Loss Code

When you see loss code in a deep-learning project, don't try to memorize every line. Instead, identify these pieces:

1. Actual values
        ↓
2. Predictions
        ↓
3. Loss formula
        ↓
4. Calculation
        ↓
5. Final loss

For example:

actual = ...
prediction = ...

        ↓

loss = ...

        ↓

print(loss)

Why Put the Code Inside a Function?

Instead of repeating the same code, we can create a reusable function.

def mean_squared_error(actual, prediction):

    errors = []

    for y, p in zip(actual, prediction):

        error = y - p
        errors.append(error ** 2)

    return sum(errors) / len(errors)

Now we can call it whenever we need it:

actual = [10, 20, 30]

prediction = [12, 18, 29]

loss = mean_squared_error(
    actual,
    prediction
)

print(loss)

Functions make code reusable and easier to organize.

Understand return

return sum(errors) / len(errors)

return sends the calculated value back to the code that called the function.

loss = mean_squared_error(
    actual,
    prediction
)

The returned MSE is stored inside:

loss

One Important Problem With Manual BCE

There is one practical issue with the educational BCE code.

math.log(0)

This is not valid because the logarithm of zero is undefined.

A safer educational implementation can keep predictions away from exactly 0 and 1:

epsilon = 1e-15

prediction = max(
    min(prediction, 1 - epsilon),
    epsilon
)

Production deep-learning frameworks use more robust, numerically stable implementations than this simple demonstration.

What You Normally Do in Real Projects

In real deep-learning applications, you normally do not calculate the loss manually.

For example, Keras can calculate BCE:

model.compile(
    optimizer="adam",
    loss="binary_crossentropy"
)

Or categorical cross-entropy:

model.compile(
    optimizer="adam",
    loss="categorical_crossentropy"
)

The framework handles the optimized loss calculation.

We learned the manual Python version because understanding the calculation makes framework code much easier to understand.

How This Connects to Neural Networks

Loss calculation is one part of the complete training process.

Input
  ↓
Neural Network
  ↓
Prediction
  ↓
Loss Function
  ↓
Loss Value
  ↓
Backpropagation
  ↓
Gradients
  ↓
Update Weights
  ↓
New Prediction
  ↓
New Loss
  ↓
Repeat

The goal is to adjust the network's weights so that the loss generally becomes smaller on the training data.

Complete Example to Remember

Imagine a binary classification model predicting whether an email is spam.

Actual = 1

Prediction = 0.2

The model is not very confident that the email is spam.

BCE ≈ 1.609

After training:

Prediction = 0.7

BCE ≈ 0.357

After more training:

Prediction = 0.9

BCE ≈ 0.105

The code has not changed. The model's prediction changed.

Prediction
0.2
 ↓
0.7
 ↓
0.9

Loss
1.609
 ↓
0.357
 ↓
0.105

Lesson 5 Summary

Loss Function
    ↓
Measures prediction error


MSE
    ↓
Commonly used for regression


Binary Cross-Entropy
    ↓
Commonly used for binary classification


Categorical Cross-Entropy
    ↓
Commonly used for multi-class classification


Python
    ↓
Implements the mathematical formula


Training
    ↓
Uses loss to improve model weights