Bayes' Theorem

What Problem Does Bayes' Theorem Solve?

Imagine you are trying to determine whether an email is spam.

Before looking at the email, you already know that some emails are spam and some are not.

Initial belief:

Spam = 20%
Not Spam = 80%

Now you see that the email contains a suspicious word such as "winner".

New evidence:

Email contains "winner"

This new evidence should change our belief about whether the email is spam.

Bayes' Theorem gives us a mathematical way to make that update.

The Basic Idea

Don't start by memorizing the formula. First understand the process.

Initial belief
      ↓
New evidence
      ↓
Update the belief
      ↓
New probability

This is the main idea behind Bayes' Theorem.

In simple words:

"I believed this before.

Now I have new information.

How should my belief change?"

A Simple Everyday Example

Imagine you hear a loud noise outside your house.

You might initially think:

Possibility:
A car crashed

Initial probability = 10%

Then you hear people shouting outside.

New evidence:
People are shouting

Now you may think a crash is more likely.

Before evidence:
Crash = 10%

After evidence:
Crash = maybe 40%

The exact numbers are not important here. The important idea is that new evidence changed your belief.

Bayes' Theorem Formula

Now that the idea is clear, we can look at the mathematical formula.

The formula contains four important parts:

P(A)     → Initial probability of A

P(B|A)   → Probability of B when A is true

P(B)     → Overall probability of B

P(A|B)   → Updated probability of A
           after seeing B

The most important part for beginners is:

P(A | B)

Probability of A
given that B has happened.

Understanding "Before" and "After"

One of the easiest ways to understand Bayes' Theorem is to think about probabilities before and after receiving evidence.

BEFORE

What did we believe?

        ↓

NEW EVIDENCE

What did we learn?

        ↓

AFTER

What should we believe now?

The probability before receiving evidence is often called the prior probability.

The updated probability after considering the evidence is called the posterior probability.

Prior Probability

The prior is what we believe before seeing the new evidence.

For example, suppose only 10% of emails received by a company are spam.

P(Spam) = 10%

This is our starting point.

We haven't looked at any particular email yet.

Evidence

Now suppose we receive an email containing the word "winner".

We know that this word appears frequently in spam emails.

Evidence:

Email contains "winner"

This evidence gives us additional information.

Bayes' Theorem tells us how to use that information to update our original 10% probability.

A Complete Simple Example

Let's use 1,000 emails to make the calculation easier to understand.

Suppose:

Total emails = 1,000

Spam emails = 100

Not spam = 900

Therefore:

P(Spam) = 100 / 1000

P(Spam) = 10%

Now suppose the word "winner" appears in:

80 spam emails

and

45 normal emails

That means 125 emails contain the word "winner".

80 + 45 = 125

Now we receive an email containing "winner".

Out of the 125 emails containing that word, 80 are spam.

Probability of spam
given "winner":

80 / 125

= 0.64

= 64%

So our belief changed from:

Before evidence:

Spam = 10%

        ↓

After seeing "winner":

Spam = 64%

This is Bayesian updating.

Why Did the Probability Change So Much?

Initially, we only knew that 10% of all emails were spam.

Then we received useful evidence: the email contained "winner".

Without evidence:

Spam probability = 10%

With evidence:

Email contains "winner"

Spam probability = 64%

The evidence was strongly associated with spam, so our probability increased.

This is exactly why Bayes' Theorem is useful.

Bayes' Theorem in AI

AI systems often need to make predictions from incomplete information.

For example, an AI model might try to determine whether a customer will purchase a product.

Initial probability:

Customer will purchase = 20%

Then the AI receives new information:

Customer added product to cart
Customer viewed the product 5 times
Customer spent 10 minutes on the page

This evidence may change the probability:

Updated probability:

Customer will purchase = 75%

Again, the exact numbers are only for understanding the idea. The important concept is that the AI updates its prediction when it receives new evidence.

Medical Diagnosis Example

Bayes' Theorem is also useful for reasoning about medical test results.

Suppose a disease is relatively rare.

Initial probability of disease = 1%

A patient receives a positive test result.

New evidence:

Test result = Positive

We now want to know:

What is the probability
the patient actually has the disease
given the positive test?

Notice the direction:

We know:

Positive test

We want:

Disease given positive test

P(Disease | Positive)

This distinction is extremely important. A common mistake is to confuse P(Positive | Disease) with P(Disease | Positive).

The Most Important AI Idea

Bayes' Theorem teaches an important idea that appears throughout AI:

Start with a belief
        ↓
Receive evidence
        ↓
Measure how useful the evidence is
        ↓
Update the belief
        ↓
Make a better prediction

AI systems constantly receive new information and use it to improve their predictions.

Connection to Machine Learning

The Bayesian way of thinking is useful far beyond simple probability exercises.

Customer data
       ↓
Evidence
       ↓
Probability
       ↓
Updated belief
       ↓
Prediction

Similar ideas appear in spam filtering, recommendation systems, medical diagnosis, fraud detection, classification, and many other AI applications.

Modern machine learning does not always explicitly use Bayes' Theorem, but the underlying idea of reasoning under uncertainty is fundamental.

Probability Section Complete

You have now covered the main probability concepts needed before moving into statistics.

Probability Basics
        ↓
How likely is something?

Conditional Probability
        ↓
How likely is something
given information?

Probability Distributions
        ↓
How probability is spread
across possible outcomes?

Bayes' Theorem
        ↓
How do we update probability
using new evidence?

These concepts form the foundation for understanding how AI handles uncertainty and makes predictions.