Probability Distributions

What Is a Probability Distribution?

Imagine rolling a normal six-sided dice.

Possible outcomes:

1
2
3
4
5
6

Each number has the same probability of appearing:

P(1) = 1/6
P(2) = 1/6
P(3) = 1/6
P(4) = 1/6
P(5) = 1/6
P(6) = 1/6

A probability distribution puts all these probabilities together and describes the complete set of possible outcomes.

Why Do We Need Probability Distributions?

Sometimes we don't know exactly what will happen. There may be many possible outcomes.

Instead of saying:

"The result will definitely be 6."

we can describe all possible results and their probabilities.

Outcome     Probability

1           16.7%
2           16.7%
3           16.7%
4           16.7%
5           16.7%
6           16.7%

This gives us a much better way to represent uncertainty.

AI deals with uncertainty constantly, so probability distributions are very important.

A Simple Real-World Example

Imagine an AI system predicting tomorrow's weather.

It may not be certain about the result.

Weather prediction:

Sunny   → 60%
Cloudy  → 30%
Rainy   → 10%

The AI is not saying that tomorrow will definitely be sunny.

Instead, it is saying that based on the available information, sunny weather has the highest probability.

These possible outcomes and their probabilities form a probability distribution.

Probabilities Add Up to 1

One important rule is that the probabilities of all possible outcomes must add up to 1, or 100%.

Sunny   = 0.60
Cloudy  = 0.30
Rainy   = 0.10

0.60 + 0.30 + 0.10 = 1.00

Or as percentages:

60% + 30% + 10% = 100%

Why?

Because the distribution should account for all possible outcomes.

Discrete Probability Distribution

A discrete probability distribution is used when the possible outcomes can be counted.

A dice is a simple example.

Possible values:

1, 2, 3, 4, 5, 6

These are separate, countable outcomes.

Another example could be the number of products purchased by a customer.

Products purchased:

0
1
2
3
4
...

These values can be counted, so they can be represented using a discrete probability distribution.

Continuous Probability Distribution

Sometimes a value can take many values within a range.

For example, temperature:

25°C
25.1°C
25.15°C
25.157°C
25.1578°C
...

There can be infinitely many possible values between two numbers.

These types of values are commonly represented using continuous probability distributions.

Examples include:

Temperature
Height
Weight
Time
Speed

Distribution as a Map of Possibilities

A useful way to think about a probability distribution is as a map showing where the probability is concentrated.

Possible outcomes

Low probability
      ↓
   [     ]

Medium probability
      ↓
   [       ]

High probability
      ↓
   [           ]

The distribution tells us
where the probability is concentrated.

This becomes especially useful when working with large amounts of data.

Probability Distribution in AI

Now let's look at a simple AI example.

Suppose an image classification model receives an image of an animal.

The model might produce:

Cat       → 0.80
Dog       → 0.15
Rabbit    → 0.05

The model is not simply saying:

"This is a cat."

It is expressing its confidence across several possible classes.

Cat       = 80%
Dog       = 15%
Rabbit    = 5%

This is a probability distribution over the possible classes.

Why This Is Useful in Machine Learning

A machine learning model often has uncertainty about its prediction.

For example, suppose a model is trying to identify an email.

Spam       → 0.90
Not Spam   → 0.10

The model is highly confident that the email is spam.

Another email might produce:

Spam       → 0.55
Not Spam   → 0.45

Now the model is much less certain.

The probability distribution allows us to see this uncertainty instead of hiding it.

Probability Distribution vs Single Probability

These two ideas are related but not exactly the same.

Single probability:

P(Rain) = 30%

This tells us the probability of one event.

A probability distribution gives us the probabilities for multiple possible outcomes.

Sunny   = 60%
Cloudy  = 30%
Rainy   = 10%

The distribution gives us a complete picture of the possible outcomes.

A Simple AI Prediction Flow

Input Data
    ↓
AI Model
    ↓
Calculate probabilities
    ↓
Probability Distribution
    ↓
Choose / evaluate possible outcome
    ↓
Prediction

For example:

Image
  ↓
AI Model
  ↓
Cat       80%
Dog       15%
Rabbit     5%
  ↓
Cat is the most likely prediction

What Is a Distribution Telling Us?

A probability distribution helps answer three basic questions:

1. What outcomes are possible?

2. How likely is each outcome?

3. Where is most of the probability concentrated?

These questions are extremely useful when an AI system has uncertainty.

Connection to the Next Lesson

Probability distributions become even more useful when we understand how probabilities can be updated using evidence.

Probability Basics
        ↓
Understand probability

Conditional Probability
        ↓
Probability given information

Probability Distributions
        ↓
Probability across many outcomes

Bayes' Theorem
        ↓
Update probability using evidence

This is why probability distributions are an important foundation for understanding Bayes' Theorem and many machine learning concepts.