Standard Deviation
Standard deviation tells us how far the values in a dataset usually are from the mean. It is one of the most useful ways to understand how spread out data is.
Standard deviation tells us how much the data varies around the mean.
A small standard deviation means the values are close to the mean. A large standard deviation means the values are more spread out.
What Is Standard Deviation?
Standard deviation is a measurement of how spread out numbers are around their average.
For example:
Dataset A 48, 49, 50, 51, 52
These values are very close to the mean. Therefore, the standard deviation is small.
Dataset B 10, 30, 50, 70, 90
These values are much farther from the mean. Therefore, the standard deviation is larger.
Why Do We Need Standard Deviation?
The mean tells us where the center of the data is. But the mean does not tell us how spread out the data is.
Consider these two datasets:
Dataset A 48, 49, 50, 51, 52 Mean = 50
Dataset B 10, 30, 50, 70, 90 Mean = 50
Both datasets have exactly the same mean. But their spread is completely different.
Standard deviation helps us describe that spread.
Simple Example
Let's use a small dataset:
2, 4, 6, 8, 10
First, calculate the mean:
Mean = (2 + 4 + 6 + 8 + 10) / 5
= 30 / 5
= 6
So:
Mean = 6
Find the Distance From the Mean
Next, we calculate how far each number is from the mean.
Number Difference 2 2 - 6 = -4 4 4 - 6 = -2 6 6 - 6 = 0 8 8 - 6 = 2 10 10 - 6 = 4
These values show the distance of each number from the mean.
Square the Differences
Some differences are negative and some are positive. We square them so that they do not cancel each other out.
(-4)² = 16 (-2)² = 4 ( 0)² = 0 ( 2)² = 4 ( 4)² = 16
So the squared differences are:
16, 4, 0, 4, 16
Calculate the Variance
The average of the squared differences gives us the variance.
Variance = (16 + 4 + 0 + 4 + 16) / 5
= 40 / 5
= 8
Therefore:
Variance = 8
From Variance to Standard Deviation
Standard deviation is the square root of variance.
Variance = 8
Standard Deviation = √8
≈ 2.83
Therefore:
Standard Deviation ≈ 2.83
This means the values are typically around 2.83 units away from the mean.
Why Take the Square Root?
This is an important reason why standard deviation is often easier to understand than variance.
Remember that variance uses squared differences. That means its unit is also squared.
Original data: Age → years Variance: years² Standard deviation: years
Taking the square root brings the measurement back to the original unit of the data.
That makes standard deviation easier to interpret.
Small vs Large Standard Deviation
The simplest way to understand standard deviation is to compare how close the values are to their mean.
Small Standard Deviation 9, 10, 10, 10, 11 Values are close together → Small spread → Small standard deviation
Large Standard Deviation 1, 5, 10, 15, 20 Values are more spread out → Large spread → Larger standard deviation
Why Is Standard Deviation Used in AI?
AI models work with numerical data. Different features can have very different amounts of spread.
For example, imagine a machine learning model using:
Age 25, 26, 27, 28, 29 Income 20,000 50,000 80,000 1,50,000 3,00,000
Age has relatively little variation, while income has much greater variation.
Standard deviation helps us understand this variation numerically.
Standard Deviation and Feature Scaling
Standard deviation is also important when preparing data for machine learning.
One common technique is called standardization. It uses the mean and standard deviation to transform values.
z = (x - mean) / standard deviation
This helps put numerical features onto a more comparable scale.
For example, age might range from 18 to 80 while income might range from 20,000 to 500,000. Standardization can make these features easier for some machine learning algorithms to work with.
Standard Deviation in Python
Python provides functions for calculating standard deviation.
from statistics import pstdev numbers = [2, 4, 6, 8, 10] result = pstdev(numbers) print(result)
Output:
2.8284271247461903
This is approximately:
2.83
We are using population standard deviation here because the example treats all five values as the complete dataset.
The Complete Process
The entire calculation can be remembered as a simple flow:
Original Data
↓
Find the Mean
↓
Find Difference From Mean
↓
Square Each Difference
↓
Find Average
↓
Variance
↓
Square Root
↓
Standard Deviation
The important connection is:
Variance = Average of squared differences Standard Deviation = √Variance
Mean vs Variance vs Standard Deviation
These three concepts answer different questions.
Mean "Where is the center?" Example: 10, 20, 30 Mean = 20
Variance "How much is the data spread out?" It measures the average squared distance from the mean.
Standard Deviation "How far are the values typically from the mean?" It is the square root of variance.
A Real-World Example
Imagine two students take several tests.
Student A 78, 79, 80, 81, 82 Average = 80 Scores are very consistent.
Student B 50, 65, 80, 95, 110 Average = 80 Scores are much more spread out.
Both students have the same average, but Student B has a much larger spread.
Therefore, Student B will have a larger standard deviation.
This is why looking only at the mean can be misleading. Standard deviation gives us additional information about the behavior of the data.
Standard deviation tells us how spread out the data is around the mean.
First find the mean, calculate the differences from the mean, square those differences, calculate the variance, and then take the square root. A small standard deviation means the values are close to the mean, while a large standard deviation means the values are more spread out. In AI, this concept is especially useful for understanding data variation and preparing numerical features for machine learning.