Choosing the Number of Clusters
K-Means needs us to choose K before it creates clusters. The challenge is deciding whether we should use 2, 3, 4, or another number of clusters.
Don't choose K blindly.
We can try different values of K and compare how well the data is grouped. One common technique for doing this is called the Elbow Method.
Why Does K Matter?
Remember that K tells K-Means how many clusters to create.
The algorithm creates two clusters.
The algorithm creates three clusters.
The algorithm creates four clusters.
Choosing a different K can produce completely different groups.
What Happens If K Is Too Small?
Suppose the data naturally contains several different groups, but we choose:
K-Means is forced to put everything into only two groups.
The clusters can become too broad and may not represent the structure of the data well.
What Happens If K Is Too Large?
Now imagine choosing a very large number of clusters.
K-Means can create many small groups.
Instead of finding useful broad patterns, we may create too many tiny clusters.
Try Different Values of K
Instead of guessing, we can test several values.
For each value, we measure how tightly the data points are grouped around their cluster centers.
K-Means provides a value called inertia, also known as within-cluster sum of squares (WCSS).
What Is Inertia?
Inertia measures how far data points are from the centroid of their cluster, using the sum of squared distances.
In simple terms:
As we increase K, inertia normally decreases because more clusters give the algorithm more centers to work with.
The Elbow Method
The Elbow Method is a common technique for choosing K.
We plot:
A typical graph looks something like this:
The curve drops quickly at first and then starts to flatten. The point where the improvement noticeably slows down is called the elbow.
Simple Example
Suppose we test these values:
Notice the improvements:
This does not mean K = 3 is automatically "correct." It means the Elbow Method gives us a useful reason to consider K = 3.
Python Example
We can test several values of K using Python.
from sklearn.cluster import KMeans
inertias = []
for k in range(1, 7):
model = KMeans(
n_clusters=k,
random_state=42
)
model.fit(X)
inertias.append(model.inertia_)
print(inertias)
Here we test:
The value:
gives the inertia for that particular K.
We can then plot the K values against their inertia values and look for the elbow.
Don't Automatically Choose the Lowest Inertia
This is an important point.
Suppose we have:
K = 5 has lower inertia than K = 3.
But that doesn't automatically mean K = 5 is better.
Adding more clusters will generally make the points closer to their assigned centers, so inertia tends to keep decreasing.
Choose K by looking for a useful balance.
Try several values of K, calculate the inertia for each, and use the Elbow Method to find where adding more clusters starts giving much smaller improvements.