How KNN Works
KNN makes a prediction by finding the data points closest to a new example and using their labels to decide the answer.
KNN finds the closest examples, checks their labels, and lets them vote.
The process is simple: give KNN a new data point, find its nearest neighbors, look at their classes, and use the majority class as the prediction.
The KNN Process
When KNN receives a new data point, it follows a sequence of steps to make its prediction.
Give KNN a data point that needs a prediction.
Find the known points closest to the new point.
Look at the classes of those neighbors.
Count how many neighbors belong to each class.
The majority class becomes the prediction.
Step 1 — Start With a New Data Point
First, we have a new data point whose class is unknown.
Let's use a student example with two features:
We don't know yet whether this student will Pass or Fail.
Step 2 — Look at the Known Data
KNN already has examples where the correct class is known.
KNN compares the new student with these known examples to find which ones are closest.
Step 3 — Find the Nearest Neighbors
Now KNN determines which known students are closest to the new student.
These examples are closer to the new student than the students with much lower study hours and attendance.
Step 4 — Choose K Neighbors
K tells KNN how many nearby examples it should use.
For this example, let's choose:
That means KNN will use the 3 closest students to make the prediction.
Step 5 — Check Their Labels
Now KNN looks at the known labels of the three nearest neighbors.
5 hours / 85%
6 hours / 90%
7 hours / 95%
All three nearest neighbors are labeled Pass.
Step 6 — Let the Neighbors Vote
For classification, the nearest neighbors can vote on the class of the new data point.
Pass has the majority of votes, so KNN predicts:
The Same Process With Mixed Neighbors
The neighbors do not always have the same label. That is where majority voting becomes important.
Suppose K is still 3, but the closest neighbors are:
Count the labels:
Pass wins because it has more votes than Fail.
Why Does "Nearest" Matter?
Imagine asking ten random students whether a new student will pass. Their answers may not tell us much.
But if we look at students who are similar to the new student, their outcomes can be much more useful.
Their feature values are similar to the new example.
Their feature values are less similar to the new example.
KNN in One Example
Let's put everything together.
KNN Does Not Use All Data Equally
This is an important idea.
KNN has access to many training examples, but for a particular prediction it focuses on the nearest examples rather than treating every point as equally relevant.
One Important Detail: Distance
KNN needs a way to decide which points are close and which points are far away.
It does this using a distance measure.
We will look at exactly how this distance is calculated in the next part of the lesson: Distance Between Data Points.
KNN = Find → Select → Check → Vote → Predict
KNN finds the closest known examples, selects the requested number of neighbors, checks their labels, and uses the majority class to make the prediction.