Build KNN With Python
Now let's build a real KNN classification model in Python. We will use scikit-learn to train the model, give it a new data point, and make a prediction.
Give KNN examples, choose K, then ask it to classify a new example.
Python and scikit-learn handle the distance calculations and neighbor selection for us. We mainly need to prepare the data, choose K, train the model, and make a prediction.
What Are We Building?
We will build a simple model that predicts whether a student will Pass or Fail.
We will use two features:
Number of hours the student studies.
Student attendance percentage.
Step 1 — Import KNN
First, we import the KNN classifier from scikit-learn.
from sklearn.neighbors import KNeighborsClassifier
It imports KNeighborsClassifier, the
scikit-learn class we will use to create our KNN
classification model.
Step 2 — Create the Training Data
Next, we create some example students. Each row contains the student's features.
X = [
[2, 60],
[3, 65],
[4, 70],
[5, 80],
[6, 85],
[7, 90]
]Each row represents one student.
Notice that X contains only the
features. It does not contain the
Pass/Fail answer.
Step 3 — Create the Labels
Now we tell the model the correct class for each training example.
y = [
"Fail",
"Fail",
"Fail",
"Pass",
"Pass",
"Pass"
]
The first value belongs to the first row in
X, the second value belongs to the second
row, and so on.
Step 4 — Choose K
Now we decide how many neighbors KNN should look at.
For this simple example, let's choose:
Use the 3 nearest students.
model = KNeighborsClassifier(n_neighbors=3)It tells KNN to use the three closest training examples when making a classification.
Step 5 — Train the Model
Now we give the training features and labels to the model.
model.fit(X, y)
This connects the feature values in X
with their known labels in y.
Study hours + attendance
Pass + Fail
Ready to classify new data.
With KNN, this does not mean that the algorithm creates a complicated equation. It keeps the training examples available so it can compare new points with them.
Step 6 — Give the Model a New Student
Now suppose a new student has:
5 study hours and 82% attendance.
We don't know whether this student will Pass or Fail. That is what the model needs to predict.
new_student = [[5, 82]]Step 7 — Make the Prediction
We use predict() to ask the model for
the class of the new student.
prediction = model.predict(new_student)
print(prediction)The output will be:
['Pass']The three nearest training examples produce a majority Pass vote.
What Happened Behind predict()?
Although Python gives us the answer with one line, KNN is conceptually doing the same process you learned on the previous pages.
Complete Python Code
Now put all the pieces together.
from sklearn.neighbors import KNeighborsClassifier
# Training features
X = [
[2, 60],
[3, 65],
[4, 70],
[5, 80],
[6, 85],
[7, 90]
]
# Training labels
y = [
"Fail",
"Fail",
"Fail",
"Pass",
"Pass",
"Pass"
]
# Create the KNN model
model = KNeighborsClassifier(n_neighbors=3)
# Train the model
model.fit(X, y)
# New student
new_student = [[5, 82]]
# Make prediction
prediction = model.predict(new_student)
print(prediction)['Pass']Understand the Five Important Lines
You don't need to memorize the entire program. Understand these five lines first.
KNeighborsClassifier(...)
Creates the KNN classifier.
n_neighbors=3
Sets K to 3.
model.fit(X, y)
Gives the model the training data.
model.predict(...)
Asks the model to classify new data.
print(prediction)
Displays the prediction.
A Second Small Example
The same idea can be used for something completely different.
Suppose we want to classify fruits as Apple or Orange using weight and size.
from sklearn.neighbors import KNeighborsClassifier
X = [
[150, 7],
[160, 7],
[170, 8],
[180, 8],
[190, 9]
]
y = [
"Apple",
"Apple",
"Apple",
"Orange",
"Orange"
]
model = KNeighborsClassifier(n_neighbors=3)
model.fit(X, y)
new_fruit = [[175, 8]]
prediction = model.predict(new_fruit)
print(prediction)Only the features and labels changed.
Building KNN in Python is mostly about preparing the data and using the KNN classifier correctly.
Put your features in X, your labels in y, choose K, create the classifier, fit it with the training data, and use predict() for a new data point.