Feature Selection
Feature selection means choosing the features that are useful for solving a machine learning problem and leaving out features that are unnecessary, irrelevant, or potentially harmful.
Keep useful information. Remove unnecessary information.
A dataset can contain many pieces of information, but not every piece of information helps the model make a better prediction.
Why Do We Need Feature Selection?
Imagine we want to predict the price of a house.
We might have the following information:
May have a strong relationship with house price.
Can provide useful information about the house.
Can strongly affect house prices.
Identifies the row but usually does not describe the value of the house.
The goal of feature selection is to focus the model on information that actually helps solve the problem.
A Simple Example
Suppose we want to predict a student's exam score.
6 hours
8 tests
58291
Study hours and practice tests could provide useful information for predicting an exam score.
Student ID identifies the student, but the number itself does not normally tell us how well the student will perform.
More Features Does Not Always Mean a Better Model
A common beginner mistake is to think:
That's not necessarily true.
Some features may contain irrelevant information. Adding unnecessary features can make the dataset more complicated without giving the model useful information.
Some may be useful, while others may be irrelevant or redundant.
The model focuses on information that is more relevant to the problem.
Feature Selection Is Based on the Problem
A feature is not automatically useful or useless. Its usefulness depends on what we are trying to predict.
For example, consider house location.
Location can be very useful.
Location may be less directly useful than other information.
So you should never ask only: "Is this feature good?"
Ask: "Is this feature useful for the prediction problem I am solving?"
Irrelevant Features
An irrelevant feature is information that has little or no useful relationship with the prediction we are trying to make.
For example, suppose we are predicting house prices.
Removing clearly irrelevant information can help keep the model's input cleaner and simpler.
Feature Selection Does Not Mean Guessing
We should not simply remove a feature because it looks unimportant to us.
In real machine learning projects, feature selection can be supported by data analysis, domain knowledge, model evaluation, and statistical or algorithmic techniques.
For this course, the important concept is:
A Complete Example
Suppose we want to predict whether a customer will buy a product.
The selected features become the information the model uses to learn and make predictions.
Feature Selection Means Choosing the Right Inputs.
A dataset can contain many features, but we should focus on features that provide useful information for the prediction problem. Unnecessary or irrelevant features can be removed when appropriate.
Which Features Would You Keep?
We want to predict a house's price.
House size and number of bedrooms are reasonable features to keep because they can provide information about house prices. A random record ID usually does not describe the house and would generally not be useful.