What Are Features?
Features are the pieces of information that a Machine Learning model uses as input to make a prediction or decision.
Features are the information we give to the model so it can learn useful patterns.
If we want a model to predict house prices, features might include house size, number of bedrooms, location, and age.
Start With a Real-World Problem
Imagine that we want to build a Machine Learning model that predicts the price of a house.
We have many historical house records and each record contains different pieces of information.
These pieces of information can be used by the model to make a prediction.
These Inputs Are Features
In the example above, the house size, number of bedrooms, age, and location are all potential features.
Features describe the example that the model is looking at.
Another Example: Spam Detection
Suppose we want to build a model that predicts whether an email is spam.
The model could use information such as:
How many links appear in the email?
How long is the email?
Information about the sender.
Certain words or patterns found in the message.
Features Are the Inputs
A useful way to remember the idea is:
Information about the example.
Finds relationships in the data.
The result produced by the model.
One Row Can Contain Many Features
Consider one house record:
This single row represents one example, and each relevant input column can act as a feature.
Features Can Be Numbers
Many features are numerical.
square feet
bedrooms
years
Numerical features are often directly represented as numbers, although preprocessing may still be needed depending on the model and problem.
Features Can Also Be Categories
Not every feature is naturally a number.
For example, a house location might be:
Machine Learning algorithms usually require data in a numerical representation, so categorical information often needs to be encoded before it can be used by a model.
More Features Do Not Automatically Mean a Better Model
This is an important point.
You might think that adding every available column will automatically improve the model.
It won't.
Irrelevant, duplicated, noisy, or misleading information can make learning harder and can sometimes cause the model to perform worse.
House price prediction may benefit from house size, but a random internal record ID probably provides no useful information about price.
A spam classifier may benefit from message patterns, but an unrelated database identifier should not automatically be treated as a useful feature.
Features Should Be Available at Prediction Time
Another common mistake is using information that would not actually be available when the model makes its prediction.
"Will this information actually exist at the time I need the prediction?"
If you want to predict whether a customer will purchase tomorrow, using tomorrow's purchase information as a feature would be meaningless.
If you want to predict whether a transaction is fraudulent before approving it, using information that only becomes available after the transaction is investigated would create leakage.
Features in a Dataset
Suppose our dataset looks like this:
| Size | Bedrooms | Age | Location | Price |
|---|---|---|---|---|
| 1800 | 3 | 8 | City | $320,000 |
| 2200 | 4 | 5 | Suburb | $410,000 |
| 1400 | 2 | 15 | Town | $240,000 |
In this example, the first four columns can be used as features for predicting the house price.
Features vs the Thing We Want to Predict
Notice that the table contains another important column: Price.
Price is not an input feature in this example. It is the value we want the model to predict.
Size, bedrooms, age, location
Value we want the model to predict
The target is commonly called the label in supervised Machine Learning.
The Simple Mental Model
Information given to the model.
Patterns and relationships.
The value or category we want to predict.
Features Are the Model's Inputs
Features describe each example and provide the information from which a Machine Learning model can learn patterns. The value we want to predict is usually called the target or label.
Which Ones Are Features?
You want to predict whether a customer will purchase a product.
Your dataset contains:
- Customer age.
- Number of previous purchases.
- Product category.
- Whether the customer purchased the product.
Customer age, previous purchases, and product category can be features.
Whether the customer purchased the product is the target or label we want to predict.