A Simple Dataset Split
We now know why training, validation, and test data have different jobs. The next step is to see how one dataset can actually be divided into these three parts.
One dataset can be divided into separate portions for different purposes.
A common simple example is to divide the data into training, validation, and test sets. For example, we might use 80% for training, 10% for validation, and 10% for testing.
Start With One Dataset
Imagine that we have collected information about 10,000 houses.
Each house has information such as its size, number of bedrooms, location, and actual selling price.
All the data we collected for our machine learning problem.
We do not want to use all 10,000 records for training. We need to keep some records aside so we can evaluate the model later.
Split the Dataset
For this example, let's use a simple 80 / 10 / 10 split.
Used to teach the model.
Used during development to compare and improve the model.
Kept aside for the final evaluation.
Notice that the three portions together still contain all 10,000 records.
What Happens to Each Part?
Each portion now has a different job.
The model learns patterns from these examples.
We compare model choices and make development decisions.
We use these examples for the final evaluation.
Why Not Use 90% for Training?
You might wonder why we don't simply give the model more training data.
More training data can be useful, but we still need separate data to help us make development decisions and evaluate the finished model.
We need to balance these purposes rather than giving everything to training.
The Split Is Not Always 80 / 10 / 10
The 80 / 10 / 10 example is useful for understanding the idea, but it is not a universal rule.
The appropriate split depends on things such as the amount of available data, the type of problem, and the machine learning workflow being used.
Understand the purpose of each portion first. The actual proportions can change depending on the project.
Another Simple Example
Imagine we have 1,000 customer records and want to build a model that predicts whether a customer will buy a product.
Learn patterns from previous customer behavior.
Compare and improve model choices.
Perform the final evaluation.
The numbers are different from the house example, but the idea is exactly the same.
The Important Part Is the Purpose
Don't focus too much on the exact percentage. Focus on why each portion exists.
What Happens After the Split?
Once the data has been separated, we can begin the machine learning process.
This gives each portion of the dataset a clear role instead of using the same examples for everything.
Split the Data Based on Purpose.
A simple example is 80% training, 10% validation, and 10% test. The exact percentages can change, but the important idea is that different data is reserved for learning, development decisions, and final evaluation.
Let's Split 10,000 Records
Suppose you have 10,000 customer records and use an 80 / 10 / 10 split.
How many records go into training?
80% of 10,000 = 8,000.
How many records go into validation?
10% of 10,000 = 1,000.
How many records go into testing?
10% of 10,000 = 1,000.