Validation Data
Training data teaches the model. Validation data helps us check the model while we are developing it and helps us decide how to improve or configure it.
Validation data helps us make decisions about the model.
We do not use validation data to teach the model its main patterns. Instead, we use it during development to compare different choices and see which version of the model performs better on data it did not train on.
Why Do We Need Validation Data?
Suppose you train a model and its predictions are not good enough.
You may want to change something about the model. For example, you might try a different algorithm or change one of its settings.
You need a consistent way to compare these choices. Validation data provides that checkpoint during model development.
Simple Example: House Price Prediction
Imagine that you have 10,000 house records.
You use some of them to train your model. You keep another portion aside as validation data.
The model learns relationships between house features and prices.
We use it to compare and improve model choices.
The model learns from the training data, while the validation data helps us decide whether the current version of the model is performing better or worse.
Validation Data Is Like a Practice Exam
A simple analogy is a student preparing for an exam.
The student learns the material.
The student checks how well they are doing and decides what needs improvement.
The student can use the practice exam results to change their study approach before the final exam.
Similarly, validation results can help us make decisions about the machine learning model before the final evaluation.
Validation Helps Compare Models
Suppose we are trying two different models for predicting house prices.
Average prediction error is higher.
Average prediction error is lower.
Based on the validation results, Model B may be the better choice for further development.
The important point is that the validation data gives us information we can use to make this decision.
Validation Helps Choose Model Settings
Machine learning models often have settings that we can choose before training. These are commonly called hyperparameters.
We can try different settings and use validation results to help choose a good combination.
Validation accuracy: 88%
Validation accuracy: 92%
Validation accuracy: 95%
We could choose the setting that performs best according to the validation results and continue developing the model with that choice.
Hyperparameter tuning will be covered in much more detail later in the course.
Training Data vs Validation Data
The simplest way to remember the difference is to ask: What is this data being used for?
The model learns patterns from these examples.
We use the results to compare models and make development decisions.
Training data directly participates in learning. Validation data helps us judge different choices while developing the model.
A Common Mistake
One mistake is to think that validation data is simply another batch of training data.
It has a different purpose.
The model should not simply learn from the validation examples as if they were normal training examples. Otherwise, the validation data loses its usefulness as an independent development check.
Training Data Teaches. Validation Data Helps You Decide.
Training data is used to learn patterns. Validation data is used during development to compare models, test choices, and help improve the model before the final evaluation.
Which Data Should You Use?
You trained two different house-price models and want to decide which one performs better during development.
The validation data can be used to compare the two models and help decide which model or configuration should be developed further.
Validation is not the final exam.
Validation data helps us make decisions while developing the model. We still need separate test data for a final, more independent evaluation. That is the next topic.