Validation Data

Why Do We Need Validation Data?

Suppose you train a house-price model.

After training, you discover that you have several choices:

• Model A.
• Model B.
• Model C.

You need a fair way to compare these choices using data that was not used to fit the model.

That is where validation data becomes useful.

Training vs Validation

A Simple Example

Imagine that we have 10,000 house samples.

We might divide them like this:

The exact percentages are not universal rules. They depend on the problem and the amount of available data.

Compare Two Models

Suppose we train two different models using the same training data.

Based on this validation result, Model B looks like the better choice.

The important point is that we did not choose the model based only on its training performance.

Validation Helps With Model Choices

Machine Learning often involves making decisions during development.

For example:

• Which model should we use?
• Which settings should we choose?
• Which approach performs better?
• Should we change the model?

Validation data gives us evidence that can help answer these questions.

Don't Train on the Validation Data

The purpose of validation data is to provide an independent check during development.

If you train the model directly on the validation examples, you weaken the meaning of the validation result.

Validation Is Not the Final Test

This distinction is extremely important.

If we repeatedly compare models using the validation data, we are making decisions based on that data.

Therefore, the validation data is no longer a completely untouched final evaluation.

The test set should remain separate until the final evaluation.

A Real-World Analogy

Think about preparing for a driving test.

Validation Data in Python

Later, we may explicitly create training and validation sets using tools such as scikit-learn.

from sklearn.model_selection import train_test_split

X_train, X_validation, y_train, y_validation = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)

The exact implementation can change depending on the project. The important idea is that the validation examples are kept separate from the data used to fit the model.

Validation Helps Us Improve

Imagine we try three different approaches.

Based on the validation results, Model B currently looks like the strongest candidate.

We can then take that development process forward before performing the final test.

The Biggest Mistake to Avoid

A common mistake is repeatedly checking the test set and changing the model based on the result.

For example:

The test set is gradually becoming part of the model development process.

A better approach is to use validation data for those development decisions and save the test set for the final evaluation.

Which Dataset Should You Use?

You trained two house-price models and want to decide which one performs better before the final evaluation.

Should you use the training data, validation data, or test data to make this development decision?