Preparing Data

Why Does Data Need Preparation?

Imagine collecting information about houses to build a house-price prediction model.

Your raw data might look like this:

There are already problems here.

• One house has a missing size.
• One bedroom value is written as text.

Giving this raw data directly to a model may cause errors or lead to poor results.

Step 1: Find Missing Values

Real datasets often have information that is missing.

For example:

Depending on the situation, we might remove the row, fill the value using a reasonable strategy, or use another method.

The important idea is that we should not blindly send incomplete data into the model.

Step 2: Remove Duplicate Data

Sometimes the same record appears more than once.

For example, suppose a customer accidentally appears twice in a dataset:

If these are actually the same transaction, keeping both can give the model an inaccurate view of the data.

Duplicate records should therefore be identified and handled appropriately.

Step 3: Fix Incorrect Data

Data can also contain values that are entered incorrectly.

For example, suppose a dataset contains:

A value like 350 might not make sense if the field represents a person's age.

Data preparation involves identifying these kinds of problems and deciding how they should be handled.

Step 4: Handle Categorical Data

Some data is represented using words instead of numbers.

For example:

Many machine learning algorithms work with numerical representations rather than raw text categories.

Therefore, categorical information may need to be transformed into a suitable numerical representation.

Step 5: Deal With Very Different Scales

Different features can have very different numerical ranges.

For example:

In some machine learning algorithms, features with very different scales can affect how the model learns.

A technique called feature scaling can be used to bring numerical features onto a more suitable scale.

We will study feature scaling in more detail later.

A Simple Before-and-After Example

Let's put the ideas together.

Complete Example: House Price Data

Suppose we want to predict house prices.

We collect thousands of houses, but the data is messy.

Only after this preparation should we move toward training the machine learning model.

Do Not Change Data Without Understanding It

Data preparation does not mean deleting anything that looks unusual.

An unusual value may be a genuine real-world value, not an error.

For example, a house that costs $5 million might look unusual compared with most houses in a dataset. That does not automatically mean the value is wrong.

Is This Data Ready?

You are given a customer dataset containing 10,000 records.

Some records are duplicated, some ages are missing, and the city is stored as text such as "New York" and "Chicago".

Can you immediately give this raw dataset to your machine learning model?