How Decision Trees Make Decisions

Start With a Question

A Decision Tree starts with a question about the input data.

For our student example, the tree might ask:

The answer can be either Yes or No.

Follow the Correct Branch

Once the question is answered, the model follows the branch that matches the answer.

Suppose the new student studied for 6 hours.

Study Hours = 6

Is 6 > 3?

Yes

Because the answer is Yes, the tree follows the Yes branch.

What If the First Question Is Not Enough?

Sometimes one question cannot separate all the data. In that case, the tree can ask another question.

Imagine that the tree first asks:

Now suppose we also use attendance.

The tree can therefore make a decision in stages.

A Complete Example

Let's follow a complete decision from beginning to end.

A student has:

The tree asks its first question:

Is Study Hours > 3?

5 > 3

Yes

The tree follows the Yes branch. It now asks the next question.

Is Attendance > 80%?

90 > 80

Yes

The tree reaches the final prediction:

The Tree Does Not Ask Every Question

An important point is that the model does not necessarily follow every branch in the tree.

It only follows the path that matches the input.

How Does the Tree Know Which Question to Ask?

This is where Machine Learning becomes important. We do not normally write every decision by hand.

During training, the Decision Tree examines the training data and looks for splits that separate the classes effectively.

The exact way the tree chooses the best split will be explained later when we study Splitting Data.

Decision Tree Prediction in Simple Words

You can think of prediction as walking through the tree.

Another Real-World Example

Imagine a bank wants to classify a transaction as Normal or Suspicious.

A transaction does not automatically become suspicious just because it is expensive. The tree can use additional information before making its final decision.

Decision Tree vs a Normal If-Else Program

At first, a Decision Tree may look exactly like ordinary if and else code.

For example, we could manually write:

if study_hours > 3:
    prediction = "Pass"
else:
    prediction = "Fail"

This is a normal programming rule.

The important difference is that a Machine Learning Decision Tree can learn useful decision rules from training data instead of requiring us to manually write every rule.

Check Your Understanding