What Does the Machine Learn?
We have seen how Machine Learning can use examples to make a prediction. But there is an important question:
The machine is not learning facts in the same way a human learns a subject. It is learning useful patterns and relationships in the data that can help it produce an output for new inputs.
It Is Not Simply Memorizing the Examples
Let's return to our house price example.
Suppose the training data contains these houses:
Now suppose we give the system a new house:
The exact 1,350 sq ft house was not in the training examples.
A useful learning system should still be able to make a reasonable prediction.
The system needs to capture something about the relationship between the input and the output so that it can handle new examples.
The Machine Learns Patterns
A pattern is a relationship that appears repeatedly in the data.
In our simplified house example, we might observe:
This relationship is useful because it can help us estimate the price of a house that the system has not seen before.
A Pattern Is a Relationship Between Data
Machine Learning often tries to discover relationships between different pieces of information.
For example:
Larger houses may generally have higher prices.
More study time may be associated with better scores.
Certain combinations of email characteristics may be associated with spam.
Certain customer behaviors may be associated with purchasing a product.
What Goes Into the Learning Process?
The system needs examples.
Each example contains information that can help the algorithm discover a relationship.
1,200 sq ft
$500,000
The algorithm looks across many such examples instead of relying on just one house.
One Example Is Usually Not Enough
Suppose we only show the machine one house:
There is not enough information to understand the general relationship between size and price.
But if we provide many examples:
the algorithm has more information from which to identify a useful relationship.
What Changes During Learning?
This is where Machine Learning differs from simply running a normal program.
During training, the learning algorithm uses the examples to determine values that help describe the relationship in the data.
Those learned values are then used when making predictions.
Remember the Simple Math Example
Earlier, we manually found a simple relationship between house size and price.
We calculated that relationship ourselves.
In Machine Learning, an algorithm can use training data to determine the values that make the relationship useful for the problem.
The algorithm is trying to find values that allow the learned relationship to produce useful outputs for the examples and, ideally, for new data as well.
The Goal Is to Work on New Data
This is one of the most important ideas in Machine Learning.
A model that only performs well on the examples it has already seen is not very useful.
We want the learned relationship to also be useful when new data arrives.
What Does It Not Automatically Learn?
A Machine Learning system does not automatically understand everything about the real world.
If an important factor is missing from the data, the model cannot magically recover that information.
If location is not included in the house data, the model cannot use location directly.
If the training examples are unreliable, the learned relationship can also be unreliable.
The machine learns useful relationships from examples.
It does not simply memorize the training examples. The goal is to learn a relationship that can be used to produce useful outputs for new inputs.