Evaluate a Regression Model
A model can make predictions, but predictions alone are not enough. We need to measure how close those predictions are to the actual values.
Model evaluation tells us how good or bad the predictions are.
We compare the model's predicted values with the actual values and use evaluation metrics to measure the error or overall performance.
Actual Value vs Predicted Value
Suppose our model predicts exam scores for students. We compare what the model predicted with what actually happened.
The predictions are not exactly equal to the actual values. That difference is the prediction error.
What Is Prediction Error?
Prediction error is the difference between the actual value and the predicted value.
For example:
A smaller error generally means the prediction is closer to the actual value.
Why Do We Need Evaluation Metrics?
Looking at one prediction is not enough. A model usually makes many predictions.
For example:
We need a single measurement that summarizes how well the model performed across all these predictions.
Mean Absolute Error — MAE
MAE stands for Mean Absolute Error.
It calculates the average size of the prediction errors, ignoring whether the errors are positive or negative.
Suppose our errors are:
2
3
2
4First calculate the average:
This means the model's predictions are off by about 2.75 units on average.
Mean Squared Error — MSE
MSE stands for Mean Squared Error.
Instead of simply taking the absolute error, MSE squares each error before calculating the average.
Suppose the errors are:
2
3
2
4Square them:
2² = 4
3² = 9
2² = 4
4² = 16Because errors are squared, large errors receive much more weight.
R² Score
R² is called the R-squared score.
It gives us another way to understand how well the model explains the variation in the target values.
A simple way to think about it is:
For example, an R² value of:
The model explains a large portion of the variation in this particular dataset.
The model explains much less of the variation.
R² can also be negative when the model performs worse than the mean-prediction baseline.
MAE vs MSE vs R²
Lower is generally better. Easy to interpret.
Lower is generally better. Large errors matter more.
Higher is generally better, but context matters.
Evaluate the Model With Python
Scikit-learn provides functions for calculating these metrics.
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
actual = [50, 60, 70, 80, 90]
predicted = [48, 63, 68, 84, 87]
mae = mean_absolute_error(actual, predicted)
mse = mean_squared_error(actual, predicted)
r2 = r2_score(actual, predicted)
print("MAE:", mae)
print("MSE:", mse)
print("R²:", r2)The important idea is that we give the evaluation metric two things:
Evaluate a Trained Model
Usually, we don't manually create the predicted values. We get them from the trained model.
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
X = [[1], [2], [3], [4], [5]]
y = [45, 55, 65, 75, 85]
model = LinearRegression()
model.fit(X, y)
predicted = model.predict(X)
mae = mean_absolute_error(y, predicted)
mse = mean_squared_error(y, predicted)
r2 = r2_score(y, predicted)
print("MAE:", mae)
print("MSE:", mse)
print("R²:", r2)Here the process is:
A Very Important Mistake to Avoid
You should not judge a machine learning model only by evaluating it on the same data it used for training.
Why?
That is why machine learning uses separate validation and test data.
We covered this idea earlier in Training, Validation, and Test Data.
Don't just ask, "Can the model predict?" Ask, "How accurate are those predictions?"
MAE measures the average absolute error, MSE gives more weight to large errors, and R² describes how much variation the model explains relative to a mean-prediction baseline. For evaluating real model performance, use unseen validation or test data rather than relying only on training data.