The first chart to learn is predicted against actual

I was practicing machine learning on my own computer in VS Code. I started with a script and then switched to Jupyter when I saw it would be easier cell by cell. The first figure that came out of the notebook was a parity plot, which is also called predicted against actual.

It is a simple chart. Each run is one point, with the measured value on the horizontal axis and the value the model predicted for that run on the vertical axis. Then you draw the 45 degree line. If the model predicted every run exactly, every point would sit on that line. It is the same chart you would draw to compare a flow meter against a reference, with the model in place of the meter.

import matplotlib.pyplot as plt

fig, ax = plt.subplots()
ax.scatter(y_measured, y_predicted)
ax.axline((0, 0), slope=1)   # the 45 degree line
ax.set_xlabel("measured")
ax.set_ylabel("predicted")
ax.set_aspect("equal")

axline draws a line through a point with a given slope, so you do not have to work out the axis limits first. The code above is a general version and not the cell from my notebook.

A score like R2 is one number for all the runs. On the plot you can see each run, so you can see which runs the model misses and in which direction. With these axes a point above the line is a run the model overpredicts. If the cloud of points is flatter than the line, the model is pulling everything toward the average, so the high runs are predicted low and the low runs high. If a group of points sits off the line together and they are all one feed or one reactor configuration, then I think the model has not learned that feed or that configuration. If there is one point far from the others, I would look that run up in the lab notebook first, before deciding anything about the model.

The predictions on the plot should come from runs the model was not fitted on. scikit-learn has its own version of the chart, PredictionErrorDisplay, and its documentation says to use it “preferably on held-out data points”. The axes are also not the same everywhere. With kind="actual_vs_predicted" scikit-learn puts the observed values on the vertical axis and the predicted on the horizontal, so above and below the line mean the opposite of what I wrote here.