What this notebook does
Start with explicit reference labels and invented model scores. Apply a threshold, count the four kinds of outcome, and calculate accuracy, precision, recall, F1, and a weighted error cost.
The data matches the interactive threshold lab, so you can compare the browser results with your own calculations.
Requirements
- Python 3.10 or later.
- Jupyter Notebook, JupyterLab, or Google Colab.
- No third-party Python packages are required by the notebook.
Open the downloaded .ipynb file in Jupyter, or upload it to Colab. Run the cells from top to bottom.
Things to explore
- Compare thresholds of 0.50 and 0.30.
- Hold the threshold fixed and change the relative cost of a missed positive.
- Try 1.00: why is precision undefined when there are no positive predictions?
- Add an invented example and recompute the counts.
Scope
This is a standalone educational resource. The scores and labels are synthetic, and the notebook does not train a model. It is separate from the full course notebooks.
A standalone resource by Ackley Will. About the author · Share a question or observation