Build a Decision Tree
Sort a basket of fruit using features you can see. Compare splits and discover why depth matters.
Which question should the tree ask first?
A useful split makes the child groups more homogeneous. Compare candidate splits using weighted impurity, then decide how much complexity the evidence can support.
Make a prediction.
Then move a control.
Apple, orange, or banana?
Use color, shape, and size to classify 15 training fruits. Then evaluate on 9 separate examples. Every fruit and measurement is invented.
Increase the maximum depth to 3. Does a perfect training score improve the test result?
The learned tree
Follow a feature, then a branch.Why choose that first feature?
Compare the impurity before and after grouping by each feature. The reduction is the gain: larger means purer groups. This lab uses one branch for each categorical value.
| Feature | Before | Weighted after | Gain | Eligible? |
|---|
Inspect all training and test fruits
The tree sees the “Observed label” column for training. Test references are held separate from fitting. These 9 examples illustrate a mechanism; they are too small to establish general performance.
| Set | Fruit | Color | Shape | Size | Reference | Observed label | Prediction |
|---|
What evidence would justify a more complex tree? Compare depth 2 and 3, then remove the labeling error. Does every extra split help?
Illustrative categorical tree. This is a transparent teaching model, not an implementation of scikit-learn’s binary CART tree. Empty or unseen categories fall back to the node’s majority class; tied class counts use alphabetical order.
An educational experiment with illustrative data. The controls run locally in your browser.
The idea,
in Python.
A compact implementation of the core concept. Open the code when you are ready to connect the visual to the algorithm.
Open the core implementation Python ↗
from collections import Counter, defaultdict
# Illustrative categorical fruit examples: the lab uses multiway splits.
fruits = [
{"label": "apple", "color": "red", "shape": "round"},
{"label": "apple", "color": "green", "shape": "round"},
{"label": "orange", "color": "orange", "shape": "round"},
{"label": "orange", "color": "yellow", "shape": "round"},
{"label": "banana", "color": "yellow", "shape": "long"},
{"label": "banana", "color": "green", "shape": "long"},
]
def gini(rows):
return 1 - sum((n / len(rows)) ** 2
for n in Counter(r["label"] for r in rows).values())
for feature in ["color", "shape"]:
groups = defaultdict(list)
for row in fruits:
groups[row[feature]].append(row)
weighted = sum(len(g) / len(fruits) * gini(g) for g in groups.values())
print(feature, "weighted impurity:", weighted,
"reduction:", gini(fruits) - weighted)Explain the change.
Test your reasoning.
- Predict the best first split before revealing the impurity values.
- Change the feature used at the root. Which fruits remain mixed?
- Increase depth. Distinguish a clean training split from evidence of generalization.