What is a Validation Curve?
A validation curve answers: "How does model performance change as I vary one hyperparameter?"
In Step 0 you wrote a manual loop over max_depth values. validation_curve() does the same thing but:
| Advantage | Why it matters |
|---|---|
| Uses cross-validation | Each depth is evaluated with K-fold, not a single split |
| Returns both curves | Training and validation scores for every parameter value |
| Handles the loop | You specify parameter name and range — it does the rest |
from sklearn.model_selection import validation_curve
from sklearn.tree import DecisionTreeClassifier
import numpy as np
param_range = np.arange(1, 21)
train_scores, val_scores = validation_curve(
DecisionTreeClassifier(random_state=42),
X_train, y_train,
param_name='max_depth',
param_range=param_range,
cv=5,
scoring='accuracy'
)
# Shapes: both (20, 5) — 20 depths x 5 foldsReading the Three Regions
| max_depth | Train score | Val score | Diagnosis |
|---|---|---|---|
| 1–2 (shallow) | Low | Low | High bias — model too simple (underfitting) |
| 3–5 (moderate) | High | Peaks here | Best complexity — choose this depth |
| 10–20 (deep) | 1.000 | Drops | High variance — memorising noise (overfitting) |
The validation curve is a complete diagnostic: it shows the underfit zone, the sweet spot, and the overfit zone in a single plot.
Plotting with Standard Deviation Bands
import matplotlib.pyplot as plt
train_mean = train_scores.mean(axis=1)
train_std = train_scores.std(axis=1)
val_mean = val_scores.mean(axis=1)
val_std = val_scores.std(axis=1)
plt.figure(figsize=(10, 6))
plt.plot(param_range, train_mean, 'b-', label='Training')
plt.fill_between(param_range,
train_mean - train_std,
train_mean + train_std, alpha=0.15, color='blue')
plt.plot(param_range, val_mean, 'r--', label='Validation')
plt.fill_between(param_range,
val_mean - val_std,
val_mean + val_std, alpha=0.15, color='red')
plt.xlabel('max_depth')
plt.ylabel('Accuracy')
plt.title('Validation Curve')
plt.legend()
plt.show()The shaded bands show the standard deviation across folds. Wide bands mean the model's performance is unstable — another sign of overfitting.
Picking the Optimal Parameter
# Find the depth with highest mean validation score
best_idx = np.argmax(val_mean)
best_depth = param_range[best_idx]
print(f"Optimal max_depth: {best_depth}")
print(f"Val accuracy: {val_mean[best_idx]:.4f} +/- {val_std[best_idx]:.4f}")
print(f"Train accuracy: {train_mean[best_idx]:.4f}")
print(f"Gap: {train_mean[best_idx] - val_mean[best_idx]:.4f}")This is the evidence-based way to select hyperparameters. No guessing, no "more is better" — the data tells you the optimal complexity.
Think Deeper
The validation curve shows that max_depth=5 gives the best validation score. Your manager asks you to use max_depth=15 because 'more is better.' How do you respond?