Overfitting: train vs validation
Select an architecture and watch how training accuracy and validation accuracy diverge. The gap between them is the overfitting signal.
Accuracy curves
Overfit gap (train - val)
How to fight overfitting
| Technique | How it works | When to use |
|---|---|---|
| Early stopping | Stop training when val accuracy peaks | Always — cheapest fix |
| Dropout | Randomly zero out neurons during training | Large networks |
| Reduce capacity | Fewer neurons/layers | When params >> samples |
| More data | Collect more training examples | Best long-term solution |
Loading...
Loading...
Loading...
Think Deeper
Try this:
At what epoch does the validation accuracy peak for the medium network? What should you do about it?
Check the chart — val accuracy typically peaks around epoch 20-30 then plateaus or drops. You should stop training at the peak. This is called early stopping — a key regularisation technique.
Cybersecurity tie-in: An overfit intrusion detector memorises specific attack patterns from training
instead of learning general attack behaviour. It catches known attacks perfectly but misses new variants.
Early stopping and validation are essential in security ML.