Highly recommend Andrej Karparthy’s blog post on this subject.
He breaks it down into four phases:
- Be one with the data
- Set up an end-to-end training and evaluation pipeline + get dumb baselines
- Overfit
- Regularize
N.B.
Don’t forget to also agree to and lock in to a benchmark dataset for training and evaluation purposes.