Finally got a good model without overfitting!

CNN. Overfit, so next time add dropout.

Uses 40% dropout from each layer. This drops the accuracy significantly.

The accuracy is good but the difference with the validation accuracy suggests overfitting. Adding more complexity, layers, and weights isn't going to improve the model - we need to fix the over-fitting. This model is also slower to run.

Data normalized. The validation accuracy drops below the accuracy, which could be a sign of overfitting

Used one hot encoding

Changed function to sigmoid and epoch to 6. But data set is imbalanced and always guessing "not 5" would give us 90%. So that's what this model learned to "improve"