There's no reason that finetuning just the last layer in this should create a good statefarm model. The imagenet model was created for a very different purpose (recognizing objects in images, vs identifying whether a driver is distracted). So I think to get a good result in this competition you'll need to take a somewhat different approach! Maybe you can experiment with a few different things on the sample and see if you can find something that looks promising...

Hi Chris,I replicated this process using a fine-tuned Vgg16 model and my prediction dimensions were (79726, 10) . If your final layer has an output of 10 then that is what you should be getting back. Try running it again, if you get the same problem copy and paste your notebook into https://gist.github.com/ and save it with an .ipynb extension so we can see exactly what you did.

That validation set won't work correctly - it'll give very (very!) optimistic results. The competition data page says: The train and test data are split on the drivers, such that one driver can only appear on either train or test set. That means you need to do the same thing for your validation set creation!

My model seems to get very poor results no matter what I do. The accuracy and validation accuracy seems to float around 10% and never increases. This is even more obvious when I look at the confusion matrix and it's guessed that every image was of a single category.

I have moved 3 different drivers into my validation set thinking that was what Jeremy has been hinting at. I have set the layers from the first dense layer onwards to trainable since this data set isn't predicting categories from imagenet.

I assume there is something fundamentally flawed with my model that it is only predicting 1 category. I have been considering using the techniques (data augmentation, batch normalization, dropout) but I am under the understanding that these techniques only help once you have a relatively good model. Is there something simple I'm missing?

@brianorwhatever the challenge of getting a model to train at all on this dataset is part of what makes it such a good learning experience. So the process you're going thru will hopefully be very rewarding

It sounds to me like you're on the right track. I'd suggest trying a variety of learning rates. Perhaps your learning rate is too high. I would suggest using batch normalization if you can - it makes initial learning much easier.

Brian I'm with you. My model is perfect..... at classifying everything into the first class! I'm using a sample training size of 2000 and a validation set of 500 (drivers not in training set). Learning rate of .01. I popped the final VGG layer and added my new dense. I even tried going back a few layers but still everything in the first cat. Even though this is a small training set I figured I could at least get to 11% .. then I wouldn't feel like this was a bug on my part.