Day #97: Neural network models

Today should be an interesting day for me. I plan to do a deep dive into the world of neural networks to better understand how they work and the best type to apply to my data. There are a few, quite a few in fact that I could use and really it depends on how I want to frame my problem, so let’s look at what I’ve got planned.

Artificial neural nets have just recently come into vogue. If neural nets were a hammer, everything would be a nail. This doesn’t mean that they don’t have a wide range of uses, it’s just not always the best solution to every problem. It’s the same issue that blockchain has, everything can be solved with blockchain (spoiler, no it can’t).

Like any good tool, there is a right way to use it and a wrong way to use it. Unfortunately, if you don’t know a whole lot about the tool or how to use it you might start applying it to things that don’t make sense. I mean sure I’ve used the random screwdriver handle as a hammer before (I say, who amongst us hasn’t?), but that doesn’t mean I don’t understand how the tool works, I’m just getting creative with the use case.

That would be a good analogy for neural nets, if you understand how they work, you can adapt them to problems that might have been solved some other way, but maybe this way was easier (or you’re too lazy to try and find a hammer). In any case, the point is this, if you’re going to properly use a tool you need to understand it. If you want to modify the use case of that tool, you really need to understand not only how the tool works, but how you can change it to fit your case.

I’m not a neural nets person. Yes, I’ve been tinkering with them the past few weeks, but it’s just that, tinkering. Now, it is time to get serious and instead of applying a nicely packaged solution for an adjacent problem to my dataset, I need to either modify the tool, or create a new one. Thankfully there are tools online to help me get started.

I’m a big python kind of guy, but I’ve been using a lot of MATLAB lately, so that will be my toolbox of choice for this problem. I’ve been using the neural net UI, which is nice, but not perfect. I’ve also found out that it trains my neural net using randomly subsampled data, this is not okay and I will explain why.

If you recall from previous posts, I’ve been training models and testing them. I would get 99.9% accuracy, but crap outputs when I would feed in my test data the model has never seen. I originally attributed this to overfitting, but after reading the documentation more thoroughly, I’ve learned that the default is to randomly sample my data to create the validation set (the set my model uses when training to avoid overfitting). This wouldn’t be a problem for data that didn’t have a temporal aspect to it (timing is important here), however my data is highly correlated so as an example it would be like trying to train a model to write text by pulling out random letters from an example text and expecting it to have some meaning.

This discovery meant it was time to step away from the UI (no!!!!) and start using the command line. Not a huge deal, but I really just wanted an out of the box solution to my problem, was that too much to ask for?

This also means it’s time to read up on how to build my own neural net. In this case a NARX feedback neural network. Matlab has some great documentation on it and the network looks something like this…

Basically I’ve got a lot of work ahead of me, but I’ve got a promising start and with my qualifying exam date set, I have a few weeks to get ready. You’ll probably be seeing me talk alot about neural nets in the weeks going forward, especially if I don’t hear back about my data request.