Changing accuracy value and no change in loss value in binary classification using Tensorflow

am trying to use a deep neural network architecture to classify against a binary label value - 0 and +1. Here is my code to do it in tensorflow. Also this question carries forward from the discussion in a previous question

Once you pre-process your data into the wrong shape or range in a ML training task, the rest of the data flow will go wrong. You do this multiple times in different ways in the code in the question.

Taking things in order that the processing occurs. The first problems are with pre-processing. Your goals here should be:

X values (input features) in tabular form, each row is an example, each column is a feature. Values should be numeric and scaled for use with neural network. Test and train data need to be scaled identically - that doesn't mean using same .fit_transform because that re-fits the scaler.

Y values (output labels) in tabular form, each row is example matching the same row of X, each column is the true value of an output. For classification problems the values are typically 0 and 1, and should not be re-scaled since they represent class membership.

This re-write of your create_feature_sets_and_labels function does things correctly:

Scaler is fit once then applied to features (major difference, if you scale train and test data separately, you are not predicting anything meaningful)

Scaler is not applied to outputs (major difference for classifier, you want the train and test values to be 0,1 for meaningful training and reporting accuracy)

There are also some problems with your training code for this data:

y = tf.placeholder('float') should be y = tf.placeholder('float', [None, 1]). This makes no difference to processing, but correctly throws an error when y is the wrong shape. That error would have been a clue much earlier that things were going wrong.

n_nodes_hl1 = 500 and n_nodes_hl2 = 500 can be much lower, and the network will actually work much better with e.g. n_nodes_hl1 = 10 and n_nodes_hl2 = 10 - this is mainly because of you using large initial values for weights, you could alternatively scale the weights down, and for more complex data you might want to do that instead. In this case it is simpler to reduce number of hidden neurons.

As we discussed in comments, the start of your train_neural_network function should look like this:

. . . this is a major difference. By using sigmoid_cross_entropy_with_logits you have committed to using the pre-transform value of the output layer for training. But you still want the predicted values to measure accuracy (or for any other use of the network where you want to read off a predicted value).

For consistent measure of loss, you want to have mean loss per example, so you need to divide you sum of mean-per-batch by the number of batches: 'loss:', epoch_loss/(len(train_x)/batch_size)

If I make all those corrections, and run this with a few more epochs - e.g. 50, then I get a typical loss of 0.7 and accuracy measure of 0.5 - and this occurs reasonably reliably, but does move a little due to changes in starting weights. The accuracy is not very stable, and possibly suffers from over-fit, which you are not allowing for at all (and you should read up on techniques to help measure and manage over-fit, it is an important part of training NNs reliably)

The value of 0.5 may seem bad. It is possible to improve upon it, by modifying network architecture or meta-params. I can get down to 0.43 training loss and up to 0.83 test accuracy for example by swapping tf.nn.relu for tf.tanh in the hidden layers and running for 500 epochs.

To understand more about neural networks, what to measure when training and what might be worth changing in your model, you will want to study the subject in more depth.

Email codedump link for Changing accuracy value and no change in loss value in binary classification using Tensorflow