Bad accuracy on MNIST dataset

[d4nst]

Commit 2a4fe4b made the accuracy on the MNIST dataset decrease from 6-7 degrees to 26-27 degrees.

The version before that commit can be used as a workaround for now: https://github.com/d4nst/RotNet/tree/1d934ba4e810806d264cbbbb94d165515887988a

[suzhenghang]

@d4nst
Hi,
I got the accuracy 29-30 degrees after 50 epoch. It is right?

[d4nst]

Sounds about the same that I get with the latest version of the code. As mentioned in the first comment, I used to get an error of 6-7 degrees before commit 2a4fe4b. I suspect that the old rotate method introduced some artefacts that made it easier for the network to figure out the angle. I'm not planning to investigate the issue further but if you find the reason please let me know :)

[AlexZot]

@d4nst Hi. It's off topic, but can You explain why you are using Sigmoid activation in output Dense layer in regression task? I've always thought that regression uses no activations on output.

[d4nst]

Yes, you are right. It is not recommended to use sigmoid as an output when training with L1/L2 loss. I believe that I tried to train the model without the sigmoid and the result was very similar though.

[luisfelipe18]

My tensorflow stops without erros at epoch 6.