The formula for the logistic activation function is often written as:
netoutput = 1 / (1+exp(-netinput));
But this formula can produce floating-point overflow in the exponential
function if you program it in this simple form. To avoid overflow, you can
do this:
if (netinput < -45) netoutput = 0;
else if (netinput > 45) netoutput = 1;
else netoutput = 1 / (1+exp(-netinput));
The constant 45 will work for double precision on all machines that I know
of, but there may be some bizarre machines where it will require some
adjustment. Other activation functions can be handled similarly.

User Contributions:

Comment about this article, ask questions, or add new information about this topic: