That button has two modes; it can be set to act like an on/off switch, in which case you press it to lock in exposure and/or focus. The lock stays until you push the button again. Or you can set it such that it needs to be constantly depressed to keep the lock. It's a menu option. Check the manual. Choose whatever works best for you.

I thought this only applies to semi auto modes ( ie., A and S).The AE-L/AF-L button function as quick focus button to active one touch auto focus in Manual focus mode. It doesn't lock exposure or focus... or I am wrong.

So the issue has nothing to do with the setting selectoin in the menu option. Or I am wrong?