Sometimes it does happen when your device is detected but is not recognized by your Eclipse ADT. Each Android device has a Development Device ID which is used to uniquely identify your android device. You can view this ID by going to

System settings -> Developer options -> Development Device ID

(It is right below option to enable USB debugging)

Screenshot for the same is provided below(It is of my device running Android 4.0.4/ICS).

Now coming back to our main problem.Whenever it occurs you will see your prompt asking for device as follows -

You will see something like ???? which means your device is detected but is not recognized. You can check this issue by going to adt-bundle-windows-x86\sdk\platform-tools and typing the following command

adb.exe devices (Windows)

./adb devices (Linux)

In both cases you will see an entry in list of devices but again some ????? with no permissions . All you need to do to get things to work again is restart your server. Detailed steps to do so and it's effect before and after is provided in a snapshot below.

Note that the device is now listed properly. Another interesting thing to note here is when you kill the server a reconnect thread is started in your Eclipse ADT. It will continuously try to reconnect to the server and will stop once server is up. Screenshot for the same is given below.