If you constantly have a "New Voicemail" notification on your Microsoft Lumia despite checking your Voicemail Box and verifying that there are no messages, here is how you can clear the stuck notification:

Call yourself using a different line and let it ring until you land on your voicemail

Leave yourself a voicemail message

Once you've left yourself a voicemail, go into your voicemail box

Delete the message you left and any other messages remaining in your mailbox

Power your device Off, wait 10 seconds then turn it back on and it should be gone!