Yeah it's very odd. It's not a caching issue, well the website has only just gone live, and I can't find a work around so I've had to go live with the bug present for now. I've found a workaround to make the mobile version better by changing the width of the fly-out menu, but it hasn't fixed it.

Now after I close the menu, this w-nav-overlay div seems to be sitting over a section of the page. Anything in the highlighted area below isn't clickable. So if I try and re-open the menu for example, I can't because of that overlay that isn't disappearing.