The above ticket pointed me in the right direction. The solution here it to move the cursor outside of the HTML comment when switching views. This is already done for HTML tags and shortcodes.

I've also found that this will help with those editing ex-gutenberg posts, as this bug could mess up a post very badly otherwise.

Testing is fairly straightforward. Create HTML comments, and place your cursor inside of them while switching between visual and text mode. Patch incoming, and props to @davidsword for the regex help :)