I had a linux mint guest system installed with virtualbox 5.1.30. I recently uninstalled its guest addition and want to migrate linux mint to virtualbox 5.2.2. But when I tried to mount the guest addition of virtualbox 5.2.2 in order to install it, it complains "Error mounting /dev/sr0 at /media/john/VBox_GAs_5.2.2: command-line `mount -t "is09660"-o ... mount: unknown filesystem type 'iso9660'`", does this mean I cannot make modifications on the guest addition with newer virtualbox release? How can I possibly fix it?

Also another question is when installing guest linux mint 18.3, it would render flickering dark screen when setting up, (actually many other linux systems have the same issue, this is why I finally landed on linux mint 18.2), is this an issue with vb or linux mint itself?

You need to post the actual mount statement you used if you did so manually. Otherwise the unknown file system usually means it is corrupted. It would also help if you posted the guests log file ( as an attachment )