Your user account is not allowed to install to the
system RubyGems. You can cancel this installation
and run:
bundle install --path vendor/bundle
to install the gems into ./vendor/bundle/, or you
can enter your password and install the bundled gems
to RubyGems using sudo.

After double checking the environment variables in fish I figured I should
follow the instructions to the letter and also add the variables to .bashrc.

And it worked!

So, when using fish you also need to set GEM_HOME in a .bashrc for bundle
to work properly as it seemingly uses bash or the bash config internally.

I’m guessing we might see more issues like this pop up when macOS changes its
default shell to zsh in Catalina.