You can generate a key on your server as Clinton suggests, or you can enable SSH Agent Forwarding.

This means you won't have to generate a new SSH key on your server. You use the the key already on the machine you're deploying from and the key is forwarded for use on the server to connect to your git server.

You can add ssh_options[:forward_agent] = true to your capistrano recipe or add a Host config to your ~/.ssh/config like this: