Why are there caps on FPS anyway? What does a developer gain by putting a limit on what their customer can render at?

Lazy development.
Having 30fps cap or any specific cap allows for everything to be in sync like physics-audio-movement-controls etc which requires less work on optimisation since most modern CPUs/GPUs can handle 30fps.

We know how important having options that best suit your PC’s performance capabilities are. The team has been hard at work ensuring that Mafia III’s performance is consistent across all platforms, and right now the game runs at a solid 30 frames-per-second. We are currently working on an update for PC that will give players the option to play Mafia III at higher framerates. We’ll share more details about the update, and when you can expect it to arrive, very soon. In the meantime, the team welcomes feedback and would like to hear from you about what performance adjustment options are important to you. We’re actively considering a number of additional post-launch improvements on PC that would give players more control over how they can tailor the game’s experience to suit their rig’s performance. It’s been an incredible journey developing Mafia III and we couldn’t be more excited to get the game in your hands and hear what you think!

There are already 300+ pre-release reviews on Steam right now and the game unlocks in a little more than 4 hours for everybody, so I doubt that the fps problem will be fixed in time as they aren't even ready to tell people when the update will arrive.