First, we're going to enable the Apple Push Notification Service for our app on Apple's Developer site. So go there, open Identifiers, App IDs and choose your app ID. (Create an ID for your app if you don't have one already.)

After clicking the Edit button, you'll be presented with all the services you can enable for your app. Enable Push Notifications and click Create Certificate….

Now, this step is the most important and the most easily forgotten:
To create the correct certificate, when you use Keychain Access to create the CSR you have to include your private key. For this, the menu item has to say "Request a Certificate From a Certificate Authority With “iOS Developer: [your_name]”…" and not simply "Request a Certificate From a Certificate Authority…".

To do this, in Keychain Access, choose the login Keychain on the left, then Certificates on bottom left, find your iPhone Developer: [your_name] certificate in the list, expand it and select the private key iOS Developer: [your_name] as shown in the picture.

Upload the generated CSR and download the certificate. Then, create a new provisioning profile for your app. Download your developer certificate again, just in case. You need to have everything up to date.

Do not forget to select your app's target in Xcode, go into Capabilities and enable Remote Notifications in Background Modes

Open your AWS console, choose Create New App and in the Push Platform list, select APNS_SANDBOX. (This assumes that you are setting up Push for development, of course)

As you can see, it asks you for a P12 File. So open Keychain Access, select login then Certificates again but this time choose the Apple Push certificate called Apple Development IOS Push Services: [your_app_id] and right click on it (i.e. the certificate, not the private key). Choose Export “Apple Development...”…, save it and give it a password.

On SNS, choose the exported certificate, enter your password, click Load Credentials from File and then Add New App.