Adding additional MX records

I need to add additional MX records to a domain
for a secondary backup mail server,
I know that to start I need to add a new MX record with the secondary mail
server, but then I'm lost ... do I need to add an A record for the secondary server as well?