Again this year when outside temprature reaches -20C (-4F) and my car isn't used for nearly 24 hours, clock resets and comfort access doesn't work.

This happened to me last year too, but the car sat for a longer period of time (2-3 days). I noticed this time around even the tire pressure warning (red) came on and had to be reset.

Now when I set the time, it still lost the next morning. Is this a good indicator for needing a new battery, or just needs to be driven longer to regain charge?

Possibly. -20C is COLDDDDDDDD.

I would plug in a CTEk battery charger overnight in those temperatures. New or old battery your battery will love you.

Plus these cars are PIGs when it comes to battery recharging so you really need to drive it to charge. at -20C the battery initially won't even accept a charge from the alternator so your 30 minute drive barely begins to recharge the battery.