Driving is not more dangerous than snow sports and the "by particpant" metric is a rediculous metric. You may ride 5-10 miles a day max. 90 minutes of actually standing on the base if the board maybe? Most participants do that 3 times a year or so.

Most of us drive 30,000 miles a year.
Its the reason you always see the traffic fatality number drop when gas prices spike. Not a decrease in total driver but a decrease in miles driven.

You should compare with driving a bike, where you are as exposed as riding a board. Not in a car. Would you ride a bike without helmet?