Gravity is 9.8m/s^2 not newtons. Therfore if theta is 45 degrees the magnitude of acceleration =ma*sqrt(2)/2m the mass makes no difference until you get to terminal velocities. The inertia can make a difference however. Theoretically they would roll down together, unless these balls had different inetria( one was hollow).

this may be true, but you also need to consider the net force on the object and thus with an increased mass the force normal at that angle would be higher on the larger mass object resulting in an larger frictional force as long as the force applied are reasonalby the same. I would then have to think about the forces applied on ther masses to determine the rate of accleration of each mass. when we look at all of this our equation for each acceleration would be something like:

a= [ (sin 45(fg)) + -(cos 45(fg)) ] / m

where fg is the wieght of each ball. this will allow you to consider the rate of each as it rolls down a displacement.

Are the balls actually rolling, or are they simply sliding down the incline?

If they're simply sliding, your initial analysis is correct (not sure why it confused you!). The equation ends up being independent of the mass. Of course, this is for the case where there is no friction and a different analysis would be required if friction were present.

If they are actually balls rolling down the incline, then you need to do a rotational analysis of the situation as well. In physics, rolling has a special meaning, so if it is just referring to sliding, then the question is worded poorly.