Years ago,I can't say when,The American Rifleman tested a variety of bullets of various weight in 308,30-06,and 300 magnum.

These were penetration and cavity tests.

What may seem counterintuitive,at close range,the 308 gave the most depedable deep penetration.Another way of saying it,with the bullets available for the test,the .308 might be the best rifle to have in your hands if you had to deal with a bear at 10 yds.

Why? Because it is far easier to make a bullet that performs properly at 2600 or 2700 fps than it is to make a bullet that performs at 3200 fps,especially since there will be the expectation that same bullet will perform at a much lower velocity at 500 yds.

General trends,(understand,one can make a heavy 308 or a light 300 magnum)
But,trends,A 308 will be a lighter rifle that will kick less and make less noise(easier to shoot accurately) It will be cheaper to shoot,and the barrel will last longer.There are variables,but,generally you will destroy less edible meat with a .308 wound channel.

With a good 165 gr bullet and a well placed shot,a 308 kills elk just fine at 300 yds.

But modern bullets require a certain minimum velocity to expand properly,generally around 2000 fps.It varies per the manufacturer.The 308 is limited in its ability to deliver the high velocity at longer ranges.The 300 magnum has the advantage.

The other advantage to the higher velocity is a flatter trajectory.That makes hitting easier at extended range.

Handloading is a whole new skill set,not to be taken lightly,but what you can do is tailor reduced velocity loads so that your 300 magnum will shoot at 308 velocities,or even less.Then you would be eliminating a lot of rifle induced variables.