First off, the Gravemind says he has listen to "much talk...through rock and metal and time."

That tells me that he's being alive for quite a long time. If the transmissions from the BWU are to be interpreted as from Delta Halo, and the breakout is to have happened right after the Halos activated (100,000 years ago according to the combined time records of both transmissions), this Gravemind HAS existed for 100,000 years.

"I am a monument to all your sins..." may be referencing the Forerunner through the Chief with a collective "you", considering the Chief is the inherited descendant of the Forerunner. But what sin of the Forerunners? Their sloth and inaction. If the Gravemind really is that old, then he is composed not only of the new human and Covenant corpses that have arrived, but of Forerunner bodies from the past. The ones taken over because they could not defend themselves. And that was their own fault.