In the maps of the Second and Third Age it may look like the Grey Mountains were but a northern arm of the Misty Mountains, but in truth this mountain range was far older, stemming from the creation of Arda, whereas the Misty Mountains had not been raised until after the Years of the Lamps.

Where the Grey Mountains met at their western end with the Misty Mountains lay Mount Gundabad, an ancient Dwarven holy site and the place where Durin I awoke.

The stretch of mountains west of the Misty Mountains which still formed one range with the Grey Mountains was known as the Mountains of Angmar, another remnant of the Iron Mountains.

The eastern end of the Grey Mountains was split into two branches, and in between lay the Withered Heath, where dragons still bred. After that was a long gap, until the Iron Hills continued the old line of the Iron Mountains again. The Lonely Mountain, was not part of either range and was entirely separate.

Of old the Grey Mountains had been mined and inhabited by Dwarves of Durin's Folk, but by the Third Age all Dwarven strongholds had been abandoned or raided by dragons, and the Grey Mountains served only to divide Forodwaith from Wilderland.

Another line of Grey Mountains in Middle-earth are seen on the Ambarkanta map: these are a series of mountains which continue the line of the Blue Mountains as the western edge of Endor, but on the southern half of the continent. Since no maps of the entire world exist after the First Age, it is unknown if this mountain line still existed in the Third Age. In any case they do not appear in any narrative.

Yet a third line of Grey Mountains start at Umbar and run southwards, following the western coast of Harad. Of old they lay directly east of Numenor. These mountains are labeled on the map on pages 38 and 39 of The Atlas of Middle-earth. It is possible that these are the same as the above that have been moved by a Change of the World.