Prussia's territory roughly corresponded to the historical region of Prussia, from which the province and the kingdom derived their names, that had been divided by the Kingdom of Poland since the Second Peace of Thorn in 1466. The region was divided into Ducal Prussia, a fief of Poland, and Royal Prussia, which was an integral part of Poland under direct rule. Ducal Prussia became part of Brandenburg-Prussia in 1618, and became the Kingdom of Prussia upon the coronation of King Frederick I of Prussia in 1701. After the coronation, the term "Province of Prussia" was used as a designation for East Prussia to differentiate the territory of the former duchy within the larger kingdom. Royal Prussia was annexed by the Kingdom of Prussia from the Polish-Lithuanian Commonwealth in 1772 during the First Partition of Poland, placing the entirety of the Prussia region under Prussian rule. The following year, Prussia converted the former Royal Prussia and Ducal Prussia (including Ermland) into the provinces of West Prussia and East Prussia, respectively, with minor border changes.

On April 1, 1878, the Province of Prussia was divided back into the provinces of West Prussia and East Prussia.