Which were the first religions to be created with well known scriptures, ceremonies and traditions? Was it that of the ancient Egyptians or the old kingdoms of the Middle East? What about Africa or the East?

Many illiterate societies developed pretty complex systems of cult.. Hence, there will be no definitive answer to the part of your question regarding ceremonies and traditions, because of lack of sources. Judging "development" of religion having fragmentary archeological sources is highly questionable. The first religions that left some kind of holy scriptures will be those of ancient Egypt and Sumer.
–
soliloquyyJan 26 '13 at 17:13

Unsurprisingly, the first literate societies, Sumer and Egypt, used their literacy to record some of their religious practices.

The oldest written work of a major world religion that is still in widespread use today is probably the Rigveda (sacred to the Hindus), which we now believe was written sometime between 1700 and 1100 BC.

In terms of organised religion on a state level I believe the ancient Mesopotanians where the first to truly structure and organise religion. Commission the building of temples paying priest out of state coffers etc.