When I first encountered the work of MC Escher, I couldn't understand how he managed to depict the seemingly impossible. I was nine, and the two pieces that puzzled me were Waterfall and Ascending and Descending. In the first, water at the bottom of a waterfall flows along a channel back to the top without defying gravity in a never-ending cycle. The second is even more striking, with one set of monks climbing an endless staircase while another group walk down it without either ever getting any higher or lower. Years later I learnt that both works were inspired by Roger Penrose.

As a student in 1954, Penrose was attending a conference in Amsterdam when by chance he came across an exhibition of Escher's work. Soon he was trying to conjure up impossible figures of his own and discovered the tri-bar – a triangle that looks like a real, solid three-dimensional object, but isn't. Together with his father, a physicist and mathematician, Penrose went on to design a staircase that simultaneously loops up and down. An article followed and a copy was sent to Escher. Completing a cyclical flow of creativity, the Dutch master of geometrical illusions was inspired to produce his two masterpieces.

Doing what most find impossible has long been Penrose's stock in trade in mathematics and physics, even when it comes to publishing. His previous book, The Road to Reality, was a 1,049-page bestseller, although it was mostly a textbook. Penrose doesn't do "popular", as he peppers his books with equation after equation in defiance of the publishing maxim that each one cuts sales in half. By that reckoning Cycles of Time will have about four readers, though it's probably destined to be another bestseller. As Penrose puts forward his truly Extraordinary New View of the Universe, that the big bang is both the end of one aeon and the beginning of another in an Escheresque endless cycling of time, he outlines the prevailing orthodoxy about the origins of the cosmos.

In the late 20s it was discovered that the light from distant galaxies was stretched towards the red end of the visible spectrum. This redshift was found to be greater the further away the galaxy was, and was accepted as evidence of an expanding universe. This inevitably led theorists to extrapolate backwards to the big bang – the moment of its birth some 13.7bn years ago, when space and time exploded into being out of a single point, infinitely hot and dense, called a singularity. That at least was the theory, with little more to back it up until 1964, when two American scientists discovered "cosmic background radiation" – the faint echo of the big bang. In the decades since, further evidence has accumulated and theoretical refinements made to accommodate it. Yet in recent years a few physicists have challenged the big bang model by daring to ask and answer questions such as: was the big bang the beginning of the universe?

Traditionally such questions have been dismissed as meaningless – space and time were created at the big bang; there simply was no "before". Although it's possible to work out in incredible detail what happened all the way back to within a fraction of a second of the big bang, at the moment itself the theory of general relativity breaks down, or as Penrose puts it: "Einstein's equations (and physics as a whole, as we know it) simply 'give up' at the singularity." However, he believes we should not conclude from this that the big bang was the beginning of the universe.

Acknowledging that he's not the first to think such heretical thoughts, Penrose looks at earlier "pre-big bang proposals". Finding them "fanciful", Penrose looked anew at the big bang, because of an unsolved mystery at its heart involving the Second Law of Thermodynamics. One of the most fundamental in all of physics, it simply says that the amount of disorder, something that physicists label "entropy", increases with the passage of time. Herein lies the mystery for Penrose. The instant after the big bang, "a wildly hot violent event", must have been one of maximum entropy. How can entropy therefore increase? Penrose thinks he has the answer; there must be a pre-big bang era that ensures that entropy is low at the birth of the universe. And here's how.

In what Penrose calls "conformal cyclic cosmology", the beginning and the end of the universe are in effect the same, since these two phases of its evolution contain only massless particles. Between now and a far off distant future, everything from the tiniest particles to biggest galaxies will have been eaten by black holes. They in turn lose energy in the form of massless particles and slowly disappear. As one black hole after another vanishes the universe loses "information". Since information is linked to entropy, the entropy of the universe decreases with the demise of each black hole.

The strangest thing about massless particles is that for them there is no such thing as time. There is no past or present, only "now", and it stretches for all eternity – but since there is no tick of the clock, what eternity? With some mind-numbing maths, Penrose argues that as time ends in the era of massless particles, the fate of our universe can actually be reinterpreted as the big bang of a new one: "Our universe is what I call an aeon in an endless sequence of aeons." Escher would have approved.

Manjit Kumar's Quantum: Einstein, Bohr and the Great Debate about the Nature of Reality is published by Icon.