The source material of It's Gonna Rain consists entirely of a tape recording made in 1964 at San Francisco's Union Square.[2] In the recording, a Pentecostal preacher, Brother Walter, rails about the end of the world,[3] while accompanying background noises, including the sound of a pigeon taking flight, are heard. The piece opens with the story of Noah, and the phrase "It's Gonna Rain" is repeated and eventually looped throughout the first half of the piece.

For the recording, Reich used two normal Wollensaktape recorders with the same recording, originally attempting to align the phrase with itself at the halfway point (180 degrees). However, due to the imprecise technology in 1965, the two recordings fell out of sync, with one tape gradually falling ahead or behind the other due to minute differences in the machines, the length of the spliced tape loops, and playback speed. Reich decided to exploit what is known as phase shifting, where all possible recursive harmonies are explored before the two loops eventually get back in sync. The following year, Reich created another composition, Come Out, in which the phrase "come out to show them" is looped to create the same effect.

The work is in two parts of roughly equal length, the first using the "It's Gonna Rain" sample as mentioned above, the second using a separate section of the speech with short phrases cut together and the resultant pattern then phased as in the first part, but with additional tape delay to create a more processed sound.