The Stone represents death. Literally, it is a gravestone. We do know that death is the motif, because of the common theme of blue that Dave uses in song to signify death. The subject expresses guilt over a death, and over a death they may have caused.