I think some care must be exercised when looking for examples of human atavisms in the form of tails. Not all tails contain any bone structures, which is less impressive (for an atavism) than a human tail consisting of both flesh and bone. I don't know if the ones without bone structures are also atavisms, but the ones with bone structures are no doubt better examples.

I took some interest in atavism a while ago and therefore know that there are a few articles on the subject out there. To see if I could find them again (and newer ones) I did a PubMed seach. Unless otherwise indicated, I have only read the abstracts of the articles I cite here:

This article reports an atavistic return of the mouse inner ear to an earlier synapsid state. (N.B. I read the Rijli and Pollock articles some time ago along with some follow-ups and I seem to remember that one of designations as an "atavism" was challenged in a later article. I think it was Rijli's atavism that was challenged. Not being a developmental biologist I couldn't tell who was right, but check this before using the mice atavism in an argument.)

This article is just outside the range of issues available online on the journal's homepage. It appears to contain either an argument that some atavisms are not atavisms or, perhaps more likely, a comparison of the modern view to such an argument.

Then there are these ones:Hall B (1984) "Developmental mechanisms underlying the formation of atavisms", Biological Reviews, 59:89-124

I read these long ago. The first is a pretty long review of some known atavisms. Of particular interest is the section on atavism of whales (some whales are even born with toes!) and atavistic returns of the hindlimbs of modern birds to their ancestral, Archaeopteryx-like state. The 1995 article includes a picture of an unfortunate human child born with a lot of hair all over his body, but other than that it doesn't contain much that isn't covered in the 1984 review.

Relaxed selective pressure on an essential component of pheromone transduction in primate evolution.

Liman ER, Innan H.

Quote

The vomeronasal organ (VNO) detects pheromones in many vertebrate species but is likely to be vestigial in humans. TRPC2(TRP2), a gene that is essential for VNO function in the mouse, is a pseudogene in humans. Because TRPC2 is expressed only in the VNO, the loss of selective pressure on this gene can serve as a molecular marker for the time at which the VNO became vestigial. By analyzing sequence data from the TRPC2 gene of 15 extant primate species, we provide evidence that the VNO was most likely functional in the common ancestor of New World monkeys and Old World monkeys and apes, but then became vestigial in the common ancestor of Old World monkeys and apes. We propose that, at this point in evolution, other modalities, notably the development of color vision, may have largely replaced signaling by pheromones.

So we have an organ, the VNO, which appears to require a gene that has been rendered a pseudogene.