We’re so used to hearing about the importance of screening for cancer — the earlier tumors are detected, the better they can be treated — that it seems anathema to learn that in some cases, screening may not necessarily save us from cancer.

At least, it may not necessarily save enough lives on a population level to justify their costs, in both health complications as well as for the test itself. In 2009, two eye-opening reports claimed that routine prostate-cancer screening does not seem to save lives, or at the very least that any longevity benefit is modest and comes with serious risk of over diagnosis. Then, later that same year, an influential federal panel recommended that women under 50 stop getting regular mammograms, the go-to test for detecting breast cancers.

Today a new study adds yet another skeptical voice to the debate on screening. The study is a follow-up to one of the two in 2009 that looked at prostate-specific antigen (PSA) testing, and it confirms that men who receive regular PSA tests do not live longer than men without regular screening. Both the 2009 study and the follow-up, published in the Journal of the National Cancer Institute (JNCI), use data from the same Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial. But the new study updates the old one, confirming the finding that there’s still no mortality difference between the screened and non-screened groups, even after the roughly 75,000 men in the study have been followed for 13 years.

This new finding is likely to cause confusion and anger — we’re trained to think it’s irresponsible to skip our tests. And it goes against the strong logic behind screening. In fact, many people figure that the new studies and recommendations against screening must some kind of ploy to ration health care.

Yet a growing number of medical experts really do believe that screening procedures, in some select cases, may do more harm than good. Why?

To begin, it’s helpful to think about when cancer screening works the best. To date, perhaps the most successful cancer screening program is the pap smear for cervical cancer, one of only two cancer-screening tests that the U.S. Preventive Services Task Force “strongly recommends” (at least for young and middle-aged women with a cervix). The test is cheap and has virtually no side-effects. It detects cancerous and pre-cancerous cervical abnormalities with reasonable accuracy. And, crucially, it helps people to find potentially lethal cancers long before symptoms appear — the point when patients would otherwise notice something was wrong — thus letting patients take advantage of treatment early, when the chances of success are highest.

Compare that to the PSA test for prostate cancer. It too is cheap and has virtually no side-effects. However, its accuracy is only mediocre, which means that many patients who undergo a PSA test get referred on to more invasive screening because the PSA results were ambiguous. Only about 25% to 20% of men with a high PSA score who get a biopsy turn out to have cancer. So most men who test high may be exposing themselves to unnecessary risk of infection, bleeding, and more.

Of course, that might be perfectly reasonable if early detection of prostate cancer was likely to save a life. But many prostate cancers are slow-growing, which means that men may end up dying of other causes long before the cancer becomes a threat. Current screening techniques cannot distinguish well between prostate cancers that will soon be dangerous and those that will not. As a result, many patients undergo treatment — which also carries risks, including impotence and incontinence — for a disease that may not have ever caused any symptoms had they chosen to ignore it.

In the end then, the evidence stacks up to provide at best tenuous support for routine PSA testing. Screening leads to a good deal of suffering, and it doesn’t save a lot of lives.

The USPSTF — that same influential federal panel that recommended against mammograms for younger women in 2009 — currently says there is insufficient evidence to make a recommendation either way for most men, but recommends against PSA testing in men 75 or older. The American Cancer Society suggests men discuss PSA testing with their doctor; for men without a family history of the disease, the talk can start at age 50; for those at higher risk, the discussion should begin at age 40. A new USPSTF draft recommendation, released in 2011 but yet to be approved, would recommend against routine PSA testing among healthy men of all ages.

Still, the debate over PSA tests — and over other controversial screening procedures — remains thorny. Everyones knows someone who detected a cancer from a screening test, which potentially saved that person’s life. No one is denying that screening can save lives, but when it comes to screening policies for a population, public health officials must weigh the medical and economic costs and benefits across people of different risk levels. There are risks associated with screening. And there are risks associated with not screening too. It’s worth remembering that screening guidelines are just that — guidelines that should serve as a starting point for a discussion between doctor and patient about that individual patient’s need for the test. As evidence such as the latest PSA results continue to emerge, everyone will have more information than ever to decide whether screening is really worth it.