Previous Topic

Next Topic

Look at the science

If you want to use a cancer treatment with a proven track record, look at how the treatment was tested. The way tests are set up can affect the outcome, and sometimes can make a treatment look like it works when it really doesn’t.

Lab and animal studies: Pre-clinical tests

Lab studies

Scientists usually start by testing a new treatment on cancer cells in a dish in the lab, to find out if the treatment has any effect there. If it doesn’t, they may change the formula or use different types of cells to try it again. If they find the effect they want in the dish, they may move on to animals to find out if the substance is absorbed from the stomach or intestine, and look at how it’s distributed in the body. They look for good and bad effects. Because some of these study reports are published, you may hear about them on the news. But often the headlines do not clearly say what kind of study is being reported. Of all the substances that are tested in these early stages, only about 1 in 1,000 look promising enough to even be tested in humans.

This means that if you are looking at a report of a research study – even one that says a treatment “stops the growth of cancer cells” – you may notice that there’s no mention of people. Some of these lab studies use human cancer cells, but other times the cancer cells are from animals. (These kinds of studies are called in vitro studies.)

At this point, anything that stops cancer cell growth may sound like good news. But there are many things that can stop cancer cells from growing in a lab dish that do not work when people take them.

Some reasons a treatment may not work for people are that the substance also hurts or kills normal cells, or because the body cannot absorb it and get it to the cancer. Sometimes, even if the substance can be absorbed, can reach the cancer, and doesn’t harm normal cells, the amount of the substance that reaches the cancer isn’t enough to stop the cancer cells. There are many hurdles between lab studies and human ones.

Animal studies

If the researchers find the effect they want in cells in a dish, they may move on to animal tests. This can help them find out if the substance can be absorbed from the stomach or intestine, and learn how it’s distributed in the animal’s body. They may look for good and bad effects. Because some of these study reports are published, you may also hear about them on the news. These are called in vivo studies. This means that they were studied in living creatures.

If the study was done in living animals, good outcomes may sound promising. Keep in mind, though, that drugs that work in animals don’t always work when they are tested in people. Animal studies sometimes help scientists know which drugs may be toxic to people, and which may show unexpected effects. Sometimes, the drug works in almost the exact same way in people as animals. But as any veterinarian can tell you, there are many drugs people use that don’t work on animals, and vice versa. Some animal drugs can hurt people, and some human drugs can hurt animals. So while animal tests can give researchers certain types of valuable information, they still may not show how the drug will affect people.

News stories on lab and animal studies can mislead

In both lab studies and animal studies, the research report may be published. Usually, the researcher’s own report will make it clear that more studies need to be done to see if the substance makes a difference in people. But if a news group picks up the story and publishes it, they may not mention how the study was done.

Often the headlines, and sometimes even the full story, do not clearly say what kind of study is being reported. Sometimes the news reports on this very early research may make it sound like the compound will work in people, which can lead to confusion. This is why it helps to look at the whole printed story, and then see if you can find out more about the details of the research. Always keep in mind that there’s a huge difference between positive results in lab or animal studies and good results in human studies.

Human testing

After testing in the lab, and sometimes in animals, the treatment may be ready to be tested in humans. Before this decision is made, the results from lab and animal tests are carefully looked at. Other evidence may be weighed, such as the effects of related drugs, what’s already known about the class of chemicals, and other such information. If the drug is a new substance, the company must tell the FDA why they think the drug will work in humans, and share research from the lab and animal studies. They must also have some evidence that it will be safe for people to take. If the FDA approves human testing, researchers must decide how to design clinical trials and find volunteers who are willing to take the drug.

This is why careful cancer researchers make sure they know exactly what disease each person in their study has before any treatment is given. They want to know how the drug affects that disease. After the treatment, they watch for side effects, and regularly check on how the patients are doing. Researchers can then see how people who got the treatment are doing for months or even years after getting the treatment. All this information becomes part of the study record.

What’s a clinical trial?

Clinical trials are research studies in which people volunteer to help doctors find ways to treat a disease or improve care. Today’s cancer treatments are mostly based on what was learned in past clinical trials. Cancer treatments have improved because of clinical trials. It’s important to continue this kind of research, so that cancer treatments can keep getting better. Here are some things to think about when looking at the results of clinical trials.

Who were the study subjects?

If you got your information from a study done on people, this is a good start. Still, a treatment must go through many stages in human tests before it goes into widespread use as a cancer treatment. It’s possible that the study you heard about is an early (preliminary) study, or a pilot study. These are small studies in which a drug or treatment is tested on a few people just to see if it’s worth testing on larger numbers of people. If the treatment you heard about is already being tested in large groups of people, there’s still a chance that the treatment can only be given to those who are in the study.

When a drug is being looked at for FDA approval, there are several stages of clinical trials, starting with smaller studies and building up to bigger ones. For new drugs, there are phase I trials, then phase II and phase III trials which must be completed before a new treatment is approved. This can take a number of years. (For more information on the stages of clinical trials, see our document called Clinical Trials: What You Need to Know.)

Was it a controlled study?

Did the study have a control group? If it did, it means that those taking the test treatment were compared to people who took a proven treatment (or a placebo, a sham treatment, if there’s no standard treatment available). This helps researchers find out if the people who get the new treatment do better, live longer, or have fewer symptoms than those who got the old treatment or the placebo. A study that has a control group is called a controlled study.

Not every clinical trial has a control group. This means that the treatment being tested cannot be directly compared with other kinds of treatment. There are other pitfalls to not having a control group. For instance, if the cancer is a type that stops growing for a while without treatment, a study without a control group can make it look like the treatment is what stopped the cancer’s growth. A control group would show that the cancer stopped growing (and probably started back later) no matter what treatment was used.

Some cancer treatment centers almost always have better outcomes than others, which can pose problems when you compare treatments in research studies. A good outcome may be partly because of better supportive care (like quicker treatment for infections, better use of other medicines, expert nurses, and other factors). It may also be because they are known as excellent treatment centers, and people travel great distances for care there. When this happens, the people in their clinical trials may be healthier (if they were healthy enough to travel a long way to the cancer center, for instance), have other health care problems under control (because they have more money or better insurance), and be better-informed health care consumers (so they may be more willing to stick with treatment). All of these factors can improve survival, no matter which treatment the group gets.

Still, control groups from the same cancer center help to balance out these advantages, as long as the control group is like the treatment group (see “Randomization,” below). If there’s no control group from the same cancer center, you can’t take results from one clinical trial and compare them to results from a different treatment that was tested at another cancer center.

Were the subjects randomly assigned to study or control groups (randomization)?

Randomization means that the treatment is compared using similar groups of volunteers who were chosen completely by chance to be in one group or the other (they were randomized). This reduces the risk, for instance, that the sicker people end up in one group, which could change the study outcome.

For example, if more people who are healthier (such as those who are younger, or have an earlier stage of cancer) end up in the group getting the new treatment, it may make the new treatment look better than it really is. In the same way, if more people with worse disease end up in the new treatment group, that group may fare worse than the control group. This could make the new treatment look less effective, because it was tested on people who were sicker to start with. Because of this, researchers put people into one group or the other by doing something along the lines of flipping a coin (although it’s really done by computers instead of a coin toss).

Randomizing the study volunteers into groups lowers the odds that one group will be different enough from the other to affect the outcome.

Was it a blind study?

A blinded study is one in which the patients did not know which treatment they got. This helps avoid responses that are based on what was expected. Sometimes people who think a treatment is going to work will appear to do better over the short term. (For more information, see our document called Placebo Effect.) If the patient knows which treatment he or she is getting, the study is called an open label study.

One advantage to a blinded study is that it can help researchers learn more about true side effects. Patients who know that they are getting placebos might not bother to report health problems to the study coordinator. Those who know they’re getting the test drug or treatment are more likely to report nausea, headaches, and fever, even if the problems turn out to be from food poisoning or the flu. The same is true for serious illnesses, which also can happen with no known reason but may end up being blamed on whatever person is taking. When a study is blinded, both groups have similar rates of unrelated illness.

Was it a double-blind study?

A double-blind study is one in which neither the doctors nor the patient know which treatment the patient got until after the treatment is completed and the observations are on record. This helps to avoid bias, in which a doctor or researcher expects one group of patients to do better and that bias affects his or her observations. In clinical trials, observations are carefully measured and written up. After the study, the code is broken so that the people who are analyzing the data can find out which group (if any) did better than the other.

There are times when double blinding isn’t possible; for instance, when people get treatment at entirely different times in the control group than the test group (like before surgery instead of after surgery). In studies like this, the doctors must know when to give the treatments and do the surgery, so they cannot be blinded. Even then, the doctors reading the scans or looking at tumor samples might not be told which treatment the patient got, which would still help reduce the risk of biased results.

Did the results have statistical significance?

The data are carefully looked at to see if the differences between the groups could be due to chance. This is called a test of statistical significance. It means that if one group came out better than the other by a large enough margin it’s very unlikely that the differences were by chance. In this case, the results are said to be “significant.” Keep in mind this kind of test alone cannot prove that factors besides random chance didn’t bias or confound the results. Careful study planning and precise measurements are used to avoid those factors.

Is the study published and peer reviewed?

Publishing the findings in a respected, peer-reviewed journal means that the methods and information from the study were looked at by other doctors or scientists. When they look at the information, they want to be sure that the scientific procedures were properly followed. They also keep an eye out for any bias or other factors that would make one group do better than the other for some reason other than the treatment being studied.

Sometimes you will find news about studies that comes from other sources, such as magazines or journals that are not peer reviewed, or books and letters that are supposed to have been written by experts. Be extra careful of this information.

The highest standard of proof that a treatment works is a double-blind, randomized clinical trial on humans that has met the strictest standards of scientific method. If blinded studies are not possible, scientific procedures must still be carefully followed to be sure that any difference in outcomes are due to the treatment, and not other factors. This usually allows the study to be published in a respected, peer-reviewed medical journal.

Good science is cautious

It takes more than one study to prove something works

Even the best-sounding ideas take a lot of testing to show that they work. Since many good ideas don’t pan out for cancer treatment, the failure rate can be high. Science builds on the studies in the lab, and sometimes animal studies. If the treatment seems to be safe enough, it may next be tested on a small group of people. Getting to this point often takes many years.

One study with a good outcome, even a human study, doesn’t mean a treatment works. Future studies that try the same thing sometimes get different results, even if they do their study the same way. This can happen because the second clinical trial tests the method on a different group of people that doesn’t respond the same way as the first group. Or the treatment may be used in a slightly different way, or with some other small difference that may not have been noticed. Or the treatment looks good in the first study based on random factors that can’t be controlled. Sometimes a treatment looks great on the first study, but then no other study gets the same outcome—meaning that real-life patients couldn’t expect those great results either.

It’s rare to find a respected cancer researcher who wants to use a new treatment based on just one study. But even if it looks that way, behind that study is almost always a lot of other information that has been built up over a long time. When one large clinical trial makes the news, keep in mind that if a treatment actually works, it usually has a good track record from earlier studies.

Other questions about new treatments

Can a clinical trial find a cure?

Among possible cancer drugs being studied for FDA approval, most never get past the first human clinical trial. In fact, less than 1 out of 5 medicines that make it to the stage of clinical trials will ever be approved to treat cancer.

Even when a drug or treatment works well, rarely will a researcher or doctor say that a treatment cures cancer. In order be sure of a cure, studies would have to keep up with each patient until he or she died of something else. This is much harder to do than it sounds—and think about how many years it would take before the study results could be announced.

Even if a treatment is good enough to cure some patients, it’s not likely to cure everyone. But for certain types of cancer that haven’t spread, or that are found very early, treatment can sometimes get rid of the cancer so that it doesn’t come back. Because it can be hard to tell in advance which cancers will come back, there’s always uncertainty, even with well-tested medical treatments.

What about treatments that are not in line to be approved by the FDA?

There are many herbs and food extracts that are advertised as having an effect on cancer. As long as these food-related products are generally regarded as safe, there are few restrictions on their sales. Many are simply packaged and sold. Because some of these supplements have been found to not contain what’s listed on the label, and others have been found to include substances that were not on the label, the FDA set up new rules for dietary supplements that took effect in 2010.

These rules are intended to help people be sure that the supplement contains what it says on the label, with no extra ingredients or impurities. The rules still do not require those who make or sell the product to submit proof that the herb or supplement is safe or effective, and they do not address the supplements’ effects on the body. Despite the new rules, many supplements have been recalled due to impurities or extra ingredients that were not listed on the label.

Because there’s growing interest in dietary supplements, researchers have started studying some of them using the same methods used for cancer treatments. Large sums of money are not usually available for clinical studies to look at herbs, so these studies tend to be smaller. But because the safety of the substance isn’t called into question, there’s less need for safety testing.

When looking at studies of herbs or supplements in people, look at the same questions you would use for clinical trials. Keep in mind that until recently, many of these studies were poorly designed and many did not get published. Or, if they did, some were published someplace other than peer-reviewed scientific journals.

On the Internet, at conferences, and in health food stores, those who sell herbs sometimes try to use lab studies or animal studies as evidence that the herbs work. Some sellers will refer to studies that are not published in peer-reviewed journals. The studies may be written up in a “natural cures” book or posted on a Web site. Sometimes sellers will show letters on a doctor’s or hospital’s letterhead, often from another country where it’s impossible to check the facts. This leaves you with no way to know that the studies were done as they are presented.

It also happens that researchers will isolate a chemical from an herb and test it in the lab to find out if it affects cells. Because the effects of the isolated chemical might be different (especially in large doses), this kind of treatment is tested from the ground up, like a new drug. This type of study is usually written up in scientific literature and can be found there.

If sellers of an herb learn about scientific studies done with herbal extracts, some of them will talk about the study’s findings as if the success in the study were a reflection of the whole herb, not a concentrated extract of one of its parts. Purified extracts do not work the same way as the whole herb, and these 2 types of studies cannot stand in for one another. You can get more information in our document Dietary Supplements: What Is Safe?

What if different clinical trials show different outcomes?

Different outcomes can be very confusing, especially at first. When there are just a few studies, as there may be on a compound that’s generally thought to be safe, tests on humans may be done first. This means there may not be much understanding of how the substance might work from lab studies or animal studies.

Even when the studies are set up well, these clinical trials often end up showing very little difference, if any, between the people who took it and those who didn’t. When the compound really doesn’t have any effect, random chance will often tip the scales in one direction or another—sometimes enough that the results seem to be significant. This means that sometimes the placebo group will do a bit better than the test group, while at other times, the group that gets the new treatment does a little better.

In most cases, when results conflict with one another like this, it means that the treatment really has very little effect. But sometimes conflicting results can be caused by study design problems, or other factors that affected the outcomes.

This leads to another problem that can creep in as studies are published. Sometimes, the studies that show no difference between the treatment and placebo, or the ones that show the placebo group doing better, are not published. After all, it isn’t exactly exciting news when something doesn’t work. But these kinds of studies could really help people who are trying to decide whether it’s worthwhile to get the treatment.

Worse, if the only clinical trials published are the ones that show the treatment helps, a person reviewing the published information won’t be able to find those studies that showed no difference. He or she might conclude that the treatment was helpful, because those are the only studies that were published. This is an example of what is called publication bias.