Working the night shift can throw off your body clock, leaving you feeling tired and sleep-deprived. But, increasingly, evidence suggests that shift work harms health in other ways, raising the risk of diabetes, obesity and even cancer.

In a study published in the journal Occupational and Environmental Medicine, researchers in Denmark found that women who worked night shifts were up to four times more likely to develop breast cancer than women who didn’t work nights.

Johnni Hansen, an epidemiologist at the Institute of Cancer Epidemiology at the Danish Cancer Society, and his colleague, Christina Lassen, studied 18,500 women who were in the Danish Army between 1964 and 1999. The researchers followed-up with 141 women who had developed breast cancer by 2005-06, and compared their medical histories with those of 551 women in the military who did not develop the disease. All of the participants filled out a detailed 28-page questionnaire asking about their working habits and whether they considered themselves “morning” or “night” people.

The researchers also asked the women about other factors that could affect breast cancer risk, including whether they used contraceptives, how many children they had, whether they used hormone replacement therapy if they were past menopause, and if they sunbathed.

After accounting for these potential confounding effects, Hansen still found that working night shifts was associated with a 40% increase in breast cancer risk. The effect was cumulative: women who worked at least three night shifts a week for six years had twice the risk of breast cancer as those who worked one to two night shifts a week.

Most surprising, though, was the fact that women who worked night shifts and described themselves as being “morning” people — that is, they preferred to wake up early, rather than stay up late at night — had a four times higher risk of breast cancer than women who worked during the day. “The four times higher increased risk surprised us. It was very unusual,” says Hansen.

Previous studies of shift work and breast cancer risk, which focused on nurses, have also found an increased risk of disease linked to night work, but researchers could never fully adjust for other factors that could have influenced the women’s breast cancer rates, such as the fact that nurses are exposed to higher-than-usual levels of potential cancer-promoting factors like radiation and other treatments.

That’s why Hansen and Lassen decided to review records of military personnel, who, like nurses, commonly work night shifts, but tend not to be affected by the same potential confounders.

The nurse studies had pointed toward a possible explanation for the association: because night workers labor under artificial light, they may be exposed to less natural sunlight and, therefore, less vitamin D from the sun’s rays than day workers; lower levels of vitamin D have been linked to increased risk of breast cancer. Based on the questionnaires, however, Hansen found that the Danish night workers actually spent more time outdoors and had higher rates of sun exposure than day workers, since they were free during the day when others were indoors at work.

That suggests a more biological explanation for breast cancer risk, perhaps related to changes in hormone levels due to disruptions to the body’s internal clock. Working at night and sleeping during the day reverses the body’s circadian rhythm, which in turn alters levels of melatonin. Normally, melatonin rises at night, in the presence of darkness, but artificial light at night can suppress melatonin levels, which can affect other hormones that influence breast cell growth. Breast cancer patients tend to have lower levels of melatonin than women without the disease.

Genes may also play a role, since whether a person is a “night owl or a “lark” is partially determined by his or her genetic makeup. Hansen’s finding that shift work can be particularly risky for “larks,” people who are more alert and active in the morning than at night, makes sense, since genetics may make it harder for a morning person to adjust to hormonal and metabolic changes that come with working at night. “It’s much worse to be a morning person and have night shift work when it comes to breast cancer,” says Hansen.

Despite the health risks, night shift work may be unavoidable for many women. The good news is that Hansen’s study suggests that for most women, one or two shifts a week probably won’t have a significant effect on breast health.

Alice Park is a writer at TIME. Find her on Twitter at @aliceparkny. You can also continue the discussion on TIME’s Facebook page and on Twitter at @TIME.