We know that smoking contributes to a variety of health problems, including heart disease, emphysema and lung cancer, among others. But now there’s growing evidence that using cigarettes can also affect the brain: the latest study shows that smoking is associated with cognitive decline as early as age 45, and that male smokers may be more vulnerable to these mental effects than women.

For the study published in the journal Archives of General Psychiatry, researchers led by Severine Sabia, a research associate in the department of epidemiology and public health at University College London, gave cognitive tests to 7,236 middle aged men and women three times between 1997-99 and 2007-09, when they were 44-69 years old, 50-74 years old and 55-80 years old.

The researchers also collected the participants’ 20-year smoking history through regular self-reported questionnaires.

And when they compared the cognitive scores to smoking status, they found that men who smoked showed faster decline than nonsmoking men over 10 years.The size of the effect associated with smoking was similar to that of 10 years of aging. Even after Sabia and her colleagues adjusted for the effects of heart disease, stroke and lung function on mental abilities, the effect of smoking remained strong.

The more men smoked, the greater their decline. What’s more, the study showed for the first time that the smoking-related cognitive declines may begin as early as age 45.

Women, on the other hand, did not show any differences in cognitive scores over the same 10-year period. “The result among women was not particularly expected,” Sabia wrote in an email responding to questions about the study. But she says, it might easily be explained by the fact that there were fewer women in the study (2,137) than men (5,099), or because female smokers tended to smoke less than the men.

As far as the male smokers were concerned, the drop in cognitive functions, which included their scores on tests of memory, vocabulary, reasoning, verbal fluency and other executive skills, confirmed earlier studies that showed similar declines among those who lit up. But Sabia’s study is the first to find the effect in smokers as young as middle age. That suggests that tobacco can have an influence even on relatively young brains, and that its impact isn’t primarily due to the weaker neural networks of an already aging brain.

But there was good news in the findings as well, especially for those who quit smoking. Men who had kicked the habit more than 10 years before the study began actually had better cognitive scores than men who had never smoked. Sabia credits this effect to the fact that people who quit smoking tend to become healthier overall, and may have adopted other lifestyle or diet changes that promoted better mental health.

That’s certainly encouraging, but Sabia cautions that the association between smoking and cognitive decline needs to be studied further. Because smokers tend to drop out of long-term studies of cognitive function due to disease, it’s possible that previous studies on the subject have underestimated the effect of lighting up on the brain. Sabia’s study is the first to use a statistical model to make up for this potential bias, but additional studies will need to confirm the relationship. If the research bears out, perhaps future public health messages about smoking will include warnings about cognitive harms as well as damage to the heart and lungs.

Alice Park is a writer at TIME. Find her on Twitter at @aliceparkny. You can also continue the discussion on TIME’s Facebook page and on Twitter at @TIME.