Middle-aged men who drink heavily are more likely to experience memory issues and other mental troubles at a younger age than their non-drinking counterparts, a new study suggests.

Researchers found that drinking heavily effectively added up to six years of mental aging in regards to a man’s memory, per the Associated Press. It aged men two years, meanwhile, in other mental activities, such as reasoning.

The study, published Wednesday in the journal Neurology, sampled about 5,000 British civil servants. The study didn’t replicate the findings in women, as it included too few female drinkers to deliver statistically significant results.