Men have been warned not to take a pair of popular vitamin and mineral supplements after research showed they can dramatically increase the risk of life-threatening prostate cancer.

Overdosing on the mineral selenium by taking supplements raised the chances of developing high-grade cancer by 91%, scientists found.

Vitamin E pills also boosted the risk of aggressive cancer, more than doubling it for men lacking selenium.

The researchers believe selenium can turn toxic when present in the body at excessively high levels.

At the same time, the mineral appeared to protect against the harmful effects of too much vitamin E.

The US study was a follow-up of Select, the selenium and vitamin E cancer prevention trial, which originally recruited more than 35,000 men to see if the supplements could help prevent prostate cancer.

Researchers stopped the trial three years early in 2008 after there were hints that instead of protecting men, vitamin E was putting them at greater risk, while selenium showed no benefit.

Study leader Dr Alan Kristal, from the Fred Hutchinson Cancer Research Centre in Seattle, said: "These supplements are popular – especially vitamin E – although so far no large, well-designed and well-conducted study has shown any benefits for preventing major chronic disease.

"Men using these supplements should stop, period. Neither selenium nor vitamin E supplementation confers any known benefits, only risks."

Selenium supplements had no effect on men who started out lacking the mineral, but were harmful when added to baseline levels that were already high. For aggressive, high-grade cancers, the risk went up by 91%.

Among men with low selenium status at the start of the study, vitamin E supplements increased the overall risk of prostate cancer by 63% while the high-grade risk rose by 111%.

Of the men in the study who developed prostate cancer, 489 were diagnosed with high-grade disease.

Dr Kristal said people were often misled by the supposed benefits of dietary supplements.

"Many people think that dietary supplements are helpful or at the least innocuous," he said . "This is not true. We know from several other studies that some high-dose dietary supplements - that is, supplements that provide far more than the daily recommended intakes of micronutrients - increase cancer risk.

"We knew this based on randomised, controlled, double-blinded studies for folate and beta carotene, and now we know it for vitamin E and selenium."

The new findings appear in the latest edition of the Journal of the National Cancer Institute.

In their paper the scientists warn that "men aged greater than 55 should avoid supplementation with either vitamin E or selenium at doses that exceed recommended dietary intakes".

Men taking part in the research were given 400 international units (IU) of vitamin E a day and a selenium dose of 200 micrograms.

Dr Matthew Hobbs, deputy director of research at Prostate Cancer UK, said: "It is very difficult to draw any useful conclusions from this paper that would be applicable to all men. We would need to see data from much longer studies that look at the total health impact of selenium or vitamin E supplements before we could say if the small effects on the chance of getting prostate cancer suggested by this study outweigh any general benefits to health.

"If men are concerned about their prostate cancer risk, rather than worrying about selenium and vitamin E supplements, they should talk to a GP or visit our website to find out more information."

• This article was amended on 24 February 2014 to correct a quote from Dr Matthew Hobbs.