'X Factor' Contestant WarnedYou Drink, You Smoke, You Lose

11/8/2011 12:06 AM PST

Josh Krajcik's chance at winning $5 million bucks on "X Factor" could go up in smoke -- because the singer's doctor believes cigarettes and booze are sabotaging his shot at victory ... TMZ has learned.

A source close to the production tells us, Krajcik was having issues with his voice giving out during practice last week -- a doctor told him boozing and smoking were to blame ... and Krajcik should stop both IMMEDIATELY.

At that point, we're told that Josh's mentor, Nicole Scherzinger, stepped in to tell him to cut down before he gets eliminated ... but while Josh is trying, quitting ain't easy.

The source said Josh "knows he needs to if he has his sights set on winning."