What would you do if you discovered that your best friend’s wife was cheating on him, and that telling him would probably ruin the little business that you and he run?

Correct answer: Do nothing. Don’t break his concentration on his important work, and let him and his wife work out their troubles. End of dilemma.

But no: this is just the beginning of the middle, and the muddle, of The Dilemma, a bizarre experiment about the lies — little and big, white and darker — that can tax any relationship. The presence of box-office magnets Vince Vaughn and Kevin James might prime their fans for wacky character comedy; the addition of Jennifer Connelly as Vaughn’s girlfriend and Winona Ryder as James’s wife suggest a more ornery, indie-minded drama. The movie tries to be both, but it succeeds as neither. Director Ron Howard — momentarily deferring his Oscar-aspirational films (Apollo 13, A Beautiful Mind, Frost/Nixon) for a return to his social-comedy daysof the previous millennium (Gung Ho, The Paper) — can’t find a coherent tone.

In another movie age, the question of whether to keep or reveal a toxic secret about a friend’s sex life would have been set in a sorority house or a beauty parlor; in the 1930s it was the plot motor for Clare Boothe Luce’s all-female play and film The Women. But in what I pray is the depleted end of the bromance cycle, guys must also adhere to this exalted confusion of loyalty and gossip. At least that’s the apparent theme of the script by Allan Loeb. He may be the current hot screenwriter — his style is so versatile, or his agent so persuasive, that recent Loeb scripts have been made by Oliver Stone (last year’s Wall Street: Money Never Sleeps) and Adam Sandler (next month’s Just Go With It) — but all he’s done here is stretch to feature length a sitcom notion that How I Met Your Mother could resolve in 22 minutes with higher levels of hilarity and moral purpose.

Nick (James) is the creative whiz of a small engineering company, and Ronny (Vaughn) its super-salesman — think Steve Wozniak and Steve Jobs from the early Apple days, except these guys are around 40 and haven’t yet struck gold. Friends from college, they are both involved with smart, strong women: Nick with his wife Geneva (Ryder), Ronny with definite bride material Beth (Connelly). The two men have dreamed up a way to add machismo to the environmentally sensible but, in Ronny’s phrase “totally gay” electric car: by adding a motor that revs like a ’66 Ford GT-350. Never mind that the same effect could be accomplished by stereo speakers with a vroom sound; this is their ticket to the stratosphere. Ronny’s pitch mesmerizes one of the auto companies; a consultant (Queen Latifah) giddily tells him, “I wanna have sex with your words. I wanna bang your brains.” All Nick has to do is complete his research in a few days, demonstrate the engine and close the sale.

While Nick has his head under the hood, Ronny plans his marriage proposal to Beth. But in the local Botanical Gardens, he notices Geneva kissing an elaborately tattooed dude named Zip (Channing Tatum). Shocked and chagrined, he’s determined to tell Nick the awful news, though it may unhinge his friend and scuttle the big deal. His attempts to prove the charges against Geneva are calamitous: they lead him up a couple of trees and into the hospital. Ronny is also walking around with a few thousand dollars in cash (for an engagement ring), which worries Beth, since her beau is a recovering gambleholic. Beth is involved in some subterfuge of her own. And Nick’s innocent victimhood might be compromised by his patronage of a Vietnamese masseuse.

O.K., a standard romantic-comedy imbroglio gets darker. Nothing wrong with that; Billy Wilder often led his characters into poisonous jungles of obsession, virtually daring the viewer to keep sympathizing with a decent fellow as he slips toward derangement. Occasionally The Dilemma adjusts your mental remote-control apparatus from a wishful Fast Forward to a startled Pause, with a plot U-turn (intervention counseling? huh?) or a weird emotional eruption — as when Ronny shambles down city streets talking out loud to himself or God. But even these apostrophes are upstaged by repeated on-screen plugs for the the lingerie company Urban Fox. That’s a clear signal that the movie, whatever its bold ambitions, can be counted on to renounce risk for commerce.

Midway through, The Dilemma looks to be going instructively mad along with Ronny: when he confronts Zip one evening in an argument that escalates from shouts to a fight to car-bashing and gun-waving. (In an accidental but creepy evocation of the Tucson massacre, Zip tells Ronny, “I’m only gonna pump three [bullets] in you, like a calm gunman would.”) Here’s a good place for the movie to ratchet up from the domestic to the apocalyptic; but Howard directs the scene with feeble comedy tropes, like Vaughn trying to shoot incriminating photos with his camera’s lens cap on, and ends with a screaming match on residential streets whose neighbors, oblivious to the mayhem, must have been drugged or on vacation. The Dilemma never stops being a movie, and never starts being a good one.

The film is so schizophrenic, it suggests that Howard gave different signals to different actors. Vaughn and James, who could pass for Mutt-and-Jeff brothers, rattle through their dialogue as fast the anchors on Onion Sports Dome, while the women dig earnestly for subtext that isn’t there. Connelly seems especially tense when she’s playing relaxed; and Ryder, in a comeback role, goes for broke in a scene where she has to counterfeit big aggrieved tears — the veins stand out eerily on her forehead like bas-relief highways on a road map.

Nor are the two sexes suited iconographically. Vaughn looks way over his normal fighting weight; Ryder and Connelly are rather severely under theirs. Only James — the pudgy sitcom star whose endearing shtick is to under-react to disaster — appears comfortable with the material. Then again, given that he acquitted himself well in the otherwise negligible I Now Pronounce You Chuck & Larry and Paul Blart: Mall Cop, the definition of a failed comedy may be that Kevin James is the best thing in it.

The Greeks defined a dilemma as a problem with two possibilities, neither of them acceptable. That’s this Dilemma: neither comedy nor drama, neither bitter nor sweet, neither inspiration nor satire or some inventive mix of the two. By the end, an exasperated spectator will be wishing that Ronny and his friends — and Ron Howard and Allan Loeb — had done what good friends might do in real life, and kept this secret to themselves.