Condé Nast’s own stars compare their glossy empire to the MGM of Old Hollywood. But no one would wish it the same fate.

S.I. Newhouse Jr., chairman of Condé Nast, falls in love with his editors. His romance with Joanne Lipman began over lunch at his U.N. Plaza apartment, with its beige carpets—no red wine allowed—and paintings by Warhol, de Kooning, Cézanne. Lipman, 47 years old, who’d spent her entire career at The Wall Street Journal, is a serious journalist with a serious mien, and long legs, which she likes to show off with short-skirted power suits. Lipman is “attractive,” in Newhouse’s vernacular—“He uses the word like others use the word spiritual,” says a former editor. The two brainstormed at a small dining-room table. Newhouse, in his standard worn New Yorker sweatshirt, told her he had an idea for a business magazine. Newhouse didn’t say much more; he rarely does. He asks questions. But Lipman excitedly filled in the details.

Newhouse’s pursuit of Lipman was unusual. In most cases, someone else winnows future editors, presents the possibilities to Newhouse, shapes the conversations. But Newhouse, this time, made a point of doing it himself—Portfolio was very much his thing. And by the end of the day, he’d decided he wanted her to be editor of the magazine he planned to launch, which would be called Condé Nast Portfolio. Newhouse pledged patience and breathtaking resources—said to be more than $100 million over five years.

It was a great romance even if, like many great romances, others shook their heads about it, wondering whether Newhouse’s passion for Lipman was entirely rational. Business magazines were, after all, in decline. And soon, turmoil in Portfolio’s offices, along with incessant leaks to blogs and tabloids, made Lipman seem a caricature of the imperious Condé Nast editor, ruling from on high, out of touch. Even factions within the Newhouse family believed Si was blind to the real situation at Portfolio—“a good idea, badly executed,” was how one person described the magazine.

Finally, Newhouse himself couldn’t ignore the economic realities. Portfolio was on track to lose $15 million in a year; the total cost may have ballooned to as much as $150 million. On April 27, Newhouse summoned Lipman, this time to his eleventh-floor office, with its giant Andreas Gursky photograph of the NASDAQ sign on the outside of the Condé Nast building, to deliver the difficult news. In the past, Newhouse’s breakups had been unsentimental. The past was over—he moved on. His editors sometimes saw it on TV or heard it from others. This one was different. “I love Portfolio,” he told Lipman, with obvious feeling.

“I love it too,” Lipman replied.

A star-crossed romance. “It was painful,” says one person close to him. “It wasn’t just a financial investment. He had great hopes for it.”

Newhouse has never been one to show much emotion. But in the last two years, he has had to close Jane, House & Garden, Men’s Vogue, Golf for Women, Domino, and finally Portfolio. At Condé Nast, the rumor mill, accurate or not, continues to grind. Which will be next? Wired? Architectural Digest? Does the company really need two food magazines? The grim work has taken a toll. His own personal wealth has declined by half, to some $2 billion, but personal wealth was never the point. “Without Condé Nast, he would cease to exist,” says a person close to him. “It’s where he comes alive.”

So when it dies a bit, he does, too. “I’ve never seen him so depressed,” says one person on the publishing side. On his next birthday, he’ll be 82, and Portfolio may have been his last great fling. Who knows whether he’ll get to launch another magazine?

Si Newhouse is nothing like his magazines. Short, physically unimposing, dressed for the office in khakis and beat-up loafers, he’s the opposite of glamorous. “He’s always had the luxury of being himself,” says a friend. He’s notably inarticulate, speaking softly, with long, excruciating pauses between words. A decision to commit millions of dollars might be communicated with a “very, very quiet whispered yes,” says one of his former editors.

It’s a type of decision Newhouse, one of the great media entrepreneurs of the past three decades, has made with breathtaking regularity. In 1979, when magazines like McCall’s, Good Housekeeping, Redbook, and other sensible books were leading women’s titles, Newhouse started Self magazine for a new generation of restless, body-proud female readers and bought GQ for a new style-empowered man. Four years later, he relaunched Vanity Fair, which—after years of huge losses amid editorial floundering—channeled and helped create the arriviste dreamscape that took off in the eighties. Along the way, he purchased The New Yorker, then brashly rebuilt it, grafting its sedate Shawnian DNA to Tina Brown’s topical buzz, creating a fascinating Frankenstein that still is at the core of the magazine’s identity. He also remade Details, a trend-dipping downtown title, and bought Wired, the champion of the technological revolutions that now nip at his empire.