Richard Saul Wurman lounges by the hotel pool, sweat collecting under the crease of his pectoral muscles as the heat of a southern California afternoon rises in step with the anticipation surrounding the build-up to his latest conference, WWW.

It is only a few hours before the $16,000-a-ticket, three-day experimental event begins, and Wurman, a master at creating conferences – including the now legendary TED or Technology Entertainment and Design events – is on edge. Around 40 high-profile speakers, including David Blaine, Frank Gehry and Quincy Jones, are already arriving at the hotel. Over the next two days, at a nearby auditorium, they will engage in one-on-one conversations in “an energetic exploration of the lost art of conversing”.

Wurman, 77, puts on a shirt in between last-minute phone calls and finds some shade under an umbrella. A waiter comes by the pool to pick up the plate with remnants of the cheeseburger, no bun, that Wurman ate for lunch, and asks, with rehearsed professionalism, “How did everything come out?”

Wurman bristles. “Everything,” he says through a clenched jaw, “was fine.” The waiter has broken one of Wurman’s rules of conversation: “Never ask anyone, ‘How is everything?’: it’s not a question that can be answered,” he says. In fact, he continues, he has a list of other taboo phrases, among them “How are you?” and “what’s the best ... ?”

The worst is “uh-huh”, which, according to the former architect, cartographer and urban designer, is not a harmless social cue but a meaningless tic used by bad listeners pretending to understand someone when they don’t.

For Wurman every detail is an important part of the experience he is trying to create: a series of conversations, threaded together by themes and meals and musical breaks that, taken together as a whole, feels like a piece of theatre.

It is something he feels he achieved for several years with TED, which he first hosted in 1984 in Monterey, California. The event grew and grew, and since Wurman sold it in 2001 for $14m, has developed cult status, attracting thousands of followers jockeying for its $7,500 entry tickets for four days filled with highly produced 18-minute “talks of a lifetime”. TED Talks, a series of lecture videos posted online, have received more than 800m views to date. University professors assign them as required course material. Some airlines, such as Delta, even have a TED channel on their in-flight entertainment systems.

But, in Wurman’s opinion, TED today has become over-orchestrated, too “slick”. He intends WWW to be the opposite, to be an exercise in improvisation through conversation or, as the conference tagline runs, “intellectual jazz”. The event’s title, WWW, does not have a single meaning, says Wurman, suggesting instead a long list of words beginning with “w”, including “wanderlust”, “warming” and “wizardry”.

Wurman originally formed TED by “subtracting” elements common to other conferences: introductions, lecterns, suits and ties. “I took away CEOs who legally can’t tell the truth,” he says, “and politicians who can’t tell the truth because they serve so many constituencies.”

In recent years, though, too many things have, he feels, crept back in. “Now every speech is auditioned, rehearsed, edited, rehearsed again,” he says. “The spontaneity is gone and there’s a lot of selling of charities. There’s the selling of being PC.”

Megan Smith, a Google vice-president and a guest speaker at WWW, agrees: “TED talks today are very prepared, which respects people’s time, but there’s not as much of that raw information.”

To achieve “intellectual jazz” at WWW, Wurman puts together two people – sometimes friends, sometimes strangers – such as Simpsons creator Matt Groening and New York Times columnist David Brooks, or cellist Yo-Yo Ma and hip-hop star will.i.am, in the hope that, through the course of an unplanned conversation, a few “good moments of honesty” will emerge.

“It’s not selling a charity or selling guilt or a movie or a book,” he says. “I thought, ‘How do I pair people so they’re not competing with each other to see who’s smarter.’”

In a world where people are more connected than ever through technology, where the internet has created an information democracy, giving people access to ideas from around the world, including the most accomplished, untouchable experts and leaders, there is still a booming business for in-person conferences where the high-powered come in search of inspiration.

There’s the World Economic Forum in Davos, Switzerland, for financiers, the Sun Valley retreat for media moguls, and Renaissance Weekend for the political set. Then there’s a range of TED offshoots and wannabes, from EG and TEDMED, both started and later sold by Wurman, to the ultra-hip techie fests such as South by Southwest, PopTech, and Techonomy. Wurman will add another three to the mix in the next three years: Prophesy 2025, the Geeks and Geezers Summit, and FEDMED.

The business model of hosting conferences has become increasingly appealing to various business sectors, particularly the media industry, as newspapers, hit by a downturn in advertising, look for new ways to squeeze money out of their brands. To the attendees who pay four to five figures to attend, sometimes six for premium tickets, and to the invitees who agree to be the bait for those who pay, the lure is the promise of being on the cutting edge of a new thought, one that hasn’t yet been committed to the internet or spread virally on social networks.

Even those charged with thinking of big ideas every day are desperate for new ideas. John Gage – one of the earliest employees at Sun Microsystems, a venture capitalist, long-time TEDster and guest at WWW – says these kinds of gatherings give executives a break from being the boss. “It’s not just introducing people to new ideas,” Gage says. “It’s enriching your ability to appreciate them.”

For those coming to WWW, there is no promise of new ideas, just a pledge, a mere hope, that putting accomplished scientists, musicians and architects in conversation together would yield enough “nuggets of truth” to justify the $16,000 entry ticket.

Two grey sofas with red cushions face each other on the stage, with an armchair in between – Wurman’s throne for the next two days. The sofas have been designed specifically for this conference. To the left are a series of glass sculptures by artist and entrepreneur Dale Chihuly, eight floral shapes in green, purple and orange, set atop black pedestals 7ft above the floor. To the right is a mini-grand piano and two chairs, where cellists Yo-Yo Ma and Mike Block will conduct their scheduled “conversation” in music. The backdrop is a grey cement wall – a blank slate. The musical interludes or a momentary pondering of the flower sculptures are intended to reset the intellectual mind to a more contemplative state.

The logistics of WWW have all been carefully thought through. Wurman picked the auditorium – at Esri, a mapping software company in Redlands, California – because of its location, a small desert city 63 miles east of Los Angeles and sufficiently remote from any business meetings or lunches that might lure attendees away. Even the hotel, a fairy tale of Spanish mission architecture, where all the speakers and attendees are staying, is 30 minutes away by car, and there is only one bus to the conference centre – at 7am sharp – and one shuttle back – at 8pm.

The success of the format, and the measure of new ideas, ultimately lies with each attendee, as the reactions to the conversations are as varied as the conversations themselves. Some flow freely back and forth, as when Brooks and Groening compare their creative processes.

The talk between Herbie Hancock, pianist and composer, and will.i.am, pop star and business guru, turned into more of a one-man show with will.i.am dominating the stage. “We don’t need another musician coming from the ghetto,” says will.i.am, referring to the neighbourhood outside Los Angeles where he grew up. “We need a Mark Zuckerberg.”

For David Blaine, the Houdini-style endurance stuntman, finding truth is a matter of forgetting. Besides a repertoire of card tricks, he has been buried alive in New York for a week and spent 44 days in a transparent box in London with nothing but water. When Julie Taymor asked him why he treads a line of death, he replies: “When I push as hard as I can, I forget all the nonsense.”

Other talks revert to one speaker interviewing the other. This format draws lots of details from cable TV mogul Mark Cuban on what it’s like owning the NBA’s Dallas Mavericks but it leaves the audience not much wiser about the talents of Dan Ariely and his research in psychology or behavioural economics.

As for yielding nuggets of truth and wisdom, Lee Larson, a philanthropist and one of few attendees to pay for his ticket, says, on that count, the conference was “a lost opportunity”. He says he saw more “haphazard ramblings” than real conversations. “Deep ‘wow’ moments coming out of this intellectual jazz – nope! ... I would much rather have given the money to charity.” Wurman later says that he, too, regretted the price tag – $10,000 would have been more appropriate. He wouldn’t say how many tickets were sold but it was too few to break even on his costs. Most attendees are friends of Wurman’s who attended at his request, at no charge, to help fill the auditorium. As a business venture, this particular conference might appear a failure but that isn’t the point Wurman is out to prove.

“I strongly encouraged him not to do it,” says Wurman’s friend Nicholas Negroponte, founder of the MIT Media Lab and the One Laptop per Child programme, who spoke on stage with Google’s Megan Smith. “He created these other conferences, sold them, and got in big fights with all the new owners. He’s persona non grata at all of them. I thought the motivations were wrong. That he was trying to get back at them.”

Though Wurman’s new conference has lots of bigwig guests, and a handful of attendees willing to pay a premium to mingle with them, it lacks the high-tech lighting of TED, the beanbag chairs and ceiling-mounted television screens in the hallways, and the rows of refrigerators stocked with soft drinks and herbal iced tea.

But then this is what Wurman wants: he calls WWW the “great leap backward”, both from TED and from the hyper-connected, hyper-busy modern world. “WWW could have taken place 2,500 years ago, with Aristotle and Socrates on stage,” he says.

During the Wednesday mid-morning break, after hit television producer Norman Lear tells DreamWorks chief Jeffrey Katzenberg stories of skimming rocks across the water with his grandfather, a group of women gathers in the ladies’ room. They debate the format’s merits, with some arguing that Wurman inserts himself too much in the conversations, not letting them unfold naturally, and that his presence in the centre of the stage is distracting. Others appreciate his interruptions or, at least, excuse them. The circle continues to grow as more women come in and join the conversation, which inevitably includes a question so often whispered at these conferences: why are there so few women on the programme?

On the first day, only one woman, the director Julie Taymor, was scheduled to sit in discussion on the grey and red couches, out of 24 total speaker slots. The next day, there were four women out of 23 slots.

Taymor asks Wurman why. “I asked the best people I know,” he tells her. He repeats this to me outside the auditorium during an afternoon break on Wednesday. He takes a bite of a tortilla chip and hummus, and adds: “When I ask the best people that are women, they generally turn me down. One after another turns me down. They don’t mind being on boards but they don’t see speaking in front of an audience as their way of participating.”

Or, they’re simply maxed out with speaking engagements. All the competing conferences ask the same women at the top of their fields to speak, and often the same men for that matter. Even TED is running out of speakers to invite and, therefore, running out of big ideas. This spring, it resorted to reality television strategies, hosting a series of American Idol-style auditions to identify speakers for future TED events.

At WWW, in conversation with Yo-Yo Ma, David Brooks blames the lack of new talent on the era. The last golden age of non-fiction thinkers was, he says, between 1955 and 1965. Back then, writers were grappling with an “all obsessive” cultural problem, shifting away from the group mentality of the second world war to assert their individuality. Today, there is no such cultural struggle. “Our culture has become more morally diffuse,” he says.

Ma agrees and draws a parallel involving the period when luthiers crafted the best violins and cellos but today, he says, there is less motivation for musicians and composers to create. “There doesn’t seem to be an enlightened way of thinking that people can follow and say, ‘This is something we can believe in,’” he argues.

Perhaps that is why people are so hungry for the entertainment and inspiration of being exposed to new ideas. Matt Mullenweg, a founding developer of the free blogging platform WordPress, and, at 28, the youngest attendee who bought a ticket, says that, over the course of the conference, he has ordered 14 books on Amazon written by the various speakers.

Mullenweg left the University of Houston after his second year to build his tech company, so he sees conferences such as WWW or EG as a university surrogate, a place to learn, a place to see ideas from vastly different fields juxtaposed. For him, the $16,000 is like a tuition fee.

“I always leave feeling inspired and creatively charged,” he says. “As an executive, you have to seek out opportunities to learn because it’s not something that’s going to happen naturally. Because you’re part of a very fast-growing enterprise, you have hundreds of emails a day, you’re managing hundreds of people, these can easily take all your waking hours, you’re not left with time to contemplate. This is taking a step back.”