Like freshman everywhere, Xander Manshel and his Middlebury College classmates found themselves in their first year of college pondering some of life's biggest mysteries—like how to play Quidditch if you can't, like Harry Potter, fly?

The solution: race around in capes and goggles with broomsticks between your legs, while shooting balls through mounted hula hoops. Their version of the game, first played in 2005, was modeled on matches described in J.K. Rowling's novels.

"Quidditch was this bridge between the fantasy world of the books and the more concrete world of college," says Mr. Manshel, who has graduated and now teaches English. "For us [playing] was a way to have both."

But now Harry has grown-up—and so has the sport. There are tournaments, new rules and special brooms for competitive play. The "Quidditch World Cup" is moving this year to the Big Apple from Middlebury's idyllic campus. More than 60 college and high school teams have registered to compete Nov. 13 and 14—up from 20 last year—at a park in Manhattan.

"Our hope is that it will be a real coming out party for the league," says Alex Benepe—one of the sport's founders and president of the newly formed nonprofit International Quidditch Association. It's now played at hundreds of schools, he says.

But just as Harry experienced growing pains, so has the "muggle" version of the game. (Muggle is the books' term for nonmagical people.) Some players want it to become more serious—with coaches, training and cuts to make the team. Others prefer to retain its innocence and inclusiveness, even for the un-athletic.

The game is "organized, but has a free spirit," says Kate Olen, a senior at Middlebury and its Quidditch commissioner. Initially, the game was more popular with Potter fanatics, but now, "more athletes are coming up," she says.

Valerie Fischman, who plays Quidditch at the University of Maryland, would like to see it go much further. She's been finding out what needs to be done to get the sport NCAA status. That, she says, could "be a stepping stone" to becoming an Olympic sport.

The National Collegiate Athletic Association says typically 40 to 50 schools need to sponsor a varsity sport for it to consider sponsoring a national championship. The most recent sport to gain such status: women's bowling.

Calls to Ms. Rowling's literary agent in London weren't returned. The Quidditch group doesn't have a licensing agreement with the author but has invited her to this year's and previous World Cup championships.

Kristen Howarth, 23, who founded a Quidditch team with her twin sister at Texas A&M, says initially there were snide comments from other organizations on campus. But they say it's gaining acceptance. "Some people still think it's a joke, but when they watch it, they're shocked at how physical it is," says Aimee Howarth.

She worries that if it gets too intense, it might lose some of its whimsical roots. "It's good to be competitive, but we need to keep in some of our original values," she says.

The co-ed game isn't for the timid—pushing, tripping and some tackling is allowed. New rules are designed to prevent some of the broken bones that occurred at last year's World Cup.

Ziang Chen, a sophomore at Purdue University, started a team there last year after seeing videos of the sport. "When I saw how brutal the sport is, I thought I would like to try it," says the former high school football player.

Some see a niche business. A company called Alivan's sells brooms including the "Scarlet Falcon" for $59 and the "Sienna Storm" for $79. Its website says the company "is proud to provide the official broomsticks of Intercollegiate Quidditch." It also notes its brooms "do not fly."

With the World Cup moving to Manhattan, says Will Bellaimey, 22, a former Middlebury player, "We're bringing the championship to the biggest stage in the world."

President of the sport's association, Alex Benepe, says he hopes the World Cup will be a 'coming out party' for the league.
Indeed, Brant Herman, waiting recently to play baseball at Manhattan's DeWitt Clinton Park, wondered why the tournament was going to be held there rather than "closer to Times Square"—the theater district.

His teammate John Hamilton didn't mind that the tournament would be taking place there, "as long as there are no broken brooms left on the field."

At this year's event, there will be owls and wizards on the sidelines, as in previous years, but also entertainers, some more used to performing in subways. Teams registered range from Ivy League Yale to football powerhouse Ohio State. Some get school funding, while others are unofficial squads, scrambling to find equipment.

When New York University sophomore Sarah Landis heard the championship was coming to New York, she decided it was time to get a team going. More than 60 students attended the first meeting, she says.

"We all secretly wanted to play this sport since we read about it" in the books, says Ms. Landis, who found $3 brooms at a Halloween store near the campus.

On a recent Sunday, more than 50 students, one in a cape, lined up for Middlebury's weekly Quidditch practice. A few dressed all in yellow because they played the role of the "Golden Snitch"—which in the books is a winged ball that usually needs to be caught to end the game.

There's nostalgia on campus about the World Cup, which drew about 2,000 people last year. "It's like being an empty nester," says Anika James, 21, a Middlebury player. "We've seen it grow beyond our borders, but it's sad too."

Soon, Mr. Benepe, of the Quidditch organization, will get the spray-painted plastic trophy cup out of a Middlebury storage closet. (The winning team gets its name written on the cup with a Sharpie pen.)

The group is selling T-shirts and collecting team-entrance fees and donations to raise the roughly $20,000 it says is needed for the tournament. Long term, Mr. Benepe wants to focus on getting students of all ages to play.

But could popularity make the Quidditch magic disappear? Mr. Benepe doesn't think so.

"A lot of sports" he says, have "become more like work. Quidditch is just about playing a game. It's just about having fun."