Staff with models at the opening of Abercrombie & Fitch's London store in 2007. Photograph: Richard Young/Rex Features

Even before I reach Abercrombie & Fitch's flagship London store, three separate groups of European teenagers stop me to ask where it is. This is destination shopping. I follow one group of girls in and they rush up to the topless man by the entrance to have their picture taken with him. He stands in front of a huge black-and-white photograph of two half-naked, muscled young men embracing. One of the mysteries of this brand is why so many straight teenage boys, not a group known for its tolerance of homosexuality, are not put off by the brand's overtly homoerotic imagery. Another is why anyone would pay £94 for a hoodie, or £50-plus for a polo shirt.

The air is pumped with nausea-inducing fragrance, the music is nightclub-loud and it is so dark that I keep bumping into cabinets and stray tweens who have become separated from their parents. "You get used to it," says a ridiculously handsome sales assistant with a perfect smile. There is no room for negativity in the Abercrombie & Fitch universe.

"It's a loathsome experience," says Andy Pike, 47, a risk analyst for a bank, who leaves with a present for his niece. "It's so contrived, it's overpriced. It was so dark I couldn't tell what colour something was." And the music is so loud! He nods. "That makes us sound really old doesn't it? I'm not the target customer."

This store, on the street that adjoins Savile Row, London, opened in 2007, in an 18th-century Grade II listed building with blacked-out windows. They were not exactly welcomed by the neighbours. One of Savile Row's bespoke tailors told the Times: "If the bespoke businesses were driven out by crappy retail stores selling poor-quality clothes, Savile Row's name would be irreparably damaged." This week a protest was staged to stop it opening a second store – this time selling the Abercrombie childrenswear brand – on the Row itself.

"This is one street in a city full of chain stores and it's not the right place for Abercrombie & Fitch," says Gustav Temple, editor of The Chap magazine ("a journal for the modern gentleman"), who organised the protest. "I think it's a very cynical move to try to muscle in on the heritage and sell something that doesn't reflect Savile Row in the slightest. It's a shop selling kids' underpants and T-shirts with slogans on them." He sighs. "They're probably gloating, thinking, 'Wow, more publicity.' I hate to think that."

The company, which also owns the Hollister and Gilly Hicks lingerie brands, has been the subject of boycotts from feminist groups – for T-shirts that read "Who needs a brain when you have these?" splashed across the chest – and has outraged people for selling thongs for children with "wink wink" printed on the front, and last year for selling "push-up" (its description) bikini tops for girls as young as seven. Its advertising campaigns, shot by fashion photographer Bruce Weber, are regularly criticised by decency campaigners in America for being far too suggestive. The company seems to almost delight in winding up activists and the religious right.

In 2009, in the UK, a former employee, Riam Dean, took it to an employment tribunal. Dean, who was born with the lower part of her arm missing, claimed she was forced to work in the London store's stockroom because she didn't fit the company's strict "look policy", a guide to the appearance of its shopfloor staff. She was awarded £8,000 for unlawful harassment (though the tribunal ruled she did not suffer disability discrimination). Last month, a union in Italy claimed staff in the Milan store had been made to do press-ups as punishment for mistakes.

In 2003, a bigger storm erupted over its hiring policy. Several Hispanic, black and Asian employees and applicants sued the company in the US, saying they had been put into less-visible backroom jobs. In 2004, the company paid a multi-million dollar settlement, though admitted no wrongdoing. It appointed a diversity and inclusion officer to increase the number of non-white employees in its stores, from 10% in 2004 to 53% in 2011, but there have been two recent lawsuits filed by women who alleged they were discriminated against for wearing a hijab.

From a public relations perspective, these allegations should be damaging – and numerous commentators and bloggers are routinely critical of A&F – but the company appears to survive them (it also made headlines last year for offering to pay stars of a reality show not to wear its clothes). But surely these kind of allegations are as bad as they come?

"I am fascinated by this," says Mark Borkowski, a PR expert. "They've got a weird arrogance. In textbook terms, it should hinder them, but in a bizarre way it's helped them. They ride the storms and manage not to focus on any key figure – I don't recognise any spokesperson who regularly turns up, the focus is on the brand. They've got a very keen identity of who they are, what they want, who they want to consume their products, and they've stuck to it."

Today's Abercrombie & Fitch would be unrecognisable to its founders. David T Abercrombie opened Abercrombie Co in Manhattan in 1892 as a gentleman's outfitters and outdoorsmen's shop; one of his customers, wealthy lawyer Edzard Fitch, bought into the company and it was relaunched as Abercrombie & Fitch in 1904. It dressed Charles Lindbergh, Amelia Earheart and Katharine Hepburn, sold guns to Ernest Hemingway and blazers to JFK. In 1976, it went bankrupt, and was revived by another company in 1978, who then sold it to The Limited, an apparel firm. It brought in Mike Jeffries, who has transformed it from a company losing $25m a year to one worth $4bn today.

For the chair and chief executive of such a high-profile brand – and one of the US's highest-paid CEOs – Jeffries keeps a low profile. In 2005, he gave interviews to Salon.com and Business Week. Both painted him as hardworking, focused and driven, but decidedly odd. The Business Week piece detailed his idiosyncrasies: "Jeffries leaves his black Porsche … at the same odd angle at the edge of the parking lot. Everyone knows why: Jeffries is superstitious about success. That's why he always goes through revolving doors twice. Associates have learned not to pass him in stairwells."

Benoit Denizet-Lewis, a writer for the New York Times magazine, recalls the two days he spent at The Home Office – the name given to the A&F headquarters in Ohio, where youthful staff zip along on scooters and hang around a bonfire as if at summer camp – for his Salon piece. "On the plus side I found him to be refreshingly honest, compared to other fashion retailers," he says. Jeffries told him: "We go after the attractive, all-American kid with a great attitude and a lot of friends. A lot of people don't belong [in our clothes], and they can't belong."

"At first you're taken aback," says Denizet-Lewis, "but many retailers believe that, they just won't say it out loud. He very quickly gets defensive if you try to talk about some of the many public controversies that have affected Abercrombie. I was asking some very basic questions and he … said he was not going to cooperate any more." (A&F also declined to comment for this piece.) Denizet-Lewis says he was struck by the appearance of the then 61-year-old Jeffries, with his bright white teeth and bleached blond hair, dressed in flip-flops and torn jeans. "He has turned himself into a caricature in some ways. He has tried very hard to look like this group that he really idolises – young, attractive, social, carefree."

It was Jeffries who emphasised youth and sex, creating an idealised image of clean-cut, frat-boy hunks, and a conventional, cheerleader-type look for girls, a Gatsby-style, affluent, youthful clique, updated in brightly coloured sweatshirts and logo'd T-shirts. Jeffries brought in Weber, known for his own idolisation of athletic, semi-naked men, to create the brand's imagery, and introduced the store's "look policy" – shopfloor staff are called "models", not sales assistants, and only beautiful people are hired (more average types are relegated to stacking shelves). They are often aspiring models, dancers or actors who work part-time. Potential employees are scouted in the street and in stores, and a chosen few are selected to be photographed by Weber for advertising material.

"In this country its appeal is limited to teenagers and cliquey students," says fashion writer and Grazia contributing editor Melanie Rickey. "People who are interested in fashion don't wear Abercrombie, at least not in public. If I were 16, I would make my pilgrimage down to the shop. It's a rite of passage for all teenagers."

It "is one of the best examples of a brand that is incredibly focused and disciplined about its retail execution," says Karl McKeever, brand director of retail marketing consultancy Visual Thinking. "It's very targeted, precisely executed, nothing is left to chance. It is multi-sensory." The stores are pumped full of its fragrance, "an idea many brands don't pursue. Music plays an incredibly important part. They create this intense, nightclub sound. It's very high-energy music, with high beats per minute, and one track is mixed into another so there is this continuous high-energy environment which helps to elevate the heart rate, keeps the audience in a heightened state of emotion. This translates into participation – ie they are buying more."

The darkened stores mean lighting is focused on the products, "illuminated so highly and brightly, they're like sweets in a sweetshop. And the products are very lavishly displayed – the tables are full, it's abundant. It's very attractive and polished, it's giving this ideal, perfectionist view of the world as told by Abercrombie & Fitch." It's an aspiration that is easily sold, he says, because the target market is "a highly impressionable younger audience."

"The shares had a bit of a wobble last year, because having opened stores in Europe in flagship locations, those stores have a strong opening and then sometimes they go backwards," says a former retail sector analyst. More than half of Abercrombie's revenue now comes from its cheaper Hollister brand stores, which are "being rolled out more rapidly than Abercrombie & Fitch itself. The strategy seems to be A&F in the flagship location; Hollister works in the malls. Hollister is pitched slightly cheaper and in the UK market, when you look at Abercrombie's prices, they have basically done pound-for-dollar so it is more expensive in the UK than in its home market. The other brand they are just starting to push is Gilly Hicks." Next month, flagship Hollister and Gilly Hicks stores are opening in London.

It is a mixed picture. Outside the London store, I stop two young women who are unimpressed. "It used to be exclusive, but not any more," says Audrey Vanderstraetten, a marketing student from Belgium. "It's too expensive," says her friend. Then there are the two other teenage girls who skip out of the store, shrieking and giggling, clearly thrilled by their Abercrombie & Fitch experience. They are each clutching a photo of themselves with the topless A&F model, but, I notice, no carrier bag.