Lady Ashton removes her spectacles before facing the foreign affairs committee of the European parliament in Brussels. Photograph: Olivier Hoslet/EPA

Last week she was in Madrid and Moscow, Kiev and Bruges, as well as chairing a Brussels meeting of EU foreign ministers and wining and dining Avigdor Lieberman, Israel's pugnacious foreign minister. The week before it was Bosnia, Serbia and Kosovo, mulling over whether she can make the Balkans a better place. This week she is in Haiti grappling with Europe's role in resurrecting that benighted country after declaring she does not do disaster tourism.

Catherine Ashton is on the move. Yet despite her punishing schedule, the knives are out for the Labour peer, not because of the places she has been, but because of the meetings she has missed.

In Paris and Berlin, The Hague and Brussels, the whispering campaign against the former leader of the House of Lords is getting louder.

Ashton was thrust into the international limelight in November as Europe's first foreign policy chief. She did not ask for the job. She was taken aback when named as a result of a classic EU political fix that had nothing to do with merit or suitability for the post.

"Lady Ashton has come under an unusual amount of criticism and attack in the first few months," said Thomas Klau, head of the Paris office of the European council on foreign relations. "It was obviously one of her handicaps that she wasn't put in a position to expect the nomination."

At the weekend, Ashton marked 100 days since European leaders named her as the EU's first high representative for common foreign and security policy. She was not the only one who was surprised. Eurocrats gulped with incredulity that a neophyte with no foreign policy pedigree could be awarded the post.

It was said to be the best job going in Europe. It is rapidly turning into the worst.

"Baroness Ashton has been given an absolutely impossible task," said Alexander von Lambsdorff, a German liberal MEP on the parliament's foreign affairs committee.

The critics are already feeling vindicated. Early French vitriol is widening into stronger and more substantive reservations about her judgment and early policy moves.

Brave face

Three decisions in the past 10 days have sparked widespread discontent in Brussels and across Europe. Ashton is in charge of creating the EU's diplomatic service under the Lisbon treaty and of making the key appointments.

With decision-taking in Brussels in a vacuum, José Manuel Barroso, the European commission chief, moved deftly to place his longtime chief of staff and fellow-Portuguese João Vale de Almeida as the new EU ambassador in Washington.

Last Monday, Ashton sought to put a brave face on things, insisting she and not Barroso had made the appointment while admitting that "one or two" foreign ministers had complained about being bypassed on such a key job.

Diplomats said 12 foreign ministers had voiced their consternation. She was warned that the incident should not be repeated. "Ashton should not have let Barroso impose that," said an EU official. "It has undermined trust in her."

The other appointment she made was of Vygaudas Usackas, a former Lithuanian foreign minister, as the new EU envoy to Afghanistan. Usackas was supported by Britain. Diplomats from other countries complained that Ashton was doing Britain's bidding.

The official added that the biggest blunder came last Thursday when Ashton, juggling a busy travel schedule, was seen to have made the wrong call by staying away from a meeting of EU defence ministers and Nato officials in Majorca.

Ashton's job puts her in charge of European security policy. The Majorca meeting was the first under the new regime and under her auspices. Anders Fogh-Rasmussen, the Nato chief, was there. Ashton went to Kiev instead.

"A bit rich," complained the French. "Conspicuous by her absence," said the Dutch. "Regrettable," noted the Spanish.

"That was an extraordinary mistake," said the European official. "It reinforces the impression of a total lack of understanding of the job. I really hope she can recover."

Criticism

Ashton's defenders argue she is being subjected to a barrage of snide criticism that is quite unfair and vastly premature. "Cathy says it's a marathon, not a sprint, and she should be judged on the results," said her spokesman, Lutz Guellner. "She wants to be judged on what she achieves and not on the perceptions. She has a job that needs to be defined, that didn't exist before."

A sympathetic diplomat added: "She's been put in charge of the most profound institutional change in the EU for years and she does not yet have a machine she can rely on."

Ashton is a foreign minister without a foreign ministry, responsible for building one not exactly from scratch, but from very disparate elements of the EU apparatus in the commission, in the council of the European Union and in the diplomatic services of the 27 member states.

She has a private office of 12 officials working on different parts of the world headed by James Morrison, a former Foreign Office diplomat who has worked with her since she replaced Peter Mandelson 18 months ago as European trade commissioner.

Mandelson wanted the job Ashton has now, and some of the early bitching in November came from London. But the whispering campaign was quickly hijacked by the French, with Jean Quatremer of Libération, a prolific blogger who trades in high-grade Brussels political gossip, marshalling what many saw as a smear campaign delivered by French officials invariably speaking anonymously.

"Lady Qui?" was the headline of a two-page spread in the leftwing Paris paper in February.

Despite working in Brussels for 18 months, Ashton continued to live in hotel rooms, Quatremer scoffed. Perhaps sensitive to the jibe, she has for the first time rented a property in Brussels and is said to have accepted that she will be spending less time in Hertfordshire. But Ashton is also aggrieved that other new European commissioners who regularly dash home for the weekend are seldom subject to the same attacks.

Quatremer berated Ashton for her "amateurism, incompetence even", while Pierre Lellouche, France's outspoken Europe minister, complained of "the current impression of a void". The criticism is not confined to the French in Paris or Brussels, nor is it the preserve of male officials and diplomats, despite the suspicion that sexism is a factor.

Miguel Angel Moratinos, Spain's foreign minister, who wanted the job himself, complained bitterly, if privately, about Ashton's performance at an international security conference in Munich in February. The Germans are known to be sceptical, but are being more discreet.

Ashton attended the Munich conference to rub shoulders with the American, Chinese, Russian and Iranian foreign policy leaders. She was generally seen to have delivered a lacklustre speech.

"There is an issue here," admitted a supportive European commission official. "She's going to have to start prioritising and start performing."

French grumble

Such perceived failings aside, Ashton is handicapped by the magnitude of the job coupled with the lack of experience and the temporary absence of support systems.

While being EU foreign policy chief, she is also a vice-president of the European commission, in effect doing a job performed until November by three senior people – Javier Solana of Spain doing foreign and security policy, Benita Ferrero-Waldner of Austria as European commissioner for external affairs, and the foreign minister of whatever country was holding the EU's six-month rotating presidency and chairing monthly meetings of EU foreign ministers.

If the workload is formidable, she is also burdened by being a relative unknown internationallyand not being plugged into the global foreign policy networks. There is also plenty of muttering in Brussels and beyond about the calibre of her staff. The French grumble, for example, that they have only one relatively junior person on her staff of 12. Only 100 days in, Ashton has not had the happiest of starts in one of the biggest jobs in Europe.

The burden of expectation is heavy, the mud-slinging is hurtful. Unlike the stellar crew of (male) European grandees who coveted her job, from Mandelson to Moratinos to Carl Bildt of Sweden, she is said to have little ego.

But in the chancelleries of Europe, especially in France, there is a palpable concern about the purpose of the new Lisbon regime and the possibility of chances lost.

"In Paris," said Klau, "there is a sense of mixed expectations and a sense of apprehension. The fear that Lisbon will not lead to a more forceful Europe is a real source of worry."