Britain is facing its most severe ever terrorist threat and fresh attacks in the country are inevitable, according to the head of Britain’s normally secretive domestic intelligence service in a rare public speech.

Andrew Parker, the director general of MI5, said the UK had seen “a dramatic upshift in the threat” from Islamist terrorism this year, reflecting attacks that have taken place in Westminster, Manchester and London Bridge.

The spy chief said: “That threat is multi-dimensional, evolving rapidly and operating at a scale and pace we’ve not seen before.”

He added: “It’s at the highest tempo I have seen in my 34-year career. Today there is more terrorist activity, coming at us more quickly, and it can be harder to detect.”

MI5 is under pressure to demonstrate its effectiveness after four Islamist terrorist attacks escaped its detection this year.

Parker’s speech to specialist security journalists on Tuesday was his chance to frame the debate about Britain’s battle against Islamist terrorism at a time when the agency’s staff numbers are already expanding from 4,000 to 5,000.

Parker said MI5 had stopped far more terror plots than those that caused mass casualties this year. He said 20 plots had been thwarted in the last four years.

Seven plots had been stopped before jihadists could strike in the last seven months alone, Parker added. “The threat is more diverse than I’ve ever known. Plots developed here in the UK, but plots directed from overseas as well. Plots online. Complex scheming and also crude stabbings; lengthy planning but also spontaneous attacks. Extremists of all ages, gender and backgrounds, united only by the toxic ideology of violent victory that drives them.”

He also defended the decisions made by his staff about which suspects posed the most danger to the public. “They get up and come to work every single day to make terrorist attacks less likely and to keep the country safe. They are constantly making tough professional judgments based on fragments of intelligence; pinpricks of light against a dark and shifting canvas. That is the job of MI5.”

Parker added that military defeat in Syria and Iraq for Islamic State did not mean its threat would wane. “Meanwhile, Daesh [Isis] itself is under military pressure and is rapidly losing ground in its heartland in Syria and Iraq. So much so that it’s now advising would-be fighters to choose other countries … At the same time the Daesh brand has taken root in some other countries where areas of low governance give it space to grow.”

He said 100 Britons were believed to have died fighting for Isis and fresh danger was posed by the potential return of 850 more who had travelled to its territory, although a large influx had not yet materialised.

Asked if a future attack in the UK was inevitable, Parker said: “I think we have to be careful that we don’t find ourselves being held to some sort of perfect standard of 100% because that just isn’t achievable.” He said it was likely attacks would occur.

The formal terrorism threat level remains at severe, meaning an attack is highly likely. Twice this year it has been raised to its maximum level, meaning an attack could be imminent, after the Manchester and Parsons Green attacks. In both cases there were fears among officials that suspects or bomb related materials could be at large.

Amid concerns that Brexit could damage security cooperation, Parker said working with European partners remained crucial to MI5’s mission. “We share intelligence. We run joint operations. Every single day.”

The spy chief was careful in his tone towards technology companies such as Google, Facebook and YouTube which have been accused of not doing enough to help thwart criminals and terrorists using their technologies.

Parker said that as technology advances, social media companies have an “ethical responsibility” to do more to help suppress terrorism. He warned that “an unintended side-effect is that these advances also aid the terrorists, whether it’s the ease of online purchasing, social media content or encrypted communications. No company wants to provide terrorists with explosive precursors.