Disaster strikes Norwegian government web portal

Altinn is a web service run by the Norwegian government, on which citizens can find, fill out and deliver forms electronically. Every year Norwegian citizens can also log in to check their tax results. Since the birth of Altinn, the public has complained that the service is too slow, and every year the server has crashed due to high traffic from people wanting to check their taxes.

2012 has proven to be no different. The tax results were published at around 6:00 AM local time on Tuesday the 20th. By 9:00 AM over 200,000 people had tried to log on, and as a result the server crashed.

This was the status until noon, where traffic evened out and the server was stable again. Logging in is fairly simple: you type in your social security number and a personal password, and you receive a pin-code that you need to type in. At 6:17 PM local time, every single user who tried to log in went right past the login screen, and found themselves logged in as Kenneth, a 36 year old man from Oslo.

Users then had access to all financial data of this unfortunate person over two years back in time, in addition to the financial information of his wife and the company he worked for. Altinn shut down some 15 minutes later, and has been down since.

It is not known how many people got access to this information, or if any data were copied or downloaded. According to Jørgen Ferkinstad, communications director for Altinn, Kenneth had logged in and his information got stored in the server’s cache memory.

It is unknown how long Altinn will be down, and what is being done to prevent this from happening again. Kenneth had at 8:00 PM contacted his lawyer, and refused to give any statement. Brønnøysundregisteret, the company responsible for the web portal, were assembled for a crisis meeting at 11:00 PM. To make matters worse, DNV, a Norwegian company responsible for quality assessment and certification, published a report in the beginning of 2012, stating:

“Altinn is a rushed solution, testing has been lackluster at best, the service has very few options for future upgrades and the overall quality is considered to be below average. Furthermore, we question the competence and preparation of the publisher to manage such a complex system as Altinn.”

According to this report, there were no plans for backup, and the service was not built to handle the scale of the requests seen on Tuesday morning.

21 Mar 2012 ~ 12:40pmJokke
You know we got too much money when the development and launch cost one BILLION Norwegian Kroner ($173 681 978), paid by the end user, the taxpayers..... I feel sort of cheated..

22 Mar 2012 ~ 5:51amPelotard
You should note that in Norway, your income statement is a public record (at least as far as I know; I live in Sweden where this is definitely the case, and Norway has a very similar system). This means that anyone can walk into the tax office and demand to see any piece of paper, including anyone's tax returns, and they're not allowed to ask who you are or why you want to see it. The reason for this (which to most Anglo-Saxons seems like having a nude picture of yourself posted on your garage door) is that the taxman is regarded, in a very real sense, to be a servant of the people. That is, the people - every single citizen - employ and own the authorities, and is entitled to inspect their behaviour on a whim. "By and for the people" isn't just an empty phrase.

23 Mar 2012 ~ 12:58pmJokke@Pelotard: Yes, the tax records are (somewhat) public. You get to see a person's income before tax, how much was paid in taxes, and how much value (estates, cars, savings etc) that person has. HOWEVER, these data are all from the pre-correction phase. This mean that in cases where people document that they get xxxx amount of money returned, it will not show on these data. Also, the data Altinn managed to display on "Kenneth" were way more comprehensive than anything you will ever see from checking a persons tax record.

Update: It is estimated thet 1500-2000 people got access to "Kenneth"'s information during the 17 minutes before the site was shut down. It's determined that server caching caused the problem. Altinn was back up at 11am, local time on Friday. Caching has been disabled on this server which means it will perform even worse than before. Companies, the biggest usergroup of Altinn's services has got an extension on the deliver date for their taxes. At one point thursday evening, it was decided to print every tax statement and send them to the recipients by snail mail. This will cost 2 million NOK.It is unknown whether this will have any repercussions on the people responsible.

26 Mar 2012 ~ 11:14amAnonymousPunter
...so the "server" caching Kenneth's data, was actually the F5 BigIP load balancer (). I'm no fan of Accenture, the distinguished vendor behind the unscalable Altinn-behemoth, but let's give "credit", where "credit" is due - this was a case of the BigIP-boxes performing "queue management" (because the back end system couldn't handle the load) failing under the pressure. Caching was never supposed to be enabled on these boxes - it "kicked in" under very high load. It was an unknown bug in the BigIP-software - off-the shelf components, used in lots of large installations. "Performance" (if you can apply such a word to Altinn) won't be affected by this caching being disabled.

9 Apr 2012 ~ 5:33pmJokke
The company responsible is now looking for 29 new employees in the ICT-sector, among them technical project managers, senior architects, and test managers. Something for currently unemployed/IC-members looking for new jobs?