More than 72 hours after it first went down, the PlayStation Network is still offline, annoying millions of loyal PlayStation 3 customers who can't play their games over the network. No Portal 2 co-op, no Mortal Kombat matches, no Home. Your PlayStation 3 is offline. In an update over the weekend, Sony's Patrick Seybold confirmed that the outage is due to one or more hackers attacking the system. Here's the full statement.

An external intrusion on our system has affected our PlayStation Network and Qriocity services. In order to conduct a thorough investigation and to verify the smooth and secure operation of our network services going forward, we turned off PlayStation Network & Qriocity services on the evening of Wednesday, April 20th. Providing quality entertainment services to our customers and partners is our utmost priority. We are doing all we can to resolve this situation quickly, and we once again thank you for your patience. We will continue to update you promptly as we have additional information to share.

The good news? Games can still update! I'm in the process of installing three different updates for a game that's not out yet. As you can see, I'm all about looking on the bright side of these situations.

It also seems that this outage has caused Sony to beef up security. "Our efforts to resolve this matter involve re-building our system to further strengthen our network infrastructure," Seybold wrote in yet another update. "Though this task is time-consuming, we decided it was worth the time necessary to provide the system with additional security."

That's all well and good, but the system has been down for days, Sony has confirmed that it is under malicious attack, and the PlayStation Network has access to my credit card information. Wouldn't it be great to know if that information is secure? It would also be nice if we had some kind of estimated time on when we will be able to play our online games again. Yes, the PlayStation Network is a free service, but pointing that out seems like an excuse. Gamers expecting to be able to play their games online, especially on a holiday weekend, are bound to be upset. This isn't just a case of getting what you pay for.

The utter lack of transparency is galling. Will we be able to play online soon? Do we need to cancel our credit cards? Answers to these questions would be wonderful.

By disabling said quality entertainment services for 72 hours. Really I'm just bitter because a friend and I scheduled Thursday morning to play a game that we've been trying to play together for months. Now we'll just have to wait a little longer, I guess...

With the prevalence of all these attacks happening at a person's/group's whim, just shows how chaotic the unrestricted internet is. Unlike in real life where there is that threat of policing bad behaviours, you can pretty much get away with anything on the internet. Oh well

I bought Portal 2 on the PS3 just for activation on PC. (Because I could trade in some old Xbox games for it, which I can't do on Steam.) Unfortunately that was Saturday so I haven't been able to play on the PC yet.. The code that comes in the box requires you to play on the PS3 first and link accounts before becoming valid.

Interesting this happened at around the same time as a much, much more serious one. While I'm sure the downtime is irritating to Sony's customers (and it appears that Penny Arcade had no reason to fear and post today's comic early "just in case they get the service back up"), I'm honestly surprised this is getting so much coverage considering the huge outage of Amazon's EC2 that happened the past few days. That knocked out or at least temporarily messed up a lot of different major services (Reddit, Quora, etc) and some more minor ones that nevertheless are likely known to the Ars audience at least (for example Cydia was offlined), yet not even a single mention of it on Ars. I guess I'm just honestly surprised, I'm used to seeing that sort of major tech news at least mentioned here, so the dead silence was weird. I'll definitely be curious to hear about Amazon's postmortem, because there really is a service that's expensive and has full SLAs associated with it.

But this also presents a nice contrast of how differently companies can handle a service disruption. Amazon was better from the start, but they also quickly learned their lesson about lack of constant updates, in contrast to the apparent tone-deafs at Sony have. Per Amazon (from their AWS Service Dashboard):

Quote:

For customers who are still waiting for restoration of the EBS control plane capability in the impacted AZ, or waiting for recovery of the remaining volumes, we understand that no information for hours at a time is difficult for you. We've been operating under the assumption that people prefer us to post only when we have new information. Think enough people have told us that they prefer to hear from us hourly (even if we don't have meaningful new information) that we're going to change our cadence and try to update hourly from here on out.

Sony should take a page from Amazon's book here and set up a similar service site with frequent updates about what they're doing and what they know. At the very least needs to get the financial part (credit cards) straightened out ASAP, as the article rightly points out. People may not have much right to expect true 99.5% type uptime from a free service, but even if the service is free they absolutely due have a right to expect security of their financial information, and to receive prompt warnings to take action if that system is compromised.

It probably got less coverage because the causes were completely different. (Also it started on the weekend, not before it). I forgot where, but Amazon said it basically went down because a bug in their system caused their servers to basically make endless backups until all of their storage space was full. (Vast simplification, but the right concept) "Someone fucked up some 0s and 1s" is less newsworthy than "a bunch of idiots DDoS'd someone again"

Why the hell do I need to be logged into the PSN to watch a movie with my Netflix account?

Sony.

You don't. After it fails 3 times you'll have access to Netflix regardless. Annoying, yes, but it does still work.

Nope. When it asks you to sign in to the PSN, when starting up Netflix, and it fails & tells you that you can't sign in to the PSN at the moment, just hit Back. It lets you into Netflix. No "after it fails 3 times" required.

Nope. When it asks you to sign in to the PSN, when starting up Netflix, and it fails & tells you that you can't sign in to the PSN at the moment, just hit Back. It lets you into Netflix. No "after it fails 3 times" required.

Nope, not how it has been working here. When I hit back Netflix starts loading again and then the login screen pops up again. At this point I can see Netflix loaded in the background. So I try to sign in again, it fails, and then a few seconds later the login screen pops up again. After this third time it finally goes away for good. It does this every time for me as well as many others... thus the confusion from people thinking it doesn't work. Most probably give up after the second login attempt.

It probably got less coverage because the causes were completely different. (Also it started on the weekend, not before it). I forgot where, but Amazon said it basically went down because a bug in their system caused their servers to basically make endless backups until all of their storage space was full. (Vast simplification, but the right concept) "Someone ****ed up some 0s and 1s" is less newsworthy than "a bunch of idiots DDoS'd someone again"

Agreed. I'm also thinking Sony trying to cover everything up and not really explaining anything isn't helping. Since then you just get a bunch of people asking 'why?' and the occasional conspiracy theory. And a lot angry people without answers who just want to use the service. Sony seems like the quintessential example of poor PR these days...

Hmmmm, I just wish they'd come clean about the security of our info. I've already had Amex call me about my card being compromised this weekend, with half a dozen online charges from $7 to $900. I've got a new card coming but if this Sony issue was the cause, consumers need to know.

Exactly! Please Ars/Ben... remember that there's a pay version of PSN.

Except PSN Plus isn't a pay version of PSN. It's a discount program for stuff on the PlayStation Store. The store has been unavailable for several days now, and Sony may want to extend PSN Plus memberships to comp for that, but it's unrelated to access to network multiplayer, etc., like it would be if X Box Live went down for Gold members.

Every company has internal weaknesses and external threats. That being said, my Company XYZ is far less a target of external threats than Amazon or Google. I appreciate the idea of "cloud", but everybody and their brother are after the white whales, while I'm an insignificant target. Thus, my chances of going down due to an external threat are far less if I host internally.

That's a generalization, of course, but I believe it holds true for the most part.

Exactly! Please Ars/Ben... remember that there's a pay version of PSN.

Except PSN Plus isn't a pay version of PSN. It's a discount program for stuff on the PlayStation Store. The store has been unavailable for several days now, and Sony may want to extend PSN Plus memberships to comp for that, but it's unrelated to access to network multiplayer, etc., like it would be if X Box Live went down for Gold members.

Hmmmm, I just wish they'd come clean about the security of our info. I've already had Amex call me about my card being compromised this weekend, with half a dozen online charges from $7 to $900. I've got a new card coming but if this Sony issue was the cause, consumers need to know.

Makes me glad I de-linked my account info before I went overseas and left the PS3 in the hands of my folks. (They needed a Blu-ray and wanted Netflix streaming.)

For what it's worth, five seconds of explanation: "let it go until it fails and then hit back" was enough for my folks to continue using Netflix this weekend.

The thing isn't whether it's 'free' or not, but whether this is the service people expected when they got their PS3 to begin with. You did indeed 'pay' for it with your choice to get the system based on the claims of the service.

Exactly! Please Ars/Ben... remember that there's a pay version of PSN.

Except PSN Plus isn't a pay version of PSN. It's a discount program for stuff on the PlayStation Store. The store has been unavailable for several days now, and Sony may want to extend PSN Plus memberships to comp for that, but it's unrelated to access to network multiplayer, etc., like it would be if X Box Live went down for Gold members.

Oh, and cloud saves too.

The only annoyance I've had is that I'd like to transfer data from my old to new PS3, but with PSN down, I can't deauthorize the first console. Oh well, not the worst thing to have happen.

The CEO of Rackspace made an interesting point the other day. Statistically, you're far more likely to die in a car accident than a plane crash, yet which are people more afraid of and which makes the news? Granted, he's biased as all hell, but the point stands, it has a much more obvious effect when it goes down but the total cost is far less than the usual outages that happen at companies all the time.

Amazon going down will shift some minds, but it should only change them towards backup plans rather than simply dumping cloud services, they are still more reliable. Anyone who paid for greater security (to have hosting through multiple Amazon locations in this kind of situation) didn't go down during this, basically they paid for insurance. Those who took the cheaper option (startups who want to cut costs as much as possible) will have to think about whether expanding without limits at the lowest cost is really the best move.

The thing isn't whether it's 'free' or not, but whether this is the service people expected when they got their PS3 to begin with. You did indeed 'pay' for it with your choice to get the system based on the claims of the service.

Yep. Plus every time I buy a game with online features I am indirectly paying for the PSN. Same as when you buy a game through Steam. You don't pay for Steam directly, but you expect it to work as the service is part of the offering when you buy the games.

That being said, my Company XYZ is far less a target of external threats than Amazon or Google. I appreciate the idea of "cloud", but everybody and their brother are after the white whales, while I'm an insignificant target. Thus, my chances of going down due to an external threat are far less if I host internally.

Correction, your chances of being attacked by an external threat are far less if you host internally. Given that you are unlikely to be able to invest nearly the the amount of resources that Amazon has in security and redundancy, your chances of going down to an external threat are actually far more than Amazon. Proof? Amazon still hasn't gone down to an attack. (They went down to an internal software bug)

Every company has internal weaknesses and external threats. That being said, my Company XYZ is far less a target of external threats than Amazon or Google. I appreciate the idea of "cloud", but everybody and their brother are after the white whales, while I'm an insignificant target. Thus, my chances of going down due to an external threat are far less if I host internally.

That's a generalization, of course, but I believe it holds true for the most part.

I'm not sure I follow your logic. If Amazon had a lot of single points of failure to external attacks, that theory might make sense. But I've not seen evidence their individual datacenters are any more vulnerable than the average company's (or your company's).

There are plenty of other good reasons to host your own stuff, though.

They do. The issue is one of control, or at least perceived control. In-house, the management can be in direct communication with the people doing the restore. They can get constant updates and possibly direct more resources to solve the problem faster. You also know that your people are spending 100% of their time trying to get your stuff back up.

If you're using someone on the outside, you're almost certainly dealing with a representative, who is going to limit the information you get. You also don't have the opportunity to direct more resources at the problem, and you don't even know if your stuff is a high priority for them to fix.

the service may be free, but that is hardly an excuse. i do believe people use that freeness when touting how much better it is than xbox live (they do calculations like it has the same value). also the system isn't free and neither are the games. but most of all, hulu plus isn't free and it doesn't seem to work either. (i don't know if i am just doing it wrong as i am kinda new to this disappointment of a platform)

Every company has internal weaknesses and external threats. That being said, my Company XYZ is far less a target of external threats than Amazon or Google. I appreciate the idea of "cloud", but everybody and their brother are after the white whales, while I'm an insignificant target. Thus, my chances of going down due to an external threat are far less if I host internally.

That's a generalization, of course, but I believe it holds true for the most part.

I'm not sure I follow your logic. If Amazon had a lot of single points of failure to external attacks, that theory might make sense. But I've not seen evidence their individual datacenters are any more vulnerable than the average company's (or your company's).

There are plenty of other good reasons to host your own stuff, though.

My point is, the "white whales" (Amazon, Google, etc) are huge targets for hackers because they house more data both in quantity and (arguably) quality. As such, that's where the hackers want to go. Company XYZ is nobody... so they're not on anybody's target list. As such, Company XYZ is easily a lower external target threat. It just makes sense. It doesn't mean if Company XYZ is targeted it won't go down, it just means the chances of it being targeted are far less, thus the chances of going down are far less (from an external threat).

That being said, my Company XYZ is far less a target of external threats than Amazon or Google. I appreciate the idea of "cloud", but everybody and their brother are after the white whales, while I'm an insignificant target. Thus, my chances of going down due to an external threat are far less if I host internally.

Correction, your chances of being attacked by an external threat are far less if you host internally. Given that you are unlikely to be able to invest nearly the the amount of resources that Amazon has in security and redundancy, your chances of going down to an external threat are actually far more than Amazon. Proof? Amazon still hasn't gone down to an attack. (They went down to an internal software bug)

Your correction is unwarranted as it makes my point.

I said in my post that "Every company has internal weaknesses and external threats. That being said, my Company XYZ is far less a target of external threats than Amazon or Google." In other words, as you said, Company XYZ's chances of being attacked by an external threat are far less if hosted internally.

Beyond that, I didn't make any statement about if Company XYZ is attacked could it hold out any better than Amazon, Google, etc. If Anonymous wanted to attack Company XYZ I'm sure it could take it down fairly easily. But that's not the point. Company XYZ simply isn't on Anonymous' radar so the EXTERNAL THREAT is lower.

EDIT: After re-reading my posts, it appears I really shouldn't have been so general in my original "Company XYZ" statements as it leads to much speculation and hair-splitting. I had an idea of "Company XYZ" in my mind that I clearly didn't articulate at all, and that isn't fair to anyone who wants to chime in pro or con on the argument. Sorry folks!

This and the Amazon outages have got to give pause to anyone thinking about moving their entire company into the cloud.

Perhaps I'm mistaken, but I have seen no mention in any of the news articles that Sony is hosting this with one of the major "cloud" providers. My money would be on the exact opposite.

Someone on Amazon might be thinking twice, or they might just compare their uptime on Amazon to what they used to have in-house and find that it's similar (or better). Or they might put stuff on standby at Rackspace "just in case". That's kind of the whole idea of this nebulous "cloud hosting", isn't it? Elastic capacity?