Posted
by
BeauHDon Tuesday September 27, 2016 @06:10PM
from the internet-of-things dept.

MojoKid writes: If you thought that the massive DDoS attack earlier this month on Brian Krebs' security blog was record-breaking, take a look at what just happened to France-based hosting provider OVH. OVH was the victim of a wide-scale DDoS attack that was carried via a network of over 152,000 IoT devices. According to OVH founder and CTO Octave Klaba, the DDoS attack reached nearly 1 Tbps at its peak. Of those IoT devices participating in the DDoS attack, they were primarily comprised of CCTV cameras and DVRs. Many of these devices have improperly configured network settings, which leaves them ripe for the picking for hackers that would love to use them to carry out destructive attacks.The DDoS peaked at 990 Gbps on September 20th thanks to two concurrent attacks, and according to Klaba, the original botnet was capable of a 1.5 Tbps DDoS attack if each IP topped out at 30 Mbps. This massive DDoS campaign was directed at Minecraft servers that OHV was hosting. Octave Klaba / Oles tweeted: "Last days, we got lot of huge DDoS. Here, the list of 'bigger that 100Gbps' only. You can the simultaneous DDoS are close to 1Tbps!"

The sad fact is that it's already too late. The problem is that there are loads of these insecure devices out there now, and they will likely be online for years to come.

Even if every new IoT device that was sold starting tomorrow was actually secure, we have a huge pool of susceptible devices that are already in place just waiting to be exploited.

Our best hope is that these craptastic devices fail quickly and are replaced, but I'm not going to hold my breath hoping that their replacements will be any more secure. Frankly, I have no reason to believe that IoT device makers will ever do anything to make their devices secure. We'll be seeing this shit 10 years from now, only worse.

The sad part is that it was too late before the devices were even built. This is really no different than any other zombie botnet.

What is needed, IMO, is a standardized system for being able to report problems upstream—an ICMP response that says, in effect, "Suppress all traffic from x.x.x.x to y.y.y.y for five minutes" that propagates upstream. Ideally, it should use a three-step handshake to prevent forged block requests from being viable, where the recipient of that message waits until it sees a packet directed to y.y.y.y, (to avoid amplification attacks), then sends a packet that says, "confirm block id xxxx" and it responds "yes xxxx" after which it drops the traffic. If it gets no response, it should try three pings (with exponential backoff), and if they fail, it should assume that the server is saturated and it should block the traffic as requested. If they succeed and a subsequent confirmation fails, it should assume that the server doesn't actually support blocking requests, and that the blocking request was spoofed. If the response is "no xxxx", then the blocking request was spoofed, and the packet passes through with only that small extra bit of latency, and the blocking request is discarded.

If such a scheme were in place, then each botnet member joining in a DDoS attack would get blocked by their closest router, or at a bare minimum, by the router at their ISP, and would basically be unable to do any real harm.

Typical Windows PCs in botnets (a) are never updated & therefore decay until they implode and are reinstalled, wiping out the zombie and (b) at least at re-install time, they get updated so the old exploit doesn't work anymore.

The current SOP for IOT manufacturers, however, breaks BOTH of these things at once: These badly-designed devices none the less usually run a well-designed underlayer (*nix), which means

If you have an automated way to block traffic, then someone will abuse that system for the same goals as the original attack...The goal of a ddos is to take something offline, a system which is blocking traffic is offline.

Except that what I described is carefully designed to make abuse almost impossible. Any fake blocks are removed almost immediately, and unless the server is actively being DDoSed, assuming it supports the protocol, such removal causes at most one additional packet to get sent in each direction, which means there's no amplification if the server supports the protocol, ignoring situations where packet loss causes a retry.

If the server doesn't support the protocol, there's typically only a 2x amplification (

Actually, now that I think about it, I did forget to mention one small bit of the protocol. Each router that passes on the original request should immediately ACK the request to the previous router so that the previous router knows that it does not need to handle the blocking itself. It should then sent it towards the attacker's IP, and if it does not get an ACK from any router that's closer to the attacker in a timely manner, it should handle the blocking request itself and send back a confirmation requ

Only for an hour, though I guess you could send a new blocking request every 45 minutes.

It would also let me block those idiots who keep trying to sign in to my servers via SSH. You'd think that when they send the original request (for authentication-free login) and the server says that it only accepts private key authentication, they wouldn't send thousands of password-based login attempts, but apparently the people who write those bots don't understand the SSH protocol very well, or else they just like w

It is time to blacklist these devices and prevent insecure devices that participate in DDoS permanently. This may mean things like MAC-based blocking on ISP-level. In order to make ISPs do this, we may have to drop a few ISPs from global routing first though.

Another option would be to make hacking them to take them down legal, but that is hugely problematic.

Anyways, with the damage these idiots allow the DDoSers to do, terrorism begins to seem kind of irrelevant.

It is time to blacklist these devices and prevent insecure devices that participate in DDoS permanently. This may mean things like MAC-based blocking on ISP-level.

But all your ISP sees is your router...so they'd have to start cutting people off from the internet left and right. And many, many people won't know what to do when that happens because all the ISP can tell them is that "some device" is sending traffic out.

Is it their thermostat? One or more light bulbs? The washer or refrigerator or the furnace? Maybe it's little Johnny's Speak-N-Spell or Sally's Barbie Dream Castle. Maybe it's the TV or the DVR or the the remote-viewing doorbell.

They'll have to unplug their whole house, bit by bit, checking with the ISP each step of the way. How is Joe Sixpack or Grandpa going to know what to do? And what if two or more devices are the culprit?

Shit, the more I think about it, the more I realize that this shit is going to be way worse than I imagined, and I'm pretty pessimistic to start with.

Yes. That's EXACTLY what they need to do. They need to figure out WHICH part of their SHIT is breaking the world for everyone else.

This is the same stupid kind of shit that causes entire neighborhoods to burn down because some idiot is too stupid to know not to put a space heater under the curtains in their house, get their house blazing, then (by the sheer idiocy of the developers) set ablaze the other houses that are only six feet away.

Take some damn responsibility for the shit you buy. Don't go buy a gun if you're too stupid to know you can accidentally kill someone with it. Don't buy a stupid Internet connected piece of shit if you're too stupid to know you can bring down the Internet with it.

On the plus side it might finally lead to home routers getting some more interesting IP accounting features. That is one thing that has always annoyed me ever since I stopped having a Linux gateway - the home routers typically have no useful feedback as to what device is responsible for traffic.

Even a simple counter table would be incredibly useful, but I don't really see any reason why it would be hard to have good real-time graphs showing the current and total data usage from each IP on the network.

One interesting challenge though - what happens if you have an IoT device that is thoroughly pwned and keeps changing IP addresses (and/or MAC addresses!) specifically to make identifying it internally even more complicated?!

One interesting challenge though - what happens if you have an IoT device that is thoroughly pwned and keeps changing IP addresses (and/or MAC addresses!) specifically to make identifying it internally even more complicated?!

Or if you have multiple pwned devices working in concert to trade off the traffic so as to try and stay below the radar. What if there were 5 or 6 or 10 devices, all infected...they could each share the load in random rotation. Each would would behave normally except for a few seconds or minutes a day when it would act maliciously. I would think that would be fairly tricky to nail down.

What do people do now if their home gets infested with pests?I think that a new kind of professional bugbusters could arise as a result.

Sure, but how much would this kind of service cost? Maybe as much or more than just replacing the suspect gadgets (not a refrigerator or furnace, obviously, but still...). And who's to say they won't get reinfected the next day?

I can see it now: "Norton Anti-Virus For Home Appliances". "Mcafee HomeGuard Extreme DoubleSecure". Ugh.

Frankly, I have no reason to believe that IoT device makers will ever do anything to make their devices secure. We'll be seeing this shit 10 years from now, only worse.

As someone who owns a company that makes IoT devices and properly secures them, there are companies that do take security serious. The problem is that security is all too often seen as just a cost, not a feature you can charge money for. You need dedicated security people, incorporate security form the start, etc. and lots of companies just don't want or have the money. It makes the cost of the device go up, you get longer time to market, etc. and that's a hard sell to investors.

We actively try to educate on security, but it is going to take several more of these and some big losses before the majority will take security serious.

As someone who owns a company that makes IoT devices and properly secures them, there are companies that do take security serious.

I know, but for every one that does take security seriously there are a hundred that don't. I applaud you for thinking of security, but you're the one out of a hundred. It's the other 99 I'm worried about.

And how are consumers supposed to identify which devices are more secure at the pre-sale stage, and which vendors take security seriously?

Also in what way do you take security seriously? A lot of vendors go to great lengths to prevent anyone (including the legitimate owner of the device) from loading alternative firmware or gaining shell access to the underlying system etc. Vulnerabilities will still be found, but if you can't replace the firmware and the original vendor no longer produces an update or bund

And how are consumers supposed to identify which devices are more secure at the pre-sale stage, and which vendors take security seriously?

They can't, and I never said they could. We try to educate them. One thing we do for example is analyze potential devices for customers and figure out if there are any security issues. For example, GPS trackers that you buy cheaply on eBay or Alibaba all have major security issues. We show this to customers and have independent parties verify this before they decide to buy them. Granted, we usually don't deal with individual end users, but with re-sellers or distributors and industry, but each one of them g

Yeah, easily, if you lay in some plastic wrap or something. Actually it's easier than most things as the sieve is the right shape to hold water, and the holes are pretty easy to cover - the water will even help you do it!

Well in TV-land a hacker can just send a huge EMP to the device until smoke starts coming out of it and the screen melts.Not sure what happens after that, it's usually where I choose a different show to watch.Would be cool if the passwords on these devices could be reset to a random value from a remote hack tho.

The IoT is, by design, a security risk. Who the hell needs their oven, thermostat, refrigerator and each individual light-bulb connected to the Internet? I have no pity for anyone who gets their speaker-included light-bulb hacked, and I truly believe the companies whose products are involved in this DOS should be held completely responsible. CEOs and CTOs should be fired and charged with computer crimes.

If you can't see advantages and demand for controlling your house from your phone, regardless of if you're home, then you're very short sighted and not a good futurist.

Bullshit. There is a safe way to do this: Don't let any of the devices have direct access to the internet. None. Put them on their own dedicated wireless router, connect that wireless router to your real router and then set a firewall rule that doesn't allow anything from the IoT router to route outside your LAN. If you want to check the status of the devices when you aren't on your local LAN, VPN into your house and check them.

You don't need to trust shady vendors that don't give a shit. You don't need to open a billion insecure ports in your firewall to expose devices. Consider the devices 100% insecure, configure your network in a sane way and setup a VPN or use an SSH tunnel.

I'm sure there must be a simple way to require an inexperienced new user to load up a phone app and initialize each new device before enabling its network connection. The app could even supply a GUID or something as the password, so said inexperienced new user doesn't even need to be bothered with thinking of one, and all of his IoT devices could share the same unique activation code.

The mythological they should have set a standard and enforced this from the beginning.

How... then would the vendors sell a phone app to naive users to change their thermostat settings when they're on vacation?

They shouldn't. None of this should be happening. What should be happening is that vendors should be selling "IoT-enabled" routers that are highly secure and will generate a VPN connection package for a device type. I run an Untangle appliance and it will literally generate a unique Windows installer package for a VPN to your home network. And it's very easy to do. There is no reason why it couldn't generate a VPN package for any device you wanted to use outside your home. In fact, I would say that if you are connecting to random wifi networks without initiating a VPN to a more trusted network (like your home), you are doing it wrong.

Yes this is how it should work, although because of NAT and the difficulties of setting up a VPN etc most of these products talk to an external server somewhere and then your mobile app communicates with that.What's worse is that these devices often communicate with random target addresses (eg the vendors host their stuff on amazon and just allocate more machines on new ips as load increases) so you cant set up sensible firewall rules.

By that logic why limit it to only IoT. Everything connected to the net should be held accountable which starts with ISP's holding each other and their customers accountable. ISP's need automated ways of telling each other about unwanted DDoS traffic in real time, or even just identifying members of botnets after an attack, and then demanding that those customers be warned/taken offline until they secure their local networks. If an ISP fails to act then their peering links would start getting throttled progressively more until either they fix the problem or they get cut off entirely.

Forgot to mention that the ISP's could also pressure any device manufacturer to secure their products better and all the customers with devices that are inherently insecure could take legal action against the device manufacturers for a defective product.

Which is how it should work. But the problem with that is that many/(most) ISP's don't do source address filtering. Which means that if the attack nodes also use source address spoofing, once the traffic gets to the target you don't know which ISP it came from.

If you knew which ISP the traffic was coming from you could indeed grab them by the throat and work backwards, but unfortunately the target doesn't know that.

But in reality it's not likely these are all home devices, which are typically behind NAT routers with at least some basic firewall features. I suspect most of these are devices that aren't firewalled.

But most of those devices have some "check for updates" functionality built in, and if you can intercept that and feed false data back to the device, it will gladly download bogus firmwares or execute commands injected in the data stream. And now the attack starts behind the NAT/firewall, and this direction is not in any way filtered at most sites, but set to In->Out Allow All.

Yerrrr! fucking technology, taking our jobs. I remember when Jeeves would stand there and sing to me whilst holding a candle, I didn't need no speaker light bulb. Jeeves would never attack me as he knew his place unlike these internets, good old Jeeves, I miss him. Damn slavery laws, fucking god damn liberals and their "progress"!

No one needs what you describe. But on the other hand that us only a small tiny part of what IoT is. Please stay away from consumer marketing material when discussing conceptual technologies with a wide breadth.

The complaints aren't being "ignored". You try to deal with as many customers as they have while still turning a profit and see how many complaints you get and what your response time is. Besides, if OVH disappeared today, all the spammers would flock to the next-cheapest hosts, and then Amazon or Microsoft or Hetzner or whoever would be the #1 spammer, and we'd all be complaining about them.

Where's that "So you think you have a way to block spam?" fill-out-form joke?

A website, or a game server, is EXACTLY the kind of machine that receives a significant portion of its requests from people it's never seen before.

On top of that, a DDoS doesn't care if you "block" it. It's still consumed 1Tb of traffic. Even if every single packet never reaches the server, the DDoS will knock you offline by swamping your connection.

You can "firewall" it right at the first point that your connection comes in. It

If your equipment is found to be part of a DDoS attack, taking you offline removes teh DDoS, and you get the necessary incentive to fix your security. Once word gets around that having brand X VoIP/Camera/IPTV/Printer device causes you to lose internet access, people stop buying them, and at this point the manufacturer is incentiviced to fix their shit.

There are a few options but all of them require high-jacking IoT devices.

If I were feeling more energetic I'd pull out some comments from here I left a decade ago talking about a guild of Internet engineers and a trust system where certified operators could send cryptographically-signed messages upstream to shut off attacking ports (or requests to do so - that's a local detail).

Yes, we're decentralized, and that's good, but we also need to cooperate.

There is actually a fourth option: Turn the IoT devices against their local LAN. Pretty innocuous in the grand scheme of things but, if you discover that you can't watch Netflix when you have your IoT lightbulb plugged in, it might make you wonder about the value of IoT devices.

You know, the third amendment prevents you from having to quarter troops in your house. Why buy all these "Internet of Things" devices, and quarter the troops of a cyber war? DDoS provides the censorship dreamed of by the worst governments and the casual keyboard tyrant alike. These "things" are just malicious tools.

I don't understand how this sort of thing happens anymore. In every one of these DDoS threads, a fellow slashdotter (anon, of course) is giving "expert" advice on how to easily manage such DDoS activities by configuring Windows NT [slashdot.org].