Comments (70)

John Honovich

To play devil's advocate, outside of random embarrassment, what will really drive video surveillance manufacturers to rectify this? Have we not seen enough evidence that manufacturers can have serious issues with near zero practical consequences? e.g., the still legion of vulnerable, not upgraded, Axis cameras out there...

Create New Topic

Undisclosed Distributor #1

I attended Black Hat this summer and one of the largest topics was the general ease someone with modest skills could infiltrate a network via IoT devices especially network cameras. The proliferation of IoT devices is almost making it almost too easy for hackers, if the vendors in our space ever catch up the targets will just shift to the next “smart” device.

Create New Topic

Undisclosed Distributor #1

I would not for someone new to the Cybersecurity market, it is radically different from something like ISC or ASIS.

It is mostly focused on the educational track rather than the show floor and can be fairly expensive to attend. If someone isn't already fairly well versed in this space they might be over their head at this event.

Create New Topic

Undisclosed Distributor #1

Extremely, it is such a fast moving market in terms of the type of attacks and the products being released to defend against those it is about the only live venue to see the latest from both sides. If you are on the defense side there isn't anything like IPVM out there.

The other big show on the cyber side is the RSA Conference, I haven't been to it but I am planning on attending in 2017. From what I hear it is more like a ISC or ASIS with 30-40K attendees each year.

Create New Topic

Undisclosed Integrator #6

John, Black Hat is really for folks in the field/studying to be in the field (i.e., cybersecurity / information assurance). It is very informative and educational... not to mention a bit spooky to see what these guys can do. Happens every July (end of) in Las Vegas... lots of fun!

Create New Topic

John Bazyk

So what's the recommendation right now, no port forwarding and stick to VPN's for remote access?

It seems at this point; no camera manufacturer is secure. Are the cameras that have been hacked on a private network where the only point of access to the outside world is through the server where the VMS is running? Or are these cameras directly connected to the internet using default security credentials and default ports.

For us to understand how we can better secure our customers, I think it's important to understand how these attacks are happening and how the systems are setup. If no one is getting hacked using non-default credentials, then we need to know that too.

I realize this is a substantial question but the more I learn about the surveillance industry, and it's vulnerabilities, the more diligent I want to be.

Create New Topic

Undisclosed Integrator #2

I'm with John above, it would be very helpful to know more about how these devices are being compromised. Do they know if it's only devices with default UN/PW exposed to the internet via open ports? this is what the industry is usually suggesting but that still makes me wonder my next question.

Even with the UN/PW known, how are they changing or executing code on the cameras through the web interface? Assumption here is that any other port would be blocked, so how are they actually modifying the cameras to run the attacks later? Just because there is an open web interface shouldn't mean a 3rd party can modify the firmware. If they are actually going in and uploading new firmware, then maybe the manufacturers should only allow updates if connected locally. Or better yet, create a secure connection back to their servers to make sure firmware is legitimate. Gray market owners won't like that but then again, who knows where the gray market firmware is coming from.

Or could they be infecting PCs on the same local machine that can access more vulnerable ports on the devices to make changes like SSH?

I'd like to also add, how do we know if a camera has been compromised? does a simple firmware update fix it?

Create New Topic

Undisclosed Integrator #2

This is a good reason why certain features in the web GUIs should only be accessible locally and not remotely which should be set in the firmware itself. The two main things that come to mind are the firmware update option and any shell access. This at least makes it more difficult for cameras connected to the internet to be infected.

Create New Topic

Brian Karas

So what's the recommendation right now, no port forwarding and stick to VPN's for remote access?

In a perfect world, maybe yes. From a practical standpoint you likely realize that is hard to implement.

A general recommendation for the purposes of securing any network of devices is to first minimize your attack surface. This means reduce, as much as possible, the number of devices that can be remotely accessed without requiring something like a VPN or higher-level service that restricts anonymous access.

For a network of cameras you can reduce your attack surface by installing an NVR (or similar device) that provides access to multiple cameras through a unified interface and login credential. This alone will not make your network "secure", but if it does get compromised you have reduced the compute power available to the attacker (this is assuming the attacker wants to use the device for a DDoS or similar purpose), and you have also reduced the amount of post-hack cleanup you have to do (rebuild/reinstall one device instead of 16).

Secondly, this NVR(like) device should ideally come with some sort of certification from its manufacturer that shows it has undergone some amount of basic PEN testing and can withstand common known attacks. This is not a guarantee that it is unhackable forever, but it shows that at least for now it should be able to withstand casual to moderate attack efforts.

If the device is not certified (and at this stage, most will probably NOT be certified) you can take some time on your own with common pen testing tools and see if a particular brand or model of NVR passes more tests than others. You may be able to find something that with moderate configuration can be more secure than average.

At this stage you are probably in the top 10 percentile of secured devices, meaning that your network is going to be harder to penetrate than many others. Because this is a game of numbers/volume, the attackers looking to exploit devices will try to script as much of this as possible. If your components do not look "interesting" or weak from a casual scan, you stand a decent chance they are passed over entirely.

You can probably take a giant lead forward from there by installing a firewall that can block/blacklist large blocks of IPs, potentially by subscribing to a IP blocklist provider. At a minimum you can probably block out IP ranges from incoming connections that do not correlate to your local region. This is on the premise that your users are likely to be relatively close to home when accessing devices remotely.

For users that travel, you may need to reduce the blocklist to just foreign subnets (Russia, China, etc. (unless of course you are in Russia...)), or require the use of a VPN when the user is "remote". This makes it more convenient to casually check on things most of the time, and then fire up the VPN when you're truly remote.

A step beyond all that is a firewall that can automatically detect and block port scans, or potentially even a device like a web application firewall, which can scan traffic at a lower level and look for common exploit attempts. This is probably overall for non-enterprise applications though.

Create New Topic

John Lineweaver

Brian, great points and I completely agree (Aim to make it as difficult or unattractive as possible for an attacker.)

Our cameras reside on a infrastructure that is pre-approved by our Network Engineers. The NVR’s sit in-between the cameras and the Network Switch. The Network is only accessible by VPN and our company employs a constant training / retraining program to educate the weakest link (end user).

We know we can’t stop all attackers however we are trying very hard to keep from being an attractive target.

Create New Topic

Steve Mitchell

When you are required to port forward to a device behind your firewall, you're by definition providing public access to said device and in the process transforming it into a publicly accessible internet server. This commitment requires responsibility on your part both in the initial secure configuration and ongoing maintenance, monitoring and care. At MultiSight something like 10% error rates of API requests due to probing by malicious sites was not uncommon. In summary, hang a device off the public internet and (congratulations!) you've become the administrator of a public site with all the implications thereof.

(Obviously, we're assuming a lot about the vulnerabilities being exploited here--that the devices were configured on the public internet via port forwarding or otherwise)

Your advice, Brian, is great. But should it be necessary? It's not if you're running a type of IoT device that doesn't follow this same architectural model for remote access.

Many devices--what I would characterize as "true" IoT devices--connect outbound from behind a NAT and establish a connection to a cloud-based service where they then poll for commands or retain an open two way connection for ongoing communication. The end-user interacts with the device with this cloud-based service as an intermediary. No firewall holes required. Assuming the service provider maintains a secure environment, there is little if any attack surface area at the device on the end-user customer's network.

This is how most of the devices I'd label as IoT work today. The practice of putting the device behind a firewall then poking holes in the firewall is an anachronism. I understand why it's often most practical, given the needs of low latency high bandwidth video streaming and the state of the offerings from existing manufacturers. But alternatives to the latter exist and should become more common in the future.

Create New Topic

Jon Dillabaugh

My first response would be if you are installing any devices on an IP network, you should be IT savvy.

That said, a properly configured, professional firewall will block a majority of the issues. Nothing is 100%, in liu of not connecting the system to the Internet. But you can restrict which hosts have access to the camera network. You can also restrict the outbound traffic from the camera network as well.

Create New Topic

Brian Karas

You can also restrict the outbound traffic from the camera network as well.

True, blocking traffic to remote/foreign networks could be beneficial, though in theory if your device is trying to communicate OUT to an undesirable network that likely means that someone undesirable has already gotten IN. Exceptions here would be reflector attacks, but that outbound traffic could be targeted to more local networks (local being a datacenter in your same geographic area) and may not have hit a restriction filter anyway).

Overall, blocking outbound traffic will have the same challenges as blocking inbound traffic, you have to find the right set of networks to block that increases securing without decreasing the ability for the customer to easily use the system.

Create New Topic

Jon Dillabaugh

So are we working on the assumption that these cameras weren't compromised BEFORE install? That they weren't installed with tampered firmware? We know for sure that they were tampered with post install?

Create New Topic

Brian Karas

In thinking more about these problems, one thing manufacturers could do is offer a kind of port-knocking-as-a-service.

Port knocking is a stealth way to enable remote access by sending a set of packet requests that "knock" on a firewall. If the requests follow the right sequence, the firewall opens a port. Kind of like a "secret knock".

The layout would roughly be:

Camera system is setup so that it is whitelisted to only allow traffic from a remote IP, which is an IP address associated with a cloud server the manufacturer controls.

The corresponding app has the login details/info per usual, but is configured to first send a request to the cloud server the manufacturer maintains. That request effectively says: my current IP is x.x.x.x and I want to access the device known as "Home NVR" with "username/password". If the info checks out, the manufacturer cloud server sends info to the on-site NVR (via its white-listed IP) that says "allow an incoming connection from x.x.x.x on port y" (the port y part is dynamically assigned). After that command, the cloud server sends a response to the app that says "ok, go ahead and connect on port y".

The other component here would be that the NVR would auto-expire this access request after some amount of time and/or inactivity.

With this you would have a dynamically managed IP whitelist, without needing to expose your NVR directly to the "raw" internet. It would solve the majority of remote attacks/exploits, with very little overhead on the cloud server since you are not routing all video through the cloud, just an initial connection request.

Create New Topic

Brian Karas

I would envision it as an HTTPS connection (for simplistic encryption) between the cloud server and the onsite device. It would be a simple data exchange, but still two-way (not just a direct "open this port for x.x.x.x"), and over TCP. This would pretty much eliminate the option to spoof the IP of the cloud server and send an incoming connection configuration request.

There are multiple ways this idea could be implemented, and it might not require a whitelisted IP (and the general risks/issues for the manufacturer in being so tightly tied to that IP). However by doing a whitelisted IP for the initial incoming traffic you make the NVR device nearly transparent to all casual port sniffing.

Create New Topic

Undisclosed #3

If the info checks out, the manufacturer cloud server sends info to the on-site NVR (via its white-listed IP) that says "allow an incoming connection from x.x.x.x on port y" (the port y part is dynamically assigned). After that command, the cloud server sends a response to the app that says "ok, go ahead and connect on port y".

How does the NVR open the router's firewall on a arbitrary port selected dynamically to allow the connection?

Create New Topic

Brian Karas

In traditional port-knocking the client sends a series of "knocks" (essentially packets to a pre-determined sequence of ports) and then the server opens up a defined port for a specific service, like SSH.

In my proposal the client sends a knock of sorts to a 3rd party cloud server. That server verifies the "knock" and then sends a message to the NVR to open up a port and accept an incoming connection from the remote IP of the verified client.

Create New Topic

Robert Shih

I can forward this idea up to my Dahua rep. It seems she has some pull with the R&D department. Especially, if I was able to kiss up to her enough to get her to reopen firmware development on a cam that was released back in 2012 to try and cure an RTSP security flaw. Let's see who can guess the other member I am referring to before he speaks up to say hi to me :D

Create New Topic

Robert Shih

Other differences, should they exist, are probably minute and negligible. For a more precise answer, I'll have to wait till Dahua is back from vacation. I'll grill them for specifics when that happens.

Create New Topic

Undisclosed Integrator #7

Is there an article on IPVM that goes over the specifics of basic best practice for securing our systems?

Also is it my understanding that embedded NVRs like Hikvision with cameras directly connected are harder to hack than systems run over the network?

Most of our systems are 16 channel or less and cameras plugged directly into the POE ports on the NVR and the only network access we have is plugged into the router for internet access. What kind of vulnerability does this setup have?

Create New Topic

John Honovich

As of today Dahua has become aware of a vulnerability that may have affected a small group of Dahua IP cameras. It would appear that this vulnerability exploits the affected cameras to run unauthorized code after compromising the cameras’ network capabilities. It would also appear that this vulnerability is limited only to cameras that are connected to the internet and running outdated firmware (pre-January 2015).

If you believe your cameras have been affected, please first check your firmware to make sure you are running the most up to date firmware available. You can find the most current firmware here It is very important to keep your firmware up to date on all of your devices. Additionally, we recommend only forwarding ports your devices actually need. For more tips on how to boost the cybersecurity of your network, please consult the Cybersecurity page of our website.

Dahua is continuing its investigation into this vulnerability and will work to ensure that your Dahua products will run safely, securely, and as intended. Dahua will provide any updates it has, however, if you have any questions or concerns, please do not hesitate to contact our cybersecurity team at cybersecurity@global.dahuatech.com.

Create New Topic

Zachary Spradlin

In my personal anecdotal experience; probably not Jon. Sometimes if we would push them hard we could get a firmware fix on old stuff, but most of the time it's just "move on to the next product."

Though, this is an unprecedented situation that they've found themselves in, so perhaps it would warrant a firmware upgrade from them. Does Dahua have cloud-upgrade yet? What a miserable job it would be to update over 145k compromised cameras manually...

Axis has released their first IR multi imager, the P3717-PLE, a repositionable model listing 360° IR illumination and flexible positioning,...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

Member Login

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.