The Drones and Aerial Robotics Conference happened last weekend in NYC. I was really looking forward to going but an impending deadline at my day job kept me in LA. I wasn't able to watch the livestream, but I was able to experience the conference through twitter; primarily via the #droneconf tag. I made a list of favorite tweets that resonated with my interests and ideas of what's important in drones. Ingrid Burrington/@lifewinning, Allison Burtch/@irl, Adam Rothstein/@Interdome and Ron Evans/@deadprogram especially did some good twittering.

Terminology

Many speakers are against the "d-word". Evading the word won't escape the issues. Lets keep talking about drones. #droneconf

Some people think all drones are evil. Other people think those people are making an error by conflating military drones and hobbyist UAVs. I don't think all drones are evil, but the magnitude of the societal change that both civilian and military/law enforcement drones will bring is so big that we need to discuss and debate both. Changing the name won't change the need for that debate.

Vijay Kumar

Great point here. Drones are missing operating systems to allow people to develop apps. #droneconf

Regarding "Drones are missing operating systems to allow people to develop apps", the earliest prototypes of that are coming, but wouldn't it be great if there was a Processing for drones? The open source drones are finally moving past the weak micro-controller stage, but it feels like it's still going to be years before they reach the level that the AR.Drone has set, where you can start getting creative and run interesting code on the drone: 128 MB RAM and running Linux. (And any tight control loop, like visual tracking, needs to run on the drone itself.)

Possibly the most interesting thing about DARC is that it brought together people creating drone technology and people thinking about the social effects of the technology. Knowing the technology intimately and working on it first hand is exactly why I am both excited for and scared of what the future holds. When the About page of this blog says "You now live in an age of ubiquitous flying robots. ... Cheap flying robots have already changed the world, but it took a while for people to notice" I'm hoping it communicates the combination of hope and concern that I feel.

Law

Brendan Schulman attorney's *crash course* in drone law really does not portray the FAA in a positive light. #droneconf

At the moment the only way for public agencies or companies to fly drones without getting in trouble with the FAA is to get a Certificate of Authorization or a Special Airworthiness Certificate (though if you look at this list of companies with certificates, it appears you have to be a defense contractor to get one). It will be a big step forward if the FAA really issues new guidelines in the next couple months, but I'm also nervous about the possibility of new rules that are even more restrictive than the ones we have now, especially for hobbyist activities.

When I saw Daniel Suarez talk about drones at DorkbotSF last year I was all like OMG this guy sees the same crazy drone future that I see and am kind of freaked out about! His book Kill Decision, a techno-thriller about killer autonomous drones unleashed by rogue elements of the military-industrial complex, isn't superb literature but extrapolates from current technology to capabilities that are both plausible and shocking.

Web of Drones

Drones represent the actualization of understanding that internet space is *part* of real space #droneconf

The workshop I was going to do at DARC (before I flaked) was going to be about the Internet of things + drones. This was the proposal I wrote on the fly, literally at the last minute:

An emerging trend in DIY, amateur drones is to webify & socialize them. They are given HTTP interfaces and Javascript APIs. Ground control station software lets strangers control drones remotely from anywhere on the internet, overlaying HUD and augmented reality markers on top of live video feeds, with point-and-click flying; and it can run in the web browser on your iPhone. Reams of detailed telemetry data are uploaded to web sites where they are publicly shared and mapped--Drone analytics.

The future of drones will be as egalitarian (and as risky) as the web.

That is part of the future I see for drones.

Missy Cummings

The autopilot always lands better on an aircraft carrier- and that was in the mid 90's #droneconf

Missy Cummings is Director of the MIT Humans and Automation Lab and was one of the first female fighter pilots in the U.S. Navy. In the tweets above she touches on two of my favorite themes: The idea that eventually robots will do every physical activity (and some mental activities) better than we do, and the human factors of drone control.

My Mavelous project is intended to be an extremely easy to use GCS that can run on smart phones and tablets (Missy Cummings has worked on simple iPhone interfaces as well), but I've also spent a significant part of my career making natural language interfaces to autonomous systems—the idea is to reduce operator workload by talking to your drone like you can talk to Siri (see my "Electric Familiar" post for a video).

Via sUAS News: With a 12-foot wingspan, an unloaded weight of 50-60 kg, 10 kg payload capacity and a starting bid of £500.00, this "LARGE SURVEILLANCE AIRVEHICLE" listing on eBay could be just what you need to catapult-launch (catapult launcher is included!) your city's—or country's—drone program.

“Anthony’s Wireless Airship.” A small powered blimp used in 1912 to demonstrate remote control of aircraft by wireless telegraphy. (“Professor Anthony has exhibited a method of airship control of his own by wireless. He and Leo Stephens recently gave an exhibition of starting, controlling, turning and stopping an airship by wireless which was quite a long distance from the station which controlled its action.”)

Hudson Maxim, the famous inventor, gave his views on the future of the flying machine in war. The Hon. Col. Butler Ames, M.C., described, and for the first time showed photographs and moving pictures of, his new machine, and his experiments at the Navy Yard, Washington, and on the Potomac River. M. O. Anthony gave a demonstration of his remarkable invention for the control of airships by means of wireless telegraphy. The evening closed with a fine display of moving pictures of machines in flight, the first display of the kind ever made in this country. A unanimous vote was passed urging Congress to appropriate generous sums for the development of aeronautics for the Army.

These guys were the Homebrew Computer Clubbers and Makers of their time.

Lesh brought his glider to the exhibition, and made a number of fine glides, towed by a horse and also by an automobile. It was his purpose in his last flight, in an endeavor to win the Brooklyn Eagle gold medal, to cut the tow-line when he had reached a sufficient height. He did so, but the crowd got in his way, and hampered him in landing. He fell and broke his right ankle. Unfortunately, the fractures were badly set at the Fordham Hospital, and later it became necessary to place Lesh under the care of a specialist at Hahnemann Hospital. The plucky boy had a bad time for a long while. But he is now well again, and though slightly lame, Dr. Geo. W. Roberts gives every assurance that eventually he will be all right. On his reappearance at a meeting of the Society, Lesh was at once unanimously elected a Complimentary member, and he is now again taking an active part.

There is an RC plane field not far from me with a web discussion board that, 100 years later, reads similarly to this, with the showing off of new stuff to each other, recounting of failures, and even some serious injuries (with subsequent attempts to call for aid via wireless). I bet the monoplane enthusiasts and the dirigible gentlemen of the turn of the century occasionally lost their tempers and called each other names, in the same way the modern RCers divide themselves into quarreling factions based on whether they fly helicopters or electric planes or models of military aircraft.

It might only be in contemporary SoCal, though, where someone will pull a knife on you for landing your wireless airship against the traditional field pattern.

This post is going to demonstrate how to create an augmented reality-style display of nearby aircraft on an amateur drone's video feed. It combines three of my favorite concepts: ADS-B, software defined radio, and browser-based control of drones. The goal is to show how easy it is to use ADS-B on amateur drones and to show a proof of concept for increasing pilot situational awareness.

Laurent Eschenauer's ardrone-webflight is a browser-based ground control station, similar to mavelous but focused on the specific capabilities of the AR.Drone. The AR.Drone has no GPS, so there are no maps or waypoints, but it does have the ability to transmit a high resolution video feed from its camera over wifi so webflight shows that live video as the primary UI element. webflight also lets you fly your drone with the keyboard: press t to takeoff, use the arrow keys to yaw left & right or climb & descend, w/a/s/d to move forward/left/back/right.

The video below shows the plugin in action. I took a drone outside and scanned the Los Angeles sky for aircraft.

Each black square shows the location of air traffic. It displays the callsign (if they're broadcasting one), the ICAO identifier (the unique 24 bit ID assigned to every airframe), speed, altitude and distance.

A few things to note about the video:

I calibrated the drone's compass (webflight has a calibration command), but there was still some drift. Compass drift is visible as a barely perceptible creep of the compass at the bottom of the window and a more significant movement of aircraft markers when the drone is held relatively still.

I know what type of aircraft the drone saw because I looked up the ICAO codes (the 6 digit hex numbers in parentheses) at airframes.org.

The AR.Drone doesn't have GPS so I hardcoded its location for this proof of concept. It shouldn't be hard to add GPS to the AR.drone, and other types of drones (Arducopter/Arduplane/etc.) already have GPS.

For the demo I had the ADS-B receiver connected to my laptop, but you can also connect it directly to the drone; See "Cheap ADS-B on amateur drones" for details.

This demo overlays aircraft on the drone's first-person video in a ground control station, but there are other ways the ADS-B data could be used:

The current generation of amateur drones are compact collections of sophisticated subsystems; usually at least a CPU, GPS, IMU, sonar and barometer. They typically have multiple modes of operation: manual, acrobatic, stabilized/fly-by-wire, simple, altitude hold, loiter, guided, return-to-launch, failsafe.

or a user of the drone, knowing the status of the entire system can be critical, especially if there's a malfunction. You might have a ground control station that can receive telemetry from the drone and display sensor values and subsystem status, but it's common for drones to use patterns of LED lights to communicate information to anyone who can see the vehicle. The patterns can be simple, for example green means OK, red means ERROR; or they can be complex, with a half dozen different LEDs of different colors with multiple possible flash patterns--worse than the from-zero-to-twelve beeps of the old Power On Self Test error codes.

I've collected some examples of current tatus LED practice in the gallery below. Make sure you check out the video of the continuously variable RGB status LED, which is hilariously difficult to interpret.

ADS-B is the name of a radio transponder system used by aircraft to broadcast their position, velocity and other information to other aircraft and to air traffic control. It enhances safety by making aircraft more visible to pilots and ground controllers and can even work in concert with automatic collision avoidance systems (like TCAS, the "traffic collision avoidance system").

You can see ADS-B in action at flightradar24.com, where hundreds of volunteers with special receivers collect data and send it to the site to be mapped in realtime. There are also radarspotting enthusiasts around the world who meticulously log every aircraft picked up by their radios, building databases of registry and location info, occasionally spotting prizes like Air Force 1 or rare military aircraft like the E-6B "Doomsday planes" that would take control of U.S. nuclear forces if ground-based control centers were disabled.

Not every plane uses ADS-B yet. Approximately 70% of passenger planes in Europe and 30% in the U.S. currently use it, but the FAA will require most aircraft to have it by 2020.

Nick Foster has demonstrated spoofing ADS-B messages from a flight sim and feeding them to a receiver:

While the security issues still need to be worked out, ADS-B-based sense-and-avoid technology is considered critical to plans to integrate unmanned aircraft into North American airspace. The idea is that drones, which are often hard to see and spot on radar, would broadcast their own coordinates and maintain safe separation from other ADS-B-equipped aircraft. It's an area of active research:

In February 2012, a Linux kernel developer discovered that a cheap USB digital TV receiver could be used as a software-defined radio--that is, a radio that could be tuned to any frequency between about 20 MHz and 2 GHz, including the 1090 MHz frequency that ADS-B signals are broadcast on. Blog posts like "Tracking planes for $20 or less" show how easy it is to buy one of these receiver dongles and use it to track aircraft more than 100 miles away.

You will notice that none of the messages I received included latitude and longitude. My tests on the AR.Drone while it wasn't flying were much more successful; I was able to successfully decode many more packets, including messages containing Lat/Lon coordinates:

My guess is that during flight there's a lot of RF interference, and I will have to experiment with antenna placement and receiver gain. Maybe the best configuration will be to trail the antenna below the drone, at the end of its USB cable, like a towed array sonar.

Finally, here's an example of using dump1090's built-in map server (the server runs on the drone, I connect to it from my laptop):

Players get points in Astro Drone for docking in a way that’s "rapid but controlled," with bonuses for correct orientation and low speed on the final approach. According to Guido de Croon, a research team fellow at ESA, the result of the crowdsourced game data "could be much more autonomous spacecraft that can reliably maneuver, dock, or land themselves."

Reminds me of DARPA's ACTUV, a game that collected player data to help improve anti-submarine warfare tactics.

Domestic Drone Countermeasures is an Oregon company offering "Multi-layer Drone Shields and Countermeasure Domes for Large area, Small area, Open Area, Mobile, Fixed or Built in to structures using several DDC LLC Proprietary technologies."

Domestic drones are the commercialization of military technologies to be used against the citizens of the United States by various entities. Domestic Drone Countermeasures, LLC (www.DomesticDroneCountermeasures.com) (DDC) has taken on the task of commercializing military technologies for use as domestic drone countermeasures.

DDC offers many different, large area, small area, mobile and fixed drone countermeasures and systems. All countermeasures are non-offensive, non-combative and not destructive. Drones will not fall from the sky, but they will be unable to complete their missions.

DDC’s countermeasures are highly effective and undefeatable by most current domestic drone technologies.

Due to the growth of drone technology, the domestic drone countermeasure market is expected to grow quickly. In fact, the countermeasure market is likely to be as large as the current drone market itself. DDC and others in the domestic drone countermeasure industry are likely to require significant capital investment for rapid growth to catch up with domestic drone development.

They won't reveal their technology unless you sign an NDA, but "drone shields" and "countermeasure domes" make me picture something like this, just bigger and made of mylar:

I shouldn't make fun of them, it's just that I think drone countermeasures should be open source (I made that page 2 years ago and totally forgot about it until just now).

Drones aren't just coming soon, they are already here. While military drone technology is a hot topic these days, commercial applications of drone technology are poised to make a big impact across a number of diverse industries. The commercial implications are virtually endless - drones can be used by journalists to record aerial video and photos, by law enforcement to see without putting officers at risk, by farmers to keep an eye on crops, or by civil organizations to inspect infrastructure for safety.

In the US, commercial applications of flying drones are regulated by the Federal Aviation Administration (FAA). The drone commercial era will begin with the announcement of the FAA's rules for integrating drones into the National Air Space by 2015. Much like the 1889 Oklahoma Land Rush, an avalanche of disruptive commercial drone applications and services is bound to emerge.

The panel will be moderated by Chris Anderson (CEO of 3D Robotics) and will include Helen Greiner (co-founder of iRobot and CEO of CyPhyWorks), Jonathan Downey (CEO of Airware), Matthew Pobloske (Director of Unmanned Aircraft Programs at BAE Systems) and Zack Schildhorn (Lux Capital).

After hearing about some unusual activity at a supposedly abandoned building near their model airplane flying site, Rob and his son Nate determine to build their own amateur UAV, or drone, to circle the building and see what's going on. The building is too far away to fly their video-equipped plane manually, so they must use an on-board autopilot to guide it. The video recorded from their little drone draws them deeper and deeper into a mystery that not only involves law enforcement, but puts them and the rest of their family, a mother and daughter, in great danger.

"He did a really good job explaining the workings of the model aircraft drone."—Amazon reviewer.

If you're going to have a personal drone fleet, you need an easy way to command it. The Wicked Witch of the West didn't use an RC transmitter: She said "Fly, fly, fly!"

Years ago I worked on a project for the Air Force Research Lab involving natural language interfaces. We tried to sell some of the technology to Boeing to help reduce drone operator workload, but this was before the Drone Era had really gotten started, and they weren't interested. This is a crude demo of using some of that same technology to control an AR.Drone with my voice.

In the video I'm speaking into my laptop, which is controlling the drone. At the end of the video you get to witness a stubbornly unresponsive drone—The result of a software crash.

There is interest in using speech interfaces to help ease the integration of drones into the National Airspace System. For example, one issue that the FAA (and the military) worry about is what happens if the communications link between a drone and its operator is broken. The safest thing would be if the drone could still respond to commands from Air Traffic Control, autonomously. This is why level 2 of NASA's UAS Airspace Operations Challenge requires "that the vehicle be able to communicate verbally with the Air Traffic Control system under lost link conditions."

(This would be my Drone Games entry if only there was an event in Los Angeles.)

Far more than simply building and flying RC vehicles, Sprague advises the seriously-interested officer to join an AMA (Academy of Model Aeronautics) sanctioned RC club. Merely by being at the flying field with other club members, the hobbyist will be surrounded by experienced pilots who have a wealth of knowledge and insight into design and development of unmanned aircraft. There, the new hobbyist will be exposed to safety guidelines, aviation etiquette, and the FAA.

“There is not a flying field anywhere in this country where the FAA and current UAV regulation is not a topic of discussion and education. In this world, an aspiring UAS Operator will learn the pros and cons of basic fixed wing platforms as well as rotorcraft platforms. This is an important foundation needed before venturing into the world of advanced, autonomous systems,” Sprague said.

Last year, the Pentagon asked DIY-drone enthusiasts to come up with the spy drone of the future. Twelve months later, it looks like they might need a little more time.

In an announcement posted online yesterday with little fanfare, Darpa announced that its UAVForge competition had ended — with none of the 140 teams emerging victorious in the quest to create a better spy drone. “The teams brought creativity and enthusiasm to the competition,” Jim McCormick, the Darpa program manager in charge of the contest, said in the statement. “The competition was more constructive than you might expect; there were many examples of teams helping each other.”