I share a commonly-held vision for the Internet of Things wherein mobile devices (such as robots) interact with smart objects -- objects with embedded microelectronics that perform computation, sensing, communication, energy harvesting, and energy storage. The other day, my colleagues and I published an open-access white paper that illustrates one incarnation of this vision: Robots interacting with long-range, "sensorized" RFID tags to provide IoT-style sensing -- i.e. an example of how autonomous mobile robots outfitted with UHF RFID readers could interact with sensor tags to perform tasks such as soil moisture sensing, remote crop monitoring, infrastructure monitoring, water quality monitoring, livestock monitoring, and remote sensor deployment. You can find the white paper on arXiv.org (or direct PDF). Check the paper for exact details, but I'd like to share some of the concepts here on Hizook... with plenty of pretty pictures!

The basic premise is pretty simple: Deploy these low-cost sensor tags out in the environment. Since they harvest all their operating energy from the RF signals, they don't need batteries and have virtually-limitless lifetimes. The robot, owing to its mobility, can do two useful things: (1) deploy these sensor tags in hard-to-reach locations, and (2) go back to read the sensor tags to obtain direct physical measurements.

Tag Hardware

From the paper's introduction:

UHF RFID is one compelling technology that speaks directly to the Internet of Things vision. Classic UHF RFID tags contain a small integrated circuit affixed to an antenna and mounted on a flexible substrate. These battery-free tags harvest all of their power from the wireless signals that are also used for communication. The tags provide unique identification; are extremely low cost (sub-$0.10 USD each); can be read from several meters away; can co-exist in the environment in the hundreds or thousands owing to low-level anti-collision protocols; and are produced in vast quantities each year for logistics applications. Previously, researchers developed UHF RFID tags that also contain general-purpose computation as well as sensing capabilities. Similar sensorized UHF RFID tags are now commercially available. In this paper, we explore some early prototype applications for these tags and generally explore how these tags could be a boon for robotics.

We used long-range sensor tags from Farsens for this work. These are gen2 (standards-compliant) tags that have a RFID frontend and a companion sensor chip. By default, they are battery-free (passive). But a battery can be added for improved read range (or specific applications).

For future systems, the RFID frontend and sensors could all be contained on a single IC, along with power harvesting, data logging, and general purpose computation circuitry.

Robot Hardware

We built two robots (one UAV and one UGV) to demonstrate our proof of concept applications. From the paper:

Unmanned Aerial Vehicle (UAV): We employed a commercially-available 3D Robotics IRIS quadcopter as our unmanned aerial vehicle (UAV). During flight the UAV remained tethered at all times by a 40 lb test nylon cable for both safety and regulatory compliance. The drone remained in line of sight for all tests, and also featured an emergency human override via a separate 2.4 GHz manual radio remote control (RC). A WA5VJB log-periodic antenna was mounted in a downward fashion to read tags as the quadrotor hovered nearby a tag.

Unmanned Ground Vehicle (UGV): We employed a commercially-available Traxxas Stampede RC car as our unmanned ground vehicle (UGV). The UGV included a sturdy chasis, drivetrain, suspension, rear wheel differential drive, electronic speed controller (ESC), and a servo to control the steering of the front wheels. We removed the RC unit that came with the vehicle, and mounted a forward-facing WA5VJB log-periodic antenna to detect tags while approaching their location via ground.

Hardware Common to Both Robots: Both robots employed commercial Pixhawk PX4 autopilot systems with GPS telemetry as the on-board controllers. We used a ThinkPad T430 laptop as our ground control station (GCS). We mounted a commercial ThingMagic M6e UHF RFID reader with 1 W RF output power, a WA5VJB log-periodic antenna, and an Arduino Mega to the underside of the UAV and to the topside of the UGV. The RFID reader and antenna provide the core RFID functionality while the Arduino acts as an interface between the RFID reader and the Pixhawk autopilot. All three components (RFID reader, Arduino, and Pixhawk) communicate via UART serial interfaces; a coaxial cable connects the RFID reader with the log-periodic antenna.

Applications

In the paper, we developed some rudimentary(!!) proof of concept applications using these drones and sensor tags:

Soil & Agricultural Monitoring

There's a lot of ongoing work (companies like 3D Robotics & Terravion) to use drones to monitor plants for agriculture. Many of these systems use remote sensors (eg. hyperspectral cameras).

Using sensor tags, we can obtain direct physical measurements instead of inferred measurements. This could be useful for calibration, or even for sensing on a plant-by-plant basis.

Plus, the tags can have a number of sensors to measure properties that would be difficult (or impossible) to measure with cameras: moisture, nitrogen content, solar insolation, etc.

Infrastructure Monitoring

One oft-discussed application for drones is to perform infrastructure monitoring: measuring stress, strain, corrosion, wear, etc. in hard-to-reach locations on buildings, bridges, power lines, and dams.

One big challenge with camera-based systems is that they can only explore the surface of these structures. Sensor tags can obtain direct measurements, and could even be embedded inside the structures during construction.

Water Quality Monitoring

Floating tags could be placed onto bodies of water to allow robots to obtain direct physical measurements related to water quality: clarity, contaminants, etc.

Crop Monitoring

As tags proliferate, the cost per tag should drop to the approximate cost of ID-only tags ($0.10 ea). For some high-value crops, it may be possible to tag individual plants.

Alternatively, sensor tags could be placed in hard-to-reach locations (such as forest canopies) for research and tracking purposes.

Ubiquitous Readers

We used two different robot platforms to read the tags. Since all the tags are standards-compliant, virtually any robot (or stationary reader or handheld reader) could read the very same tags.

I think it's also worth pointing out that that robots can be used to deploy sensors into remote, hard-to-reach locations. Our basic proof of concept for this idea used an adhesive-backed tag on a boom arm, which "poked" the tag into location on a high, out-of-reach wall. The drone came back later and successfully read the tag.

This set of applications certainly isn't limiting. Tags could also be used for medical sensing (human or livestock), or embedded in everyday objects (eg. spoilage sensors in the fridge; sensors in your clothes; sensors in furniture, etc).

A Few Final Notes:

This work was a direct descendant from my PhD and Postdoc research. For my PhD thesis, I built some of the first mobile robots capable of operating in real homes -- which was enabled by long-range ID-only RFID tags. I used these robots to demonstrate useful tasks such as robot-mediated medication delivery, fetching and retrieving tagged objects, and helping people with motor impairments. Indeed, ID-only tags already provide a lot of utility for robots. Then, during my Postdoc, we developed custom tags with higher data rates and specialized sensors. We used those tags for ECG (electrocardiogram) sensing and to build "cyborg dragonflies." All of this research was very cool (go check it out!), but it was still very academic.

But things are changing. There's a lot of renewed interest in item-level tagging (ID-only tags) for retail; companies such as AMS and Farsens are developing COTS (off the shelf) sensor tags; and other companies are developing their own tags and end-use applications. Coupled with readily-available hobbyist robot platforms, we realized now was an ideal time to build a proof of concept... before it becomes commonplace. Thus, this fun little side project was born.

Also, I think we can make a reasonable case that this type of sensing (whatever you may call it - IoT?) is a new sensing modality for robotics. Most sensing in robotics is either remote, contactless sensing (eg. lasers, cameras, and ultrasound) or direct touch (eg. haptics). IoT-style sensors, whether RFID-based or not, offer a compelling third sensing modality where “smart objects” or “smart environments” with embedded computation and sensing can directly measure and report salient information back to a robot. Regardless of what you call this third sensing modality (IoT?), I strongly believe that it will be hugely valuable to robots. To quote the paper:

We recognize that many of these results are preliminary. We explicitly do not address the design of sensorized tags; perform a theoretical analysis of RF propagation; examine the design of the robots themselves; address RFID system design considerations such as antenna selection; nor do we provide a detailed evaluation of any one application (eg. soil monitoring) – each of these considerations has already been covered in the literature. Instead, our focus is to show, for the first time ever, how robots can use sensorized UHF RFID tags to facilitate a number of unique applications. Through these rudimentary prototypes, we hope to impress upon you, the reader, the potential benefits afforded to robots by these tags and provide a new tool for your toolbox. We believe there are myriad possibilities for sensor tags to be applied in other areas too, such as livestock monitoring, healthcare, and home automation. As the Internet of Things pervades our lives, it may even become practicable to embed tags in everyday objects, where: cups can inform the robot what liquid it contains and in what quantity; clothes can indicate their dirtiness; or food that can indicate its freshness or spoilage. The rich information provided to robots by direct, embodied intelligence in the form of UHF RFID sensor tags could be transformative.

Finally, I would be remiss if I failed to point out... RFID is not a panacea!

While the possibilities are compelling, current long-range RFID systems are not a panacea. Perpetual concerns like read range, power budgets, and robot control still play a significant role. Even if ameliorated by battery-assist or energy-harvesting technologies, there will still be significant challenges associated with building cost-effective sensorized tags and integrating them directly into objects and the environment.

I think it's important to be technology-agnostic. While a lot of my work over the years has focused on long-range RFID, I'm equally excited by WiFi, BLE, Zigbee (etc) sensor technologies. It's all about using the right tool for the job!

PS -- Many thanks to our friends at IEEE Spectrum, HackADay, and DIYdrones for their previous coverage of our paper. We're glad you think it's worthy of sharing... and kudos for beating us to the punch on reporting it. :)

Comments

I haven't used the M6e directly, though a few of my interns did the other summer. It's my understanding that ThingMagic no longer supports low-level serial protocols, which were the primary method to interact with the M4 and M5 series of readers using lib_M5e. ThingMagic now uses their own C++ and Java APIs that must be used instead -- which I believe work with all readers (M4, M5, and M6). To the best of my knowledge, they have no intentions of releasing the low-level serial interfaces or Python bindings for the newer readers -- much to all of our dismay -- and I believe the M6 is not backwards compatible.

Thus, I'm afraid lib_M5e.py will likely never support the M6e. Sorry I couldn't be more helpful.

Superb website and I wanted to post a note to let you understand, "Great job"! I am glad I found this site. Your blog site has introduced me the majority of the strategies that I like. Thank you for sharing this.

I haven't used the M6e directly, though a few of my interns did the other summer. It's my understanding that ThingMagic no longer supports low-level serial protocols, which were the primary method to interact with the M4 and M5 series of readers using lib_M5e. ThingMagic now uses their own C++ and Java APIs

that must be used instead -- which I believe work with all readers (M4, M5, and M6). To the best of my knowledge, they have no intentions of releasing the low-level serial interfaces or Python bindings for the newer readers -- much to all of our dismay -- and I believe the M6 is not backwards compatible.

What an interesting article. I really like learning about robot and instagram technologywhat an interesting world. I always surprise and wonder how complicated the engine is, how could the engineer knows it all! I admire their work. Hope to see more updates of yours on the robots field.

I think there are many other people who are interested in them just like me! How long does it take to complete this article? I have read through other blogs, but they are cumbersome and confusing. I hope you continue to have such quality articles to share!