Why machine vision matters for the Internet of Things

Connecting machine vision systems to the IoT creates a powerful network capability. Being able to identify objects from cameras allows the local node to be more intelligent and have greater autonomy, thus reducing the processing load on central servers and allowing a more distributed control architecture. This is turn provides more efficient operation that requires much less external input.

by Mark Patrick, Mouser Electronics

Machine vision has developed in great strides over the last decade. State-of-the-art algorithms capable of detecting edges and movement within video frames, alongside advances in silicon technology relating to image sensors, programmable logic, microcontrollers and graphics processing units (GPUs), have helped bring it into a wide range of embedded applications. More sophisticated designs that can be downloaded to an FPGA are being used in conjunction with new development environments, such as OpenCV, to make machine vision much more accessible to embedded system designers.

This growing proliferation of machine vision is converging with the trend of linking up industrial systems to the Internet of Things (IoT). As sensors become increasingly intelligent, driven in part by the supporting computer vision algorithms, so the data produced is offering valuable insights into the operation of industrial systems. This in turn is opening up new ways of monitoring equipment, with autonomous robotic systems such (as drones) being connected to IoT infrastructure.

Part of the move to machine vision is driven by bandwidth considerations, while the other major motivation is the prospect of automating more parts of an industrial operation. One of the key applications for machine vision is in inspection systems. High performance camera systems with CMOS image sensors have fallen in price considerably over the last ten years, allowing higher resolution examination of boards and systems during manufacturing. These camera modules are combined with FPGAs to add more processing and decision making. This allows the camera to respond accordingly to the received data, reducing the need to send video over the network and enhancing overall operational efficiency.

Robotic machine vision system

Connecting the machine vision elements of inspection equipment to the IoT provides more data for the enterprise systems that are undertaking analysis on the performance of the factory. Rather than raw data, machine vision can provide information at a level of abstraction that is suitable for such enterprise systems. This markedly reduces the bandwidth overhead both for the servers and the network as a whole - the enterprise systems are handling millions of data points coming from the IoT, so any reduction in the load on the servers will help to make more timely decisions.

The rise in machine vision uptake has also opened up the market for robot guidance systems in automated factories. Inspection machines can thus bypass the central enterprise servers and communicate directly with other equipment in the factory, based on the results derived from the machine vision systems. This increases efficiency and again reduces the load on the network and servers. Machine vision is also being used to control automated equipment, particularly in materials handling. This encompasses everything from the control systems for autonomous robots transferring material around a plant through to the automated picking machines in warehouses identifying products.

For autonomous materials handling robots, machine vision can be as simple as determining a line on the floor to follow from one location to the next. However, it can also be used to detect people or obstacles in the way, allowing factory operatives and robots to work together safely and more efficiently. As already mentioned, product picking is now also employing machine vision - with items being identified via their barcode and then a robotic gripper being aligned to capture a particular item and place it into a basket. The camera and local processing to accompany this are both essential, and the pickers/autonomous robots are also constantly monitored as part of the wider IoT.

This has even extended to the air, with Unmanned Aerial Vehicles (UAVs) increasingly relying on machine vision. UAVs have proven to be a highly effective method for carrying out inspections in hard to reach areas, such as oil pipelines and gas installations. As well as allowing the UAV to identify a particular target area and approach it, so that it can be examined more closely, machine vision is being utilised for anti-collision purposes - avoiding fixed obstacles and even other UAVs by linking the camera system to the on-board flight controller.

Robotic machine vision system

Then there is the surveillance market. The increasing use of machine vision has tremendous implications here. Instead of feeding back Megabits of video data every second for an operator to examine, video can be processed locally and alarms triggered without needing any human intervention. The machine vision algorithms running on FPGAs are becoming increasing accurate. As a consequence they are much better able to differentiate between the movement of an intruder, an animal or leaves on a tree (for example), allowing an operator to simultaneously support a larger number of surveillance nodes. Furthermore, surveillance cameras can themselves instruct other machines to respond to an alarm. The combining of autonomous ground and air vehicles, such as UAVs, potentially changes the whole way in which surveillance operates. Rather than fixed cameras that can be avoided, imaging systems are instead mounted on airborne craft that are constantly monitoring the area while in flight. These UAVs then return to a charging base as their batteries run down, and other drones are sent out to replace them. This means constant surveillance cover can be provided. More advanced machine vision algorithms are able identify potential threats and then summon other air and ground craft to the area to further monitor the situation - all without the involvement of an operator. The same type of scenario applies equally well to agricultural applications, where the machine vision algorithms on an airborne craft can monitor the condition of crops and direct an operator (or an autonomous tractor) to the target area if an issue arises that it deems needs some form of responsive action.

The applications outlined here have been enabled by progress made in the underlying hardware and software technologies. The structure of machine vision algorithms is increasingly sophisticated and they can be downloaded to the latest FPGAs and GPUs. These devices can handle 8 or 16 channels at a time, supporting rates of 60 frame per second. They can also be coupled with high level software such as OpenCV.

Originally focussed mainly on research and prototyping, in recent years OpenCV has increasingly been used in deployed products on a wide range of platforms - from cloud to mobile. The latest version, OpenCV 3.1, has just been released. The previous version, 3.0, was a major overhaul, bringing OpenCV up to modern C++ standards and incorporating expanded support for 3D vision and augmented reality. The new 3.1 release introduces improved algorithms for important functions such as calibration, optical flow, image filtering, segmentation and feature detection.

Future possibilities

Machine learning is the obvious next stage after machine vision. Computer vison algorithms are able to provide deterministic analysis of still images and video content, but machine learning is applying other neural network approaches to ‘teach’ a system what to look for. The latest version of OpenCV, for example, now supports for deep neural networks for machine learning.

The increased performance of FPGAs and GPUs is opening up new opportunities for machine learning. This relies on a training phase, where the neural network is shown many different images that are tagged with the objects of interest and is usually handled by a large server system in a lab or in the cloud. It produces a series of weights and bias data that is then applied to the same network implemented in the embedded design. This ‘inference engine’ uses those weights to assess whether new data it is seeing contains those objects. For example, the latest surveillance cameras are using neural network machine learning algorithms to go beyond traditional functions (like monitoring and recording) and offer additional video analysis features (such as crowd density monitoring, stereoscopic vision, facial recognition, people counting and behaviour analysis). This local processing can then be delivered into the IoT and thereby integrated into broader analysis software within the cloud.

Edge computing in IoT is opening new opportunities for embedded designers. FPGAs can be used to aggregate data, and once in place can also process that data and deliver real time analytics. Coupled wi...

The Internet of Things is a quantum technology leap for the manufacturing industry, enabling business opportunities and competitive inroads that were not possible before. While IoT has long been heral...

Are you ready for digital transformation? With one universal app platform, one commerical-ready security model, and one deployment and management approach, Windows makes it easier to manage IoT device...

For a long time now, many product or device-oriented companies have been attempting to shift the primary focus of their business from shipping hardware to delivering the value and services customers d...

LTE-Advanced (LTE-A) is the latest enhancement to LTE. As its name implies, LTE-A is well suited to applications that need the next level of performance. Find out how to improve practical data rates a...

The Internet of Things (IoT) promises to deliver a tremendous amount of data from the billions of sensors that will serve as the eyes and ears of the IoT. The number of sensors is already growing dram...

The emerging Internet of Things presents tremendous opportunities for innovative companies to deliver products and services to make industry more efficient. However, these same powerful tools can be m...

With the ever-increasing functionality incorporated into the latest wearable devices, power management remains a key innovation area. Specially designed ultra-low power management is required to best ...

With the ever-increasing functionality incorporated into the latest wearable devices, power management remains a key innovation area. Specially designed ultra-low power management is required to best ...

As our world transforms into a highly connected architecture, security has become a strategic concern. Impacts extend beyond financial losses into the physical realm. This video will give you a quick ...

The latest enhancement to LTE, known as LTE-Advanced or LTE-A, is an extensive set of new features that improve performance and increase reliability. Of these many features, carrier aggregation is per...

Machine-to-Machine (M2M) Communications used to be a niche topic. But today it is regarded as the key technology of a networked, mobile future. On this microsite we present a collection of White Paper...

For companies with global deployments, there is no single right answer for planning a scalable long-term wireless strategy. The best option, focus on solutions that allow for reuse of development, tes...

A common goal of portable devices and smart sensors in the IoT (Internet of Things) is to achieve a superior user experience through a long application run time from the given battery supply. Long sta...

Technological improvements are having an impact on test, measurement, and simulation systems which results in an increasing number of test and measurement tasks that require not only more powerful sys...

In harsh industrial environments, can high quality motherboards from brand manufacturers do the job as dependably as purpose-designed embedded motherboards? What is really needed for operation in thes...

Fieldbuses will not vanish completely from industrial automation in the foreseeable future, yet the Ethernet standard for time sensitive networking (TSN), in conjuncture with the platform independent ...

Image processing is one of the key technologies in automation, robotics and the smart factory. With image-based artificial intelligence, vision systems provide accurate analyses of environments and ob...

At the time of the official launch of the new AMD Ryzen Embedded and AMD EPYC Embedded processor families, the company had already garnered support from more than 20 partners. This article explains wh...

Embedded systems play a vital role in both characterizing and developing the Internet of Things as well as in creating new processes in automation. They address several requirements of IoT solutions a...

In a much larger stand than in previous years, at embedded world 2019 SECO showcases its wide range of solutions and services for the industrial domain and IoT. Among the main innovations, in this vid...

Since about two years Portwell is part of the Posiflex Group. Together with KIOSK, the US market leader in KIOSK systems, the Posiflex Group is a strong player in the Retail, KIOSK and Embedded market...

The DIN-Rail system from MEN is a selection of individual pre-fabricated modules that can variably combine features as required for a range of embedded Rail Onboard and Rail Wayside applications. The ...

The adoption of graphics in embedded and AI applications are growing exponentially. While graphics are widely available in the market, product lifecycle, custom change and harsh operating environments...

With increasing complexity of applications, no single type of computing core can fulfill all application requirements. To optimize AI performance at the edge, an optimized solution will often employ a...

The UDE Multi-Target Debug Solution from PLS provides synchronous debugging of AURIX multi-chip systems. A special adapter handles the communication between two MCUs and the UAD3+ access device and pr...

Andrew Bickley, Director IoT EMEA at Arrow Electronics talks about challenges in the IoT world and how Arrow is facing those through the Sensor to Sunset approach. Over the lifecycle of the connected ...

AAEON is moving from creating the simple hardware to creating the great solutions within Artificial Intelligence and IoT. AAEON is offering the new solutions for emerging markets, like robotics, drone...

In this video Infineon talks about their new CoolGaN 600 V e-mode HEMTs and GaN EiceDRIVER ICs, offering a higher power density enabling smaller and lighter designs, lower overall system cost. The nor...

In this video Frederik Dostal from Analog Devices explains a very high-efficiency charge-pump demonstration at their boot at electronica 2018 in Munich, Germany. Able to achieve an operating efficienc...

ams is at the forefront of developing Micro Lens Array (MLA) technology for image projection, light distribution and illumination applications in automotive. In this webinar, we discuss the technical ...

Secure data encryption is essential for a wide variety of mission-critical applications pertaining to both civilian matters and national security. These sectors both require comprehensive safeguards t...

Artificial Intelligence (AI) and the Internet of Things (IoT) are in a process of merging into what we define as AIoT. With edge computing, computational power is moving to the edge where IoT devices ...

This article describes the value of test & measurement systems based on examples from various real-world applications. The most important requirements and criteria for everything from system architecture and mechanical systems to enclosures with a small form factor and 19" systems with electronics cabinets are explored.

Autonomy and electrification are demanding significant changes to electrical and electronic architectures within vehicles. This is due in part to the introduction of high voltages, increased safety considerations and significant weight reductions needed to maximize vehicle range from electrification...

There are many challenges in circuit design today and understanding the key differences between digital isolators and opto-coupled isolators in circuit design helps developers design more effectively. This whitepaper reviews the challenges of modern circuit design...

In addition to signal frequency and bandwidth, the signal linearity and chirp duration determine radar performance. It is therefore important to analyze automotive radar signal parameters such as chirp length, chirp rate and frequency deviation.

Driven by the need for ever greater efficiency, lower cost, and improved safety, automobiles are undergoing a revolution in power distribution network (PDN) design. From architectures to semiconductor solutions and underlying processes, this one-hour EETimes University course will provide attendees with an overview of what’s changing, why, and what designers can do about it as they approach next-generation automotive PDN design.

Developers today want to add rich Wi-Fi and Bluetooth connectivity to their products to enhance their applications and end-customer experience, but they face many challenges that can slow down the process and delay time to market. This whitepaper discusses the unique challenges presented...

Industry is placing ever-greater emphasis on automation, particularly with the increasing deployment of Industry 4.0 technology. The latest developments in machine vision systems open up opportunities for the manufacturing sector to raise quality and throughput simultaneously. Furthermore, new applications are starting to appear in other industries, e.g. object detection and...

A smart meter is an electronic device that records and reports, via some kind of communications network, the consumption of a utility service, such as the supply of electricity, gas, water, or heating/cooling. In this whitepaper, we explore the fundamentals of smart metering and some of the benefits and challenges that come with...

The future of artificial intelligence (AI) relies on advances in the underlying machine learning technology, and an effective ML architecture. Through research with organizations such as Google, Arm, and leading universities that focus on data science, GigaOm reports on two important factors required to broaden...

There are many reasons to give your embedded system “eyes” and transform it into an embedded vision system. Image aquisition can turn many devices into ‘smart’ products or enable entirely new possibilities. This isn’t just a trend for systems that should work more autonomously; often it’s also an expedient step to make systems more intelligent...

First launched in 2005, the Arduino platform is responsible for making embedded microcontroller development accessible to all. With its simple to use yet comprehensive approach, Arduino has eased the often difficult task of interfacing an embedded application to the real world of hardware devices. Arduino also heralded in an open source era which accelerated widespread adoption of the platform’s design, spawned a separate market of add-on extension boards and a host of developer communities. In this article we take a quick look at the Arduino’s evolution so far and review some of the most recent additions to the Arduino ecosystem of open source hardware.

Nowadays “Industry 4.0” machines are interconnected, which means, they operate with other machines. In this time of mass customization, you need configurable and smart controllers, which can operate different tasks, by adding or removing features or modules. This presentation will explain how to design and build modular and scalable machines based on a Plug&Play EtherCAT stack. A use case will be presented to illustrate this approach.

Challenges common to industrial automation systems – like the demand for power-efficient and industrial grade components with guaranteed long-term availability – influence which solutions best suit industrial applications. Known as the Fourth Industrial Revolution, or Industry 4.0, the future beholds a growing number of industrial automation applications powered by smart semiconductor solutions.

Bluetooth Angle of Arrival (AoA) and Angle of Departure (AoD) are new technologies that establish a standardized framework for indoor locationing. With these technologies, the fundamental problem of locationing comes down to solving the arrival and departure angles of radio frequency signals.

Over the last two decades, Wi-Fi has transformed connectivity with more than 15 billion installed devices in the world. Such a broad and pervasive installed base, along with its throughput capabilities and native IP capabilities, makes Wi-Fi an attractive protocol for IoT wireless applications. However, a key limitation of most Wi-Fi based IoT systems is high power consumption.

Non-stop growth in design size and complexity makes it more difficult than ever for verification teams to keep up with project demands and product goals. According to the Synopsys 2017 Global User Survey, “Verification taking longer than planned” is the top reason for tapeout delays, and “Simulation runtime performance” is the top challenge for verification.

Enterprises in every industry are gearing up for AI’s integration with IoT at the edge. Analytics and cloud-based applications are crucial foundations for the AIoT infrastructure. But even more importantly, AIoT requires complete, real-time access to the data in fulfill the needs of highly responsive edge computing applications.

This paper provides an overview of RADAR technology as used in ADAS applications, including a summary of the relevant signal processing, typical computation requirements, and design parameters. This paper also considers approaches to estimating computational complexity and processing resource requirements and addresses some other system design tradeoffs.

n this Webinar you will learn how Arrow’s design tools can accelerate your design and development process by providing you with solutions based on the latest technologies and protocols. Made in cooperation with leading suppliers, Arrow’s development boards enable rapid integration of the newest technologies in your design.

This talk discusses Edge Computing and the migration of Artificial Intelligence & Machine Learning from Cloud-hosted methodologies to running on IoT and edge devices in the field. When it comes to the Internet of Things, we all know everything from your toaster to your car is becoming more connected and intelligent.

Learn how Arrow enables customers to address big business challenges and start the journey to a digital transformation of their business with advanced tools and services. Arrow’s Distribution capabilities now extend to supporting customers with complete Technology Platforms and Solutions.

This webinar introduces Winbond’s High-Performance Serial NAND technology which is the first in the world to run at 166MHz, and is well-suited for center-display and cluster segments in next-generation Highly-Automated Vehicles-HAVs, and other applications that require instant-boot and high throughput.

This White Paper explores how by using SDR, engineers can build radios that can react more readily to interference, employ more sophisticated channel-coding schemes to increase data rates, and take advantage of other advanced RF technologies. English Version / German Version