Day: April 16, 2019

This is a crash course in getting the Movidius NCS2 neural compute stick up and running with a benchmark application. Even though only the benchmark app is covered, the same steps can be used to compile any of the other apps included with the OpenVINO toolkit.

Environment:

Download a container pre-loaded with OpenVINO:

I created a container on dockerhub which already has the OpenVINO toolkit installed. Download it as follows:

docker pull jonaswerner/movidius_nc2_with_openvino:2018.5.455

Run the container in privileged mode

Privileged mode is required as the Movidius compute stick changes from USB2.0 to USB3.0 and is re-enumerated by the OS once the ML model is loaded into it. The container need to be able to access the “new” USB3.0 device once it appears.

Output has been shortened for brevity. If all goes well it will finish with the following message:

Demo completed successfully.

This verifies that the Movidius NCS2 (referred to as “MYRIAD” in the command above) is working as expected. It will also have downloaded the sample code to multiple applications, including the benchmark_app we will build.

Download the Alexnet model

We now download the Alexnet model which will be used when executing the benchmark_app. Then we optimize it for FloatingPoint 16 (Movidius NCS2) and for FloatingPoint32 (CPU) so we can run benchmarks against both.

When we use the model optimizer (mo.py) to convert model to Inference Engine format we end up with a pair of files – one XML and one BIN.

Compile the benchmark app from the sample source code

Note that in this case we’re doing the benchmark app but there are many interesting application samples included in the same directory.

cd ~/inference_engine_samples/benchmark_app/
make

After compiling the resulting binary can be found here: ~/inference_engine_samples/intel64/Release/

Run benchmarks for MYRIAD and CPU for comparison

Note that even though we run the inferencing against the same image (“car.png”) we have to change the model optimizer between FP16 for MOVIDIUS and FP32 for CPU depending on which of the two we intend to benchmark.

That is all for this blog post, but it should have provided the required information to compile any of the other sample applications as well as the necessary instructions for how to download and optimize models required for some of the apps.