Last year we discovered Kendryte K210 processor with a RISC-V core and featuring AI accelerators for machine vision and machine hearing. Soon after, Sipeed M1 module was launched with the processor for aroud $10.

Then this year we started to get more convenient development board featuring Sipeed M1 module such as Maixduino or Grove AI Hat. Seeed Studio sent me the last two boards for review. So I’ll start by showing the items I received, before showing how to get started with MicroPython and Arduino code. Note that I’ll be using Ubuntu 18.04, but development in Windows is also possible.

Unboxing

I received two packages with a Maixduino kit, and the other “Grove AI HAT for Edge Computing”.

Grove AI HAT for Edge Computing

Let’s start with the second. The board is a Raspberry Pi HAT with Sipeed M1 module, a 40-pin Raspberry Pi header, 6 grove connectors, as well as connectors for camera and display. The USB-C port is used for programming and power in standalone mode.

Click to Enlarge

There are also headers for the board, but failed to see them during unboxing. We’ll see them when I insert them into the board.

The bottom side does not have much expect numbering the I/O header pins, and another mount point for the camera.

Later on I received the camera for the board as well.

Maixduino kit

The second package includes a Maixduino Arduino compatible board with Sipeed M1 module and an ESP32 module. The camera is already attached to the board, and a 2.4″ color display (QVGA resolution) is also part of the package.

Click to Enlarge

The bottom side of the board just include silkscreen for the header pin names, as well as a block diagram for the board.

Getting Started with Maixduino Kit using MicroPython

I’ll start with Maixduino first, trying to following some of the instructions from the Wiki, and other resources from the Internet since it’s somewhat incomplete or inaccurate at times.

[INFO]Defaultbaudrate is115200,later it may be changed tothe value you set.

[INFO]Trying toEnter the ISP Mode...

**

Greeting fail,check serial port(Attempting touseaport that isnotopen)

I asked Seeed Studio, and I was told to use the other port, and it worked!

Looking at the kernel log above, I can actually see both ttyUSB0 and ttyUSB1 assigned to Maixduino, and with hindsight, that’s probably because we can access both ESP32 and Kendryte K210 processors over different serial ports through a single USB cable.

However when I connected to /dev/ttyUSB0 via minicom (115200 8N1), there seems to be an issue with the firmware:

This gives list of modules, and we can help(‘<module_name>’) to get a list of functions for each module. That’s not the most user-friendly way to learn, so let’s get back to the Wiki, and check the sample to capture a photo and show it on display:

Python

1

2

3

4

5

6

7

8

9

10

11

12

importsensor

importimage

importlcd

lcd.init()

sensor.reset()

sensor.set_pixformat(sensor.RGB565)

sensor.set_framesize(sensor.QVGA)

sensor.run(1)

whileTrue:

img=sensor.snapshot()

lcd.display(img)

Actual output in terminal:

Shell

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

>>>import sensor

>>>import image

>>>import lcd

>>>lcd.init()

>>>sensor.reset()

init i2c2

[MAIXPY]:findov2640

>>>sensor.set_pixformat(sensor.RGB565)

True

>>>sensor.set_framesize(sensor.QVGA)

>>>sensor.run(1)

True

>>>whileTrue:

...img=sensor.snapshot()

...lcd.display(img)

...

We can indeed see the camera output on the attached display.

As your program grows, the REPL interface where you type commands in the serial console becomes impractical. But good news the company also designed MaixPy IDE to more conveniently write and load code to the board.

I wanted to use the latest MaixPy IDE 0.2.4 which required firmware 0.4.0 or greater. I had no luck with 0.4.0.39 I could see a new release had just been build on August 16th, so I flashed it:

Click on the green chain link icon on the bottom left of the ID and select /dev/ttyUSB0 to connect to the board, and once it’s done press the “Play” button underneath to load and run the Hello World sample on Maixduino.

Click to Enlarge

The top right zone shows the actual frame buffer, and the three zones underneath the R, G & B histograms.

It’s all good but the highlight of K210 processor is its KPU AI / machine vision accelerator. There’s documentation to help us.

First let’s download the models:

Shell

1

wget-r--no-parenthttp://dl.sipeed.com/MAIX/MaixPy/model/

There are three:

Shell

1

2

3

ls

face_model_at_0x300000.kfpkgmobilenet_0x300000.kfpkg

mobilenet_7_5_224_tf.h5

“Face model” is a pre-trained tiny Yolo-v2 model to detect (human) faces, and the other two are mobilenet. I’ll run the Yolo-V2 demo.

We can get back to kflash_gui, load the model, and click Download to flash it to the board.

The first line loads the model we’ve just flashed at address 0x300000. The second line defines the anchors for Yolo V2. Those are predefined width & heights for Yolo “boxes”. More details about what Yolo anchors mean can be found on Github. The third line initializes Yolo on the KPU with five parameters:

kpu_net: kpu network object

threshold: probability threshold

nms_value: box_iou threshold

anchor_num: number of anchors

anchor: anchor parameters previously defined

The while loop is pretty much self-explanatory: capture image, run yolo, if a face is detected draw a rectangle, push the result to the display, and loop again.

But does it actually work?

Click to Enlarge

Yes, it does. Note recognition is done at around 11 fps in the IDE, but it will be faster when running standalone. So I’ve added code I found in Tiziano Fiorenzani’s MaixPy Dock demo to show the fps in the top left of the display:

I’ve then called an associate to help testing multiple faces detection, and shot a video.

Face detection/tracking is done at around 15 to 18 fps, and my panda friend did not get detected, but it’s probably more of a feature than a bug since the model must have been trained with human faces. When I tried before the panda could get tracked from time to time, but not quite as well as my face.

To go further you may want to use ESP32 features like WiFi and/or Bluetooth connectivity, but sadly there’s no documentation for it. I tried my ESP32 Micropython tutorial to setup WiFi but it did not work:

Shell

1

2

3

4

5

>>>import network

>>>sta_if=network.WLAN(network.STA_IF);sta_if.active(True)

Traceback(most recent call last):

File"<stdin>",line1,in<module>

AttributeError:'module'objecthas no attribute'WLAN'

I guess that’s normal since I’m running this in K210 instead of ESP32. I was hoping they may have adapted it. Looking at the help there are only a few function implemented:

Shell

1

2

3

4

5

6

>>>help(network)

object<module'network'>isof typemodule

__name__--network

ESP8285--<class'ESP8285'>

ESP32_SPI--<class'ESP32_SPI'>

route--<function>

It’s not possible to use /dev/ttyUSB1 to connect to ESP32, so it may require a bit more work..

Getting Started with Grove AI HAT

Standalone mode

Let’s switch to the other board: Grove AI HAT which works both in standalone mode or connected to a Raspberry Pi. I’ll mostly follow the Wiki that shows how to use the board in standalone mode with the Arduino IDE, skipping the GPIO / Grove stuff, and jumping directly to the camera, display, and computer vision section.

I’ve connected the fisheye camera, inserted the pin headers, and borrowed the display from Maixduino kit. Note that you could also use Maixduino camera with Grove AI HAT, and I’ve tried but noticed face detection and tracking did not work as well.

I’ll assume you have already installed the Arduino IDE in your host computer. For reference, I’m using Arduino 1.8.9 in Ubuntu 18.04.

First we need to go to File->Preferences, and edit the “Additional Boards Manager URLs” with the following URL to add support for Seeed Studio boards:

Now go back to your working directory and create a folder and empty sketch with the same name, in our case “face_detect”:

Shell

1

2

3

mkdirface_detect

cdface_detect/

touchface_detect.ino

Copy the content of face_detect from the SDK:

Shell

1

cp../kendryte-standalone-demo/face_detect/*-rf.

Now open face_detect.ino in Arduino IDE. This will open a bunch of other files automatically, including board_config.h which we need to modify to enable OV2640 sensor and LICHEEDAN board as illustrated below.

You can go to main.c to study the code, but it will require a bit more efforts to understand the code than Maixduino’s MicroPython sample. If you’d just like to try it out, you can check whether it compiles. I had a few warnings due to unused variables:

The board will now capture video and show a red rectangle when a human face is detected in a similar fashion to the Maixduino MicroPyhon demo.

The display was up-side-down though. A small change to the code fixed that (see line 5 below):

Arduino

1

2

3

4

5

6

7

8

#if BOARD_LICHEEDAN

#if OV5640

lcd_set_direction(DIR_YX_RLUD);

#else

lcd_set_direction(DIR_YX_LRUD);

/* lcd_set_direction(DIR_YX_RLDU);*/

#endif

#else

That’s basically like Maixduino demo above but instead of relying on MicroPython, we did so in the Arduino IDE.

Grove AI HAT and Raspberry Pi 4

Since we have a HAT board that would be a shame if we did not also use it with a Raspberry Pi board. So I inserted the board into a Raspberry Pi 4 SBC, after disconnecting the power supply obviously. Talking about power, please make sure RPI 5V jumper on the top of the Grove AI HAT board is set to ON.

One of the first things I thought would be interesting was to develop code directly on the Raspberry Pi 4. So I installed Arduino 1.8.9 for Arm 32-bit, and loaded Seeed Studio board URL based on the instructions above.

Click to Enlarge

But when I tried to install “Grove AI HAT for Edge Computing by Seeed Studio” I quickly encountered a roadblock:

The message above is “Tool kflash is not available for your operating system”, but I also got “Tool kendryte-standalone-sdk is not available for your operating system”. It should not be surprising that Kendryte K210 tools are not (yet) available for Arm.

As a side note, I’m using Raspberry Pi 4 with 1GB, and I quickly found out it was not a good idea to run Chromium, even with just one tab, and the Android IDE side-by-side, as you quickly run out of memory, and it even completely filled the 512MB swap on the microSD card. Even with ZRAM there’s no hope, so if you ever considered running the Arduino IDE on Raspberry Pi 4 (e.g. for Arduino boards) while also checking out documentation, do yourself a favor and get the 2GB or 4GB RAM version.

Nevertheless, another way to use Grove AI HAT with Raspberry Pi is to display data returned by the expansion, as Seeed Studio explains in their face count demo tutorial. Let’s try it.

We’re off to a good start since we already setup the Arduino IDE above, and installed the face demo firmware. Now we can just install the Face Count program on the Raspberry Pi:

Shell

1

2

3

4

5

6

cd~

git clonehttps://github.com/LynnL4/face-detected.git

cdface-detected

sudo chmod-R777installer

cdinstaller

sudo./setup.sh

However, if I run it a segfault occurs:

Shell

1

2

sudo./face-detected.sh

Segmentation fault

Let’s have a look at setup.sh:

Shell

1

2

3

4

5

6

7

8

9

10

11

12

13

14

#!/bin/bash

#start pi spi

pwds=/boot/config.txt

sudo sed-i's/dtparam=spi=off//g'$pwds

sudo sed-i's/dtparam=spi=on//g'$pwds

sudo sed-i'$a dtparam=spi=on'$pwds

#reboot后生效

CurDir=$PWD"/face-detected.desktop"

ExecDir=$PWD"/face-detected.sh"

IconDir=$PWD"/Pyte_Data/icon.png"

sudo sed-i"4c Exec=$ExecDir"$CurDir

sudo sed-i"5c Icon=$IconDir"$CurDir

sudo cp$CurDir/usr/share/applications/face-detected.desktop

cp$CurDir/home/pi/Desktop/face-detected.desktop

We can see the Grove AI HAT communicate with the Raspberry Pi board over SPI, and they set it by modifying /boot/config.txt file. So I decided to reboot, but the problem is the same. The installer directory contains many binary libraries which were tested on Raspberry Pi 3 B+ with Debian Wheezy, so switching to Raspberry Pi 4 with Debian Buster probably screwed things. I’ve let Seeed Studio know about it and they are looking into it.

Let’s complete the review by showing a video of the face detection demo running on Grove AI HAT, again joined by a few “friends”.

It works roughly as expected with the demo filtering out most of my non-human friends, and catching up to 3-4 faces simultaneously from a photo.

Purchasing the Boards, Display and Cameras

I’d like to thanks Seeed Studio for sending the board and let me experiment with Kendryte K210 / Sipeeed M1 platforms using MicroPython and Arduino. If you’d like to reproduce my setup, you can buy the Grove AI Hat for $28.90, the OV2640 fisheye camera for $7.60, and/or the Maixduino kit with the board, a display and a camera for $23.90.

I can confirm that the latest firmware release (24/08/2019) maixpy_v0.4.0_46_gf46e4c4.bin works with the IDE AND the face detection example on Maixduino board. So no need anymore for a downgrade to v 3.2 .

Vote Up0Vote Down Reply

17 days ago

Guest

TPass

firmware: maixpy_v0.4.0_46 (latest)

I tried to convert movies with ffmpeg but their own movies don’t work. Even their own supplied ‘badapple.avi’ movie fails with OSError: [Errno 22] EINVAL

The lvgl library (graphocs) is not there. When I try the examplles, I get: ImportError: no module named ‘lvgl’

Embedded Systems Jobs

You are a bright and passionate C++ Software Engineer who will work as an Engineer or Designer on the design and implementation of state of the art projects both onsite with the client and at our own offices. You will work on projects in our differen [...]

We're looking for someone to develop tools in C# to advance analysis of established in-house processing and 3D modeling software. You'll get to join one of the strongest teams out there and help deliver a true team effort to help improve the securit [...]

Inspects, cleans, transforms, and models data with the goal of highlighting useful information, suggesting conclusions, and supporting decision making - Possesses knowledge across the entire field of software technologies and engineering, in [...]

We're looking for someone to combine FPGA analysis with software development to support FPGA analysis, developing tools in C# to implement analytic needs and further automate processing. You'll get to join one of the strongest teams out there and he [...]