Overview

The Zynq CPU/FPGA and host operating system

The main CPU of the E320 is a Xilinx Zynq SoC XC7Z045. It is both a dual-core ARM Cortex A9 CPU and Kintex-7 FPGA on a single die. The CPU is clocked at 1GHz (speedgrade 3).

The programmable logic (PL, or FPGA) section of the SoC is responsible for handling all sampling data, the 1/10 GigE network connections, and any other high-speed utility such as custom RFNoC logic. The processing system (PS, or CPU) is running a custom-build OpenEmbedded-based Linux operating system. The OS is responsible for all the device and peripheral management, such as running MPM (see section The Module Peripheral Manager (MPM) Architecture), configuring the network interfaces, running local UHD sessions, etc.

The STM32 microcontroller

The STM32 microcontroller controls various low-level features of the E320 series motherboard: It controls the power sequencing, reads out fan speeds and some of the temperature sensors. It is connected to the Zynq via an I2C bus.

It is possible to log into the STM32 using the serial interface (see Connecting to the microcontroller). This will allow certain low-level controls, such as remote power cycling should the CPU have become unresponsive for whatever reason.

The SD card

The E320 uses a micro SD card as its main storage. The entire root file system (Linux kernel, libraries) and any user data are stored on this SD card.

The SD card is partitioned into four partitions:

Boot partition (contains the bootloader). This partition usually does not require any modifications.

A data partition, mounted in /data. This is the only partition that is not erased during file system updates.

Two identical system partitions (root file systems). These contain the operating system and the home directory (anything mounted under / that is not the data or boot partition). The reason there are two of these is to enable remote updates: An update running on one partition can update the other one without any effect to the currently running system. Note that the system partitions are erased during updates and are thus unsuitable for permanently storing information.

Note: It is possible to access the currently inactive root file system by mounting it. After logging into the device using serial console or SSH (see the following two sections), run the following commands:

The device node in the mount command will likely differ, depending on which partition is currently already mounted.

Getting started

This will run you through the first steps relevant to getting your USRP E320 up and running. Note: This guide was creating on an Ubuntu machine, and other distributions or OS's may have different names/methods.

Assembling the E320

Unlike the X300 or N200 series, there is no assembly of required since it is a monolithic board.

Checklist:

Connect power and network

Read security settings

Connect clocking (if required)

Updating the file system

Before doing any major work with a newly acquired USRP E320, it is recommended to update the file system. For the OEM/Board-only version of E320, the SD card is physically accessible and filesystem update can be accomplished directly by using Mender or externally by manually writing an image onto a micro SD card and inserting it. For the enclosure version of E320, Mender update is required as there is no direct physical access to the device. For details on using Mender, see Section Mender: Remote update capability .

Manual updating is simply loading an image on the micro SD card. The first step in that process is to obtain an image.

To obtain the default micro SD card image for a specific version of UHD, install that version of UHD (3.13.0.2 or later) on a host system with Internet access and run:

$ uhd_images_downloader -t e320 -t sdimg

The image will be downloaded to <UHD_INSTALL_DIR>/share/uhd/images/usrp_e320_fs.sdimg, where <UHD_INSTALL_DIR> is the UHD installation directory.

To load an image onto the micro SD card, connect the card to the host and run:

$ sudo dd if=<YOUR_IMAGE> of=/dev/<YOUR_SD_CARD> bs=1M

The <YOUR_IMAGE> is the path to the micro SD card image (i.e.<UHD_INSTALL_DIR>/share/uhd/images/usrp_e320_fs.sdimg).

The <YOUR_SD_CARD> device node depends on your operating system and which other devices are plugged in. Typical values are sdb or mmcblk0.

CAUTION: The Linux utility dd or bmap can cause unrecoverable data loss if the incorrect disk is selected, or if the parameters are input incorrectly. Ensure you have selected the correct input and output parameters for your system configuration.

The micro SD card used can be the original SD card shipped with the device or another one that is at least 16 GB in size.

Serial connection

It is possible to gain root access to the device using a serial terminal emulator. Most Linux, OSX, or other Unix flavours have a tool called 'screen' which can be used for this purpose, by running the following command:

$ sudo screen /dev/ttyUSB2 115200

In this command, we prepend 'sudo' to elevate user privileges (by default, accessing serial ports is not available to regular users), we specify the device node (in this case, /dev/ttyUSB2), and the baud rate (115200).

The exact device node depends on your operating system's driver and other USB devices that might be already connected. Modern Linux systems offer alternatives to simply trying device nodes; instead, the OS might have a directory of symlinks under /dev/serial/by-id:

Note: Exact names depend on the host operating system version and may differ.

Every E320 device connected to USB will by default show up as four different devices. The devices labeled "USB_to_UART_Bridge_Controller" are the devices that offer a serial prompt. The one with the if01 suffix connects to Linux, whereas the one with if00 suffix connects to the STM32 microcontroller. If you have multiple E320 devices connected, you may have to try out multiple devices. In this case, to use this symlink instead of the raw device node address, modify the command above to:

On this prompt, you can enter any Linux command available. Using the default configuration, the serial console will also show all kernel log messages (unlike when using SSH, for example), and give access to the boot loader (U-boot prompt). This can be used to debug kernel or bootloader issues more efficiently than when logged in via SSH.

Connecting to the microcontroller

The STM32 microcontroller (which controls the power sequencing, among other things) also has a serial console available. To connect to the microcontroller, use the other UART device. In the example above:

It provides a very simple prompt. The command 'help' will list all available commands. A direct connection to the microcontroller can be used to hard-reset the device without physically accessing it (i.e., emulating a power button press) and other low-level diagnostics.

SSH connection

The USRP E320 devices have two network connections: One SFP port, and an RJ-45 connector. The latter is by default configured by DHCP; by plugging it into into 1 Gigabit switch on a DHCP-capable network, it will get assigned an IP address and thus be accessible via ssh.

In case your network setup does not include a DHCP server, refer to the section Serial connection. A serial login can be used to assign an IP address manually.

After the device obtained an IP address you can log in from a Linux or OSX machine by typing:

$ ssh root@ni-e320-311FE00 # Replace with your actual device name!

Depending on your network setup, using a .local domain may work:

$ ssh root@ni-e320-311FE00.local

Of course, you can also connect to the IP address directly if you know it (or set it manually using the serial console).

Note: The device's hostname is derived from its serial number by default (ni-e320-<SERIAL>). You can change the hostname by modifying the /etc/hostname file and rebooting.

On Microsoft Windows, the connection can be established using a tool such as Putty, by selecting a username of root without password.

Like with the serial console, you should be presented with a prompt like the following:

root@ni-e320-311FE00:~#

Network Connectivity

The RJ45 port (eth0) comes up with a default configuration of DHCP, that will request a network address from your DHCP server (if available on your network).

The SFP+ (sfp0) port is configured with static address 192.168.10.2/24

The configuration for the sfp0 port is stored in /etc/systemd/networkd/sfp0.network.

Note: Care needs to be taken when editing these files on the device, since vi / vim sometimes generates undo files (e.g. /etc/systemd/networkd/sfp0.network~), that systemd-networkd might accidentally pick up.

Note: Temporarily setting the IP addresses via ifconfig etc will only change the value until the next reboot or reload of the FPGA image.

Security-related settings

The E320 ships without a root password set. It is possible to ssh into the device by simply connecting as root, and thus gaining access to all subsystems. To set a password, run the command

$ passwd

on the device.

Updating the FPGA

Updating the FPGA follows the same procedure as other USRPs. Use the uhd_image_loader command line utility to upload a new FPGA image onto the device. The command can be run on the host to load the image via RJ-45 network connection or it can be run on the device.

A common reason to update the FPGA image is in the case of a UHD/FPGA compat number mismatch (for example, if UHD has been updated, and now expects a newer version of the FPGA than is on the device). In this case, simply run

$ uhd_images_downloader

to update the local cache of FPGA images. Then, run

$ uhd_image_loader --args type=e3xx,addr=ni-e320-311fe00

to update the FPGA using the default settings. Replace ni-e320-311fe00 in the addr with the correct device address. If a custom FPGA image is targeted for uploading, use the --fpga-path command line argument. Run

$ uhd_image_loader --help

to see a full list of command line options. Note that updating the FPGA image will force a reload of the FPGA, which will temporarily take down the SFP network interfaces (and temporary settings, such as applied via ifconfig on the command line, will be lost).

Using an E320 USRP from UHD

Like any other USRP, all E320 USRPs are controlled by the UHD software. To integrate a USRP E320 into your C++ application, you would generate a UHD device in the same way you would for any other USRP:

ref_locked: This will check that all the daughterboards have locked to the external/internal reference clock.

fan: get fan speed (in rpm)

Remote Management

Mender: Remote update capability

Mender is a third-party software that enables remote updating of the root file system without physically accessing the device (see also the Mender website). Mender can be executed locally on the device, or a Mender server can be set up which can be used to remotely update an arbitrary number of USRP devices. Mender servers can be self-hosted, or hosted by Mender (see mender.io for pricing and availability).

When updating the file system using Mender, the tool will overwrite the root file system partition that is not currently mounted (note: every SD card comes with two separate root file system partitions, only one is ever used at a single time). Any data stored on that partition will be permanently lost. After updating that partition, it will reboot into the newly updated partition. Only if the update is confirmed by the user, the update will be made permanent. This means that if an update fails, the device will be always able to reboot into the partition from which the update was originally launched (which presumably is in a working state). Another update can be launched now to correct the previous, failed update, until it works. See also Section The SD card.

To initiate an update from the device itself, download a Mender artifact containing the update itself. These are files with a .mender suffix.

Then run mender on the command line:

$ mender -rootfs /path/to/latest.mender

The artifact can also be stored on a remote server:

$ mender -rootfs http://server.name/path/to/latest.mender

This procedure will take a while. After mender has logged a successful update, reboot the device:

$ reboot

If the reboot worked, and the device seems functional, commit the changes so the boot loader knows to permanently boot into this partition:

$ mender -commit

To identify the currently installed Mender artifact from the command line, the following file can be queried:

$ cat /etc/mender/artifact_info

If you are running a hosted server, the updates can be initiated from a web dashboard. From there, you can start the updates without having to log into the device, and can update groups of USRPs with a few clicks in a web GUI. The dashboard can also be used to inspect the state of USRPs. This is simple way to update groups of rack-mounted USRPs with custom file systems.

Salt: Remote configuration management and execution

Salt (also known as SaltStack, see Salt Website) is a Python-based tool for maintaining fleets of remote devices. It can be used to manage USRP E320 remotely for all types of settings that are not controlled by UHD. For example, if an operator would like to reset the root password on multiple devices, or install custom software, this tool might be a suitable choice.

Salt is a third-party project with its own documentation, which should be consulted for configuring it. However, the Salt minion is installed by default on every E320 device. To start it, simply log on to the device and run:

$ systemctl start salt-minion

To permanently enable it at every boot, run (this won't by itself launch the salt-minion):

$ systemctl enable salt-minion

To make use of Salt, both the device needs to be configured (the "minion") and, typically, a server to act as the Salt master. Refer to the Salt documentation on how to configure the minion and the master. A typical sequence to get started will look like this:

Install the salt-master package on the server (e.g. by running apt install salt-master if the server is an Ubuntu system), and make sure the Salt master is running.

Add the network address / hostname of that server to the /etc/salt/minion file on the device by editing the master: line.

Launch the Salt minion on the USRP by running the command systemctl start salt-minion.

The minion will try to connect to the master. You need to authorize the minion by running salt-key -a $hostname where $hostname is the name of the minion.

Once the device is authorized, you can try various commands to see if the communication was established:

ni-n3xx-$serial:

eth0:

|_

[...]

Theory of Operation

E320 is on the MPM architecture (see also: The Module Peripheral Manager (MPM) Architecture). Inside the Linux operating system running on the ARM cores, there is hardware daemon which needs to be active in order for the device to function as a USRP (it is enabled to run by default).

A large portion of hardware-specific setup is handled by the daemon.

Modifying and compiling UHD and MPM for the E320

E320 devices ship with all relevant software installed on the SD card. Updating UHD and/or MPM on the SD card is typically easiest done by updating the filesystem image (see Section Mender: Remote update capability). However, it is certainly possible to compile UHD and MPM by hand, e.g., in order to modify and try out changes without having to build entire filesystems in between. At Ettus R&D, this mode of operation is often used for rapid iteration cycles.

Compiling MPM natively

In general, compiling natively is not a recommended way of compiling code for the ARM processors. However, in the case of MPM, the amount of C++ code that needs to be compiled is very little, and a full compile of MPM will take a few minutes even on the device. First, you need to get a copy of the MPM source code onto your device. If you have an internet connection, you can use git to pull it directly from the Ettus repository (all commands are run on the device itself, inside the home directory):

$ git clone https://github.com/EttusResearch/uhd.git

You can also SSHFS it from another computer:

$ mkdir uhd # Create a new, empty directory called uhd
$ sshfs user@yourcomputer:src/uhd uhd # This will mount ~/src/uhd from the remote machine to ~/uhd on the device

Now, create a build directory and use the regular cmake/make procedure to kick off a build. It can be advantageous (especially for slow network connections) to create the build directory outside of the repository directory:

$ mkdir build_mpm
$ cd build_mpm # You are now in /home/root/build_mpm
$ cmake ../uhd/mpm
$ make -j2 install # This will take several minutes

Note that this overwrites your system MPM. You can install MPM to another location by specifying -DCMAKE_INSTALL_PREFIX, but make sure to update all of your paths appropriately.

If you prefer cross-compiling MPM the same way as UHD, refer to the following sections and adapt the instructions for UHD appropriately.

Obtaining an SDK

The recommended way to develop software for the E320 is to cross-compile. By running the compiles on a desktop or laptop computer, you will be able to speed up compile times considerably (compiling UHD natively for the E320 would take many hours).

SDKs are distributed along with other binaries. They contain a cross-compiler, a cross-linker, a cross-debugger, and all the libraries available on the device to mirror its environment.

If this doesn't work, the executable permissions of the file might have been lost (this can occur with some versions of Python). In that case, add those permissions back before executing the .sh file:

$ chmod +x oecore-x86_64-cortexa9hf-neon-toolchain-nodistro.0.sh

Executing the .sh file will prompt you for an installation path. Please ensure you have sufficient disk space, as each of the SDKs may require several gigabytes of disk space (depending on the image flavor selected).

This will allow you to compile UHD as well as (depending on the image flavor) other software.

Please note, that while several toolchains can be installed in parallel, they have to be installed to different directories.

SDK Usage

Having installed the toolchain in the last step, in order to build software for your device open a new shell and type:

$ . $SDKPATH/environment-setup-armv7ahf-vfp-neon-oe-linux-gnueabi

This will modify the PATH, CC, CXX etc, environment variables and allow you to compile software for your USRP E320 device. To verify all went well you can try:

$ $CC -dumpmachine

which should return 'arm-oe-linux-gnueabi'.

Building UHD

Type the following in the build directory (assuming a build in host/build):

$ cmake -DCMAKE_TOOLCHAIN_FILE=../host/cmake/Toolchains/oe-sdk_cross.cmake -DCMAKE_INSTALL_PREFIX=/usr .. # Add any CMake options you desire
$ make # You can run make -j12 to compile on 12 processes at once

Note: The UHD you are cross-compiling will not run on your host computer (the one where you're doing the development). Compiling UHD regularly on your host computer (with MPMD enabled) will allow you to talk to your E320.

Several GNU Radio components depend on running binaries built for the build machine during compile. These binaries can be built and used for cross compiling, but this is an advanced topic.

E320-specific Features

Front and Rear Panel

Like the USRP X300 and N310 series, E320 has connectors on both the front and back panel. The back panel holds the power connector, all network connections, USB connections for serial console (see Serial connection), JTAG and peripherals, and front-panel GPIO.

The front panel is used for all RF connections, SMA connectors for GPS antenna input, 10 MHz external clock reference.

The connectors are labeled RF A and RF B and are powered by the two channels of AD9361 RFIC.

FPGA Register Map

The following tables describe how FPGA registers are mapped into the PS. This is for reference only, most users will not even have to know about this table.