Details

Project Logs

I learned about a new FOSS project a few weeks ago: apio. It's stated mission is:

Experimental open source micro-ecosystem for open FPGAs. Based on platformio. Apio is a multiplatform toolbox, with static pre-built packages, project configuration tools and easy commands to verify, synthesize, simulate and upload your verilog designs.

That sounded pretty good to me. It was easy to install on my RPi3:

pip install apio

That just installs the apio "shell". In order to get all the synthesis, simulation, and board tools, I used the command:

apio install --all

That automatically downloads and installs some system utilities, the Icestorm FPGA toolchain, the iverilog simulator, and some design examples for several types of iCE40 FPGA boards. Once that was completed, I could check to see what boards were supported:

What!?!? No CAT Board in the list!?! I needed to fix that situation. I poked around in the apio Python code and found a likely file at apio/resources/boards.json. Inside that file, I pretty much copied the existing entry for the icoboard with a few modifications:

I've been away from this project for a few months (OK, four months) building things like a new tool for designing electronics. One of the things I haven't discussed here is the time it takes to download a bitstream to the FPGA on the CAT Board.

As shown in previous logs, the FPGA is configured through one of the hardware SPI ports of the RPi. I've never considered SPI a very fast way of transferring data, so I initially set the port bit rate at 1 Mbps. That was good enough to get the FPGA going within a couple of seconds and there was no reason to push it and possibly cause errors while I debugged the board.

But once the board was working reliably, I revisited the SPI bit-rate setting. I figured there was no harm in upping it to 5 Mbps just to see what happens. I went into the litterbox.py script and changed it to:

self.spi.speed = 5000000

Then I ran the command to load the FPGA with the bitstream for the LED blinker:

sudo litterbox -c blinky.bin

The download to the FPGA completed more quickly than before and the LED started blinking. Success!

OK, I hadn't expected to get even close to 200 Mbps. With a little trial and error, I finally found the maximum speed I could use was 199,999,999 bps. The reason for that becomes clear later.

Now, was I actually transferring bits at 200 Mbps, or was the software making a promise that the hardware couldn't keep? To test that, I wrote some code to time the transmission of a 10 MByte payload and compute the effective bit-rate while I also observed the maximum SPI clock frequency and duty cycle with an oscilloscope:

spi.speed (Mbps)

Actual Speed (Mbps)

Fmax (MHz)

Duty Cycle (%)

4

1.8

2.5

---

10

3.3

6.25

54.0

25

8.8

25

36.1

50

10.7

50

21.7

67

11.3

67

17.9

100

12.0

100

12.1

150

12.0

100

12.1

200

12.8

200

6.4

As can be seen, the actual transmission speeds are quite a bit lower than the speed setting. The reason for that is the overhead in the python-spi module that copies and converts the individual 4096-byte packets of the payload before sending them to the SPI driver. Even though each packet gets transmitted at a high clock speed, there's a significant "dead time" (2.3 ms) while the software readies the next packet. As the raw speed increases, the packet transmission time decreases and the dead time (which stays constant) consumes a larger percentage of the time to send the full payload. That's why the duty cycle decreases as the speed setting increases.

To decrease the overhead, I modified the python-spi code as follows:

The data payload is checked and no conversion or copying is done if it is already in the form of a string of bytes.

The address of the current packet within the payload is sent to the SPI device driver rather than making a copy of the packet.

After these two changes, setting spi.speed to 100 Mbps resulted in an actual transmission speed of 65 Mbps (an increase of 540%).

There's no reason to set the spi.speed to a value greater than 100 Mbps. The table indicates the RPi is generating the SPI clock by dividing a master 200 MHz clock by an integer. Any setting between 100 and 199 Mbps will result in an SPI clock of 100 MHz, and going to 200 Mbps has already proven too fast for sending an FPGA configuration bitstream. (The iCE40HX datasheet also shows the SPI clock in slave mode should not exceed 25 MHz, so getting to 100 MHz is really pushing it already.)

A transfer rate of 65 Mbps opens up some interesting possibilities. That means there is an 8 MByte/second channel between the CAT Board FPGA and the RPi that uses only a few pins of the GPIO connector. I have some Xilinx-centric VHDL modules and a Python library that provide a printf-like debug interface for FPGA designs through the JTAG port. I can modify these to use the SPI port so the CAT Board + RPi will have the same capabilities. I'll be working on that next. I think. Maybe.

In my previous post, I showed how to blink an LED on the CAT Board using Verilog. Now I'll do the same thing using MyHDL, a hardware description language based on Python.

I'll assume a starting point of a Raspberry Pi running the Raspbian OS with the yosys, arachne-pnr and icestorm FPGA tools installed. If you're following along and haven't already got that, read the previous post. The stuff I describe here won't work without it.

Install MyHDL

Before you can use MyHDL, you have to install it. Installing the latest release is as simple as:

sudo pip install myhdl

But I like to use the development version because that's where all the new features are. That's installed like this:

Compiling to Blinky

The blinky.py file containing the MyHDL code shown above is executed as follows:

python blinky.py

This translates the MyHDL code into Verilog that's stored in the blinky.v file. Then this Verilog file can be compiled into a bitstream using the yosys, arachne-pnr and icepack tools just like in the previous post. Once the bitstream is downloaded to the CAT Board, the LED will blink.

Raison de'Etre

Why bother doing this in MyHDL? It just adds another step, but does it add any value?

In this case, no, it doesn't. Because this design is so simple, the MyHDL code isn't any more compact or expressive than the original Verilog. The value of MyHDL is experienced when doing design exploration, i.e. trying various approaches to solving a problem. Then all the features of the Python ecosystem can be used with MyHDL to come up with a solution. I showed a few examples of this here and here.

A fundamental rite of passage is getting your embedded system to blink an LED. In this post, I'll show you how to take a Raspberry Pi 3 fresh out of the box and get an LED blinking on the CAT Board.

(If you're already experienced with the RPi, then a lot of this post will be redundant for you. However, I've found it useful to document processes so I can repeat them reliably at a later date when the details have grown fuzzy.)

Use the Win32DiskImager to write the Raspbian OS to an SD card inserted in the PC. (This will take a while.)

SSH Over Ethernet

I'm going to use the RPi by connecting to it over a network SSH connection from my PC:

Look in Network Connections to find the IP address of an Ethernet port the RPi can attach to.

Go into the SD card and make a copy of the cmdline.txt file. Then rename the copy cmdline.normal. (This saves a copy of the original configuration in case I want to go back to it later.)

Edit the cmdline.txt file to assign an IP address to the RPi that's identical to the one for the Ethernet port except for the last field. In my case, I selected 169.254.68.2 and appended it to the first (and only) line in the file to get:

Also, change the name of the workgroup to whatever is being used for the Windows PC:

workgroup = XESS

Start the Samba server running on the RPi:

sudo service smbd start

Open the Network window on the PC. The RPi should be visible there:

Double-click the RASPBERRYPI icon and the shared directory will appear. Files can be transferred between the PC and RPi there:

Reconfigure the RPi

Before the RPi can be used to program the CAT Board FPGA, a few of its configuration settings need to be adjusted:

On the RPi, type the command:

sudo raspi-config

In the initial screen that appears, select option 1 for expanding the linux partition. (The stock Raspbian OS image creates a 4GB partition. This won't be enough to compile the FPGA tools. Believe me, I tried.)As an alternative, you can also use the command-line version to do the same thing:

sudo raspi-config --expand-rootfs

Communicating with the CAT Board also requires the use of the RPi's hardware SPI port. That is enabled using the following screens:

Finally, close raspi-config and reboot the RPi to let the changes take effect.

Install the FPGA Software Tools

One of the reasons for respinning the CAT Board PCB was to get the SPI flash chip connected correctly to the Lattice FPGA. The flash+FPGA+Raspberry Pi interconnection is complicated because it has to operate in three different modes:

During development or when the FPGA is being used dynamically, the RPi has to be able to load a configuration bitstream into the FPGA.

After development is completed, the RPi may want to store the finished FPGA bitstream into the flash.

When the CAT Board is used stand-alone or when the FPGA performs some fixed function for the RPi, the FPGA has to be able to configure itself upon powerup with the bitstream stored in the flash.

All three modes have to share a single SPI bus between all the devices while using a minimum of RPi GPIO signals and extra circuitry.

Mode 3 is the easiest: when the FPGA powers up (or is reset), it checks to see if the SPI CS line is pulled high and, if so, becomes an SPI master and reads its configuration bitstream from the flash. The RPi GPIO signals are hidden behind the series resistors and can't interfere.

Mode 2 is slightly more difficult. When the RPi is storing a bitstream by sending it to the flash chip's SI input, the SPI CS line is pulled low. But that also enables the SPI interface of the FPGA which could lead to interference if the FPGA's SDO output becomes active. To prevent this, the RPi asserts the reset pin of the FPGA so its SPI port can't turn on. Problem solved.

Mode 1 is the most difficult. The RPi pulls the SPI CS line low as it removes the reset from the FPGA. This places the FPGA in slave mode so the RPi can send a bitstream to the FPGA's SPI port. But this also enables the flash chip's SPI port, which means the flash's SO output could interfere with the bitstream data. Unfortunately, there's no reset pin on the flash to keep it quiet. However, the flash does have a deep power-down mode that is entered by sending a specific command to the flash. Once in this state, it will not respond to anything until it receives another specific command to wake up. The RPi can then transfer the bitstream to the FPGA. During the transfer, there's no chance the wake-up command will be sent accidentally to the flash's SI input because 1) neither the RPi or FPGA will be driving that signal line during the configuration process, and 2) the flash only executes a command once its CS input goes high but the SPI CS line is held low for the entire duration of the bitstream transfer. So the flash will stay quiet and the RPi can send the bitstream to the FPGA in peace.

Another complication of the shared SPI bus is that the roles of the RPi's MOSI and MISO pins are reversed in modes 1 and 2. In mode 1, the RPI's MOSI output pin drives the SDI input of the FPGA and the FPGA's SDO output drives the RPI's MISO input. That allows the hardware SPI port of the RPi to be used for the SPI transactions. But in mode 2, the RPi's MOSI pin acts as an input to receive data from the flash's SO output and the RPI's MISO pin has to drive the SI input pin of the flash. That precludes the use of the RPI's SPI hardware and the SPI transfers to/from the flash have to be done using bit banging.

I searched for a ready-made SPI bit-banger program but nothing great popped up. So I just wrote one in Python using the RPi.GPIO library. It consists of a class for handling individual pin I/O, another class for SPI transactions, and a final class that handles most of the commands for the serial flash chip.

To test the code, I first tried the command to read the device ID from the flash. The manufacturer and device IDs should have been 0x1F and 0x8401, respectively. Instead, I got 0xFF and 0xFFFF. That's OK; nothing ever works the first time.

I probed with an oscilloscope to make sure the RPi was driving the correct pins of the flash. No problem there except that the SO pin was always high (naturally).

Next, I pulled out my old HP LogicDart and sampled the waveforms on the CS, SCK and SI pins. Once again, no problems:...

Well, this doesn't happen often: I assembled the new 0.2 CAT Board and it worked the first time! The Raspberry Pi was able to program it with the SDRAM diagnostic and it passed (showing four green LEDs).

The next steps are:

Write the RPi code for programming the bitstream into the SPI flash so I can test the configuration of the FPGA in stand-alone mode.

Assemble the SATA I/O. (I checked that the SATA connectors fit their new PCB footprints, but I still need to get the correct termination resistors.)

Before assembling a CAT Board with the new 0.2 PCBs from PCBCart, I thought I'd compare them to the previous 0.1 version from PCBWay.

At a macro-level, they both look like circuit boards. No discernible difference. So that's good.

Looking a little more closely, the PCBCart silkscreen is better defined than the one by PCBWay which suffers from some "blooming". But that also makes the part references on the PCBCart PCB look a bit spidery so it's probably a good idea to thicken the text a bit for those.

Now let's look at drill/via alignment. The vast majority of the small 20-mil vias had drill holes punched completely within the copper pad. But there were a few vias in each set of boards that had "breakouts" like those shown below. Thankfully, there were no cases where the drill missed the via completely in either set. For this metric, the PCBWay boards were slightly superior to those from PCBCart but there were no outright failures in either.

The above image also shows the alignment of the soldermask with the BGA pads. Bad alignment can lead to BGA balls not connecting to the pad or having a ball short to a nearby, uncovered via pad. Neither of those is a good thing. The PCBWay boards have a bit more misalignment than those from PCBCart, but not as much overall as the image above would make you believe. Still, for this metric PCBCart has a slight edge.

Another factor to consider is via size. PCBWay can drill holes as small as 8 mils but they require a 6 mil annular pad so the total via diameter is 6 + 8 + 6 = 20 mils. PCBCart drills a larger 10 mil hole but they only need a 4 mil annular pad for a total via size of 18 mils. When you're putting vias between the pads of an 0.8mm BGA and then trying to squeeze traces between them, that extra 2-mil clearance is a big advantage. Of course, the smaller pad size will also lead to more of the drill breakouts we saw above.

Regarding price, I got 11 PCBs from PCBWay for $185 and 10 PCBs from PCBCart for $166. So the per-board price is about the same. But this changes all the time, so always check before submitting a board.

Now I'm on to building some CAT Boards using the new PCBCart PCBs. That will be the true test.

My wayward PCBs that were erroneously shipped to New Caledonia have arrived! PCBCart arranged to have them shipped to me in NC for no charge (to me; I'm not sure what it cost them). That was nice of them, since they were under no obligation to do that. So +1 for customer service!

I finished testing the major systems of the CAT Board about four weeks ago. Based on what I found, I made the following changes:

Straightened out the MISO/MOSI mess between the FPGA, serial configuration flash, and the Raspberry Pi. Now the FPGA should be programmable from either the RPi (during development) or the flash (for dedicated applications).

Added series resistors between the RPi SPI pins and the FPGA/flash chips to prevent any possible contention while still letting the RPi get control of the FPGA.

Fixed the footprint for the SATA connectors which I first created in 2004 with Eagle and either didn't get right or it changed since then. (Of course, this required some adjustments to the layout since the new footprint is wider and the two connectors were already very close together.)

Rerouted the SATA I/O lines from the FPGA in order to get 100-ohm differential impedances.

Simplified the way the voltage is programmed for the adjustable VCCIO_3 bank of FPGA I/Os. (Now it's done just by changing a single resistor instead of using solder-bridge jumpers.)

I regenerated the gerber files and went looking for someone to fabricate them. I previously used PCBWay and they did a good job for $180, but 3 of the 11 boards they sent had enough soldermask misalignment to make me leery of using them. I had also considered PCBCart, but at the time their prices were considerably higher (around $230, as I recall). But when I went back and requoted them again, the cost had dropped to $167.

So I submitted the gerbers to PCBCart's online ordering system and waited. They fabricated the PCB in a week and shipped it ... to New Caledonia! Now you may not know where New Caledonia is (I know I didn't), but it's best described as "the ass-end of nowhere": hundreds of miles off the east coast of Australia in the middle of the South Pacific. The only way to get farther from North Carolina in the USA would be to ship to Samoa (which, I guess, makes that "the colon-end of nowhere".)

This screwup was caused by some interaction between the automatic address fill-in feature of Google Chrome and the PCBCart web forms, and by me not checking the PCBCart status emails closely. PCBCart got my billing address mostly correct, but the shipping address was completely whacked and I never noticed.

So my CAT Board PCBs currently sit in New Caledonia, enjoying the onset of Fall in the Southern Hemisphere. DHL will tranship them from there to me for $163. What I'm hoping is that the shipment will be undeliverable and DHL will return it to PCBCart who can then send it to me for less than that.

It's been a while. I got the SDRAM on the CAT Board working a few weeks back, so I'll use this post to document that.

I've been using SDRAMs with FPGAs since 2001. Back then, I created an SDRAM controller module in VHDL that manages all the SDRAM timing, bank and row activation, address multiplexing and refreshing while presenting an interface that looks like a static RAM to the rest of the FPGA.

To test an SDRAM, I attach the controller to a finite state machine (FSM) that sequences through a two-phase process:

Write the entire SDRAM with a pseudo-random sequence from a random number generator (RNG).

Reinitialize the RNG with the same seed and then compare its output to the values read from the SDRAM. If the outputs from the RNG and SDRAM match over the entire address range, then the test is passed.

Since I've switched to using MyHDL for the CAT Board, I needed to convert the SDRAM controller. That's a big job. Luckily, I mentored a student for the 2015 Google Summer of Code (GSOC) and he translated the controller to MyHDL. He also provided a simulation model of the SDRAM chip.

Once the simulation was working, I tried to map it to the FPGA on the CAT Board. Unfortunately, the SDRAM controller describes its I/O using interfaces instead of lists of signals. Interfaces are a (relatively) new MyHDL feature that simplifies the interconnection of modules and reduce errors (that's good!), but using them with pin assignments is undocumented (that's bad!). So I had no way to assign the individual I/O signals to the FPGA pins connected to the SDRAM on the CAT Board.

I finally resorted to exporting a Verilog version of my design using the MyHDL conversion routines. Using the port definitions in the converted Verilog code, I made the appropriate pin assignments for the CAT Board within Lattice's iCEcube2 software. Then I compiled the Verilog using the iCE40HX8K FPGA as the target device. I took the bitstream generated by iCEcube2 and transferred it to my Raspberry Pi.

From there, it was just a matter of having the RPi download the bitstream to the FPGA on the CAT Board and running the SDRAM test. Here's what happened:

So the SDRAM works! That was actually a little easier than any of my previous tests.

No, I really don't have any plans for selling this. Lattice sells $22 iCE40 boards and Olimex has one for $25. Even though these use a small HX1K FPGA, they kind of set the price point of what people are expecting. Earlier this year, I calculated the part cost of the CAT Board with an HX8K at $17. Even if I could sell it at $50, I'd make almost nothing and still not many people would buy it.

Hi Dave, very cool project. I have actually been working on similar project off and on for a while with a Xilinx FPGA. Glad to see that I am not the only one who had the idea of pairing the pi with FPGA. I haven't posted anything yet but I might in the future.

Just a quick note based on an earlier comment where you mention the SPI as being slow. I'm using this library (http://www.airspayce.com/mikem/bcm2835/index.html) to control the SPI interface. My very rough calculations put the transfer rate at about 3 Mbit/s when I'm loading my bit file, and it can probably be pushed a bit faster. I can share some code if you're interested.

Hi, Mike. Thanks for the reply. I actually sell the StickIt!-MB (http://www.xess.com/shop/product/stickit-mb-4_0/) that mates one of my XuLA Xilinx FPGA boards to an RPi. The only problem with that is I can't run the Xilinx programming tools on the RPi. Have you been able to do that?

In regards to the slow SPI, that's a problem only when I program the flash by bit-banging to arbitrary GPIO pins using Python. When I use the actual hardware SPI of the BCM2835 to program the FPGA directly, it's incredibly fast. I looked at the library you mentioned and it appears to only provide an interface to the hardware SPI port. Is the library also able to bit-bang to any set of GPIO pins?