It’s written purely in synthesizable Verilog, and uses device-agnostic inference for all FPGA primitives (though the current implementation is more optimized for Xilinx devices). I’m releasing it under a standard 3-clause BSD license.

The design is heavily pipelined. Under realistic conditions (in a highly-utilized, slowest-speed-grade part without any floor-planning), it can run around 150 MHz in Spartan-3E and Spartan-6 parts, and around 300 MHz in Virtex-6 parts. Much higher speeds (50+%) are possible under unrealistic (ideal) conditions.

The core has been verified in simulation using Verilator with SystemC testbenches. Post-synthesis results (from Xilinx’s XST tool) have been verified using a simplified Verilog testbench and Xilinx’s own ISim simulator.

Scope

Now, before everyone runs off and tries to build their own open-source Kinect, I must stress that this isn’t a complete solution just yet; here’s a block diagram of what I have implemented:

Stereo Vision Core block diagram

If we now refer back to the high-level block diagram that I presented before:

High-level system logic diagram

..we can see that this core implements all of the “Stereo Correspondence” block, some/all of the “Post-Processing” block and (had I actually included it on the original diagram) the “Pre-Filtering” block. While “Image Rectification” is the only significant missing image pipeline component, there’s still a lot of other system level infrastructure to develop (external interfaces, buses, etc.) before I can call the project “complete.”

That being said, the correspondence core easily represents the most critical, most resource-intensive and highest-performance component of the entire system. Completing it is a major milestone in the project.

As alluded to in a few of my other posts, I’m working on developing an open-source FPGA-accelerated vision platform. This post is a detailed overview of the project’s architecture and general development methodology. Future (and past) posts will elaborate on specific pieces of the project, as they’re implemented.

Stereo-vision is the main objective for the project – but once the general framework is in place, an obvious next-step would be the offloading of additional vision algorithms onto an FPGA.

Eventually, when my FPGA stereo-vision project nears its terminus, I’m going to want to produce a refined sensor board that combines the image sensors and FPGA onto a single board. In preparation for that, this board is a test vehicle to investigate what it takes to design and assemble a compact PCB with multiple BGA packages using only tools and services that are within the reach of a well-equipped hobbyist.

Looking at the top two items on that list – an FPGA in a 256-ball 1.0mm BGA package, and a memory device in a 60-ball 0.8mm BGA package – one can easily imagine that assembly is going to be the trickiest part of this project.. but this post isn’t about the assembly of the board, seeing as I’ve only just sent it off to be fabricated (this time by Laen’s 4-layer PCB service). I’ll make a follow-up post once the board is back and assembled.

Instead, this post is entirely about the design and layout of the board.

Another piece of my ongoing FPGA stereo-vision project. This board is, as the name suggests, a breakout board for Aptina’s excellent MT9V032 1/3″ VGA image sensor.

MT9V032 camera board - assembled

The board’s main purpose in life is to connect the LVDS output of the MT9V032 sensor to my FMC-LPC to SATA adapter board, which would then route the LVDS data into one of Xilinx’s Spartan-6 FPGA development boards. Multiple camera boards would be connected to support stereo vision.

At first glance, it has one major failing: a complete lack of user-friendly I/O expansion. Xilinx has put all of their eggs in one high-density, surface-mount basket: the board has ~70 FPGA I/Os brought out to a single high-speed FMC connector.

FMC-LPC to SATA adapter board - bottom

Not the most friendly looking footprint, right? While it’s no 100-mil pin header, it’s remarkably easy to work with – even on a simple 2-layer PCB.

I’m a big fan of inference, especially as it applies to writing synthesizable Verilog code for FPGAs. Properly coded, a module that infers technology-dependent blocks (e.g. block RAMs) should: be portable between devices from a particular vendor (e.g. Spartan 3E to Virtex 6), be portable between devices from different vendors (e.g. Spartan 6 to Cyclone III), and even be portable to vendor-independent environments (e.g. simulation in Icarus Verilog).

The trick is that little “properly coded” clause. Figuring out exactly the right sort of Verilog required to get a particular tool to infer the block you want isn’t always straight-forward. Figuring out exactly the right sort of Verilog to get multiple tools to infer the block you want can be even trickier. Which is, perhaps, a little bit silly – considering that the whole point behind this little exercise is to be able to write code that isn’t tied to any particular tool, device or vendor!

Using current synthesis tools from Xilinx (ISE WebPack 12.2) and Altera (Quartus II Web Edition 10.0 SP1), it’s now practical to write synthesizable device and vendor-independent Verilog code (or VHDL, if that’s your thing) that properly infers true dual-port (TDP), dual-clock block RAMs in each vendor’s respective FPGAs. There are, of course, a variety of limitations and caveats that come along with that statement.

It has now been just over 6 years since I launched the last incarnation of my website. Finally, it has now been supplanted by this – a site that can safely be regarded as superior in virtually every way (if for no other reason than its tentative embrace of the buzzword-laden Web 2.0 – yes, it is now possible to comment on my content. How novel).

Many of my past projects are now well-documented here (of particular interest: Eddie, a Mars-rover inspired autonomous robot; and Elysium, a presentable solid-state Tesla Coil). Some others are only glorified photo-galleries at the moment; these will be expanded upon in due time.

Speaking of photos, I’ve picked up another hobby in the past year: photography. As a result, expect many more and better photos of my past (those that I still posses) and ongoing projects. These will make their way to my Flickr photostream before venturing here (and often, due to sheer volume, in lieu of ever appearing here).

Anyone that is familiar with my previous attempts at websites will be skeptical of my ability to make updates in a timely fashion (that is to say, more frequently than once per leap-year). I’m hoping that this blog format will encourage me to make more frequent postings about whatever it is that I’m currently working on (be it a project, some photography tidbit, or even just a hike I’ve been on). We shall see.