Jeremy Blum, EE and Director of Hardware at Shaper, shares his experience in computer vision, the difference between engineers and designers, why helping engineer Google Glass was an "electro-mechanical nightmare", and more.

Jeremy Blum is the Director of Hardware at Shaper, a startup that is fusing robotics technologies like computer vision with handheld power tools.

From a brief glance at his CV, it’s very clear that Blum is a very capable and inventive engineer with experiences ranging from sustainable building design, robotics, to entrepreneurship.

However, the thing that stands out most about Blum is his propensity towards collaboration, which he uses to his full advantage in his career to make things better from every angle—how things are designed, how they are made and manufactured, and how they are used.

"The levels of complexity are a million times higher now and it’s not possible to design products or things in a built environment where an engineer does something, and a designer does something, and then they smash them together at the end. Things are so tightly integrated that it’s important for those two groups of people to understand each other and those are ultimately the two groups of people that make products happen."

AAC’s Chantelle Dubois had a chance to speak with Blum about his career and what makes communication and collaboration such an important element of product design.

Chantelle Dubois (AAC): Your current title is Director of Hardware at Shaper. What does your typical day look like in that role?

Jeremy Blum (JB): Sometimes I wish I could give out what a typical day looks like here, but the reality is that there is no typical day at a startup. Every day is different.

If I were to trying to give you a sense of just what are the various list of things I might be working on on any given day, it varies a lot. Some days I could spend the whole day doing R&D on a new product and putting together quick prototypes. Another day I might be spending all day in Altium working on board design. Another day I’ll be in the lab testing a new design, or reviewing a defective unit that came back to us, trying to find the root cause of failure and understand what happened and how to prevent it from happening again.

I spend a large amount of time either working remotely with our factory or physically at our factory. At Shaper, we’ve designed all of our factory tests and assembly systems in-house, so on top of making the actual product, we’ve basically designed a factory around building that product and testing it effectively, which is an enormous undertaking.

I’ll also spend time involved in investment conversations or working with investors. I’ll spend time copyediting our website, designing the manuals that ship with our products—I’m all over the place and actually, everyone at Shaper is kind of all over the place. We make electro-mechanically complex products and a very interdisciplinary product, so a lot of my job is about effective communications either with coworkers, vendors, suppliers, or our customers.

AAC: Shaper's flagship product is the Shaper Origin, a handheld CNC machine. What can you tell us about its applications?

JB: There's certainly a lot of applications. You can definitely go to Shaper’s YouTube page or go to ShaperHub where we upload designs and where users can also share their designs.

A community member using the Shaper Origin. Screengrab from Shaper Tools

We have seen people making brass jewelry with it. We have people making dining room and conference room tables, building boats, building furniture, kitchen and bathroom fit outs and designs, flooring inlays. You can mill PCBs with it, you can do a weekend project with your kids, you can build a soapbox derby car—the sky’s the limit. Anything that you can cut with a spinning blade, you can make it with Shaper Origin. It’s a wide range and there’s a reason I said "anywhere from hobbyist to professional woodworker".

AAC: The Shaper Origin notably uses computer vision to guide design. What can you tell us about developing computer vision in conjunction with hardware?

JB:I did a bunch of computer vision work when I was in college and it’s a big reason why I’m at Shaper. I think the technology is extremely promising and, in my opinion, it’s the holy grail of robotics.

Teaching a robot or another electro-mechanical system to understand something at a deep level about the images it’s seeing has an enormous amount of potential as a sensing technology. We have all these other kinds of sensors which are great for specific things. When we talk about self-driving cars, we talk about putting all these sensors on there like LiDAR, acoustic sensors, ultrasonic sensors, cameras, GPS, inertial measurement units, and all these other things.

Then, think about how humans drive. When you are behind the wheel of a car, what is the primary thing that you’re using to drive the car? Vision. You have binocular vision. You do not have an ultrasound sensor, although you do have your inner ear, which provides some level of inertial measurement. You have your ears and you can hear things. However, you can cut off all your other senses and you can rely on only your eyes to drive a car pretty well.

That should tell us something about [what would happen] if a computer is able to process visual data in the same way that a human brain can. Visual information is sufficiently information dense that I think you can do almost anything with it with an extremely high level of accuracy and with extremely low latency. It’s an exciting technology for that reason, and it’s something that has an enormous amount of potential.

The interface on the Shaper Origin utilizes computer vision to guide the CNC device. Screengrab from Shaper Tools

The really great thing about it, too, is that it’s predominantly limited by the capabilities of our software. Hardware for cameras is already pretty damn good. It’s continuing to get better, but we can make computer vision-based systems better based purely by improving our algorithms and software. That means we can put something out into the world and it will continue to improve, and that’s very true of Shaper Origin. We are putting a fixed piece of hardware out into the world but we are constantly releasing software updates that make it perform better, and I think that’s awesome.

AAC: You worked with Google X on Google Glass (and probably can’t tell us too much about the details). What would you say you learned from that experience?

JB: What I became really good at while at Google was not only how to interact with [people from different disciplines] but how to utilize my relationships with those people to build tightly integrated products.

I mostly worked with Google Glass and that product is, from a design perspective, an electro-mechanical nightmare. It’s teeny tiny; it needs to weigh almost nothing; it needs to have battery power in it; it needs to drive an extremely complicated visual optics system; it needs to basically be water and sweat proof; the electronics need to be tiny and reliable; and we need to fit all this in a svelte package that has a desirable industrial design that people would want to put on their heads.

"[Google Glass] is, from a design perspective, an electro-mechanical nightmare."

That was very challenging. From my perspective as the electrical engineer, of course, I want to put the biggest battery possible on it but the industrial designers basically want the smallest battery possible because they want this thing to be tiny.

The mechanical engineers want as much heat sinking as possible, but I want to use a CPU that is as powerful as possible. That’s going to generate a bunch of heat that’s going to require it to heat sink which is going to require things that are ugly from the industrial design perspective.

So it’s a lot of fighting requirements. Something I became good at is working with everyone and being able to pull back to a higher-level view of the product and understand what is the ultimate goal. What are we trying to address for our customers, the person who’s to actually buy this? Something I might desire from an electrical perspective is not necessarily the best decision from a product perspective.

"Something I might desire from an electrical perspective is not necessarily the best decision from a product perspective."

Working with an interdisciplinary team is about compromises. The biggest thing I got from Google is how to compromise effectively with my coworkers and think about things from a product level, which is something I employ heavily at Shaper now.

AAC: You've had to develop many skills to design, for example, rigid-flex PCBs that can be difficult to manufacture. How do you develop ambitious products while overcoming manufacturing challenges?

JB:A lot of it is about talking to vendors and understanding the capabilities. I will fully admit that sometimes I, myself, am hesitant to ask manufacturers for design advice and understanding the industry best practices for doing things. You do have to be careful about it because manufacturers will necessarily want to be conservative, so you take it with a grain of salt, but I learned a lot from talking with manufacturers and vendors about was possible.

Then, at Google, when it came to our flex or rigid-flex or rigid designs, we would take what vendors said they were capable of and then design something that shouldn’t have been manufacturable from what they told us. Then I'd say let’s figure out a way to do it together. Sometimes we could, sometimes we couldn’t, but it’s a lot of trial and error.

"...at Google, we would take what vendors said they were capable of and then design something that shouldn't have been manufacturable from what they told us. Then I'd say let's figure out a way to do it together."

I worked with a lot of really great electrical engineers at Google and manufacturing engineers who had experience with these things and taught me these things. But the rest of it has been trying to assume anything is possible. I design it and then take it to a manufacturer and ask what about the design I’ve made is going to be impossible to manufacture—and then I try to make them meet me half way.

I have found that the best way to do that is to physically go in person to meet with people who are manufacturing things for you and that human connection makes a huge difference. You can empathize and it makes it easier to find some middle ground where you can get some of what you want and they get some of what they want. That’s how we push the industry forward.

AAC: You've made many of your projects over the years open source. Why?

JB:I like my day job, I like what I do from day to day, but sometimes I’m working on other projects and things that interest me for fun or to help teach myself or help expand my knowledge. For those things, frankly, I don’t want to deal with the overhead of managing those projects or selling them. Since I’m going to do them, anyway I’d rather that some other people benefit from some of the side projects that I do and learn from them.

The RainCloud Umbrella Minder is an example open source project Blum developed in 2014. You can find the full explanation of this device's first version here.

We stand on the shoulders from giants. Most of what I’ve learned is from reading what people are posting on the Internet. The majority of my capability in terms of writing software comes from me reading other software people have posted online. If I’m going to benefit from that then I feel it’s my responsibility to share back in the same way.

"We stand on the shoulders of giants... If I'm going to benefit from [others online] then I feel it's my responsibility to share back in the same way."

I know there are a lot of people from around the world who really want to learn about electronics and want to learn about hardware, embedded software. And their ability to access educational content for doing that, especially outside of the United States, can be quite limited. I make no illusions about the fact that I’ve been very fortunate in my upbringing and what I’ve been exposed to and living in America gives me a lot of advantages that other people don’t have so I feel it’s my responsibility to share some of those things with other people.

AAC: Speaking of access, platforms like Arduino are important for bringing hardware to students all over the world. Given your technical background, it may surprise people to learn that you wrote a book about Arduino. Who is the ideal audience for this book?

JB: I would definitely say anyone who wants to build their skills in embedded design in engineering, I would encourage them to check out my book, Exploring Arduino. It’s kind of a nice middle ground for if you have experience doing hardware design but don’t have experience programming microcontrollers—it’s a good intro. Conversely, if you have software experience but want to learn a bit about the hardware your software runs on, it’s also a good intro book for that.

AAC: In addition to everything else, you've also spent much of your academic and professional career working on sustainable projects. As a student, you co-founded the Cornell University Sustainable Design (CUSD) program. What can you tell us about why you were inspired to create this institution?

JB:Most engineering schools have project teams where you can do stuff like build a car, build an autonomous submarine or airplanes, and they compete in competitions. I wanted to do something like that.

There was already a group working on building a solar-powered house for a competition, a Department of Energy competition called Solar Decathlon that would happen every two years. I thought that was really cool because it was like the other project teams in that you could kind of flex some of your engineering knowledge and apply it to actual problems. But what I particularly liked about what this team was that it seemed aimed at solving global issues, namely renewable energy and sustainability, and how we address those things. The other reason I was drawn to it was because it was the only project team that wasn’t just engineering. There were architects involved, business people, landscapers, and basically every type of person was involved.

Blum worked on Cornell's 2009 Solar Decathlon entry, the Silo House. Left: A rendering of the house's design. Image courtesy of CUSD. Right: The project. Image courtesy of Jeremy Blum.

I did that for a year. [After we competed in Solar Decathlon, I took over the team] and we decided together we can actually do more than just a competition. We can leverage what this group is good at and do different sustainable design projects, both locally and around the world. We left the competition and reformed the group into what became Cornell Sustainable Design.

We kicked off doing a number of projects. Cornell won a bid to build a new tech campus in New York City, which just opened last year, and our group was the only student group that was contributing to the design of the new buildings. This whole campus is going to be net zero energy consumption, and we worked on the designs for that.

We’ve built a self-sustaining schoolhouse in South Africa. We helped build and develop a sustainable community in Nicaragua. We’d go out to members of the community both locally and around the country, visit schools, teach younger students about sustainability and the importance of sustainable design and how that plays into engineering and architecture.

I count this as a very formative experience for me because it was my first time working on large projects with people who weren’t engineers, and I learned a lot from the architects in how to think about problems from a design perspective instead of just a logical, technical perspective—which is often how engineers look at problems. I think it’s been very beneficial for me, as an engineer, to be able to step back and see things from the perspective of a designer, and that’s made me a better engineer.

AAC: What do you see as the role of engineers in the development of sustainable devices, systems, communities, etc.?

JB: I think, especially in the last few decades and more so even in the last few years, anything that we would consider "new technology" has to be this very tight integration of design and engineering.

One hundred years ago, let’s say you were going to design a typewriter. Think about the amount of technical complexity and interaction between people who are responsible for what that thing looks like and how it functions versus a team responsible for designing something like an iPhone.

The levels of complexity are a million times higher now and it’s not possible to design products or things in a built environment where an engineer does something, and a designer does something, and then they smash them together at the end. Things are so tightly integrated that it’s important for those two groups of people to understand each other and those are ultimately the two groups of people that make products happen.

AAC: You clearly differentiate between engineers and designers. What have you, a self-identified engineer, learned by working with designers and vice versa?

JB:Something I’ve learned from working with designers is how to consider the big picture a little more, how to understand how an end user is going to be interacting with something I’m designing.

Something I hope the designers I’ve worked with have come to understand working with an engineer who is good at communicating with them is how to prototype quickly, how to ideate on ideas, and how to narrow down the design approach for something without eliminating ideas too early.

I would see a tendency to want to eliminate ideas on the chopping block early for not technically possible but sometimes you just have to try it and maybe fail along the way. So that’s an approach I bring to the table that I hope some designers have picked up on: my approach to designing and testing things.

I make something, quickly prototype it first, see how it works, try to understand what doesn’t work about it, and then iterate on it from there. As opposed to trying to rule out as many things as possible from the get-go. If you can keep an open mind at the beginning and iterate as you go, you end up with a better outcome.

AAC: Is this how you imagined your career as an engineer would be like while you were studying in college?

JB:Yeah, in a way. This is similar to how I operated when I was at school. If you think about the life of an engineering college student, I was taking four or five classes at a time, running Cornell Sustainable Design, doing research in a robotics lab, and was involved in like six or seven other things that are extracurricular in some capacity.

Anyone who tells you that they are good at multitasking is full of it because nobody is good at multitasking. I fully admit I am not good at multitasking, but I am pretty good at context switching. So, focusing on one thing, going back to another thing and that was my entire college experience.

I always expected I would be doing a little bit of everything, and I like doing a little bit of everything because it keeps things interesting and exciting.