A team from Cornell University, including researcher Huaishu Peng, has developed a hybrid 3D printing system guided by augmented reality (AR). The innovative system, called the Robotic Modeling Assistant or RoMA for short, enables users to design 3D models in situ using AR technology and have the model 3D printed simultaneously.

In a video demonstrating the novel RoMA 3D printing system, a user equipped with an augmented reality headset and a handheld design tool can be seen virtually designing an object on a circular platform. As the user shapes the 3D model digitally, a robotic arm swoops in to begin 3D printing the shape’s structure on the same platform.

And though the resulting 3D printed object is quite rudimentary—the robotic arm only builds the skeleton of the object out of 3D pen-like extruded plastic—the process is fascinating to behold.

RoMA's two handheld controllers: indicator and cutter

The idea for the technology is that it can be used to create “partially printed physical models” which can “serve as a tangible reference for the designer and she adds new elements to her design.”

The RoMA system is made up of a number of hardware components, including an augmented reality headset, handheld controllers (an indicator and a cutter), a rotating platform, and a 6DOF robotic 3D printer.

The 3D printer setup is further enabled by a custom designed AR CAD editor, as well as an “end-to-end pipeline that integrates AR and robot control,” and a proxemics-inspired mechanism that makes it possible for the human user and robot to interact.

More specifically, when the user is designing his or her model as the robotic arm extrudes plastic, if the user touches the rotating build platform, the robotic arm will automatically move away from the 3D print. Similarly, when the user leaves the vicinity, the robot resumes control of the build platform and rotates it at will.

3D printed teapot structure build using RoMA

“With RoMA, users can integrate real-world constraints into a design rapidly, allowing them to create well-proportioned tangible artifacts,” say the researchers of the innovative technology. “Users can even directly design on and around an existing object, and extend the artifact by in-situ fabrication.”

In the video below you can watch a designer create a teapot model on the spot, using her hand as a reference for the teapot’s size and her finger as a sizing guide for the pot’s handle. It is pretty amazing to see the designer and the robotic 3D printer working in perfect tandem.

Notably, RoMA can also be used to 3D print new structures onto existing objects or around them. The researchers demonstrate these rudimentary abilities by 3D printing a sort of scaffolding around a LEGO structure (and even placing a LEGO man on a second 3D printed level) and by 3D printing a protruding shape from the back of a toy lion’s head.

Overall, it’s a pretty neat system that we can definitely see being used for proof of concept or early stage design models. If the 3D printing can be further refined, the system’s potential would increase significantly.