Math for Makers

23 March 2012

Artists build on top of science.

Renaissance painters used Cartesian geometry to invent perspective. Nineteenth century photographers' adapted industrial chemistry into the photographic process. Contemporary filmmakers shoot on cutting edge cameras made possible by the latest in sensor miniaturization. Each generation of artists turns the knowledge of their time into new creative tools.

No generation of artists has ever been more dependent on scientific and technical advances than today’s. Today’s artists work on computers. Advances in computer science and related mathematical fields underlie everything that digital artists make. Recently these advances have lead to the advent of whole new creative fields like interactive art, generative graphics, data visualization, and digital fabrication.

In order to produce excellent and novel work in these new fields, artists have had to learn computational and mathematical techniques. They started with basic material like trigonometry for 2D games and graphics, the rudiments of computer vision for interactive installations, and primitive signal processing for embedded electronics.

Increasingly these new creative fields are becoming the basis of art and design across our culture. And these techniques are becoming the foundation of a new kind of art and design education. A cohort of today’s best art and design schools teach introductory programming and a survey of these basic topics as their core curriculum (NYU ITP, CMU’s Studio for Creative Inquiry, Parson’s, CIID, UCLA Design Media Arts, etc.).

However, as these new creative fields advance, driven by their own internal momentum and inspired by the advent of new developments in computer science and technology, they are beginning to require more advanced math and programming techniques. Today’s SIGGRAPH papers and academic journal articles are tomorrow’s breakthrough creative coding projects.

Topics like linear algebra, topology, graph theory, and machine learning are becoming vital prerequisites both to doing daily work in these fields and, more importantly, to inventing, popularizing, and teaching the new creative tools that are rapidly arising. Without them, artists are forced to wait for others to digest this new knowledge before they can work with it. Their creative options shrink to those parts of this research selected by Adobe for inclusion in prepackaged tools. Instead of the themes and concerns of creative work driving the selection of tools from a growing technical cornucopia, artists find themselves turned into passive users of tools that are already curated, contextualized, and circumscribed by others.

So, I want to do something about this. I want to figure out a way to teach myself and others these more advanced mathematical and computational concepts with a specific eye towards applying them in creative technology. For example, I want to be able to read SIGGRAPH papers, understand what’s going on in them, explain that to others, and create software libraries that implement their techniques. I want my peers who do data visualization to be able to implement regressions, curve fitting, statistical analysis and machine learning so that their projects are more than just pretty graphs with good typography. I want the hardware hackers who are building the next generation of DIY 3D printers to be able to turn topological algorithms and concepts into open source tool path generation software that creates more efficient gcode and enables the fabrication of previously impossible physical forms.

I don’t know the best way to go about this, but this site is intended to act as home for my experiments. Here, I’ll collect research papers with potential as creative tools. I’ll catalog the mathematical and computer science terminology and techniques used in these papers. I’ll work to explain this terminology in plain English and to translate these techniques into comprehensible, publicly available code. I’ll interview other digital artists and researchers who understand parts of this material in order to capture and spread their knowledge. I’ll try any approach that has the potential to help us gain some grasp on these new areas and I’ll collaborate with anyone’s who’s interested in helping.

To inaugurate this site, I conducted an interview with Kyle McDonald about FaceTracker. Kyle is an artist and researcher who embodies much of the spirit of this site. He’s a key member of the OpenFrameworks community and he’s done extensive work democratizing 3D scanning. Last October he released the first public results of a library for fast, accurate, face-tracking he built for OpenFrameworks. That library was based on the work of Jason Saragih, a computer vision researcher at CSIRO. Since the release of Kyle’s library, there’s been an explosion of projects in the creative coding community using his code to explore the possibilities of face-based interaction. I asked Kyle to explain how he first found Saragih’s work and to walk through how the algorithm works. I think the results represent a terrific introduction to many key areas of computer vision and to understanding FaceTracker in particular. In addition to to the video, I’ve provided a transcript of the interview that I’ve annotated with links and additional media.

Much of the rest of this site is currently under construction, but I’ll be announcing additional sections as well as further interviews and other material soon. You can follow along and get in touch with me about the project through the makematics twitter account. If you’d like to get involved, this site is on Github: atduskgreg/makematics. Contributions of all forms are welcome, from writing to code to graphic design or other media.

In conclusion, a note about the intended voice of this site. All too frequently the technical and mathematical issues discussed here are written about solely by experts for other experts. And even when it does appear in venues intended for more general consumption, the material is usually presented as if its own internal logic and rigor made it naturally comprehensible. I don’t believe this is true.

I think my greatest advantage in this effort is that I am a beginner myself. I am not an expert in computer vision, computer science, or mathematics. I’m a programmer and an artist who’s committed to struggling with this material until I understand it and can make it comprehensible and useful for myself and others. I won’t hide the frustrations and confusions that are inevitable in the process. I hope to show that a normal programmer with no special academic training can grapple with these areas of research and find a way in to understanding them and making them part of my creative work. And as I go I aim to create material that will help others do the same.