This Is Your Brain On Code, According To Functional MRI Imaging

This story contains interviews with Christian Kästner, assistant professor at Carnegie Mellon University; Amy Hirotaka, state policy and advocacy manager at Code.org; and Janet Siegmund, researcher at the University of Passau.

In order to expose more students to computer science, 17 states have passed legislation to create basic math and science requirements in the curriculum, rather than count them as electives. But lawmakers in Texas have made computer programming count toward a foreign language requirement, with Kentucky and New Mexico gearing up to follow suit.

Here's how the debate is shaping up. Those in the math and science camp argue that computer science encompasses more than just the "language" the computer can interpret—specifically, it requires learning algorithm logic that necessitates specially trained technical instructors.

On the other hand, the "code-is-a-foreign-language" proponents are fiercely stating their case, seeing an opportunity to fill in curriculum gaps left behind by shrinking foreign language departments in schools.

While lawmakers and educators alike fight over whether computer programming is considered a foreign language or a math, there has been virtually no scientific evidence to support or refute either case. That is, until now.

Researchers at the University of Passau, University of Magdeburg, Carnegie Mellon, Georgia Tech, the Leibniz Institute for Neurobiology, and the Metop Research Institute put their heads together to figure out how exactly computer programmers understand code. Their study was accepted to this year's International Conference on Software Engineering.

This is the first study in which scientists have looked into the brains of programmers, using the imaging tools of neuroscience. You can find the study here in PDF form.

Primary author of the study, computer scientist Janet Siegmund at the University of Passau, was so interested in the "language or math" debate that she decided to peer into the brains of computer programmers using an MRI, in hopes of shedding some light on the issue.

"That was actually my starting point," Siegmund says. She wanted to know how the brain registered the act of programming, which led her to her current research question, "How do coders understand code?"

In the experiment, 17 participants interpreted a few lines of code while lying down inside an fMRI machine. All of the test subjects were university students that had an undergraduate understanding of programming and the Java language. Two of the 17 participants happened to be female.

Each person read several different Java code snippets of similar difficulty so that the researchers could average the resulting brain imaging data. To rule out brain activity that was not related to interpreting code, the researchers subtracted imaging data related to an error detection task. They found that error detection did not activate the parts of the brain that dealt with the code interpretation task.

All of the test code consisted of several lines, at the end of which the program would print an output. The participants’ task was to predict the printed output after studying the code. All of the participants were capable of understanding the code within the given time frame.

"It appears to make some sense, based on what we have learned from the study," Siegmund says.

But Siegmund stresses that more research would better solidify her conclusions, especially since this was an initial attempt to answer the math-or-language question. She says, "Actually, with these kinds of studies, you should always say that more studies need to be done. But what we found is that it appears to be related."

Future studies might improve on the current experiment. "We had a very artificial kind of testing because you really had just small source-code snippets with, like, 20 lines of code," Siegmund says. Anything larger would have fallen out of the viewing area of the small mirror that was tacked onto the inside of the fMRI machine.

What’s more is that the code could not be so difficult that the subject could not finish the task within the allotted time. Real-world programs usually contain numerous lines and would likely stump the external reviewer.

Christian Kästner, the second author of the study, elaborates on why more work needs to be done. He says, "There is no clear evidence that learning a programming language is like learning a foreign language, but our results show that there are clearly similarities in brain activations that show that the hypothesis is plausible."

It is important to note that the participants did not actively write computer programs in the experiment. They merely read code chunks for understanding. So, it is inaccurate to definitively say that computer programming is more a language than a math. The truth is still out there.

Earlier attempts to understand programmers’ cognitive abilities used qualitative measures and mostly relied on self-reports from the programmers the experiments were done on. But Siegmund decided to use a hard-data method from neuroscience to directly measure a programmer’s understanding of the code using fMRI imaging.

The colored areas were the active brain regions when the participants read and understood the code.

Embracing the tool carried a learning curve. "You need a lot of experience with fMRI studies. You need to know how the machine works and what you can do with the data," Siegmund says.

A chance meeting with biologists at a conference brought in the know-how that Siegmund and her colleagues needed in order to properly plan out and interpret the data from their fMRI study. "I don’t think we would have been there if we didn’t have the new biologists on board," Siegmund says.

The study might give insight into how reading code could impact a student’s brain. Even so, it is difficult to know for certain if other parts of the brain were activated.

Last year, neuroscientists did a study on patients whose brains were implanted with electrodes. Using this invasive method, they were able to pinpoint a region of the brain that processes numerals, called the inferior temporal gyrus. They further concluded that this region is physically near the area responsible for language processing.

It is possible that the fMRI in Siegmund’s study may not have been able to detect activity in this region. Maia Szalavitz, writing for Time, wrote, "Since the inferior temporal gyrus is so close to the ear canals, functional MRI machines, which detect changes in oxygen use and blood flow by nerve cells, may not be as sensitive to the activity of neurons tucked away in that area."

Even more evidence confounds the distinction between language and calculation processing areas of the brain. A year 2000 study concluded that the same region that is responsible for processing semantics is also important when performing mental calculations. Incidentally, Siegmund and her colleagues found activity in this same region.

No matter which way you look at it, most brain imaging studies remind readers that activated areas that show up with the chosen imaging method correspond to several different processes that are open to interpretation.

Siegmund and her colleagues also concede that the way they interpreted their study’s results is open for review, writing in the paper that they might have missed important processes.

But Siegmund reiterates, "The activation pattern that we found shows very clear and really distinct areas are activated that are related to our current understanding of program comprehension."

Even if Siegmund’s and Kästner’s study has showed a relationship between reading a computer program and the brain’s language centers, it does not speak to the field of computer science as a whole.

Amy Hirotaka, state policy and advocacy manager at the non-profit Code.org, wrote in an email to us, "Computer science is more than just code. The fundamental concepts of computer science—like logic and problem solving—align well with mathematics and science disciplines, extending beyond simply learning a programming language."

Considering now that computer science could come out of a school’s foreign language department, Hirotaka sees issues with teacher quality down the line. Each department has its own ways of certifying a teacher’s credentials. It suffices to say that getting foreign language and mathematics departments to converge on one credentialing standard would create an administrative mess, even if it would give students more foreign language options.

Hirotaka added, "Counting computer science as a foreign language might sound like a creative fix, but it causes major problems when it comes to teacher certification and departmental alignment."

School policies aside, the study could pave the way for improvements in the programming field. Kästner is quick to point out that using neuroscience can help us determine what makes a great programmer different from an average one.

"We still have no clear idea how to train really good developers. For decades researchers have found that there are individual programmers who program more productively at higher quality and also do most of the communication in the project. These are often referred to as 10Xers because many studies found that they produce ten times more code, or in a tenth of the time or do ten times more communication," Kästner says.

He adds, "We know that these developers exist; we often quickly recognize them. Most of us know at least one such developer, and every tech company tries to hire them. But we have no idea how they got there or whether we could train others to excel at similar levels. Studying how program comprehension differs among novices, professionals, and 10Xers may allow us to get a better understanding of what makes a truly excellent programmer."

Siegmund imagines that the research could result in better software syntax. "Now that we actually have a better idea of what is happening inside a programmer’s brain, we found that it was related to natural language processing. Maybe programming languages should be more like natural languages," she says.

"The more domain-specific languages, like SQL, are more close to natural language processing. It would come closer than Java," says Siegmund.

All things considered, the researchers’ biggest takeaway from studying programmers’ brain activity is the possibility for even more discoveries in computer science and software engineering.

"This opens the door for many future studies in this field. The actual activations found in this study are nice and confirm mostly what we expected, but the key result is a proof of concept of using fMRI as a tool in software engineering research," says Kästner.

If you want to get your language processing centers working away, try perusing the Java snippets from the study here.

Add New Comment

13Comments

Anyone who's done a google interview knows that we have to "talk through" our code, to explain our thought processes when writing or parsing code. I wonder if the effect they found wasn't just self-talk (e.g. "Ok so with x being 4 that's 432*1 so that's factorial" ) in our native natural language, rather than reading the code like a second language.

All the "tools" e.g. flow charts and psuedocode, mentioned in the previous comments are languages. Different symbol sets to convey information about special domains of knowledge. One often translates between them. The thought process that goes in to their learning anduse makes use of one's natural language; we think in words for the most part, although some do think in images mostly. If one does not know flow charts, then learning about them and to use them is learning a new language (foreign or not as one chooses to define "foreign").

I've been a programmer for 50 years. A software engineer for 45 of that . And, a professor of IT and Comp Science for some 20+ years. I have edited many technical papers. I have also had to each elementary writing skills to college freshmen. I have a pretty good handle on the language acquisition skills needed to competently define and understand a problem and those needed to describe it properly to other stakeholders and finally notate and program it.

Usually the "language" used is rather limited - you learn it you got it - no real nuances of a foreign language. Also, you won't get far without some math concepts in reading. I can read code from many computer languages - more than I can write proficiently in. How you USE the language is what matters. Your mind is jumping all over the place with this or this or this or maybe even something new. I'd like to see how my brain relates when I write code to when I plays a musical instrument or dance. To me it feels the same.

Learning a specific programming language is a "foreign language" experience. Learning to program and related undertakings are math, logic, and science tasks. I would the expect the language processing portions of the brain to be active reading existing code -- one is "doing" language processing. Understanding a problem, designing a solution in e.g. English, creating the actual solution in a specific programming task would show both math, problem processing areas and language areas "glowing".

I am surprised the study was limited to the activity of reading and understanding existing code, versus designing new algorithms to solve problems. I suspect the results might reveal that science/math brain regions are the predominately active areas when using problem solving skills to implement new source code versus reading existing code. The study mentions that as a future direction. I think it is very relevant to answering the question. The conclusions of the study, though insightful, are incomplete for use in making assumptions on what parts of the brain are used by programmers. Granted, we do spend time reading existing code to repair it or to use it to learn by example. Even the repair process involves reasoning skills, not just reading comprehension.

What a ridiculous research! This would be like researching how good an artist is by what tools he uses. It looks to me that they want to pigeon-hole programming into the old paradigm of learning and regurgitating as required by a corporate world and the education system geared feeding this system.
Programming is ultimately a create process, programmers and developer constantly create or improve something, they are creative, using programming languages as TOOL, like a fine-art painter uses a brush. To define if programming is more a language or mathematical ability is like determining if art is all about paint techniques or color coordination.

You are still forced to learn the syntax and create logical functions (sentences) to speak with an interpreter. You still have to spell check and know math. I use around 5 languages for every project I create. You can't create something without knowing proper syntax and data structuring. You can't speak a language without knowing how to speak and read the language's syntax.

How about 'over time' -- has the practice of programming changed over the past, say, twenty years? Probably so. Have the steps and tasks involved in reaching some final product changed? Maybe far more streamlined now. Maybe it's gotten a whole lot easier than it used to be. And maybe programming is going to get immensely easier in the coming iterations of the practice. To the point of being almost totally unlike what programmers used to do.

The same is not true of language. Language rules barely change over time. We can understand perfectly well what writers wrote hundreds of years ago. What we write is quite similar to ancient authors.