Teensy can school you at Guitar Hero

[Johnny Chung Lee] put together a system that is perfect at playing Guitar Hero. He’s using the PlayStation 2 version and, as you can see above he’s combined a controller connector and a Teensy microcontroller board to communicate with the console using its native SPI protocol. This custom guitar controller receives its signals via USB from a computer that is monitoring the video from the console and calculating the controller signals necessary for perfect gameplay. [Johnny] wrote an OpenCV program that monitors the video, removes the perspective from the virtual fretboard, and analyzes color and speed of the notes coming down the screen.

As you can see after the break it works like a charm. It’s fun from a programming standpoint, but if you want a hack you can actually play maybe you should build your own Banjo Hero.

It’s cool, but apparently has problems with long notes, and I assume lacks the ability to use star power (which I assume because it analyzes the color of the notes). Also, what happens when you get the notes that give you star power? Does it play them? That’s the problem with just using the color of the notes…

@Shane: I would assume you could remedy the problem with long notes by using some logic to automatically hold all notes by as long as possible. Also I don’t believe he’s using color to identify the notes so much as their position on the screen which means star power notes would work as well.

A neat enhancement would be one that records all the buttons in the song, then analyzes it to determine when it should use star power to maximize the score.

Are you kidding? It’s not cheating, because he isn’t playing. I really doubt he’s doing this for points, he’s done it as a challenge, and he succeeded. Do you think he thinks this makes him better at guitar hero?

@Shane I didn’t look at the site, but why would it have a problem with colour? It should just look at the position (since each note is always the same position, regardless of star power. and compare ti with the background. Even star power has has the white caps on top. I can see long notes being trouble though.

I wouldn’t necessarily call this cheating … I mean, quite a lot of work went into making this work. That’s like calling a car cheating at “horse and buggy” … it’s just a next step, an improvement utilizing a different form of skill.

Yeah, that’s the only way I’d win any of these games myself, build something to play it.

The article said it used color and position — position was determine how fast it was coming, not what note it is. Position has too many variables, like how far the camera is from the TV, what angle it is, etc. It would be hard to do it like that, but I guess you could.

I absolutely agree with you this is an incredible gadget. and the thought that went into I couldn’t even begin to grasp. The question I had was why? I would love to be able to gadget like this but I have a hard time grasping at the concept itself. I think it is very creative and shows a lot of skill.

I set up something similar at one point to do fusions in Persona 3. Use OpenCV to recognize the right screen area and pull a copy of the image, then feed it into OCR. The Teensy is very nice for rapid development.

From the website “For video capture, I’m just using a $30 USB capture device. Since real-time processing was priority for me, 640×480 images @ 30Hz is plenty of resolution to start with. In fact, for this starter project, I’m just using 320×240.”