An Automated Flappy Bird Player

Flappy Bird has been ported to just about every system imaginable, including but not limited to the Apple II, Commodores, pretty much every version of the Atari, and serves as a really great demonstration of the TI-99’s graphics capabilities. Porting is one thing, but having a computer automate Flappy Bird is another thing entirely. [Ankur], [Sai], and [Ackerly] in [Dr. Bruce Land]’s advanced microcontroller design class at Cornell have done just that. They’re playing Flappy Bird with a camera, FPGA, and a penny wired up to a GPIO pin to guide the little 8-bit-bird through Mario pipes.

The setup the team is using consists of a webcam that records the screen of a smartphone, an FPGA, and a little bit of circuitry to emulate screen taps. Inside the FPGA, the team is looking at the video stream from the phone to detect the bird, pipes, and gaps. The ‘tapper’ unit is a US penny, placed right above the ‘tap’ button, wired to a GPIO port. This was found to be the ideal contact for a capacitive touch screen – taps that were too small weren’t registered, and taps that were too big registered as two taps.

For spending an entire semester on automating Flappy Bird, the team has a lot of knowledge to show for it, but not the high score: the bird only makes it through the first pipe 10% of the time, and the second pipe 1% of the time. The high score is three. That’s alright – getting the algorithm right to play the game correctly was very, very difficult, and to nail that problem down, they estimate it would take at least another semester.

The algorithm is as simple as this: Calculate the y raster position from the character position, delay it with a ringbuffer and compare it with the bird y sprite position. If too low, flap. Should be easy to adapt this to the FPGA setup.

“the bird only makes it through the first pipe 10% of the time”
Um, doesn’t the space between each pair of pipes take up more than 10% of the screen? Wouldn’t that make their algorithm WORSE than random chance?

But we don’t leave people behind, even the slow learners deserve a chance at higher education, it’s too bad if the leeches, dug addicts and people who just don’t care get to tag along and ruin it for the nerd who is actually trying to learn something.

No, it’s better than random chance. You have to keep the bird at the right height by flapping. A random algorithm would flap randomly and would never make it through the first pipe, or very infrequently.

Right, maybe 10%, if it can keep the height, so just flapping with the right frequency. But Dan K said it is worse than random. This was wrong, because if their algorithm manages the first pipe every time, it is better than 10%.

Looks to me they had nice idea, done HW part very good and then failed in SW part miserably. If you have position of the bird and the pipes at any moment then you can calculate how to tap to get bird in safe passage every time.
Like this : http://www.youtube.com/watch?v=i1_X2KdUQe8

Thought I should let you know that it wasn’t sarcasm. I don’t live in the states but I did know it was an elite university, just genuinely didn’t know if it was for hard sciences or not – in the same way as MIT or CalTech for example…

As for the project itself, I haven’t ever used an FPGA (processors FTW!) but think this would be considerably more difficult than people are giving you guys credit for. *Out of interest, why did you opt for an FPGA? BBB/RPI would surely be more suited especially with all the open source machine imaging setups for them you could work off?

I actually did something similar to this with a bit of a better outcome. Finally dug out a video of it. Essentially, it does work, got a high score of 10, gets 6 in the randomly recorded video. It would usually go through 3 ≤ n ≤ 7 pipes before failing. I wasn’t that in to the project really so I didn’t keep a good record or a proper video of it working, it was more just a demo video for friends. Since then I’ve lost the actual script which calculated the tap timing unfortunately, not sure if deleted or just lost.

Admittedly I used a full computer and multiplication (their page said they just used “basic addition and subtraction”, not sure why or if that’s just an error of wording?)

Either way, I think the problems with this approach are:
1. Guesstimates: the value for gravity acceleration and the bird’s initial upwards velocity were guessed and re-guessed until it worked relatively adequately.
2. AirPlay Mirroring, unpredictable latency jitter: This probably isn’t the AirPlay protocol’s fault, more likely just the software I was using to simulate an AirPlay display or my network.

Hi everyone,
We see a lot of comments out here for our project and we can understand the reason for the negative comments and ridicule that it is going through. We appreciate all your comments and ideas to get it working and we agree that the system was not accurate enough to play the game quite reliably as how some of you have done so.

However I would like to clear up a few points here regarding the project:
1) The time duration of the project was 4 weeks and not an entire semester like how many have assumed. If you read the results vs expectations section we have mentioned
” The foundation for the game to be played reliably have been laid out effectively. Due to time constraints, we were unable to satisfactorily tweak the game-playing algorithm to play the game perfectly”. We could not work more on the algorithm part of the project due to time constraints.
2) The project was done for a course as its final project in the final semester of our Master’s program with multiple others courses to work on in the same time frame of the project and hence we could not spend the effective “4 week” period for this project alone.
3)The purpose of the project was to put together the different concepts we had used in our course earlier and attempt to get a reasonable working model and if we were to work on this project alone for a given period of time, we are confident that we can get the system to work consistently based on what we have learnt doing the same.
4) When taking up a project, there are cases where it might work or it may not. In this case, even if the project was not an absolute success, we sure have learnt a lot from this experience.

The above points are not excuses but the actual constraints that we had while doing the project and hence all your comments undermine the effort our team has put in the completion of the project. We appreciate your feedback and comments and we would be more than happy to see someone take this forward and get a better result.