Automating Flippy Knife for Android

Every so often a friend of mine will start talking about Hearthstone and it will pique my interest. Hearthstone is a collectible card game where you acquire cards and build decks which you use to battle other players.

Thinking about it triggers thoughts of my own pubescent hobby of Magic: The Gathering and I usually instinctively cringe at the thought of myself and stop, but something about the atmosphere of the game and popularity got me interested this time and I decided to try it out.

Popped open the Play Store and started the download. While it was going another suggested game popped up though, Flippy Knife. I immediately downloaded it based on the artwork alone.

Flippy Knife is actually a collection of mini-games themed around…flipping a knife (or axe, or sword, or some other weapon). It starts off pretty simply, but it has just the right balance of frustration and rewards to keep you hooked. Another clever mechanism the game has is easily switching between the different mini-games. Instead of going back to a menu you switch games by buttons on either side of the screen. As soon as you get bored of a game, you can switch to the next one. My favorite part was of course, unlocking new knifes, I was grinding out getting coins when I got a thought. Could I cheat at this game? Would that be OK?

I’ll tell you, I wrestled with it. This wasn’t like loading up Super Mario Brothers 3 that I beat 100x in the Game Genie (Actually the first game I used with the Game Genie was TMNT2), these people make their living off of people playing the game fairly. But, in the interest of learning, I thought it was ok as long as I did not “hack” the game to unlock the knives but instead automated the playing of the game.

There are 4 mini games in the app, one where you swipe to flip. This I thought would be too hard to perfectly do and imagined there would be some randomization that prevents any level of automation.

Two others were just too complex, the “catch the drone” and “side scroller” ones had no real predictable pattern and therefore would be a nightmare to even attempt.

The one I tried was the target game. Basically you have a moving target on the right side of the screen, but it only moves on the Y axis. You swipe to throw the knife at it. As long as you compensate for the height of the target by changing the angle, you can pretty much always hit it. After about 10 hits you get coins. After about 20 hits the target starts moving, which would be bad normally but if you miss the game resets.

My algorithm was simple:

* Take a screenshot
* Find the height of the target by looking for the first red pixel along the right and side of the screen where the target would be.
* Perform some “math” using the height of the target to find the coordinates I should automate a swipe to
* Swipe from the starting point to the target

Before any of that I had to find out how to even communicate with the device. I’d never done it before but some quick Googling revealed there is in fact a way to send raw commands to a device called the Android Debug Bridge (ADB).

ADB can do pretty much anything from sending a keystroke, swipe or touch, to installing APKs and taking screenshots. Exactly what I needed.

To start, I had to install ADB I used (https://forum.xda-developers.com/showthread.php?t=2588979). Fairly old link but still seemed to work for me.

I figured this might be kind of slow so I took a slight detour at this point and looked for an npm module that had the same functionality and found adbkit. (https://github.com/openstf/adbkit)

After a few quick tests I had it running sending a single random click to the screen.

Once I knew I could send the raw events, I had to figure out how to actually process the image on the screen, so using the ADB command line tools, I took a screencap and started looking at how I could load this image into Node. This led me to pngjs (https://github.com/lukeapage/pngjs). I am sure the library has many great features but for the time being it did what I was looking for, took a PNG and gave me back the pixel data.

To my test script I added a quick loop to look for the “red target”, to my surprise it worked pretty much right away.

Here, I am passing the PNG image into this findTarget function. You can see the possible range for the targets was between 566 (at the top) and 1700 (at the bottom) for some reason I also scanned an area 11px wide although it was probably unnecessary in the end.

The comparison of the img.data to the RBG values (255,79,77) would reveal if I had found the target and would return the Y coordinate.

Now I had to wire up the taking of a screen capture using ADB and this search function.

I start by taking the screencap, which returns a promise. To that I chained a function which would pipe the PNG returned (probably in binary format) to the pngjs2 library would would give me the data in a nice usable format in JS. I could then feed that to my findTarget and aimShot functions. The only real tricky part was getting all the promises to work correctly. Normally I don’t think I would have even needed them except I wanted to put this in a loop, so I did not want to start the next iteration until I was sure the work was done.

Plumbing with a PNG

JavaScript

1

2

3

4

5

6

7

8

9

10

11

12

13

14

client.screencap(device)

.then(function(png){

returnnewPromise(function(resolve,reject){

png.pipe(newPNG()).on('parsed',function(){

varaim=aimShot(findTarget(this));

//Do the swipe!

}).then(function(){

resolve();

})

});

});

})

In actuality this took me a few minutes to get to, I am sure I had to take many smaller steps until I got this working.

Through trial and error I ended up with this equation to get the right spot to aim. Basically the lower the target you need to throw harder and more towards the center. Higher you can throw a bit lighter but it must be above the target. Since it was close enough I rounded the starting x coordinate to 1337 for extra points.

Make sure your aim is true

JavaScript

1

2

3

4

5

6

functionaimShot(start){

return{

y:start-(20-((start-566)/56)),

x:1337-((20-((start-566)/56))/2)

}

}

With the aiming done I was ready to do the swipe. ADB does this with the shell command which lets you specify the type of input and parameters.

1

client.shell(device,"input swipe 271 1876 "+aim.x+" "+aim.y)

Lastly, I needed to add another promise around that code and then wire up the loop and I was done.

The trials were great and I started to accumulate some coins, but it was kind of slow. Also it was crushing my battery and causing my phone to heat up, so it was not something I was going to run long-term.

Also, it was not getting me coins faster than I could get myself. I went back to playing the game normally but then I decided I need to give this one more try. I opened up the console and tried doing a direct swipe up in the middle of the screen on the “knife flip” game. The knife went up and landed almost perfectly. With a bit more tweaking I got it as close as I could. Basically ending up with the knife angled slightly to the left. Through the normal course of playing I knew if I then swiped the other direction the knife would land and angle the other direction. So with a bit more trial and error I found two swipes which would perfectly balance each other.

I figured I didnt really even need JS at this point since I could do everything I needed from the command line and I switched to a batch file and let it run.

1

2

3

4

5

adb.exe shell input swipe7201297740855.5

sleep0.075

adb.exe shell input swipe7201297700855.5

sleep0.075

cheat.bat

Success, the knife was flipping and because I had got it to do the shortest throw possible it was crazy fast. After a while some randomness does creep in and the knife will fly off in either direction, but not before racking up some major points.

Even though I ended up using a very trivial solution, I think it was definitely worth doing to learn some of these techniques with PNG files, piping, promises and of course ADB. I ran the cheat for a bit to see how far it would go but it really was not fun or even worth it in the end. I much prefer actually playing the game and definitely recommend it.