Using machine learning to style transfer Bitterblossom and Udnie onto the art used in the MTGO client

That’s a mouthful.

MTGO is the official MtG client. MtG is a card game with art on the cards by various artists. You probably already know that.

With the magic that is called machine learning, I can take the style of one piece of art, in this case Udnie and Bitterblossom, and apply it to all the cards used in the MTGO client.

The result is an MTGO client with a consistent art style.

How to use the images in your MTGO client:

Let’s start by mentioning that I’m not responsible for damage you cause by following these instructions. I’m providing instructions on how to replace a few images, but I’ve seen people fuck that one up as well, and I don’t want any of your complaints sent in my direction. According to the response to the mail I sent to Wizards, this mod is considered fan content as specified in this policy. Hence, I have to mention:

In order to do the same to your MTGO client, you need to locate the Tier1 folder of your MTGO installation. For me, that wasC:\Users\Lutan\AppData\Local\Wizards of the Coast\Magic Online\Images\ArtInFrame\Tier1
It’s going to be different for you.

For starters, I’ll provide two mod packages, but I’ll play around with it a bit more and see how I can improve the quality of the Bitterblossom package or what other cool styles I can apply.

Stop your MTGO client, then delete all the pictures in that folder. Download the package you want, then copy all the images from the package into the folder you just cleaned out. I noticed the MTGO client resetting the images once started, so you also want to mark them all with CTRL + A, then rightclick -> properties and check the box next to “Read-only”.

By only marking modded images as read-only this way, MTGO is free to update images that I haven’t style transfered yet. Missing images are downloaded by MTGO on demand, so further releases won’t be afflicted by the mod.

By the way, using the same method, you can update your MTGO client login screen, you just have to useC:\Users\Lutan\AppData\Local\Wizards of the Coast\Magic Online\Images\Theme\Tier1
instead. Here’s mine, for reference:

Some technical voodoo, because that’s what this blog is actually for:

I wanted to get into machine learning for quite some time now, and I kind of know the basics, but a lot of stuff is pure magic for me still. As a webdev I tried out tensorflow.js, but that still requires a big deal of knowledge. Then I found the library ml5js, which abstracts away tensorflow hard enough, that even someone without any knowledge about ML can try it out.

Fortunately, ml5 has a lot of examples on various machine learning applications, including style transfer.

Unfortunately, while appearing pretty straight-forward, the tutorial for style transfer lacked a few details that blew the setup from a few minutes up to several hours of soul crushing trial and error googling.

The first option was a docker image, which should make the whole ordeal pretty simple: I’m completely new to Docker though, which may or may not be the reason on why I couldn’t get it to work. I moved on to the next option: Using the repo to train the model. The download for the training data and the script to create and train the model are already provided, that part was pretty easy. I started the script, and alas, it actually started training! It also started killing my PC pretty hard.

The tensorflow package comes only with CPU support. It’s actually mentioned in the tutorial that you shouldn’t train on your CPU (who would’ve guessed), but failed to mention that you need tensorflow-gpu in order to train on the GPU. This is where the fun began.

I’m having a GTX970 in my computer and recently reinstalled Ubuntu, now version 18.04. After some googling, I found out that in order to use tensorflow with my GPU, I need tensorflow-gpu, and some stuff to make that work with my GTX970. That stuff turned out to be cuda and cudnn, both provided by Nvidia. Hey, that’s easy, right? Just install them, and you’re good to go!

Nah. I installed cuda, version 9.2, because that’s the only version that’s for Ubuntu 18, but tensorflow couldn’t have cared less. It turns out, tensorflow-gpu currently needs cuda 9.0, which is only available on Ubuntu 17 and 16.

I scraped that idea and thought, to hell with it, I’ll just go the cheap way out and do all that training with Paperspace. It’s a service that can train your models for you, or something like that. I followed their tutorial on how to train your ml5 model for style transfer, until I got to the part where I had to actually use the API.

Paperspace, as I’m writing this article, only supports credit cards as payment method. I don’t have a credit card.

At this point I’ve been in this for about 3 hours, but I figured, hell, let’s give it another try. Sunken-cost-fallacy or something like that.

I tried to install cuda 9.0, which despite being only supported by Ubuntu 17 and 16, was my only hope. It wasn’t having any of that though: Maybe it was some issue with the previous installation of cuda 9.2, or my inexperienced fingers trying to shove a cuda version into Ubuntu which isn’t supposed to be there, but in the end my OS ended up begging for death and didn’t allow me to boot anymore. After a ton of curses against about 7 different deities, I reinstalled.

I didn’t learn my lesson though. After the reinstall, I went directly back into getting that chaos working. I wanted my pretty MTGO client, and I was going to have it. As it turns out, one of the deities heard my cries and decided to bless me with a lucky google result: Someone smarter than me already tried setting up tensorflow-gpu with an Nvidia graphics card on Ubuntu 18. Talk about coincidents. I’ll link it here, but the gist of the article was: Install cuda 9.0 and beat it until it submits to Ubuntu 18.

After that I only had to install the correct tensorflow-gpu version and every other python package it required, et voilà, I finally was able to train a model on my own.