Author: chrizzle23

Husband, Father, Veteran, cypher punk, hacker spacer, gamer, lover of privacy, free speech, and filthy scumm pirates. My opinions are my own and do not reflect those of hive13, Cinci2600, or my current employer.

It’s been a few weeks since I exorcised HyperV from my life like an evil demon. I have replaced it with Proxmox and so far it’s been mostly great. With a couple of serious caveats.

My transition to Proxmox has been a rather involved, not so much because Proxmox is hard to set up (it’s not), but because I am tired of slapping old junky hardware together and hoping it doesn’t die, and then scrambling to fix it when it inevitably betrays me. Unlike most dudes with home servers and labs, most of my acquisitions were made years ago to support an MMO habit. Specifically multiboxing.

I call them “computers” because they are computers in the sense that they have CPU’s, RAM, and HDD’s. But they were low-budget things when they were assembled years ago. The upgrade path works something like this:

A computer begins its life as my main gaming machine that will run my favorite game at a satisfactory speed and resolution.

Then I find a new favorite and upgrade the gaming machine’s guts to run the new game.

The old gaming guts get transplanted in to my “server” where they are *barely* able to run a few VMs and things like that.

The final stage is when the server guts are no longer up to the task of running VMs. I then add a few old network cards and the “server” then becomes my “router”.

The old router guts then get donated somewhere. They’re not really useful to anyone, so they probably get shipped to Africa where they get mined for gold and copper by children at gunpoint.

Breaking the [Re]Cycle of ViolenceIn the years since then, I have taken to playing epic single player games like Skyrim. These games really only need one machine. The rest of the gear I used to run little “servers” for one thing or another, which I have slowly replaced with VMs. The problem with using old junky computers as servers is when you run them balls out 24 hours a day. In my search for a replacement VM host, I spent a lot of time researching off-lease servers. My goal was to have 8 cores and 32gb of ram, with the ability to live migrate VMs to another [lesser] host in an emergency, something that my HyperV setup was lacking. After a lot of consternation, I decided that since a single VM would never actually use more than 4 cores or 8gb of RAM, why not use 2 [or more] desktops?

I found some old off-lease quad-core Intel desktops for about the same retail price as a low end server processor. I used the RAM from my older gaming machines/VMservers and some hard drives from some old file servers to build out my “new” Proxmox cluster. With two quad core desktops running maxed-out memory(16GB each) I managed to satisfy my need to be like the other kids with “8 cores with 32GB of RAM” for about the price of an off-lease server chassis, with the added bonus having a cluster. The goal is to add nodes to grow the cluster to 16 cores and 64GB of RAM, while also adding clustered storage via Ceph to make use of old hard drives from file servers.

New hot servers is old and busted. Old busted clusters is the new hotness.
For me, the clustered model is better, in my opinion for a number of reasons. It mostly has to do with modularity:

You can build out your infrastructure one paycheck at a time. Part of the problem with off-lease servers is that while the chassis is cheap, the components that go in it are expensive and/or hard to find. The deal with servers is that the cost of the motherboard and CPU are nothing compared to what you will spend on RAM. I was looking for something I could start using for less than $200, and a refurb desktop and RAM from old gaming boxes got me going at that price point.

Desktops stack on top of each other for free. I don’t have any server or telco racks, so in addition buying ECC RAM, I would also be buying a rack, rails, and all of the other stuff that goes with them. This would easily eat up my $200 startup budget before I powered on a single box.

Moar boxes == moar resiliency. My gear at home is part lab and part production environment. Yes, I use it to hack stuff and learn new things, but my family also uses it in their daily lives. Network shares stream cartoons; VOIP phones connect friends; keeping these things going is probably as important as my day job. Being able to try bold and stupid things without endangering the “Family Infrastructure” is important to my quality of life.

Scaling out is probably more important than Scaling Up. A typical I.T. Department/Data Center response to capacity problems is to regularly stand up newer/more powerful [expensive] gear and then dump the old stuff. I guess this is a good approach if you have the budget. It certainly has created a market for used gear. I don’t have any budget to speak of, so I want to be able to increase capacity by adding servers while keeping the existing ones in play. There are still cost concerns with this approach, mainly with network equipment. In addition to upping my server game, I am going to have to up my networking game as well.

It works…ish

I have my two cluster nodes *kind of* working, with most of my Linux guests running as containers, which is very memory and CPU efficient. I am running two Windows VMs, PORTAL for remote access and dynamic DNS, and MOONBASE which I am using for tasks that need wired network access. All of my desktops are currently in pieces, having donated their guts to the “Cluster Collective” so I am mostly using my laptop for everything. I am not really in the habit of plugging it in to Ethernet, or leaving it turned on, so for now I am using a VM in place of my desktop for long running tasks like file transfers.

I say that the cluster is only kind of working because my home network isn’t very well segmented and the cluster heartbeat traffic straight up murders my little switch. It took me a while to figure out the problem. So the cluster works for a few days and then my core switch chokes and passes out, knocking pretty much everything offline. For now, the “cluster” is disabled and the second node is powered off until my new network cards arrive and I can configure separate networks for the clustering, storage, and the VMs.

Coming soon: Adventures in Proxmox part 2: You don’t know shit about networking.

My relationship with Mouse Without Borders is complicated. On the one hand I dearly love it and rely on it for a lot of my workday. On the other hand it stops working for various reasons and it drives me absolutely insane. I have used Synergy in the past with Linux and MacOS, but if you are just connecting Windows machines, MWoB is the way to go.

The reasons to love MWoB are numerous. It lets you use one keyboard and mouse to control multiple computers. This is different than using a KVM switch because there is no video involved. Instead, you place up to 4 computers side by side and MWoB lets you move the mouse off of the screen on one machine and onto the screen of another. This is significant if you use several machines at once. Most video setups support 1 or 2 monitors, but I am hardcore and like to use 3 or more screens at the same time. I like to pretend that I work at NASA.

The reason to hate MWoB is that it sits at the intersection of two explosive elements: human interface devices and Windows network security.

The keyboard and mouse are the human interface to a computer system. They are of tremendous psychological significance to the human operating said computer. If the human interface malfunctions in any way, the emotional impact on the human is swift and severe. Keyboard and mouse malfunctions are Hulk-level rage inducing. This really isn’t MWoB’s fault, but it did decided to play a dangerous game.

MWoB uses networking to connect two Windows systems together. This means that MWoB is at the tender mercy of Windows Defender, a fickle beast. Windows networking can make file shares randomly disappear; it can quit seeing print queues; it’s utter chaos. I really dread messing with firewall rules on Unix systems, but I actively avoid it on Windows. The same goes for editing Group Policy. You can spend hours tuning both just to see a Windows security update wipe all of it out. Using MWoB means you have to get two Windows systems to play nicely with each other reliably, no small task. That’s two Windows operating systems, two MWoB installs, and two panicky firewalls to appease. I have reinstalled Windows on more than one occasion just to realize that the problem that I am having is actually with the *other* computer. Sure, Windows systems and networks are easy to set up, but like a house made of sticks, they’re easy to knock down. Again, this isn’t necessarily MWoB’s fault, but it’s a piece of software that has decided to play a [doubly] dangerous game.

When you force a vital computing component like your keyboard to operate in a volatile environment like Windows networking, you get a service that alleviates a tremendous strain. However, the sudden re-introduction of that strain is is eye-gougingly frustrating.

In my previous post, I expanded on my preferred methods for gaining remote access to my home network. Since then, I have decided to quit using Hyper-V because it’s awful.

I have now decided to move to ProxMox on my server. Proxmox is pretty cool, although the documentation sucks. I recently started using Linux containers for my remote access servers instead of VMs, which ProxMox supports out of the box. A truly compelling feature of Proxmox is its integration with Turnkey Linux. You can download Turnkey Linux Container Templates directly in Proxmox and spin them up quickly. I used the Turnkey OpenVPN template to rebuild GATE, my OpenVPN server.

The performance improvement is remarkable. On Hyper-V, each Linux VM ate 512MB of RAM just to sit idle 99.9% of the time. So far I have 3 containers configured with 512MB of ram each, but they use roughly 25-50MB each and they leave the rest for the server. PORTAL, my Windows VM, still takes his share of the RAM and doesn’t give it back, but that’s nothing new.

Moar RAM == moar servers!On the plus side, efficient use of memory means that I can feel better about running a dedicated Linux box (container) for each application. Dedicated boxes mean that when I inevitably screw something up, it doesn’t affect the other applications that are running (that I haven’t screwed up yet.) Also, with pre-built containers and snapshots, you can toss machines that you screwed up without losing much time. I know, I know, rebuilding a Linux box instead of fixing it is sacrilege… but I got other shit to do.

On the minus side, containers don’t really act like VMs, especially when it comes to alternative network configurations. In particular, a Linux Container that uses a TUN or TAP interface needs some extra configuring. The TUN interface is how OpenVPN does its thing, so getting my GATE machine, the OpenVPN server that allows access to the DMZ on my internal network took a lot of fiddling with to get right. I did a bunch of Googling and I ended up with this forum post that recommends rebuilding the TUN interface at boot time with a script.

Here is the TUN script that I have graciously stolen so that I don’t have to Google it again (I didn’t even bother to change the German comments):

Then you enable the script and turn it on:chmod 755 /etc/init.d/tun
update-rc.d tun defaults

With this script, I was able to stand up a real OpenVPN server (not just an Access Server appliance) for unlimited concurrent connections! Not that I need them. I’m the only one that uses the VPN and most of the time I just use SSH tunnels anyway.

Since OpenVPN container templates make standing up servers so easy, I thought I’d build another one that works in reverse. In addition to GATE that lets OpenVPN clients route in to the DMZ, I thought I would use an OpenVPN client to route traffic from some DMZ hosts out to the Internet via Sweden. In the past, I used a VPN service to dump my Bittorrent box’s traffic this way, but I would like to extend that service to multiple machines. EVERYBODY GETS A VPN!

Öppna dörr. Getönda flörr.
I couldn’t figure out what a machine that does this kind of thing is called. It’s a server, but it serves up its client connection to other clients. It’s a router, but it just has the one network interface (eth0) that connects to a tunnel (tun0). It’s basically setting up a site-to-site VPN, but the other site is actually a secure gateway. This identity crisis led to a terminology problem that made finding documentation pretty tough. Fortunately, I found another pirate looking to do the same thing and stole his scripts 🙂

Since it’s a doorway to a VPN gateway to Sweden, I decided to call the box DÖRR, which is Swedish for “door”. I did this to maintain my trans-dimensional gateway theme (HUB, GATE, PORTAL, etc.)

Also, I would like to apologize to the entire region of Scandinavia for what I did you your languages to make the pun above.

The Turnkey Linux OpenVPN template sets up in one of 3 modes: “Server”, “Gateway”, or “Client”. “Server” is the option I went with for GATE, which allows OVPN clients the option of accessing local subnets. This is the “Server” portion of a Site-to-Site VPN or a corporate VPN. “Gateway” forces all OVPN clients to route all traffic through it, this is the config for secure VPN services like NordVPN or AirVPN. “Client” makes a client connection to another OVPN server. If you connect a “Client” to a “Server” you get the full Site-to-Site solution, but there is no documentation on Turnkey about setting up a “Site-to-Site Client” to route traffic from its internal subnet to the “Site-to-Site Server”.

What I am looking to do is configure a “Site-to-Site Client” but point it to a “Gateway”. Another important consideration when setting this up was that I didn’t want to do any meddling with the setup of my DMZ network. I just want to manually configure a host to use DÖRR as its default gateway. No need for proxies, DNSMasq, DHCP or anything like that. Just static IP’s, the way God intended it 🙂

Step 1 – The Site-to-Site Client
Once I got the container running, I had to fix the /dev/tun problem (the script above) and then make some config changes to OpenVPN.

Because this is a VPN client, and not a server, you need to get the OpenVPN client profile loaded. The bulk of my experience with OpenVPN clients is on Windows where you start the client when you need it. For this application you need to automatically run the OpenVPN connect process at boot and keep it running indefinitely.

First, you need to obtain a client config. I downloaded my ‘client.ovpn’ file from my VPN provider, and I copied it to /etc/openvpn/client.conf as root. You can name the files whatever you want, just remember what you named them because it’s important later.

cp /root/client.ovpn /etc/openvpn/client.conf

Now test the connection to make sure everything worked

openvpn --config /etc/openvpn/client.conf &

The & is important because it puts the OpenVPN process into the background, so that you get your command prompt back by pressing ENTER a couple of times. You can then test your Internet connection to see what your IP is a few different ways. You can use SSH with a dynamic port and tunnel your web traffic thru it with a SOCKs proxy. You could use curl or lynx to view a page that will display your IP. Or you could just use wget. I like to use ifconfig.co like so:

wget -qO- ifconfig.co

If all goes well, you should see your VPN provider’s IP and not your ISP’s.

Once you get the VPN client working, you then want it to start up and connect at boot time. You do this by setting the ‘autostart’ option in /etc/default/openvpn.

nano /etc/default/openvpn
AUTOSTART="client"

If you changed your ‘/etc/openvpn/client.conf’ filename, you change the name here. The AUTOSTART value is the name of that file minus the ‘.conf’

Now reboot your server and do your wget test again to make sure that the VPN connection is starting automatically.

Once that is working, you have to route traffic. This means IPTables, because OpenVPN and IPTables go together like pizza and beer.

Step 2 – De Routningen

Normally to route traffic between interfaces on Linux, you have to add IP forwarding (echo 1 > /proc/sys/net/ipv4/ip_forward etc.) In this case, the Turnkey OpenVPN template has already done that for you. All you have to do add a few forwarding rules:

Now it’s time to test them. For this you need a client computer with a static IP. For the default gateway you want to use the static IP that you assigned to eth0 on your VPN doorway server. I used 192.168.0.254 for DÖRR. If your test box also shows your VPN provider’s IP when you access a site like ipleak.net then it’s time to make those rules permanent. By saving them to /etc/iptables.up.rules. It is important to save them to that specific file because the Turnkey template calls that file when setting up the eth0 interface in /etc/network/interfaces.

iptables-save | tee /etc/iptables.up.rules

I don’t know why it’s set up that way. I’m just here to make awful jokes about Germanic languages.

Once that’s done, reboot the doorway server one last time and test with your client computer with the alternate default gateway.

Now that the my VPN client is working again, I need to rebuild my BitTorrent machine. I am going to try to save some more RAM by going with another Turnkey Linux container template.

EDIT: In my elation over getting something to work, I forgot to change the default gateway back. Unfortunately my test machine was PORTAL, which happens to control my dynamic DNS. So currently all of my hostnames are pointed at Sweden, LÖL.

A while back, I wrote about using Windows HyperV server. The reason that I set up this server was to use the combination of a Linux server and a Windows desktop to get remote access to my home network. I thought that I would elaborate on the tools that I use to get into my home network from work or while traveling.

I use several methods, each with certain advantages and disadvantages. Mostly I prefer SSH over pretty much anything else in order to connect to a Linux host, and I prefer Remote Desktop over pretty much anything else in order to connect to a Windows host. As a backup, I will use Teamviewer. It’s not ideal, but it works where other services fail.

SSH is pretty much a Swiss Army Knife of network tools. You can use it to do waaaay more with it than just log into a Unix box and execute commands. It’s a tool for creating encrypted tunnels, it just so happens that 90% of those tunnels connect to remote shells. In addition to connecting to a remote shell, you can open ports on a host. I am fortunate enough to have Cincinnati Bell Fioptics which lets me open almost any port on my firewall without any bother. I forward port 22 directly to a Linux box named HUB, and I secure it with SSH keys. I can then use SSH to tunnel traffic into my home network, be that browser traffic through a SOCKS proxy and dynamic port, or RDP traffic with a local port. This works well when I am in a restrictive network that still allows outbound SSH traffic, and as long as I have my Putty session set up ahead of time with my private key. This is the technique that I use when I am not able to access my network through NeoRouter.

Remote Desktop (RDP) is another Swiss Army Knife for connecting to computers. I use Windows as my primary desktop OS. I like to use Linux mostly for server stuff and for running specific tools like Clonezilla or Kali. As a matter of fact, I prefer Linux for servers and tools over Windows. I know, I’m an odd duck. RDP not only gives you remote access to the Windows Desktop, it lets you map drives remotely to transfer files and it lets you connect at a desktop resolution that is greater or lesser than that of the machine that you are connecting to. This is a big deal when you are using RDP on a wide-screen monitor to control a server that is plugged into an old CRT monitor, or when you are using a tiny netbook to control your multi-screen desktop. Teamviewer (and the VNC server that it is based on) cannot do that.

In order to make my SSH and RDP connections, I like to use either NeoRouter or OpenVPN. NeoRouter is technically a split-tunneling VPN solution, but I like to think of it as creating a network of computers that is independent of their actual networks. Split-tunneling VPN is a fancy term for VPN connections that don’t mess with your Internet access. There are lots of other features for split-tunnels, but under most circumstances, I want my computers to talk to each other differently than they talk to the Internet.

The NeoRouter network explorer tool lets me see which of my computers are up and connected. I run the NeoRouter server on HUB, which is sitting behind my firewall, with port 32976 forwarded to it as well. Running the server inside my firewall lets me do some neat networking tricks, like having my BitTorrent VM connect to the internal IP for HUB, instead of using the Internet. My BitTorrent box uses a VPN client to route all Internet traffic through Sweden, which really slows down my Remote Desktop session. I run the NeoRouter client on my desktops and laptops, and also on my file servers so that I can access shared folders remotely. File transfers this way can be really slow, so I also use One Drive top share big files like videos or ISO images.

OpenVPN is my tool of choice for open WiFi networks at hotels and coffee shops. I can access my home network while also securing all of my network traffic. I run OpenVPN Access Server on a dedicated VM named GATE. Access Server is easy to use and configure, and it’s free for two concurrent connections. For occasional use, especially by people other than me, it works really well. There’s even a ready made Hyper-V appliance that you can just boot up and go. I used to run OpenVPN on HUB, but the networking/subnet stuff meant that I had to remember the internal IP for the OpenVPN network segment and change it to connect to NeoRouter. So I just use two separate machines and it all works out. I have built OpenVPN servers without Access Server in the past. I like to use the Turnkey Linux OpenVPN appliance, and setup couldn’t be easier.

If I cannot get in via NeoRouter, OpenVPN, or old school SSH tunneling, then I fall back on using TeamViewer. It can get me in when pretty much all other tools fail me, but it’s not as nice as using RDP. Also, it should be noted that TeamViewer can only be used to control graphical desktops, there is no command line equivalent. In order to alleviate some of the frustrations of TeamViewer’s desktop resolution, I run a dedicated Windows VM that I call Portal. I keep the native (console) resolution fairly low, and I have RDP and Putty sessions set up so I can quickly connect to my other computers.

One other thing that I use Portal for is to move files into and out of my home network. You can use RDP or TeamViewer to copy files, but for big files like videos and ISO’s, One Drive does a much better job. I have a dedicated One Drive account that I use specifically for moving files this way. I just grab a file from somewhere, copy it to the One Drive folder on Portal, and it automagically uploads. Then, some time later, I can use the One Drive website to download the file, at much faster speeds than using RDP, SCP (SSH), or TeamViewer’s file transfer tool. It’s an extra step, but one worth taking, especially if I find myself in an oh-shit-i-forgot-that-important-file situation.

I did my best in a previous rant to point out the [possibly malicious] polarization of the national conversation. I tried to express my concern over the simplification of complex ideas into rhetorically convenient narratives. Unfortunately, my self-righteousness got the better of me and I don’t think I quite captured that the heart of the issue is a reduction of ideas. This essay really captures the issue for me without needing to watch 3 hours of Adam Curtis documentaries to achieve understanding. In essence, the essay likens the focus by Western Society on [predatory] exponential financial gain to the mutation of cells into a form of cancer:

Values and complexity are focused more and more on prioritizing exponential financial growth, led by for-profit corporate entities that have gained autonomy, rights, power, and nearly unregulated societal influence. The behavior of these entities are akin to cancers. Healthy cells regulate their growth and respond to their surroundings, even eliminating themselves if they wander into an organ where they don’t belong. Cancerous cells, on the other hand, optimize for unconstrained growth and spread with disregard to their function or context.

Reducing the American experiment down to unfettered corporate growth is a gross reduction of ideas. Reducing ideas essentially reduces people. Reducing humanity down to statistics is dangerous because it eliminates most, if not all, of the complexity that makes humanity so different from everything else. I did a fair amount of whining about the hypocrisy of our polarized media landscape without talking about how absurd it is to reduce people down to being cogs in a machine. I have touched on this idea in the past pondering the nature of freedom. This reduction is absurd because the idea of advancing one ideology (which is really just a subset of human struggle) at the expense of another ideology (which is itself another subset of human struggle) is basically pitting half of society against the other half, while deliberately ignoring everything that they have in common.

Rejecting absurdity means embracing irreconcilable ideas

Much like the geardo cargo cult that worships gun culture, there is another cargo cult that worships technology. Much like the cult of the gun that has no real concept of military doctrine, the cult of code has no real concept of software engineering. Just like guns are falsely ascribed the power to help realize white-male-power fantasies, math is also falsely ascribed the power to achieve techno-Utopian fantasies. I understand how seductive both kinds of fantasies can be. The cult of the gun seeks to take control over the violent chaos that threatens to destroy so much. The cult of code wants to cede control to powerful algorithms that promise logic and objectivity and free us from the responsibility of making hard choices. The gun can’t protect you from an uncaring universe, and algorithms just crystallize the biases of their creators.

The desire to take control from those we perceive to be tyrants and to give control over to systems we believe to be objective is a paradox. Like the idea of a benevolent dictator or a truly free market, the perfect system is a great idea that always fails when implemented. Any complex system, be it an economy or a social network, was engineered by people to be used by other people. Engineers are as flawed and fallible as anyone else ergo the systems they build will also be flawed. End users aren’t the sheep that engineers wish that they were. Any imperfect thing (i.e. all things) will eventually be misused by someone with an understanding of its flaws. This isn’t hacker self-righteousness, just look at all of the government programs that get defrauded or the tax increases that billionaires find ways of avoiding. These are flawed systems designed by people that other people have chosen to capitalize on. It doesn’t matter how much time, effort, and money you put into developing a system, someone will invest more in order to exploit it.

The problem with both cults is that neither the gun nor the algorithm is a substitute for humanity, nor is either a hedge against tyranny. In fact, with frightening frequency, both tools harm humanity and encourage tyranny. Guns empower tyrants and are mostly brought to bear against the innocent. Computers empower the reduction of ideas down to data sets, where they confine us to platforms run by big corporations, another form of tyranny. The AK-47 was supposed to be a Communist gift to liberate oppressed peoples, but they mostly got used by death squads. Facebook was supposed to help us to connect with each other, but it mostly pits us against each other while shady marketers peddle divisive media for money and political influence.

Like most of my pseudo-intellectual deep thoughts, the thought began with watching an Adam Curtis documentary but is became fully actualized by watching Rick and Morty. The Adam Curtis idea, that there is a “positive liberty” which is born out of violent struggle, and a “negative liberty” which is born out of apathy. In Rick and Morty, Rick is the epitome of both the gun and the code cults. He has all the weaponry and all the technology, yet most of what he uses it for is to allow himself to get drunk and watch TV. He literally sells a gun to an assassin so he has enough money to spend the day at an arcade. He gets involved with all manner of epic adventures, mostly so he can be an asshole to everyone close to him. He is forever ridiculing the beliefs of others, yet he believes in nothing.

This is probably where I should look to the Asian philosophies I studied as part of my martial arts training should come into play, but Confucianist parables and Taoist paradoxes just aren’t the right medicine for this species existential dread. I’m pretty fucking far from being an Uncarved Stone and trusting the process. If anything, I am an *over* carved stone, and the process is light years away from trustworthy.

I have long been a fan of playing Massively Multiplayer Online games, but I really don’t like MMO gamers because they tend to be jackasses. At the time my MMO of choice was City of Heroes, which was popular with teenagers. Needless to say, the jackass factor was high. The game is best played with others tho, so I was often stuck playing with jackasses. You do what you gotta do to unlock those badges.

My gaming experience was sub optimal. So, I did what any hacker does when he is confronted with a problem: I started hacking. I found that I could multiplex keyboard commands through some networked software and came up with a workable multibox solution. The trick was it needed multiple computers. So I cobbled together some old desktops to make barely-passable gaming machines. At one point I had 8 of them running. It took a half hour to get all my bots logged into the game and another half hour to enter an instance, but being able to play on superhero teams where everyone did what I told them to do was sheer joy. My group was all robot-themed and my supergroup was called “Automated Heroics Inc.” and all of the player-character bios read like product descriptions in a catalog. I also had macros programmed so that all of them could do “The Robot” in sync. It was hilarious. Why didn’t I get any video of that?

Multiboxing can be tricky because each MMO is different about how it handles its controls, sessions, authentication, you name it. In the case of CoH, running multiple instances of the game on the same computer didn’t work well. It was fine if I alt-tabbed between the sessions and controlled the toons manually, but having sessions in windowed mode made them crash. The software that I used, Auto Hotkey, worked well when testing scripts with notepad windows, but when it came time to run them with CoH, it was shit show.

So I decided to keep AHK, but I used some junk PCs and old video cards to run the game. AHK has some networking features that let you push groups of keystrokes out to clients, so that if I pressed ‘0’ on my main PC, it would send a series of key presses and pauses to the other 7 machines. Because I am writing this several years after I did the project, I no longer have any of the files I used. Also CoH has been shut down for years, so example code wouldn’t be all that useful even if I had it. Here are a few things to consider though:

Hopefully your game has a free-to-play or freemium option so that you can set up multiple accounts for not much money. Running just one bot toon is way different from a tactical standpoint than running seven of them.

Hopefully your game has an auto-follow function, where you target a player and your toon moves whenever and where ever the target goes. This is so important for moving all of your bots in an orderly fashion.

Hopefully your game has an assist or auto-target function, where you target a player and your toon targets that player’s target. Much like the auto-follow feature, assist keeps everyone shooting at the same thing. I found that concentrating fire on the big critters first was the most effective way to initiate combat. If you time it right, you put them down fast and then mop up the minions.

If you have both auto-follow and assist, then you can round up your bot crew by mapping a key to tell each bot to target you, follow you, and assist you. Being able to get your toons to focus on you is an essential function because targeting can cause your bots to do dumb things like take off running or shoot at the wrong thing. On my “main” pc, I mapped this script to the same key that I used to target the enemy closest to me.

Multiboxed toons work best with ranged combat, especially area of effect attacks. You will want your crew to be mostly squishy DPS types and dudes that can heal and buff squishy DPS types. My bot crew was entirely ranged. I called them “The Firing Squad.”

An AOE that is centered on the player (A Player Based Area Of Effect, PBAOE, in CoH parlance) is great for mopping up a mob once it has closed distance with your crew.

Another great use is AOE heals. Even if they’re weak, you can have two or more toons dropping their heals as part of their attack sequence. Often, your toons will either have a PBAOE attack, or a PBAOE heal. If you are dropping PBAOEs when the enemy moves into melee range, you will likely need AOE heals too, so just have everyone drop them at once.

I mostly used my bots to level my support toons that were hard to solo, like controllers and tanks. It’s decent practice for keeping a team alive, but it’s not the same skill at playing with real humans.

Multiboxing isn’t about playing an indiviual bot toon well. It’s about using the entire group of bot toons to support your main toon[s]. There are some key differences between playing a main toon vs. playing a bot toon:

Your bots will probably never be alone, so there’s no need to balance offense with defense. A “real” toon needs to be well rounded, bot toons are highly specialized insects.

Your bots should have two basic specialties: shooting or healing. They should be going pew pew pew or heal heal heal pretty much all the time.

Putting up shields and other buffs can be a pain to script but it’s worth it: Targeting a team member, drop one or more buffs on them, target the next team member, etc.

There will be multiple buffers dropping different buffs, so don’t focus so much on making each buff powerful, focus on making each buff mana/energy efficient with short cool down periods so you can lay them down fast and often. Once the buff process is scripted, running it between each mob isn’t a big deal.

In CoH, there were two character classes, the Corruptor and the Defender that both combined blasting stuff with healing and buffs. The Corruptor’s primary power set was offense and the secondary power set was support, while the Defender was the exact opposite. A third class, the Blaster, was exclusively focused on offense. I had two Blasters, four Corruptors and one Defender. The corruptors could buff everyone up before a fight, then my main toon would pull a mob, the bots would open fire, and if the mob got close, I had the Blasters drop their PBAOE blasts and then the Defender and the Corruptors dropped heals. The benefit of their damage abilities was obvious, but the shields and heals were equally important for helping to level my tank and controller. At higher levels, the bots all had a sniper-type attack that was long range, accurate, and did lots of damage with a long cool down timer. I could generally have everyone target a mob’s boss/lieutenant and drop him in order to pull the rest of the mob. I would then use my tank or controller to tie up the mob while the firing squad picked off minions one at a time. If anything survived that and actually made it to melee range, I would drop the PBAOE blasts, AKA “The Nukes”, along with the heals. The stragglers then got picked off by the firing squad and we rebuffed and took on another mob.

The things you learn about keyboardsGetting your bot toons to do things involved creating macros for each toon to execute certain actions, noting the times that certain animations took, and then mapping those macros to shortcut keys and using AHK to script the key presses for those shortcuts. You have to learn a lot about your game’s behavior, but you also have to learn about keyboards.

Keyboard behavior plays a major part in getting your scripts right. I had the hardest time getting my bots to do simple things like run because I didn’t understand that pushing a key down, and letting go of it are two different events. It was so hard to get those bastards to run, that I ended up relying on the auto-follow feature for basically all movement.

It’s hard to imagine all of the realtime events that go into pressing keys on a keyboard until you have to simulate key presses with software. One thing I wanted to do make the bots do was spread out so that they didn’t all get hit with enemy AOEs. I never did get it right, so I just kept everyone close together and used lots of heals.

I miss all my robot minions. I hope that some day a similar MMO will emerge that will let me rebuild Automated Heroics Inc. so I can record some goddamn video of my dancing robots.

When I am not playing Skyrim to stave off my existential dread, I watch TV. Needless to say, I have been watching a lot of TV. I used to consider myself more of a cinema nerd, but films just aren’t that good anymore. When I compare some of my favorite films from a long time ago, to the franchise drek that is film today, it lacks quality. Sure, there are good films here and there, like The Dark Knight and Rogue One, but there are a lot of CGI messes too, and some TV shows seem to deliver more consistent quality.

Film sucks for the most part, and I can’t binge watch Adam Curtis documentaries all the time or I will lose my goddamn mind, so I watch TV. Of course I also do family stuff, but with an infant who doesn’t sleep at night, that involves a fair amount of staying up all night holding a sleeping baby, so TV is a big part of my nightly routine.

I have been watching a few new shows and re-watching some old faves, so I’m just going to list them in no particular order and say random things about them.

Stranger Things

I watched Stranger Things for the first time a couple of weeks after it dropped on NetFlix. Since then, I’ve probably rewatched it at least 3 times. It’s a great show, full of nods to 80’s movies like E.T. and Stand By Me, but it also captures something essential about my childhood, which was playing Dungeons and Dragons in my friend’s basement for hours at a time and being bullied.

There are lots of neat things to spot in the show (like the fact that Hop’s daughter, Eleven, and Will all have the same stuffed tiger) and I am unreasonably pumped for season 2, which should be out in a few weeks. I have my own theories about what will happen, but I don’t really want to spoil anything if by some odd chance this is the thing that inspires someone to watch the show, and by an even odder chance I turn out to be right. I will say that the kids’ D&D game at the beginning of the game sort of outlines the plot of the season, and their game at the end probably outlines what will happen in the second series, or at least underlines what is still unresolved at the end of the first series.

Rick and Morty (obvs.)

The new season of Rick and Morty is awesome. It’s another show full of details and fan theories to obsess over. My existential angst is both alleviated and agitated by the show. The show’s conflicting ideas of finding meaning in uncaring universe either helps or makes things worse; I can’t tell which.

The essential point of Rick and Morty is that people with beliefs will have those beliefs tested at every turn. The show actively punishes characters for having any kind of belief, including the devil. The only person that seems to escape this punishment is Rick, and yet Rick is borderline suicidal. Rick has all the answers, and his answer is not to think about it. As power fantasies go, Rick is either the greatest expression because he is essentially all-powerful, or the worst expression because all of his power never seems to get him anywhere. Again, I can’t tell which.

True Detective (season 1)

Speaking of the dichotomy of belief and disbelief, the first season of True Detective is one of the best television shows I have ever seen. Rust (Matthew McConaughey) is incredibly intelligent and yet completely unable to interact with people, except for when he is interrogating them and luring them into making confessions. There are a number of similarities to Rick and Morty, mostly having to do with the juxtaposition of human meaning and savage cruelty, but also the juxtaposition of truth and deception, duty and corruption. There is just barely enough evidence in the show to convince you that Rust is either psychic or psychotic, and somehow not enough to convince you which one.

Rust is working to find truth, and in so doing alienating everyone and choosing to live in madness and misery. Marty on the other hand does the opposite and ends up alienating everyone anyway. The only way that they can uphold the law is to break the law. It’s existential absurdity at its finest.

Season 2 is a good show, it’s just not the masterpiece that is season 1. It’s still worth watching, I just haven’t watched it a dozen times like I have season 1. If you are going to commit to both seasons, you should probably watch season 2 first. Season 2 unfortunately lacks both the Southern Gothic aesthetic of season 1, and the Lovecraftian symbolism. Season 2 takes place in L.A. and without those motifs, it’s just weird L.A. people doing weird L.A. shit. Kind of like a darker version of Bosch.

BoJack HorsemanBoJack Horseman is another “grown up cartoon” that specializes in reflecting my own nihilism back at me. While Rick and Morty is an endorsement for not engaging in reality, Bojack Horseman is an endorsement for [shying away from] your responsibility for your own reality. Like Rick Sanchez, Bojack understands that everything is shitty and pointless. Unlike Rick, Bojack learns that he is responsible for his own happiness. Of course, Bojack does a comically bad job of handling that responsibility, but he is aware that the responsibility exists.

Watching Bojack Horseman and Rick and Morty as a matched set offers two interesting takes on the “whatever you do you will end up feeling empty inside” nature of Western Civilization. I think both shows have an interesting viewpoint: that you can either take responsibility for yourself and your place in the world around you, or you can deny it. No matter which choice you make, you can still fuck it up completely.

In an effort to stave off existential anxiety I have been staying up all night watching YouTube videos about space. This guy Isaac Arthur has a large number of really interesting videos that cover some really interesting topics about the science of science fiction. I have been listening to him talk for weeks about gravity wells, and Dyson spheres and thinking about offworld societies. All of this stuff is super interesting, and then I existential bedrock again when I started watching his videos about the Fermi Paradox and the Simulation Hypothesis.

Basically, the Fermi Paradox is this idea that there are so many planets, stars, and galaxies in the universe that sheer probability says there are other planets capable of supporting life. So why are there no aliens?

Once I started thinking about the Fermi Paradox, it didn’t take long to start applying the logic to all sorts of things. At first I started thinking about this for other fantasy technologies, like time travel. No one has come from the future to stop catastrophes, so perhaps time travel just isn’t possible, or perhaps human life on earth is extinguished before time travel can be developed.

Then I hit upon the Simulation Hypothesis. Which is that our reality could just be an elaborate simulation. It was at this point that I remembered a New Yorker article about how election night and the Oscars might indicate some sort of breakdown of a simulation. At first I laughed it off, but for at least a couple of years things have been going badly all over (shootings, riots, natural disasters, you name it) and there hasn’t been much, if any hedonic adaptation as a result. Maybe we are living in a simulation, maybe we aren’t, but something certainly seems to have happened to the hedonic treadmill. Or maybe the chronic and constant bullshit that is living for 15+ years in post-9/11 America has taken a toll on everyone’s collective psyche.

I have been unable to write for a while now, and so I figured I would just write about not being able to write. That ended up being this long rambling screed about my frustration with my friends and family over their obtusely two-dimensional socio-political views. It came across really angsty and disjointed. Then all this Nazi bullshit happened and I just kind of retreated again.

Getting into the Nazi thing is kind of a waste of my time. I’ve dealt with Nazis in one form or another for most of my early adult life. I had a few clashes with Nazi skins when I was a teenage punk, and when I went into the military I clashed with a few more in the US and in Germany. Beefing with German skins was weird. What I didn’t realize, was how prevalent these hard right-wing white power types were in the Ohio National Guard. I had a squad mate that was an ex-skin and he and a few like him were treating their Guard service as free training for their race war. They were conservative gun nuts prepping for a “Shit Hitting The Fan” scenario. This was in the late 90’s. I know these dudes didn’t go anywhere, they just got pushed into silence by polite society, and the specter of a global Islamic Jihad deflected the world’s attention from them. I figured that at some point those dudes would get their lives together, but not only did they not, I guess they all had kids?

I spent a lot of time with these dudes; I got to know them. One guy hated blacks because he was divorced and his wife was dating black dudes. Another was raised to be that way by his father who was a blue collar guy who got laid off in some recession and he blamed immigrants. All of it just sounded like fear and weakness to me. These guys hate people of color because people of color have the power to make them feel inferior. Good job Master Race! You’re doing a great job! It’s no different than those “nice guys” that hate women because women have the power to make them feel lonely and pathetic. It’s ironic when you think about it, white supremacy threatens the lives of so many, yet all it does is protect the egos of a few. Also, these dudes go on and on about liberty, but they are the worst sort of authoritarian apologists.

Maybe I am lucky to have more than one identity, or I’m just lucky that my identity isn’t threatened by women or minorities. Being a geek is a pretty white identity, but it’s not like John Boyega, Gal Gadot, or Rochelle are going to take that away from me. Glenn from The Walking Dead can kill all the zombies and bed all the white women he wants, it doesn’t affect me in any way. The same goes for female Ghostbusters, Daisy Ridley or Imperator Furiosa.

One thing that this new Nazi bullshit did was cause more debate about the First Amendment. The Intercept was nice enough to illustrate my point about how controlling hate speech ends up suppressing progressives, which confirms my bias on these things and helps me feel a little vindicated.

The other thing about Nazi bullshit that pisses me off is that it takes so much Nazi related self-aware/ironic humor off the table. You can’t deliberately invoke Godwin’s Law anymore. You can’t dismiss Xbros as “Filthy Console Peasants”. I have to work so much harder to be simultaneously pretentious and fake-edgy. It’s BULLSHIT.

What disturbs me the most about all this Nazi bullshit isn’t the presence of White Power. You can’t police thought, so you can’t eliminate hate. It’s the lack of conscience on display from leaders of all kinds. Of course the President had nothing to say, those Fashy creeps put him into office, but what about basically everyone else? A bunch of corporations took a stand, and that is probably the worst possible result. They’re Nazis For Fuck’s Sake. They are literally the worst human beings there are. The USA fought a war against them. They’re the definition of an easy target. There will never be any collateral damage when you take a swing at them. They are literally Hitler, literally. When the organizations whose General Counsel advises them against any course of action, any course of action that a corporation ends up taking is the absolute least that could be done. Where in the fuck is everyone else?

My lamentations about corporate power also seem to have been vindicated a little, again thanks to The Intercept. I get that we as a country have lost faith in the political process, but corporate oligarchy is not the answer. Let’s not forget that no one served any time for crashing the economy in 2008. Let’s not forget that everyone looked the other way in the interest of keeping “the system” stable. What else are we going to look the other way on? Using a private army to violate the civil liberties of a group of indigenous people? Those people aren’t white, BTW. Just thought somebody should know that.

chris@chrizzle23.com

Husband, Father, Veteran, cypher punk, hacker spacer, gamer, lover of privacy, free speech, and filthy scumm pirates. My opinions are my own and do not reflect those of hive13, Cinci2600, or my current employer.