Category: Coding

My home network has been having issues. Mostly it seems between the chromecast packet flooding bug and something to do with WiFi + android 8.1, but things will drop connections. I’ve eventually had to make one WiFi for my important stuff, and one for the random gadgets (which I probably should have done anyways). I’m not certain either of these things were happening, but they seemed to line up.

But on top of all that I wanted to try moving DNS off of my router and onto something I had more control over. I had heard of Pi-hole as a solution for system level ad-blocking, I was mostly hoping it would help my phone cause ads on mobile webpages really suck cause of load jumping around the page, I can generally ignore ads the rest of the time.

Docker has the usual advantage of things working out of the box. No configuring and everything because someone else did it for me.

So off I go to find an install of Pi-hole that works, and I can poke around with. It didn’t take long. https://github.com/diginc/docker-pi-hole seems to work really well. Installed it, looked pretty good. Restarted it with ports mapped so I could play with it. Still success. DNS seemed fast and zippie. Fully usable.

But I wanted more. I was reading about dns-crypt, and had heard it could encrypt your DNS requests so your ISP and such couldn’t actually track what you were doing (Not that I wanted to hide, but I liked the idea of it).

So off I go. I learn about dnscrypt-proxy, and quickly found a nice docker image. https://hub.docker.com/r/rix1337/docker-dnscrypt/

So off I go, seems pretty easy to set up. Just download, run, and point at the local proxy (there’s a list on the docker hub page).

Nope, not that simple. Cause silly me, it needs port 53 as well. Okay, no problem, let me use another port and tell pihole to use that. hrm.. nope, the runtime configuration thingie eats up the ‘#’ so I can’t specify port like you can in the dnsmasq config that pihole uses. Okay. Okay, lets try a ip address alias. That seems to work, so pihole takes the main ip, and dnscrypt takes an alias? Sweet! I can manually query things on it, time to hook everything up together.

Hrm. Nope, wall again. Apparently my docker setup can’t talk to anything but the main ip. I’m guessing its firewalld which I’m hoping to get rid of once I reinstall my system. Okay, what else can I try now?

After a bunch of reading online, I found out you create a docker network, and the various services can talk to eacho ther without needing to expose ports out to the rest of the network. That sounds perfect. Oh, wait, you need to resolve the addresses inside the containers, which totally won’t work for dns because dns wants the ip so it can resolve. Close, I mean it would probably work because docker has its own dns proxy, but again you can’t pass non ips to the pihole runtime configs. Okay whats next.

I really wasn’t sure this would actually work because in theory ips could change every time it starts up, but it seems to allocate the same ip if possible, so kinda lucked out. So now I had Pi-hole talking to dnscrypt-proxy, which meant my lookups were encrypted. Yay!

Okay, whats next? Next I want to get dnssec working again. Not the end of the world for Canada. Our government and ISP are not supposed to mess with dns results, but I wanted it anyways. Plus its nice to have when the time comes.

Oh Awesome. Pi-hole has a option for it. Time to enable it.

Enabled, success. Time to walk away.

Oh wait, things are failing. Why are they failing?

Long story short, the version of Debian that was bundled with the Pi-hole docker image was super old. So the version of Dnsmasq was super old. It wouldn’t handle any cloudflare based dns requests that had dnssec enabled (which my domain does). Okay, now what? Started to dig into how the docker image was built. Looks like it actually wasn’t that hard to get it running with latest stable instead of the old stable. Between the work I did, and a different PR the author did, we managed to get it upgraded to Debian stretch that afternoon. I tried the latest build and success, everything was resolving again. Time to walk away right?

Wrong. Suddenly I started getting all these cron errors about mirrors.fedoraproject.org not resolving. Turns out Dnsmasq also had an issue with the certs for that domain. Okay, disable dnssec and start researching again. Turns out again Dnsmasq had a new – newer version that had it fixed, but wasn’t in Debian stretch. Turned out actually to be a pretty easy fix. I had never tried to install a testing package in stable before, but for Dnsmasq that didn’t really have dependencies, it was super easy. And thus my Pi-hole image was born. Sadly it would be nice to have it in the base image. And one day I’ll clean up a patch and get it submitted, but I’m happy to be totally encrypted and verified dns now.

This post turned out to be way more rambly and disconnected than I expected, but I’m very happy with the results. I now have systemd keeping up dnscrypt (primary and backup) and Pi-hole and now have fast stable dns and my phone is no longer randomly disconnecting everything. I’m pretty happy with the results. Plus pretty graphs.

I recently switched teams at Sauce Labs. I used to be the sole person on the Integrations team, and after more than a year feeling pretty isolated, my mental state was slipping pretty hard, so when an opportunity came up to switch departments, I took it.

So now I’m on the IT team, which as a developer is kinda confusing, but its giving me the oppertunity to do some customer facing work, and a lot of internal facing work. I always love working on items that I get to actually see improving peoples work life.

So starting day 0, I had some work to finish off, but I wanted to get to know the team better, and whatnot, so I kept an eye on the internal tickets. Quickly I started to notice that a lot of tickets had to have the initial question of “Whose your manager.” so one day during our team’s huddle, I made the offhand comment that I had enough experience making JIRA plugins, I could probably take the HR data (which I had already mucked with in past projects) and add a little widget to JIRA providing basic info. I felt so exposed, I didn’t want to feel like I was intruding or taking over or anything, but no, as expected the team loved the idea.

Over the next few weeks between whatever else came up, I started working on this little project. I had seen on the atlassian developers community form that someone had made a flask-ac integration. This was perfect. This would give me an excuse to get more familiar with python (which is the main language at Sauce Labs) and even poke around with how packages were made. This module turned out to be pretty hipchat focused, but gave me a bunch of ideas.

I decided I still wanted to learn about packaging and releasing, but also wanted to get something out. So I started to hack the existing system a bit and made an integration. First version was pretty crude. Lots of hard coded stuff, but no credentials. That was my only real goal was to make sure credentials and sensitive items were never hard coded.

First release. Team was estatic. Showed name, phone number, office location and manager. This made thier lives easier because they knew managers and who was local and who wasn’t without the ackward question.

But this wasn’t enough. I really wanted to learn how to package python packages. So I started reading up on how flask plugins were created. Turns out pretty simple actually. So started to refactor a bunch of things. Then I got another jira based project at work. Okay this is perfect. Now I have 2 projects using the shared functionality.

Short story even shorter, I managed to get Flask-AtlassianConnect released. I released it under my own name because I spent mostly after hours polishing it up. Plus I didn’t really think anyone else would want to maintain it. I’m still iffy on that bit, but nobody seemed to mind. So yay! I have a published pip package.

I wasn’t done there. I wanted to get the original JIRA plugin all cleaned up and open sourced. We can’t be the only people out there that use BambooHR and also JIRA. So again, in between tasks I cleaned up the implementation. Now it had a full config screen. Let you pick what to display. Let you choose with projects it would be integrated with. Etc. Now it was ready to go. Plus now I knew how to write tests.

So I’m proud to announce (a little late here though) that the BambooHR + Jira Cloud plugin I wrote is open source. I had so much fun doing it. I hope this new position will lead to many more projects that we can open source for others to use as needed.

I am absolutely ecstatic to announce the new Sauce Labs and HipChat integration being not only released to the public, but open source as well. Its been officially out for a month now, but we just went ahead and open sourced it.

About two months ago now, Atlassian hosted their Atlassian Connect Week out in San Diego. If you do any Atlassian based development, I highly recommend going if you can. It’s so much fun to be surrounded by other developers, and be able to ask the original teams questions when you get stuck.

I went into connect week hoping to get a solution for our problem talking to jira server users behind a firewall. Someone had an amazing solution within the first couple hours for me, and I was able to bang out a working prototype in the by the end of the second day.

So what do I do now? I had most of the week left over. Well at a previous internal sauce labs hackathon, I had already started playing around with a slack integration, but was kinda disappointed by its public APIs, so didn’t really get very far. I got really excited at the earlier talks about hipchat integration to see how far I could get.

It turned out I could get something done pretty quickly. This time I decided to use the atlassian-connect-express framework so I could focus on just implementing features. And what a good choice that was. By the end of the first day, I had test results showing up in chat. By the end of the week, I had screenshots available, test information, even video working. I had a direct connection to some of the developers, so was able to play around with even more features.

Curious how it looks? But don’t really want to install it yet? Checkout this awesome animated gif one of our product team members created.

I’m so absolutely excited for this integration, and on top of that, as someone who loves contributing open source, a great example of a working hipchat integration for everyone to learn from and contribute to.

I really need to remember to post here more often. What have I been up lately? A bunch of traveling for work.

Microsoft Visual Studio Parter summit

Atlassian Connect Week

Jenkins World 2016 (Upcoming)

And the one event I went to as me, not really representing sauce labs

Cascadia 2016

I had a big blast at pretty much all of them. Most of them got me into the tinkering mood again.

Because of Cascadia, I learned about greenkeeper and have since hooked it up to a bunch of my node projects. At times the influx of pull requests to update dependancies is kinda annoying, but it forces me to keep my tests up and running. So far i’ve been pretty happy with it.

At connect week, I learned about this codegiest competition/hackathon they were hosting. I already had 2 entries due to work on some projects at sauce labs. But I really wanted to do something fun and silly. So I revisited the dance party plugin Jlipps created for our internal hubot. It gave me an excuse to use a bunch of the hipchat apis that I wouldn’t normally use, and was able to create Hipchat Dance Party. Totally available for free in the market place.

Next up would be hubot-jenkins-notifier. I’ll admit, I didn’t fully understand the bug that the one user opened, but they were willing to create a pull request. He admittedly didn’t have a lot of time to work on it, but was very receptive to feedback. After over a month of slow back and forth, I decided I was happy enough with it, but I wanted better test coverage of the project as a whole. So chatted with him, came up with a better config schema, merged everything, and started the rewrite. The code is now super cleaned up, nice objects that can easily be tested. No more horrible coffeescript. He’s happy with the result and so am I. As soon as I get a few more real life test users, i’ll be cutting a new release.

codacy-maven-plugin – Due to some of my projects being solo at Sauce Labs, I have had my eye out for various open source tooling to keep an eye on my code for me. I tried out gemnasium for a while, but that wouldn’t support java. Greenkeeper as mentioned above is great, but again, not java. Codecov has been great for reporting coverage, and I’ve used it very liberallly, Then I stumbled onto codacy. Codacy is multiple languages and builds you a report on code quality. In java, it uses findbugs, for javascript, it uses eslint. It also has a whole slew of built in checks and functionality. You can totally check out one of my projects to see some of its output. My only real complaint was that unlike codecov, which was just pip install codecov, for java code coverage, you needed to install another java package manager, then install a script, along the way there was https errors, etc. So decided it was worth it to learn a bit more about how maven goals are created, and just more about maven in general. Codacy-maven-plugin was created. This means with maven which is already installed and used to compile java projects, could also be used to upload coverage reports.

And lastly, one of my old standbys, Infinicatr, my old mobile web project that scrolls through a seemly unlimited number of cat pictures. After cascadia I was re-invigorated to learn more about service works. Its usually between salty stories and infinicatr to play around with “new” web tech. I really wanted to see if I could make infinicatr work offline. It turned out to be really easy. Using a service worker, I was able to returned cached values if flickr was unable to respond for any reason, and live network requests otherwise. This meant that once the first batch of 10 images were fetched, it would always appear to be online and working. I’m pretty happy with this result. Source

My next plan of course is to apply it to the old Salty Stories book engine so it can be properly supported offline, instead of the semi hacked version i’m depending on now.

One of my personal goals for this year is to start getting more comfortable sharing knowledge and talking infront of large crowds. As such, I volunteered to give a quick talk on testing, as its always been something I love doing, and ion the new job I thought it would be pretty appropriate.

This is my second Code and Coffee talk ever, and I can see I’m still pretty nervious giving the talk, but I think other than a few technical glitches (which were thankfully edited out) it went really well.

I have so much fun doing them. I love the fact that people come up to you afterwards and walk to talk about topics.

Totally open for more ideas on what to talk about. So far I’ve done Vagrant and now Testing in general. I’d love more suggestions.

I’ve always loved the idea of self contained environments but coming from doing perl and c/c++ a long time ago, I’ve never been able to really pull it off. It wasn’t until I encountered virtualenv for python that I finally got this working properly. I had encountered rvm’s gemsets before that, but they always seemed finicky at best.

I started to have complex vim configs, and bash rcs that tried to look at what directory you were in / file you were editing, and change configs accordingly. It never worked well. EditorConfig and vim-editorconfig helped a lot with that. No longer had to have really confusing vim configs. I could specify per project editor settings. I can’t wait till its more uniformly adopted.

That left bash configs though. Enter DirEnv. Its solved the second half for me. Makes all those configurations per project. A simple direnv edit . inside your project directory will open up $EDITOR. This lets you specify all kinds of things about that project. Anything you can do in bash you can do here. It works best with env variables but can do other things. I love the layout functions though.

1

layout node

adds node_modules/.bin to your path

1

layout python

Creates a new virtualenv and adds it to your path.
..etc

It makes it easy to work with heroku based apps as well. They do all your configurations as environment variables, its pretty easy to add a bunch of export statements to your direnv and emulate the same thing.

To set the scene. New team has been using capistrano to deploy some of its apps (all internal/behind our firewall). So new POC project comes along. Nothing sensitive but needs to be publicly accessible. I get very excited because this is the first time I’ve setup capistrano from scratch. I follow the getting started guide. Deploy to internal test. No problem there. Go to deploy to the amazon box (Go Free Tier) and fail… Riiight, can’t talk to our git server.

So spend a while scouring the internet for how to solve this problem. Can I just SCP? Its not that important right now. Hrm. That doesn’t seem to work reliably. Lots of solutions, none of which seem very clear and/or work with cap3. So I give up searching. This should be easy right? Just setup a remote tunnel. I open a new window, create tunnel, edit my config, and bang. Deployed. Yay!

But wait, this is going to get annoying fast. Plus others will have to do that manual process as well. How do I automate this? Scour the internet for a while. Again, no real automation and/or doesn’t work with cap3.

Well fine, I always need the excuse to learn more about ruby, so start digging into the code. Its using pre-existing open source modules so it shouldn’t be that hard. After a bunch of learning, and giving up, I decided to skip on the nice clean solution and just start overriding some things.

Yet Another Online Interaction Role Playing Game was a project a group of us at BCIT put together. It was for some of internet studies course (Yea I’m that old / It hadn’t been updated yet). Evan, the guy who named it, was also someone who spent a lot of time online and really enjoyed the idea of an acronym that people probably wouldn’t get.

So YAOIRPG was born. We had to do lots of paperwork. Project Docs, Design Docs, ETC ETC. But we found everyone loved playing it. I actually got killed several times while I was trying to demo the game to the class. It was fully multiplayer. Not bad for a school project. We were not expected to finish it, just document the process. There was only one other team that manged to get something demoable (I think), and they had a simple app that transferred files.

Anyways, it came up in conversation recently, mostly because I still tell the story that the teacher was heard saying he loved yaoi and we in the know at the time were so amused by this.

I realized the code wasn’t backed up, so I dug through some of my old hard drives, found the code and an old DB dump and this morning through a lot of trial and error (And actually minimal work) got it running in a recent version of php and mysql.

http://yaoirpg.halkeye.net now lives on heroku where it should forever run. Backed up to bitbucket ’cause I’m pretty ashamed of the code :D

Posts navigation

I'm a software developer at the awesome Sauce Labs. I do a bunch of other random open source development. I also play games, both board and video games and love to read.
You can usually find me on various services as halkeye