The bulk of my coding is actually back end coding and I spent a lot of time dealing with network programming issues. We’ve been running for years in a data center that I put together and every box was finely tuned. Unfortunately due to the massive issues surrounding Chef, we were never able to maintain boxes automatically so every server became a unique snowflake – well set up and such but I have no idea now what changes we made to tune each server over the years.

And while we still have that data center active, we’re now crawling on our AWS boxes and much happier but we’re still working the kinks out. This morning I started seeing this appear:

Cannot assign requested address.

It was appearing in the context of our Redis connections. And, sure enough, we had a routine which was creating a handle to redis every, single, time it was handling a url. That’s bad. So the easy fix was to pass the connection in from a higher level. But even after that it was still an issue. Some quick research brought me to:

How to Ansible-ize This

Given that we have a bunch of AWS nodes setup, I don’t really want to make this change manually so let’s script it with Ansible and run it as a role. Here’s what to do for the directory where your Ansible stuff resides:

mkdir -p roles/machine_setup_tcp_tw_reuse/tasks

touch roles/machine_setup_tcp_tw_reuse/tasks/main.yml

In your main.yml file you want this code:

# this sets it for the machine permanently after the machine restarts
- name: update /etc/rc.local for tcp_tw_reuse (faster tcp recycling) on machines which are servers
lineinfile: dest=/etc/rc.local regexp="^echo 1 > " line="echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse"
# this fixes the current machine state
- name: execute the fix on the currently running machine instance
shell: "echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse"

How to Run this

How to Verify the Fix

To verify this, you can do what I did:

ssh into a box

cat /proc/sys/net/ipv4/tcp_tw_reuse

If you see a 1 there then the fix actually worked.

Notes

The reason for the small level of granularity on this role is that I’m now using Ansible to fix issues on production running hosts. And I wanted to be able to run just this role. Ansible has tagging which I do believe would let me have this embedded within my overall machine_setup task but this felt safer since I’m still a noob at Ansible.