Life and experiences of a fresh Computer Science grad.

The Situation

Using Ansible 2.2 or older, I need to create a bash script using the template module[1] that will modify a CentOS image. Specifically, I need to conditionally create a new user on the image before injecting ssh keys if we will not be using the root user on that image. This condition will be determined by the value associated with a supplemental_user variable.

The Problem:

As you may have noticed above, the newline trailing our conditional is being absorbed when the template is rendered. A quick review of Jinja template docs[1] tells us that this is expected behavior, ‘If an application configures Jinja to trim_blocks, the first newline after a template tag is removed automatically (like in PHP).’

Template Two

The trim_blocks issue is easily resolved with Jinja by prepending the conditinoal, ‘You can manually disable the lstrip_blocks behavior by putting a plus sign (+) at the start of a block.’

Output From Template Two:

The Next Problem:

This is unexpected, we followed Jinja’s docs, right? A quick look at the docs for Ansible’s template module[2] tell us that we need to set the `trim_blocks` parameter to false when calling the template module. Unfortunately, this feature was added in Ansible 2.3.

Template Three

What do you do if your library is pinned to an earlier release of Ansible? You use an expression to interpret python directly. However if you have lots of special characters, like we do here in this complicated bash template, you will have to be very careful when you form your expression.

UPDATE: 24-May-2017

The three reviews[3] mentioned in the original post have merged and this feature is now part of OOOQ/OOOQ-E. I have updated the deployment script[10] on GitHub for your convenience.

Original Post:

Over the past month, I have been working with TripleO-Quickstart (OOOQ)[1] and TripleO-Quickstart-Extras (OOOQ-E)[2] to automate the deployment of a FreeIPA server and enable TLS Everywhere. There three reviews[3] on review.openstack.org that comprise the majority of my work. All of this is coming to a head after many, many months of work from my teammates Ozz[4], Rob[5], and Ade[6] who put together all the leg work in relevant libraries and communities. Without their effort and headaches, this wouldn’t be possible.

Automating this process is a big deal. Once this it is streamlined this will allow for greatly expanding the security and compliance coverage throughout OpenStack CI. Needless to say, I am pretty excited to be working on this topic. Without any further ado, here is the present workflow to conduct a deployment:

Assumptions:

One machine is set up[7] as the controller from which you will call OOOQ to conduct the deployment. I am deploying from a ThinkPad T450S running Fedora 24.

Once machine, referred to as the virthost, 1 quad core CPU, 24GB of memory, and 160 GB of free space. Running with fewer resources is possible but not tested, and will likely result in difficult errors difficult to diagnose.

Preparing Our Environment To deploy:

The first thing we need to do is pull down the OOOQ and OOOQ-E repositories.

Now, let’s go ahead and create a special working directory to house our deployment data, artifacts, and keep our modified repos clean in the process.

export WORKING_DIR=~/.quickstart-freeipa

mkdir $WORKING_DIR

cp -rf ~/git/tripleo-quickstart $WORKING_DIR

At this point we are ready to deploy. The only thing we have to do is ensure that we can ssh into our virthost from the controller as root. This is a requirement for Ansible.

Deployment:

cd ~/git/tripleo-quickstart

export VIRTHOST=<your virthost’s IP or FQDN>

bash quickstart.sh \

–bootstrap \

–ansible-debug \

–no-clone \

–playbook quickstart-extras.yml \

–working-dir $WORKING_DIR \

–release master \

–config $WORKING_DIR/config/general_config/ipa.yml \

–nodes $WORKING_DIR/config/nodes/1ctlr_1comp_1supp.yml \

–tags “all” \

$VIRTHOST

After calling quickstart.sh, you will see OOOQ take over and begin installing all the necessary libraries for deployment. After completing it’s environment preparations, it will call the Ansible playbook, quickstart-extras.yml[7]. Pay close attention to the two config files we are explicitly using, ipa.yml[8] and 1ctlr_1comp_1supp.yml[9]. These two configuration files specify how OOOQ deploys a single compute, control, and supplemental node (housing the FreeIPA server), as well as sets all of the various flags needed for the FreeIPA server and enabling TLS Everywhere.

The deployment itself should take around two hours to complete. While we are adding some additional time fetching the Centos image, provisioning the supplemental node, and deploying the FreeIPA server itself — the majority of the deployment time still lands on installing the undercloud and deploying the overcloud.

After deployment has completed — you have full access to the undercloud:

ssh -F $WORKING_DIR/ssh.config.ansible undercloud

as well as the supplemental node which is running the FreeIPA server:

ssh -F $WORKING_DIR/ssh.config.ansible supplemental

Please note that a log of the FreeIPA server deployment can be located on the supplemental node at ~/deploy_freeipa.log. The IPA server admin password can be located in either the Ansible logs (look for freeipa_admin_password) or in the deployment script ~/deploy_freeipa.sh (look for CA_ADMIN_PASSWORD).

Afterthoughts:

Once the reviews[3] have merged, this process will be come much simpler. In the meantime, I have a created a simple bash script[10] which automates all of the above steps requiring only a single parameter, the virthost.

Have you ever needed to specify which ssh config file rsync should use when called from Ansible’s synchronize module? I have and it was not obvious how to do so.

At work we use Ansible to deploy OpenStack in our CI infrastructure. The playbooks my work (mostly) revolve around follow TripleO. The resulting deployment will create a custom ssh.config.ansible file that specifies how to properly tunnel from the baremetal machine OpenStack was installed on to a virtual undercloud host and several virtual overcloud hosts residing on it. This config is explicitly called in our ansible.cfg. But, this is very important information is not transferred over when Ansible executes the synchronize module. The resulting call will raise an error that looks similar to:

During the summer of 2014 I worked on the OpenStack Keystone component while interning at Red Hat. Fast forward to the end of October 2015 and I once again find myself working on OpenStack for Red Hat — this time on the RDO[1] Continuous Integration (CI) team. Since re-joining Red Hat I’ve developed a whole new level of respect not only for the wide breadth of knowledge required to work on this team but for deploying OpenStack in general.

The list of deployment options for OpenStack[2a,b,c,d] is long and has a colorful history. Furthermore, there are probably only a handful of people who have developed and or used more than any given handful. I have the fortune of working with, or at least within the proximity of, several of these folks. However, even with that advantage, I find wrapping my head around all of the moving parts involved in deploying OpenStack confusing. This was a prime source of frustrations during my first several weeks back at Red Hat. Routinely I found myself having to accept the magic of a given deployment tool in order move forward with my tasks.

Presently, the RDO CI team uses Khaleesi[3][4], and by proxy Ansible, as one of the deployment tools for automating builds with Jenkins[5]. During this walkthrough I will follow the deployment steps as outlined in Khaleesi’s cookbook[6] to deploy OpenStack using RDO-Manager[7] on a single, baremetal CentOS 7.2 server.

Assumptions:

One machine is set up[8] as the controller from which you will generate the necessary Ansible configuration and execute the appropriate playbooks within Khaleesi. In my case this is a ThinkPad X1 running Fedora 22 — my work laptop.

Note: Make sure you follow all of the steps in the Khaleesi setup guide on the controller or you will run into problems when trying to use ksgen or Ansible.

One machine with a minimum of 1 quad core CPU, 12 GB of memory, and 120 GB of free space, as outlined by the RDO-Manager docs[9], that is running a clean install of CentOS 7.2

Note: Because our installer, RDO-Manager, is based off of TripleO[2d] we do in fact need to use a baremetal machine for this. I couldn’t find a detailed explanation specifying the exact reasons anywhere but the limitation is noted in both the TripleO docs[c] as well as the Khaleesi docs[10]

My setup meeting the two above requirements looks looks like this:

Configuration:

Okay, let us move to the actual setup. We will be picking up at the Configuration[11] portion of the Khaleesi cookbook. Remember that all of these commands are to be run from the machine you installed Khaleesi on — the Thinkpad X1 in my case.

We begin by copying over the Ansible config in version control, with some defaults needed across Khaleesi use cases, and then telling Ansible to use the config file that will be generated by Khaleesi, ssh.config.ansible

ssh-copy-id allows us to easily transfer your ssh keys to the CentOS box. This tool has quickly become one of my gotos as I am constantly provisioning systems and removes many of the possible human errors involved in key transfers.

The playbook we will end up calling, khaleesi/playbooks/full-job-no-test.yml will expect that the TEST_MACHINE environment variable has been set and will use it while generating the hosts file used by Ansible.

Note: If you see warnings similar to the line directly below, don’t worry. There is a set of defaults and it is simply informing you which it will be using if no respective parameter was handed to it when called.

Remember that tool we installed from within the Khaleesi repository? That was ksgen, a tool that generates a file, ksgen_settings.yml in our case, which contains most of the variables used by Ansible during the execution of Khaleesi’s playbooks. The parameters above line up with files underneath khaleesi/settings and pull in the variables respectively while magically handling any conflicts that may arise. For example, `–provisioner=manual` will include all variables located within khaleesi/settings/provisioner/manual.yml as well as khaleesi/settings/provisioner/common/common.yml as indicated by the include statement at the top of the aforementioned manual.yml.

This is a pretty basic setup. A few of the parameters are of particular note, namely:

--provisioner=manual

We have provisioned the CentOS box ourselves as opposed to using say Beaker or Foreman (both of which are supported provisioners by Khaleesi)

--installer=rdo_manager

RDO-Manager is our tool of choice here for the actual installation of OpenStack on our CentOS box.

--installer-env=virt_host

Our undercloud/overcloud deployment will be installed on virtual machines running on the CentOS box, TEST_MACHINE. Accordingly, Khaleesi will need to use respective virthost playbooks — as opposed to baremetal playbooks were we to install our nodes on actual boxes.

--extra-vars @../khaleesi-settings/…

Once upon a time all of the settings files that reside underneath khaleesi/settings lived in another repository aptly named khaleesi-settings. It still exists, we use it internally for storing sensitive data needed for our CI infrastructure that we wouldn’t want public, and it retains some things like the virtual networking settings needed for ml2-vxlan argument passed to ksgen. Why exactly does khaleesi-settings still exist upstream? To be frank, I’m not quite sure but I’ll update this post when I have a rational answer.

The result of calling ksgen is a concise YAML file, ksgen_settings.yml — you can rename it whatever you want just be sure to pass it to your ansible-playbook calls accordingly. This file is infinitely useful and will quickly become your best friend whenever you have to troubleshoot failures with Ansible.

Deployment:

Now we are ready to call Khaleesi’s playbook khaleesi/playbook/full-job-no-test.yml which will provision TEST_MACHINE, which is minimal in our case as we’ve already manually done so, and then use RDO-Manager to deploy an undercloud and overcloud in virtual machines that are hosted on our CentOS box.

If Ansible doesn’t throw an error within the first 10 seconds, indicating something is likely messed up in either you Ansible config file or in ksgen_settings.yml, feel free to go and stretch your legs as these playbooks can take awhile to finish up. The console output at the end of the playbooks execution should look similar to:

Final Thoughts:

Ansible, and Khaleesi, make it very easy to deploy OpenStack in a reproducible manner — if you have everything configured correctly beforehand. The vast majority of time I spend fixing problems while working with Khaleesi come down to mistakes related to configurations.

From the 40 or so lines we’ve entered into our shells an enormous number of subsequent actions have taken place through Khaleesi’s playbooks. I could spend days diving into each one of them. I’m sure I will eventually but it’s nice to know that I can do so as time permits me to do so thanks to Khaleesi and Ansible.

Things I’d like to write more about in the future:

A more in depth breakdown of what is happening in each of the playbooks used during this deployment — or a deployment of a similar nature.

Last night I received a neat little email from the University of North Carolina at Greensboro. I’ll leave out the verbose output from legal and finance and get to the fun bits that in short read:

Dear Harry

On behalf of the University Registrars Office, it is my pleasure to inform you that we have completed the final processing of your academic record.

Congratulations, all degree requirements have been met as of the posted graduation date,December 10th, 2015.

What a thrill it is to know that I have finally completed that damned degree. It was about a year ago that I decided to not return to school in January for the lone remaining class I required, Biology 101 (why is it always a gen. ed. class?). Instead I rode off to the great, mountainous North that is Charlottesville, Virginia.

Living in what will possibly be my favorite apartment ever, I worked as a web application developer for a small non-profit that I had previously interned with — CoS (the Center for Open Science). To them and in particular Dr. Jeffrey Spies and Dr. Joshua Carp I will be forever grateful. The slightly less than a year that I spent in that cozy, gorgeous city was one of great professional and personal development. But all states in life, aside from death I suppose, are transitionary and in October I moved back down to the warm and humid North Carolina to what has been one of my favorite cities of all time — Raleigh.

Too me Raleigh is bikes, beer, coding, music, and friends. More than that, Raleigh is the perfect blending of them in an active city that is neither too large nor too small. It is, at least for now, a good base of operations and one which I want to leave better off than I found it. I don’t know how long I’ll be here. I don’t know how long I’ll be at Red hat. But, I do know that I’m happy to have completed my undergraduate degree, be actively contributing to the open source community professionally, and living in a city that feels like home.

Have you ever tried to test POST requests to an API written with Flask? Today I was going through an old code base writing unittests that should have been written eons ago when I ran into a snag. The issue stemmed when I tried manually setting the headers Content-Type.

But before we get to that, let me show you the skeleton of the Flask route I was testing:

A quick glance at the code and I realize I’ve forgotten to set the headers Content-Type! Wait. How do I set the Content-Type? That’s a good question. After searching around for people who had run into similar problems this is what I came up with:

Hmmm, okay. Next, I decided to inspect the request coming into the flask app and found something odd in request.headers:

Host: localhost
Content-Length: 10
Content-Type:

Why is the Content-Type empty? Another search gave hinted at another possible solution. Why not just build the headers dict inline?

headers=dict(content-type='application/json') # But that's not right. We can't have '-' in a key.

By this point I’ve become agitated. Neither the Flask docs themselves nor various forums have been of much use. Then I stumbled across this.

Perhaps I missed something in the docs. Either way, I learned that you can hand the Content-Type as a parameter to the post method. It works and it looks much cleaner. Let’s revise my initial unittest accordinlgy:

Well, this has been in the works for awhile. A couple of days ago, the Center for Open Science announced the public beta release of SciNet on twitter and Facebook. I’m not going to lie, I’m pretty excited about this. Countless hours have been spent learning new technologies, testing ideas, and getting the infrastructure established for the project.

What is OSF SciNet?

To summarize, SciNet is an open source project whose goal is to crowdsource the collection, standardization, and release of a dataset containing scientific citations and their corresponding references.

Why is this important?

While many researches have conducted studies of the citation network, their efforts have been hindered by small or inadequate datasets. Services like PubMed have made research easier in the biomedical science domain, but no such solution exists that cuts across all areas of science in a fully open and transparent way. The dataset generated through this project will make it easier to see the connections in the scientific literature and to promote open science.

This is a huge task.

But, I have high hopes for the project. Since the release over 100 new users have downloaded Citelet, a Chrome extension that allows users to passively contribute to the project as the browse articles. I have to admit it feels pretty awesome to see people using our tools and watching the number of articles in our dataset increase each day. As I write this we are almost up to 112,000 articles!

Want to learn more?

If you’d like to learn more about the project, how to contribute to it, or even help develop future tools check out some of the following links:

Currently, I am attempting to trim my CV down to a length appropriate for a resume. To be honest, it’s not as easy as I would have thought. Wouldn’t it be more simple if a resume could just simply read the following:

Favorite language: Python. Why? It’s awesome and so is the community it’s built up around it.

Favorite IDE: None. I like Vim (shhhh emacs lovers). Again, why? It’s quick, powerful, and I can use it on any of my machines or while I’m remoted into some server. I value consistency.

Favorite OS: Hands down, any of the Linux derivatives. Does this require justification? Nope.

Favorite place to code: Anywhere with a nice view and lots of natural light.

Anything other questions? Let’s discuss this over a coffee or tea. And don’t forget to checkout my Github account!

Earlier today, while reading over Interactive Data Visualization for the Web, I came across an interesting fact I would like to share. We all know the preamble for web addresses, http://. I couldn’t fathom how many times I’ve typed it myself. Apparently the creator of the world wide web, Tim Berners-Lee, regrets making this part of the standard as he stated during an interview with the New York Times. To summarize, he equated it to be a convention of programmers at the time.

I wonder how many double slashes have been typed, printed, and read since the standards creation. But hey, there is always room for improvement, right?

As an aside, I highly recommend the aforementioned book, Interactive Data Visualization for the Web. Written by Scott Murray, it provides an excellent introduction to data visualization with D3 and basic web programming. What makes this even more cool? O’reilly is offering an online version of the book that includes interactive examples and it’s free! Click the previous link to view the book directly or visit this O’reilly page for more information on about the paper version!

America’s Generation Y can be divided into two distinct groups: Those who served in Iraq and/or Afghanistan, such as myself, and those who didn’t. Taking an educated guess, I assume a lion’s share of the readership of Thought Catalog are liberal arts degree bearing, student-loan debt ridden types who think those who joined the military were too stupid to go to college and were unaware cogs in the political war machine run by evil multi-national corporations with the goal of maximizing profit and exploiting the lower class. In turn, we think you’re a bunch of overly sensitive, pretentious, hyper-liberal pussies, so its even. Now, let’s begin to gain an understanding of each other’s perspective.

Our memories of our formative years are quite different. You headed out into early adulthood going to community college or university, be it full-time or part-time. You may have gotten a student loan, a scholarship, paid…