Archive for the 'english' Category

I recently came across the question how to handle SSH keys with Vagrant and Ansible.

Vagrant

Traditionally all Vagrant boxes and VMs require a fixed login (vagrant) with a fixed SSH key. This was very convenient in a development context, but could raise security issues in case Vagrant VMs were used for anything important and users were not aware of the insecure key.

In current Vagrant versions only boxes (i. e. base images) use the insecure key. When Vagrant starts a new VM it generates a new individual SSH key for this instance. Vagrant keeps these custom keys in the .vagrant/machines subdirectory; so host to guest logins (like vagrant ssh) are still possible.

To prevent this mechanism one can configure the VM with config.ssh.insert_key = False. This is necessary when modifying a box. Say you want to take a bento base box, install your development tools, and then repackage the VM as your own development box in order to distribute it to a development team. — With the default behaviour (config.ssh.insert_key = True) every repackage build would generate a new basebox with an individual ssh keypair; this makes the new boxes practically unusable, because every user would have to get and configure the custom SSH key for each box. With config.ssh.insert_key = False the box will retain the previous key; that means you users have the same experience as with a normal box.

Note: As a compromise between the two modes one can set a custom ssh key for all baseboxes with config.ssh.private_key_path. This might be useful for companies with many internal boxes. In this case one distribute one SSH key to use with all Vagrant boxes but not use the publicly known insecure standard key.

Ansible

If all VMs use the same SSH key the setup is straightforward: Configure all Vagrant VMs with config.ssh.insert_key = False or config.ssh.private_key_path and use that key for the Ansible login.

With Vagrant’s default behaviour there is no common SSH key for all VMs. In this case one has to configure Ansible to use the right keys for every VM, but the setup is still simple as long as you have the default /vagrant mount with the .vagrant subdirectory.

This subdirectory contains all Vagrant state and contains these files, right now the private_key is the important one:

I usually run one Vagrant VM as an Ansible controller (with node.vm.provision :ansible_local) and then 1 to N other VMs as installation targets. To enable access from the controller to the other VMs I include this line in the Ansible inventory: ansible_ssh_private_key_file=/vagrant/.vagrant/machines/{{ inventory_hostname }}/virtualbox/private_key

This should work just the same for running Ansible on your host machine and use it to provision the guest VMs. In this case the path would be a relative one: .vagrant/machines/....

The only important detail is the consistent naming of the VM in Vagrant and Ansible. With the ansible_ssh_private_key_file as above Vagrant’s VM name (set with config.vm.define) and Ansible’s inventory_hostname have to be the same. They are not required to match the guest’s hostname (config.vm.hostname), but I strongly recommend to either use the same value or use an obvious mapping. I usually try to use an FQDN for the hostname, and then use the short hostname (the first component) as the VM name and inventory name.

The only important detail is a consistent naming between Vagrant’s VM name (as set with config.vm.define), the VM’s hostname

Multi-Cloud Is a Trap – Brave New GeekIt comes up in a lot of conversations with clients. We want to be cloud-agnostic. We need to avoid vendor lock-in. We want to be able to shift workloads seamlessly between cloud providers. Let me say it again: multi-cloud is a trap.

The Private Cloud Has Failed Us – Chuck’s BlogPerhaps there is no deeper disappointment in life than when a cherished concept fails to produce the desired results. Such is the case with the industry’s notion of private clouds. I’m throwing in the towel, walking away – and cursing under my breath. It’s a failed concept.

What is Code?Software has been around since the 1940s. Which means that people have been faking their way through meetings about software, and the code that builds it, for generations.

Learning BASIC Like It’s 1983In 1983, though, home computers were unsophisticated enough that a diligent person could learn how a particular computer worked through and through. That person is today probably less mystified than I am by all the abstractions that modern operating systems pile on top of the hardware.

How Lisp Became God’s Own Programming LanguageLisp transcends the utilitarian criteria used to judge other languages, because the median programmer has never used Lisp to build anything practical and probably never will, yet the reverence for Lisp runs so deep that Lisp is often ascribed mystical properties.

C Portability Lessons from Weird MachinesIn this article we’ll go on a journey from 4-bit microcontrollers to room-sized mainframes and learn how porting C to each of them helped people separate the essence of the language from the environment of its birth.

Your app is an onion: Why software projects spiral out of controlYou start with the best of intentions. You hire a developer to build out your startup idea. But almost every week, it feels like the project needs tweaking. Features start creeping in, and the scope slowly expands. It’s as if the project has a life of its own, and is trying to destroy your life.

Why Are Enterprises So Slow? – zwischenzugsIn this article I want to explain a few things about enterprises and their software, based on my experiences, and also describe what things need to be in place to make change come about.

lenazun/working-remotelyWorking remotely sounds great. We think we’ll save ourselves the commute and we’ll be able to flexibly weave in and out of work and home life. In reality, work takes a different shape when there is no office, and we’re all in different environments trying to connect to other humans. […] These are some of the things we’ve learned while working remotely

I recently updated my small mailserver and finally configured DKIM. But another change was easier and still had more impact: installing postwhite. This little tool takes a list of mail domains, then uses their SPF records to derive a list of their outgoing mail servers, then writes this list into a postscreen whitelist configuration. The current default setting contains 43 domains and generates a whitelist with nearly 2000 lines (each containing an IP or subnet). Everything is nicely scripted and can run as a nightly cronjob.

This setup eliminates my biggest problem with greylisting, which is Office356. Their combination of long email resubmit intervals and using multiple cluster servers for delivery attemps always lead to long delays before I received email from Microsoft or any company using Office356. (BTW, I really like greylisting but this is its biggest design problem: it works for single SMTP servers and enforces certain behaviour, but does not and can not consider clusters.)

Revisiting Using Docker, by Gregory SzorcWhen you look at all the options for running containers in 2018, I think it is obvious that Docker – usable though it may be – is not ideal for a significant number of container use cases.

Where Vim Came FromAnd yet Vim is also a mystery. […] Despite its ubiquity and importance, there doesn’t seem to be any kind of committee or organization that makes decisions about Vim.

Linux Load Averages: Solving the MysteryLoad averages are an industry-critical metric – my company spends millions auto-scaling cloud instances based on them and other metrics – but on Linux there’s some mystery around them.

Living on the PlateauMost of us work in an environment of virtual infinities. There may be some contraints out there somewhere, but most of us know we will never bump into them. That’s what happens when you ride Moore’s trains through twenty-eight doublings.

For some reason I wanted to improve performance of a small data driven Python program and tried to parallelize it. These are a few learnings to keep around for next time.

I nearly started with the very basics, defining my own threads as well as task and result queues. But then I found the very useful concurrent.futures module which provides a high-level interface to distribute tasks to both threads and processes.

I still made the mistake to start with threads. Everything worked nicely and the tool ran along with four worker threads — but every thread received 25% of CPU time and the overall runtime did not improve. I realized I had forgotten about Python’s Global Interpreter Lock (GIL).The GIL basically prevents performance improvement using multithreading (at least of CPU-bound tasks, it is still useful for I/O). More information about the GIL:

So I had to switch to multiprocessing instead. The switch itself is really easy because it is nearly completely hidden inside concurrent.futures, I only had to replace the initialization of the ThreadPoolExecutor() with a ProcessPoolExecutor().

But with multiple processes I can no longer share variable values. Everything, including the called function itself, has to be pickled and send to the subprocess.This required some refactoring, as I had to move the function to the module top-level (as local functions cannot be pickled) and then tried to find a good minimal set of parameters and return values in order to reduce the data transfer between the processes.

I saved my code examples for different concurrent.futures invocations as a gist for later reference: mschuett/concurrent.py

Along the way I also tried the asyncio module for “Asynchronous I/O, event loop, coroutines and tasks”. That one is also quite interesting, but as the name suggests it is focussed on I/O and coroutines in a single thread; functions you need for a network server. For my use case it is not useful, because asyncio does not help to utilize a second CPU core.