Docker integration in PyCharm when using Docker for Mac

One thing I love about PyCharm is that it truly is an integrated developement environment in that you can do most of your work from creating virtual environments and installing Python modules to version control without ever leaving the IDE. However, when a crucial part of your stack updates its API or drastically changes how it works, then that tight integration becomes the worst pain in the butt.

The Problem

If you’ve ever used Docker in OS X or shall I say “macOS”, you probably have been using a collection of apps (docker, docker-machine, docker-compose) installed as the Docker Toolbox . Even if macOS shares some heritage with Unix, it doesn’t really have what it takes to run docker natively like Linux. So instead, the actual Docker app runs inside an Ubuntu virtual machine through docker-machine using VirtualBox.

Now the point of containers was to run portable encapsulated systems with very little overhead, but that point becomes moot outside of Linux-based systems. Docker tried to resolve that by using HyperKit, a lightweight virtualization solution built on top a new hypervisor since macOS 10.10 Yosemite (more about that here. This means you won’t need a separate VirtualBox installation and you don’t have to manage Docker machines because all those functions are now found in the Docker for Mac app.

But this is where the problem with PyCharm arises.

As of PyCharm 2016.3, the IDE still uses docker-machine for containers instead of hooking up to the new API of Docker for Mac. The problem is the API for the Docker for Mac virtual machine isn’t in a TCP or HTTP URL anymore but is a socket in /var/tmp/docker.sock.

If you try to be smart and use a unix prefix instead of tcp or http, it will fail. If you try to just use the predefined default, you’ll get an error about some missing TLS certificate. This is because the default certificates folder that PyCharm uses only exists when using docker-machine. Since Docker for Mac doesn’t use docker-machine, that folder doesn’t exist anymore. I have to confess that I caught the latter problem first before I stumbled on the actually cause. That’s probably why it took me quite a while to understand what was really going on.

The Solution

The solution that finally worked for me came from this issue posted in the PyCharm issue tracker. Basically, the solution involves a utility called socat which as a pipe to connect the unix socket of the new Docker for Mac API to the TCP address that PyCharm expects.

Requirements

PyCharm 2016.3

Docker for Mac (Version 1.12.3 (13776) when I was writing this)

Homebrew app (to install socat)

Steps

First, let’s install socat using Homebrew (or your preferred method)

$ brew install socat

With socat installed, let’s launch it and create a pipe with a TCP port on one end and the unix socket to the API on the other.

Now in PyCharm, go to “Preferences” and at the sidebar look for “Build, Execution, Deployment” then “Docker”. Click “+” to add a new Docker machine. On the API URL, type in tcp://localhost:2375. Note that “2375” here is the port we specified in the socat command. If you used a different port number, make sure to use that number in the API URL in PyCharm.

Then, clear the “Certificates folder” field. Click “Apply” to see if you get any errors. And finally, restart PyCharm. Now you can add a remote interpreter using Docker or Docker Compose.

Improving on the solution

One thing you might want to do is to make this procedure automatically launch whenever you log in so you can forget about it. Thankfully, another commenter on the same issue has greated a gistto create a plist to automatically execute this workaround everytime you start your Mac. Below is his code with some comments I inserted to guide you which parts to change.

Copy the code above to a text editor and save it as docker.sock.plist on your Desktop. Go to the LaunchAgents folder by right-clicking the Finder icon and choosing “Go To Folder…”. Type in ~/Library/LaunchAgents/ Now, copy docker.sock.plist from your Desktop to the LaunchAgents folder in Finder. Then, open Terminal and type launchctl load ~/Library/LaunchAgents/docker.sock.plist to execute the plist without restarting.

The PyCharm part is the same as above. Open PyCharm and go to “Preferences” and at the sidebar look for “Build, Execution, Deployment” then “Docker”. Click “+” to add a new Docker machine. On the API URL, type in tcp://localhost:2375. Note that “2375” here is the port we specified in the plist. If you used a different port number, make sure to use that number in the APIURL in PyCharm.

Then, clear the “Certificates folder” field. Then click “Apply” to see if you get any errors. And finally restart PyCharm. Now you can add a remote interpreter using Docker or Docker Compose.

Congratulations! PyCharm (tested on 2016.3) should now be able to connect to the Docker for Mac VM and detect all the containers you have.