Thoughts and notes on open scientific computing, with a focus on Python-based tools (IPython, numpy, scipy, matplotlib and friends).

Friday, September 7, 2012

Blogging with the IPython notebook

Update (May 2014): Please note that these instructions are outdated. while it is still possible (and in fact easier) to blog with the Notebook, the exact process has changed now that IPython has an official conversion framework. However, Blogger isn't the ideal platform for that (though it can be made to work). If you are interested in using the Notebook as a tool for technical blogging, I recommend looking at Jake van der Plas' Pelican support or Damián Avila's support in Nikola.

Update: made full github repo for blog-as-notebooks, and updated instructions on how to more easily configure everything and use the newest nbconvert for a more streamlined workflow.

Since the notebook was introduced with IPython 0.12, it has proved to be very popular, and we are seeing great adoption of the tool and the underlying file format in research and education. One persistent question we've had since the beginning (even prior to its official release) was whether it would be possible to easily write blog posts using the notebook. The combination of easy editing in markdown with the notebook's ability to contain code, figures and results, makes it an ideal platform for quick authoring of technical documents, so being able to post to a blog is a natural request.

Today, in answering a query about this from a colleague, I decided to try again the status of our conversion pipeline, and I'm happy to report that with a bit of elbow-grease, at least on Blogger things work pretty well!

The purpose of this post is to quickly provide a set of instructions on how I got it to work, and to test things out. Please note: this requires code that isn't quite ready for prime-time and is still under heavy development, so expect some assembly.

Once you have nbconvert installed, you can convert your notebook to Blogger-friendly html with:

nbconvert -f blogger-html your_notebook.ipynb

This will leave two files in your computer, one named your_notebook.html and one named your_noteboook_header.html; it might also create a directory called your_notebook_files if needed for ancillary files. The first file will contain the body of your post and can be pasted wholesale into the Blogger editing area. The second file contains the CSS and Javascript material needed for the notebook to display correctly, you should only need to use this once to configure your blogger setup (see below):

Configuring your Blogger blog to accept notebooks

The notebook uses a lot of custom CSS for formatting input and output, as well as Javascript from MathJax to display mathematical notation. You will need all this CSS and the Javascript calls in your blog's configuration for your notebook-based posts to display correctly:

Once authenticated, go to your blog's overview page by clicking on its title.

Click on templates (left column) and customize using the Advanced options.

Scroll down the middle column until you see an "Add CSS" option.

Copy entire the contents of the _header file into the CSS box.

That's it, and you shouldn't need to do anything else as long as the CSS we use in the notebooks doesn't drastically change. This customization of your blog needs to be done only once.

While you are at it, I recommend you change the width of your blog so that cells have enough space for clean display; in experimenting I found out that the default template was too narrow to properly display code cells, producing a lot of text wrapping that impaired readability. I ended up using a layout with a single column for all blog contents, putting the blog archive at the bottom. Otherwise, if I kept the right sidebar, code cells got too squished in the post area.

I also had problems using some of the fancier templates available from 'Dynamic Views', in that I could never get inline math to render. But sticking to those from the Simple or 'Picture Window' categories worked fine and they still allow for a lot of customization.

Note: if you change blog templates, Blogger does destroy your custom CSS, so you may need to repeat the above steps in that case.

Adding the actual posts

Now, whenever you want to write a new post as a notebook, simply convert the .ipynb file to blogger-html and copy its entire contents to the clipboard. Then go to the 'raw html' view of the post, remove anything Blogger may have put there by default, and paste. You should also click on the 'options' tab (right hand side) and select both Show HTML literally and Use <br> tag, else your paragraph breaks will look all wrong.

That's it!

What can you put in?

I will now add a few bits of code, plots, math, etc, to show which kinds of content can be put in and work out of the box. These are mostly bits copied from our example notebooks so the actual content doesn't matter, I'm just illustrating the kind of content that works.

The notebook, thanks to MathJax, has great LaTeX support, so that you can type inline math $(1,\gamma,\ldots, \infty)$ as well as displayed equations:

$$
e^{i \pi}+1=0
$$

but by loading the sympy extension, it's easy showcase math output from Python computations, where we don't type the math expressions in text, and instead the results of code execution are displayed in mathematical format:

And finally, in the same spirit, the cython magic extension lets you call Cython code directly from the notebook:

In [18]:

%load_extcythonmagic

In [19]:

%%cython-lmfromlibc.mathcimportsinprint'sin(1)=',sin(1)

sin(1)= 0.841470984808

Keep in mind, this is still experimental code!

Hopefully this post shows that the system is already useful to communicate technical content in blog form with a minimal amount of effort. But please note that we're still in heavy development of many of these features, so things are susceptible to changing in the near future. By all means join the IPython dev mailing list if you'd like to participate and help us make IPython a better tool!

13 comments:

This is realy cool. I will be using it for some work stuff soon. It would be great if the nbconvert once mature can be offered as an option in the notebook itself. then you can download it in any format you see fit. PS - the current nbconvert does not offer PDF support as stated in the readme doc? Am I correct or missing something. KEEP UP THE GREAT STUFF!

@David, I haven't tried animations yet. It would depend on how you put them in: if you have them in a video file referenced externally (say youtube) it should work. But obviously if you mean the live animation running in a matplotlib window, that won't work, as that would require running the python code itself in the blog. This is only hosting the static output, nothing more.

It almost works except it does not display math. nbconvert outputs math: \(\chi^2\) in the html file (basic template). This displays as \(\chi^2\). I guess there is some problem with my CSS since with full template it displays properly.

Also nbconvert does not output any header with css. Where should I get it? I just took this http://blog.louic.nl/wp-content/uploads/2013/09/custom.css What is missing there that would make math work?

Hi @matus: things have changed a ton in nbconvert since this post was written. I would encourage you to ask specific questions on our mailing list so that others can assist as well and the replies are visible to the whole community.

Because I hand-configured things over a year ago it still works for me, but one of our devs on the list would be far more able to help you than me on the current state of nbconvert.

Nice post, but it might be a good idea to put a notice at the top saying it's out of date, out of kindness to people like me who might put the effort into learning iPython notebooks for the purpose of blogging, only to discover it's no longer possible. As far as I can tell, there is no longer any way to do what you describe here. Even if there is, there's no step-by-step instructions anywhere, so those of us with limited knowledge of HTML will not be able to do this for the time being.