blog.dexy.ithttp://blog.dexy.it
make | docs | sexyTue, 03 Feb 2015 01:22:08 +0000en-UShourly1https://wordpress.org/?v=4.6.1MATLAB and Dexyhttp://blog.dexy.it/540
Sat, 31 Jan 2015 05:23:24 +0000http://blog.dexy.it/?p=540MATLAB is a programming language and environment which is widely used in many academic and technical fields. This blog post will cover the basics of running MATLAB code using Dexy. If you aren’t familiar with Dexy, then the Getting Started tutorial may help clarify some of the examples in this blog post.

The matlabint filter launches the matlab interpreter and sends commands, one line at a time. It returns a full session transcript including input prompts, commands and output. You can end a line with ; to prevent its output from being echoed (this is a feature of the MATLAB interpreter).

Sections and Syntax Highlighting

The syntax highlighting in this blog post is applied using the pyg filter. The .m file extension used by matlab does not uniquely identify code as matlab code within the Pygments syntax highlighter, so we need to specify the lexer manually. Here’s how:

-.m|pyg:-pyg:{lexer:matlab}

Now here’s the script we used to generate all the examples in the first section, with syntax highlighting applied:

This script is divided into named sections using specially formatted comments. The idio filter knows how to split this script up. We’ll apply this filter before we execute the code, so we can capture each section’s output separately (this concept is covered in the Getting Started dexy tutorial mentioned above).

Here’s the dexy configuration applied to this file:

-sections.m|idio|matlabint|pyg:

The idio filter splits code into sections, then the matlabint filter runs each section through the matlab interpreter, and finally the pyg filter applies syntax highlighting to the output.

The add-new-files setting tells dexy to pick up any generated files and add them to the dexy run so other documents can make use of them, and the additional-doc-filters setting tells dexy to apply the botoup filter to any new files found which use the .png extension. This means that any .png files generated by the script will be uploaded to S3.

And we specify a custom lexer for the pyg filter, this time the matlabsession lexer because we are applying syntax highlighting to the output from the matlab interpreter, rather than a matlab .m file:

-pyg:{lexer:matlabsession}

We also need to specify some other files as dependencies so they will be
available on the local file system when we run the matlab script:

We applied the botoup filter to any .png files which were generated by our script, and this filter uploads files to S3 and then returns their URL:

{{d['mesh.png|botoup']}}

Here is the URL of the generated plot:

https://s3.amazonaws.com/blog-dexy-it-2014/mesh.png

We can then use this URL to generate an image tag using Markdown syntax:

![Mesh Plot]({{d['mesh.png|botoup']}})

Or do the same thing using raw HTML:

<imgtitle="Mesh Plot"src="{{d['mesh.png|botoup']}}"/>

Referencing Individual Values

Sometimes you want to embed calculated values within the text of a document, either instead of or in addition to showing the code which calculated them. One recommended way to do this in Dexy is by writing the data to a JSON file. Here’s one way to do this in MATLAB using JSON functions provided by the jsonlab library:

The values.json file created in this script is detected by the dexy run because we have set add-new-files to true. Here’s the contents of the generated file:

{
"vars": {
"foo": 5,
"bar": 7,
"pi": 3.141592654
}
}

If you reference this file from a document template and try to access the elements within it, Dexy will automatically parse this JSON file into a data structure for you, and thanks to jinja2’s dot syntax, you can refer to elements in a JSON object using either dots (as if the values were attributes) or the more usual python dict accessors:

{%setvars=d["values.json"]["vars"]%}Foo is {{vars.foo}} and bar is {{vars["bar"]}}. Pi is {{vars.pi}}.

Here’s how it looks when this document template is evaluated and the values calculated in the script are embedded within the generated document:

Foo is 5 and bar is 7. Pi is 3.141592654.

Dexy or MATLAB Publish?

Finally, let’s compare using Dexy to using MATLAB’s built-in Publish feature. Both options let you create documents incorporating MATLAB code and the output from running it, but there are many important differences.

Dexy is a more flexible, general-purpose tool that lets you write many different kinds of documents, both in terms of document format and in terms of the content and intent of a document. With Dexy you can write a report showing just the values resulting from a calculation and generated graphs, like this:

Foo is 5 and bar is 7. Pi is 3.141592654.

Or you can write some documentation which includes a section of unprocessed source code:

With Dexy you can mix and match source code from many different MATLAB files and even include and execute code in other languages.

MATLAB’s Publish feature is much more limited, but it has the advantage of being simpler and not requiring you to install any additional software if you already have MATLAB installed.

Here’s an example of some MATLAB source code with comments written for the Publish command:

%% Hello% This is an example document.%% Patients%%% Create a table from individual workspace variables.loadpatientspatients=table(LastName,Gender,Age,Height,Weight,Smoker,Systolic,Diastolic);%%% Select the rows for patients who smoke, and a subset of the variables.smokers=patients(patients.Smoker==true,{'LastName''Gender''Systolic''Diastolic'})%%% Convert the two blood pressure variables into a single variable.patients.BloodPressure=[patients.Systolicpatients.Diastolic];patients(:,{'Systolic''Diastolic'})=[];%%% Pick out two specific patients by the LastName variable.patients(ismember(patients.LastName,{'Smith''Jones'}),:)

Here’s what HTML generated via the Publish command looks like:

Foo is 5 and bar is 7. Pi is 3.141592654.

The Publish feature is a great way to generate a presentable record of an interactive session of work, or to document a single script and show its full execution. You have a choice of several document formats and settings to control the output.

In comparison, Dexy is a more flexible and powerful tool, but it’s harder to learn and requires Python to be installed. Dexy gives you not only a powerful tool for writing documents incorporating code and results, but also a way of automating many other aspects of your project. If you have a complex workflow, Dexy can run all your scripts in the correct order, help you to share data files between them, and finally help you to communicate your results with confidence in their reproducibility.

If you have questions about using Dexy with MATLAB, please feel free to leave a comment or get in touch via one of the suggested methods here.

The rstmeta filter now works as intended, adding metadata like title and author from reStructuredText documents as document settings which can be accessed in other dexy filters. Also, document settings now are accessible as python instance attributes in addition to via the setting() method. This makes it possible to use jinja template filters like sort which work using python attributes.