Before starting discussing about Sphinx, let us remind that documentation
is an important part of software development. Even though documenting your
code takes time and request initial investments, it increases
the visibility of your work and ease future code development by other people AND
yourself.

Although there are several type of documentations, we will focus on two of them in particular:

Technical - Documentation of the code, the algorithms and the Reference guide (also called API)

End User - Manual for the end user with tutorials and examples, also called User Guide.

In this document, we will describe how to use Sphinx to create
nice and useful documentation that combines both Technical and End user manual together. We propose to
use this software in the future within all OpenAlea packages and we will explain why in the following sections.

Currently, we are using Epydoc to provide an API documentation of
the different packages in Openalea projects. For instance, you may consult thecore package API to see an instance of Epydoc's output. The HTML pages that are generated fulfill our request of having a useful and exhaustive API. However, concerning User Guides and tutorials, we use the wiki pages.

The first issue is that the API and User Guide are not located at the same place: the Reference Guide is generated by parsing the SVN archive, while the User Guide is written in the Wiki.

Generally speaking, in OpenAlea, the code are sufficiently documented to have a useful output with Epydoc. However, and this is the second issue:

we hardly have any documentation to help developer understand how the different packages works and interact with each other. Although there are efforts in this direction, we lack tutorials and examples that would help users to understand the functionalities and potential of the different packages.

In order to obtain a consistent Reference guide or API documentation
as well as a User Guide (with examples and tutorials), we decided to investigate
the Sphinx software that is being
used more and more within the Python Community (e.g., Python itself,
Matplotlib, Numpy, …). Indeed, Sphinx provides a way to put together all the documentations
into a single location (the SVN) and therefore to keep the documentation close to the code. Yet,
it provides enough flexibility to create nice and useful User Guide documentation.

This document does not attempt to cover the huge number of alternative
softwares, nor is it an attempt to exhaustively list all the positives and
negatives points of each of them. It is the results of investigations that have been
performed on OpenAlea packages (Core, Stdlib, …) as well as a few packages in
VPlants (PlantGL, Stat_tool, …)

As mentionned in the previous section, when launching Sphinx, it will search for a file called conf.py in the /doc directory. This file can be generated typing:

sphinx-quickstart

and changed to your need later on. It is used to set the HTML output, Latex output, the location of sphinx plugins and so on.

In order to have the same configuration file for each package, the project leader(s) should provide a configuration file that will be identical and placed in all the doc directories (see for instance in the openalea SVN, the file core/doc/conf.py). Similarly this configuration expect an entry file called contents.rst that will be your entry point to the documentation.

Replace the sphinx configuration file by this one, that will look for a common configuration file to all OpenAlea packages:

The source code that Sphinx expects is restructuredText, which is a textual language. Here is an example:

title
=====
This is a **bold** syntax, like in this wiki. There are also some directives to
include code, images, ... that uses this syntax:
.. image:: frogWithGlasses.png
::
After a special double column character, you can
include python code for instance. Note that there
are indentations and blank lines before and after
this paragraph.
Subtitle
--------
.. toctree::
:maxdepth:
index.rst
tutorial.rst
This toctree allows to include other //.rst// files

This proposal does not describe the restructuredText language in details. However, Sphinx expect the docstrings and your documents to be in this language. So, you will need to learn it. This is in fact quite easy and we decided that all the HTML documents that results from the Sphinx processing should contain the original source file (done in the configuration file). This will ease the ability to mimic the already existing restructuredText document that are in the SVN.

What is important here is that you can add python code (in other word examples), User Guide, and link to any number of rST files (using the toctree directive in the example above).

In particular, you can also add an API documentation. However, the API generation is not as straightforward as it is with Epydoc but is much more flexible. You have to decide exactly what to do with each module using this syntax

.. automodule:: my_best_module.py
:members:
:undoc-members:
:inherited-members:
:show-inheritance:
:synopsis: I'm sure this is the best one

But if you decide that you want to document only a particular function or a class in a module, then it is also possible (switching automodule to autoclass.

Obviously, the docstring of your python files should follow the same conventions.

Sphinx extends restructuredText quite a lot with directives such as seealso, warning that creates nice output in your HTML documents. However, note that it may not be compatible with Epydoc anymore, which knows only the pure restructuredText directives.

Here below we show three types of Docstring conventions and explain the pros and cons of each method afterwards.

Pure sphinx code

restructuredText and Sphinx

a la Numpy

"""Brief synopsis
This is a longer explanation, which
may include math :math:`\\alpha`.
Then, you need to provide optional
subsection in this order (just to be
consistent and have a uniform documen
tation Nothing prevent you to switch
the order):
:param arg1: the first value
:param arg2: the first value
:param arg3: the first value
:type arg1: int, float,...
:type arg2: int, float,...
:type arg3: int, float,...
:returns: arg1/arg2 +arg3
:rtype: int, float
:Example:
>>> import template
>>> a = MainClass()
>>> a.function2(1,1,1)
2
.. note:: can be useful to emphasize
important feature
.. seealso:: :class:`MainClass2`
.. warning:: arg2 must be non-zero.
.. todo:: check that arg2 is non zero.
"""

"""Brief synopsis
This is a longer explanation, which
may include math :math:`\\alpha`.
Then, you need to provide optional
subsection in this order (just to be
consistent and have a uniform documen
tation Nothing prevent you to switch
the order):
:Parameters:
- `arg1` (int,float,...) - the first value
- `arg2` (int,float,...) - the first value
- `arg3` (int,float,...) - the first value
:Returns:
arg1/arg2 +arg3
:Returns Type:
int,float
:Examples:
>>> import template
>>> a = MainClass()
>>> a.function2(1,1,1)
2
:Note:
can be useful to emphasize
important feature
:See Also:
:class:`MainClass1`
:Warnings:
arg2 must be non-zero.
:Todo:
check that arg2 is non zero.
"""

"""Brief synopsis
This is a longer explanation, which
may include math :math:`\\alpha`.
Then, you need to provide optional
subsection in this order (just to be
consistent and have a uniform documen
tation Nothing prevent you to switch
the order):
Parameters
----------
arg1 (int,float,...) : the first value
arg2 (int,float,...) : the first value
arg3 (int,float,...) : the first value
Returns
-------
arg1/arg2 +arg3 : the output
Warnings
--------
arg2 must be non-zero.
Examples
--------
>>> import template
>>> a = MainClass()
>>> a.function2(1,1,1)
2
Notes
-----
can be useful to emphasize
important feature
See Also
--------
`MainClass1`
Todo:
check that arg2 is non zero.
"""

do not take advantage of the automatic HTML output used with :param:, :return: and :type: syntax.

Note: there is no colored box as in the other methods for the todo, warning, seealso sections. This is deliberate, to have a uniform docstrings. However, you can use the sphinx syntax as in the first and third methods to get the colored box. The docstrings will just be non uniformed.

non robust HTML output. Do not take advantage of the automatic HTML output provided by Sphinx

doctest complains about the underlined title

Only hardcoded section such as Parameters and Examples are accepted. Put Example, and sphinx fails

buggy. After more than 30 minutes I could not make what I wanted (see the missing sections in the image)

A decision will be made soon on which docstring method will be push forward, so if you have more arguments, please give them

Final proposal (update on 13th March 2009): combine method 1 and 2 as follows

Proposal

"""Brief synopsis
This is a longer explanation, which
may include math :math:`\\alpha`.
Then, you need to provide optional
subsection in this order (just to be
consistent and have a uniform documen
tation Nothing prevent you to switch
the order):
:Parameters:
- `arg1` (int,float,...) - the first value
- `arg2` (int,float,...) - the first value
- `arg3` (int,float,...) - the first value
:Returns:
arg1/arg2 +arg3
:Returns Type:
int,float
:Examples:
>>> import template
>>> a = MainClass()
>>> a.function2(1,1,1)
2
.. note:: can be useful to emphasize
important feature
.. seealso:: :class:`MainClass2`
.. warning:: arg2 must be non-zero.
.. todo:: check that arg2 is non zero.
"""

Advantages:

Good HTML output

Nice docstring.

Since the most important sections (parameters, return, examples) use the `:keyword:` they are naturally emphasized with respect to the optional sections (todo, warnings, …)

Drawbacks:

the ..keyword:: syntax is not always recognized by epydoc for those who still want to use it.

We would like to move forward to use Sphinx as our standard tool for documentation. This is a very flexible tool, which requires to learn a little bit of restructuredText language. The advantage is that
we'll be able to include User guide, tutorials and examples closer to your documentation.

One advantage is that you can include examples and python codes within your documentation and able to test these codes during your documentation development.

Sphinx is becoming a standard tool for python documentation. Although it is still in early development (current version is 0.6.3), Sphinx is a mature tool that is already used by numerous projects, amongst which Python itself.

We've done some tests and it sounds all very promising. See for instance those draft pages that essentially contain API doc for the moment.