About internals

Documents are located using "locators". For general locators like gdu (gnome-doc-utils) or gtkdoc (gtk-doc API references) additional tweaking can be done using "decorators". Locator must be stored in locators directory, and defined as class named after its module with 'Locator' sufix. On loading, module is imported and class initialized. Dictionary containing locator arguments is passed to locator's constructor. There is a big note in file 'libgo' on how you can add new options to be passed to your locator.

Locator class must define getList() method without arguments that returns list of document objects located. "Document objects" (or docobj for short) does not have to be fully initialized (some of required properties can be empty) but it must be capable to finish initialization and do build process itself. For example of locator implementation you can look at gdu locator in locators/gdu.py and/or read a section about it in this file.

Of locators, gdu locator is implemented. It can use GNOME's CVS or SVN or CVS image obtained with jhbuild to access files and selection is done with command line arguments and access via SVN is default.

Locator can abstract IO operators using a driver for loading files into cache. Like locator, driver also must be a class, named after module in which it is located with a 'Driver' suffix. On loading, module is imported and class is initialized. Unlike locator, driver does not need to implement any specific interface but it need to be compatible with locator that is using it.

For example, CVS/SVN and dummy drivers defines same interface as they are used from gdu and gtkdoc locators:

Currently, only dummy driver is implemented. Dummy driver is using jhbuild source dir as a replacement for HEAD branch from cvs.gnome.org and can access only files for unstable release. You can notice that dummy driver can work offline, and CVS/SVN drivers will need access to GNOME's CVS/SVN server.

Decorators can be implemented for other locators as criteria and functions for applying decorators are locatator specific. Decorators are located in decorators directory, in modules named in a form of N-name.py where N is a number starting at 1. Number in a prefix enforce decorator loading sequence so cascade decorators can be written. Like locators and drivers, decorators are implemented as classes named after its module without N- prefix and having 'Decorator' suffix.

Decorators must provide interface for loading by implementing following methods:

libgo will load all decorators on initialization, and create instances. There is a function in utils module to get only decorators for a given locator. get_decorators(locator) function will return list of tuples of decorator name, its criteria and instance. Locator now can implement selection/filter function that will return instances of decorators to apply on some docobj and interface how docobj or locator interanal state can be modified. You can refer to section about gdu locator to see how gdu locator is using Makefile.am path as a criteria for selecting decorators and to see which interface is implemented by decorators for gdu locator.

And now something about "Document objects" (or docobjs for short). They are needed to provide abstraction for building documentation. In short, document object have a list of callable functions (builder functions) which is instanced by locators and decorators and that can be called in sequence to do all the dirty work of getting XHTML output in outputdir. One builder function can append another builder function to a list to be called and there is a way to pass some additional data to builder functions.

Document object also have some properties stored (which can be initialized with one of a builder functions) like list of languages, urlname (short ASCII name for urls), release,... Properties of docobj and it's interface are documented in docobj module.

For example, with gdu locator there are two functions for every docobj, one to fetch all files (xml, omf, po, figures,...) using driver and other to call xml2po and xsltproc utilities. First function take a dictionary with Makefile.am variables (DOC_MODULE, DOC_LINGUAS...) and populate docobj's properties and second takes no arguments.

gdu locator

(some parts are not implemented)

In short, gdu locators loads list of modules, for every module fetch list of branches (stable branches and unstable HEAD/trunk branch) and load main Makefile.am files for each branch of every module.

Recursively, it load sub directories from Makefile.am's SUBDIR variable and load Makefile.am from them. When all Makefile.am files are loaded, it looks for those using gnome-doc-utils and use info from Makefile to create docobj.

Criteria for decorators is list of globs and selection function load decorator if Makefile.am path matches any glob from a list. Decorators can implement following methods:

modDocobj and buildDocobj can return None, and if docobj is None after calling all decorators from a sequence docobj is ignored. You can look at implemented decorators to get better picture.

Decorators are implemented to remove modules that does not have main Makefile.am file (using special '/' path in glob criterita) and to remove po directory from list of subdirs (as it does not have main Makefile.am file). Locator can work without theese decorators but will display many unnecesary warnings about missing Makefile.am files.

Decorators that are needed are the one to remove undefined variable from SUBDIR value in Makefile.am (where undefined means that it is not defined in same file or it is defined in a conditional statement) and one for ekiga to replace @PACKAGE_NAME@ with ekiga as DOC_MODULE value. There will be a need for a few more decorators to handle some other special cases.

gdu locator can handle cases when some required file is missing (when it will skip over incorrect document) or when PO file is missing (when it will remove that language from DOC_LINGUAS and continue).

It will rebuilt only docs that are updated or everything if XSLT files or templates for static content are changed from last rebuilt.

Releases

As library.gnome.org presents documentation for both lastest and past releases of GNOME platform, docobjs continain information about release for every documentation item. Locators, can implement their own internal representations for releases (gdu is using CVS/SVN tags/branches returned from driver's getReleases(module) method), but when filling general docobj's properites there are some rules:

'release' is tuple of uppercase string representing part of GNOME
platform and indetifier of release. For example valid releases
are ('GNOME','2.14'), ('GTK','2.10')... Archived documentation is
grouped by first value from a tuple and sorted by other. After
selecting one release only documentats and categories for that release
is presented to library.gnome.org users.
'is_lastest' is True if document is from lastest stable release
Documents for lastest releases are displayed by default, merged
together whatever part they belong.
'is_unstable' is True if document is from unstable (HEAD/trunk) branch

Localization

(not yet implemented)

Documentation localization is part of GNOME Documentation project and is not an issue to solve using this program. This section is about localization of static content in libgo output like "Annotation" or "Recently updated" strings.

Localizing stylesheets

Basic styles used are from GNOME Documentation Project and are localized as a part of gnome-doc-utils CVS module.

Additional stylesheets used are localized in a same way to produce l10n.xml file from l10n.xml.in by merging it with po/LANG.po file from libgo CVS module.

Localizing static content

JavaScript scripts

js/langcookie.js.in # save/delete langauge cookie, display a note
# if language prefs are loaded from cookie
js/annotations.js.in # support for annotations in UI (add new tab under
# table of content and define behaviors)

Output is written as js/langcookie.js.LANG.js generated from *.in file merged with po/LANG.po

List of recently updated documents

After rebuilding updated documents, index files for browsing documentation are rebuilt. First, list of documents to include as recently updated is generated for each language on which at least one document exist. For all languages except English, list of recently updated documents is divided in two parts, one for localized documents and other for not yet localized documents and one olf theese can be empty.

Unix timestamp of last change in a document (stored in docobj) is multiplied with importance factor and list of top 2 documents from each channel is created for English language. Next, for every other language, if there is no localized document in each lists, lastest localized document from a channel (if there is any) is founded and stored. Now, everything is ready for generating index XHTML files.

XHTML index files

Localized index templates are loaded and info on updated documents (from XML index files) is included and XHTML index files are written to output directory.