Details

Description

This ticket is to build an intersphinx object inventory caching microservice and client-side Sphinx extension for DM documentation projects, including technotes, pipelines.lsst.io and developer.lsst.io. The main purpose of this work is to insulate DM's Sphinx builds from the real-time availability of third-party documentation projects. If a third party documentation project is down, intersphinx is not able to download the object inventory, and thus the DM build fails. By caching object inventories in a DM-operated highly-available microservice, we prevent this source of failures, and possibly improve build performance the overall intersphinx experience for developers.

The work is divided between server side and client side (sphinx extension) contexts, and it may make sense to divide this ticket into two.

Client side specification

Provide a Sphinx extension that wraps around intersphinx. The custom extension speaks to the microservice, and downloads object inventories. intersphinx itself should see these object inventories as pre-existing cached object inventories, and handle the remaining work of matching API object references to URLs during the Sphinx build.

The client may also be configured to automatically obtain inventories for all DM sphinx projects so that developers never need to manually add new technotes and user guides to their intersphinx configuration.

Microservice specification

The client sends a GET request for a list of object inventories required. The microservice responds with a URL to a single compressed set of intersphinx object inventories that the client can separately download (or maybe download straight through the client—regardless the inventories will be stored in S3).

When a new object inventory is requested, the microservice downloads the new inventory from the source.

In the background, the microservice updates third-party inventories.

The microservice pre-caches all DM Sphinx projects in coordination with the keeper.lsst.codes API.