For each criterion, he provides a five-point scale. For example, the open access scale goes from 0 for no open access to 4 for Creative Commons licensing. The full article is well worth a read, especially for David's careful explanation of the impacts of each point on the scale of each criterion on the usefulness of the content.

The article concludes by applying the evaluation to a number of articles (including itself). In this spirit, here is my evaluation of our SOSP '03 paper:

It was developed based on experience with PLOS Currents, a rapid publishing journal hosted at Google. After a detailed review of the alternatives, the developers decided to implement Annotum as a WordPress theme providing the capabilities needed for journal publishing, such as multiple authors, strict adherence to JATS (the successor to the NLM DTD), tables, figures, equations, references and review. The leverage of mass-market publishing technology is considerable. The paper describing Annotum is well worth a read.