The right to reply… paragraph by paragraph.

Monthly Archives: August 2009

We’re really pleased to help promote the launch of digress.it, the evolution of CommentPress which WriteToReply uses to allow you to comment on document paragraphs.

We’ve been in touch with Eddie Tejeda, the original developer of CommentPress, since March, and have been working with him to find funding for a complete rewrite and re-release of the original CommentPress project. You can read more about digress.it on the community website, but here’s a run down of the new features, a bit of a roadmap for forthcoming features and a shout out to anyone that wants to get involved in the project.

New Features

The original features of CommentPress can be summarised as follows:

A Table of Contents

Paragraph-level URIs

Paragraph-level commenting

A scrolling comment box

Page filters that allow you to read comments by document section or by commenter

CommentPress was a WordPress theme. digress.it is a WordPress plugin and a complete rewrite of the original CommentPress code. It adds the following features:

Floating comment box. You can now resize and position the comment box anywhere on the page.

Threaded comments. Real discussion.

Highly configurable and accepts different stylesheets

RSS feeds for comment authors. Feeds for individual comment authors is a first for WordPress.

Paragraph embedding. You can embed a paragraph on your own site. Paragraphs content is available as HTML, JSON or TXT

Real-time onsite notifications. If someone else comments on a section you are reading, the comment box will ‘pulse’ to alert you.

There still be bugs. We’re still working on browser compatibility issues with the comment box, for example. This is a first release using the version 2 codebase and we’d really appreciate your feedback from testing it by considering Amazon’s Kindle License Agreement. 🙂

RoadMap

To achieve the objectives of the JISCPress project, we’ll be continuing to fund the refinement of digress.it until November. The features we’re currently considering can be seen on our UserVoice page (please add more as you think of them). Here are some highlights, specific to digress.it:

Compatibility with IntenseDebate. This would provide a number of multimedia and reputational features.

Compatibility with PollDaddy. The ability to include polls in a document would be useful for consultations.

Paragraph and ‘comment here’ links in the RSS feed. Convenient if you read the document in your news reader or embed a feed elsewhere.

The platform is a WPMU (WordPress MultiUser) site running an evolution of DIUS’ original Commentariat theme [UPDATE: apparently it’s not an update of Commentariat, it’s a custom theme that just shares a lot of the features of Commentariat; it’d be really useful to see a comparison of the two… I also wonder if the DFID theme is available under an open license?]. You can read more about the DFID platform on Puffbox. It’s really good to see WPMU embedded in government as a platform for consultations. Did WriteToReply have anything to do with that, we wonder? 🙂 [UPDATE: apparently not – see the comments]

(Just as an aside: if you are, or are thinking of, running more than two WordPress sites under the same domain (actually, even different domains), then WPMU is a much better solution than a proliferation of single WordPress sites; as users of both, we can assure you that the step from WordPress to WPMU is tiny. It’ll take you just a half a day to understand how the WPMU platform works. Seriously… DISCLAIMER: errr, none; WordPress and WPMU are both free downloads, and we’re not on any sort of commission or payback!)

At some point, we need to do a side-by-side comparison of the CommentPress and Commentariat themes, not least so that we can provide a checklist for helping people decide which commenting theme best suits their document; but in the meantime, Steph Gray pointed out a few of the original Commentariat features in Introducing Commentariat & the POI Taskforce Report. If reading that post is still too much effort, the major difference to users is that CommentPress supports paragraph level comments, whereas Commentariat offers page level comments, and an arguably nicer navigation scheme.

Through working on JISCPress, an enhanced version of the CommentPress theme, we’ve started to tease out some principles that will guide our future work on the WriteToReply platform.

Support paragraph-level commenting. Consultation documents are generally pithy, carefully worded documents. Allow readers the option of directing comments at specific points in the section rather than at the section as a whole. The Institute for the Future of the Book have done research into online engagement with texts. It’s worth building on as Steph Gray did by including the scrolling comment box in the Commentariat theme.

Document amplification: A government consultation document is not a destination site. By using a platform that is already syndicating chunked document sections on the web (e.g. through RSS syndication), exploit the fact that they’re no longer monolithic documents. Support ‘remote publishing’ through the use of embedded quotes sourced from your original consultation document. Leverage the web to allow people to take ‘ownership’ of the pieces of the consultation that matter to them. Most sites seeking widespread public engagement provide a means for embedding content elsewhere. Work being carried out as part of our open source JISCPress project will provide tools to republish paragraph level content in a variety of formats from a family of structured, unique URIs.

Allow search engines to index your consultations: ‘robots, noindex, nofollow’ has no place in a public web-based consultation document.

Remote commenting: Pulling in discussions from elsewhere on the web can be done by publishing a unique URI for each separate paragraph in a republished document. These unique URIs can be linked to from third party blog posts and microblog posts (e.g on Twitter) allowing remote conversations to refer to very particular parts of a document, and support the generation of linkbacks and remote comments back to an individual paragraph.

Platform-wide benefits: There is no doubt that there are organisational benefits for institutions or government departments running their own consultation platform, as Simon outlines in his post. However, we also believe that there is a public benefit in hosting documents from multiple agencies on the same WPMU platform arising from platform-wide search, browsing, cross-linked related documents, thematic navigation and semantic tagging. By publishing government consultations on an open, web standards-based platform, the documents become open data. One key feature of our work is to explore the extent to which content analysis and automated semantic tagging of documents hosted on the same platform can be used to automatically generate crosslinks between related documents. For certain document ecosystems, this feature may be used to support automated content discovery or recommendations about related content in other documents.

Strength in re-use: We built WriteToReply on the CommentPress theme, in the same way that the DFID platform is built on the Commentariat theme. Our JISCPress project (another public-funded project) has in turn extended the CommentPress theme, not least in exploring the opportunities for content re-publication and third party quoting/embedding. We set up a JISCPress group to discuss our proposed extensions, and solicit further ones, so if there’s anything you like to see us working on over the next couple of months, please post your suggestions there. Remember, it’s an open source project so it’s code you can use for your own projects.

Following a brief exchange with @lesteph last night, I thought it might be worth making a quick post about the idea of content or document transclusion.

Simply put, transclusion refers to the inclusion, or embedding, of one document or resource in another. To a certain extent, whenever you embed an image or Youtube video in a page is a form of transclusion. (Actually, I’m not sure that’s strictly true? But it gets the point across…)

Whilst doing a little digging around for references to fill out this post, I came across a nicely worked example of transclusion from Wikipedia – Transclusion in Wikipedia

The idea? You can embed the content of any Wikipedia page in any other Wikipedia page. And presumably the same is true within any Mediawiki installation.

That is, in a MediaWiki wiki:

you can embed the content of any one page in any other page.

(I’m not sure if one MediaWiki installation can transclude content from any other MediaWiki installation? I assume it can???)

It’s also possible to include, (that is, transclude) MediaWiki content in a WordPress environment using the Wiki Inc plugin. A compelling demonstration of this is provided by Jim Groom, who has shown how to republish documentation authored in a Wiki via a WordPress page, an approach we adopted in our WriteToReply Digital Britain tinkerings.

One of the things we’ve started exploring the JISCPress project is the ability to publish each separate paragraph in a document (each with its own URI), in a variety of formats – txt, JSON, HTML, XML. That is, we have (or soon will have) an engine in place that supports the “publishing” side of paragraph level transclusion of content from reports published via the JISCPress/WTR platform. Now all we need is the transclusion (re-presentation of transcluded content) part to be able to transclude content from one document in another. (See Taking the Conversation Elsewhere – Embedded Quotes; see also Image Based Quotes from WriteToReply Using Kwout for a related mashup).

(Hmm, although Joss won’t like this, I do think we need a [WTR-include=REF]shortcode handler installed by default in WTR/JISCPress that will pull in paragraph level content in to one document from a document elsewhere on the local platform?)

Now this is really what hypertext is about – URIs (that is, links), that can act as a portal that can pull content in to one location from another. It may be of course that the idea of textual transclusion is just too confusing for people. But it’s something we’re going to explore with WriteToReply.

And on of the things we’re looking at for both WriteToReply and JISCPress is the use of semantic tagging to automatically annotate parts of the document (at the paragraph level, if possible?) so that content on a particular topic (i.e. tagged in a particular way) in one document can be automatically transcluded in – or alongside – a related paragraph in a separate document. (Hmm – maybe we need a ‘related paragraphs’ panel, cf. the comments panel, that can display transcluded, related paragraphs, from elsewhere in the document or from other documents?)

PS If you have an hour, here’s the venerable Ted Nelson giving a Google Tech Talk on the topic of transclusion:

Enjoy…

PPS here’s an old library that provides a more general case framework for content transclusion: Purple Include. I’m not sure if it still works though?

One of the things we discussed with respect to embedding WriteToReply/JISCPress quotes in third party applications was whether or not we should support an “imagified” embedding – that is, convert a paragraph to a JPG or PNG image format that can then be easily embedded in the third party site.

The advantage? Even if the third party site disallows script, object or embed tags, it will probably allow img tags…

However, this facility was seen as non-essential, so I looked on the web for a solution – and found it in the form of the kwout API which can be used to generate an image based representation of text found in a specified div tag (by ID) on a given web page, which can then in turn be embedded in an arbitrary web page. Although the image may be hard to read, this can work to our advantage: it might drive traffic back to the site that originated the quote 🙂

The following javascript snippet uses the Kwwout API to generate an image based representation of a single paragraph from a WriteToReply republished document:

As a step on the road to full integration (a use of the Kwout API which may or may not be in line with the stated terms and conditions? I don’t know, I haven’t read them…!) is this bookmarklet that should let you highlight a paragraph number on a WriteToReply document, and then take you straight to the Kwout embed page for that paragraph:

Actually, that looks a little cluttered, and the usability is a little off. So a better solution maybe to suggest that the user clicks on the paragraph link to get the “paragraph in focus page” page, and then click on the following bookmarklet:

As part of the JISCPress effort, one of the things we’ve been considering is the granularity of appropriate “consultation elements” or “discussion elements”, those pieces of content that people might actually want to reference, question or chat around as compared to a whole 200 page document, for example.

The page and paragraph levels fall out of the CommentPress theme (and its descendants) quite naturally – WordPress gives us the page level (along with a single item RSS feed at the page level), and the theme gives us URIs at the paragraph level.

(Hmmm… I wonder – would it also be useful to provide a multi-item RSS feed, at the page level, with a separate item for each paragraph on that page? Or do we do that already?!)

In many cases, the paragraph level seems to be the most natural chunk for discussion, particularly in an ongoing conversation about a particular document. So a major question for us is how to put those paragraphs to work?

One of the features that Eddie’s been working on as part of the JISCPress project is the ability to embed paragraphs from a document in third party web page. This feature will allow us to increase the surface area of the document by allowing third parties to re-present that content elsewhere, whilst also (hopefully) providing a means to link that external conversation directly back to the original document.

So what benefits does embedding have to offer to:

a) the person grabbing and using the embed code;
b) the publisher/whoever’s running the consultation from which the embed code was grabbed

In a discussion on the JISCPress group, Joss suggested the following:

For the user:

1. More portable transformation of document content into raw data.
2. Personalisation, presentation and ‘ownership’ of documents within their own publishing environment (which is one of the benefits of slideshare/scribd).
3. Direct joined up quoting rather than copying. More aligned with the ideals of the web and linking data. This could also be a benefit to publishers concerned about unattributed copying.

For the publisher:

1. Greater possibilities of content dissemination
2. Greater potential of attracting engagement via trackbacks
3. Further possibility of using JISCPress as an underlying ‘document store’ where authoring, dissemination and engagement occurs mostly remotely via XML-RPC, syndication, embeds and trackbacks.
4. Possibility of site analytics being hooked into embeds so the reach is measurable???? (Analytics can track document types, I’m not sure whether they are used to track embeds…)

So where are we at? Embedding is currently in testing and has the following mechanic. Hovering your mouse cursor over one of the paragraph numbers in a document raises a floating panel that contains a link to the current paragraph, and an embed code. (The panel remains open whilst the cursor is over it, so you can easily grab a copy of the code.)

Using the embed code in a third party page embeds the corresponding paragraph in that page.

For testing purposes, the pattern we are using for the embed URL is of the form:

The POSTNUMBER identifies the actual page (i.e. http://docserver.example.com?p=POSTNUMBER is a valid page URI) and the PARANUMBER identifies the paragraph to be embedded. Note that this is subject to change.

Unfortunately, the simple embed strategy does not trivially generate a linkback (such as a trackback or pingback) to the original document. For these reverse links to be generated automatically, an actual anchor tag linking back to the original page must be present in the page creating the linkback. One commonly used strategy for achieving this is to provide an embed code of the form:

<div>
<object /&gt
<a>Quoted from etc…</a>
</div>

That is, a link is explicitly included in the embed code, although it is easy enough for the person embedding the quote to strip that anchor tag out.

(Although it complicates matters, as the embedded object is being pulled from the document server, I guess that means we could, in principle, generate a linkback by observing the referrer page URIs for requests made on the server for particular embeddable objects and checking those against the current list of trackbacks? Or maybe the embedded object could generate an XML-RPC back to the trackback server itself whenever the page it is embedded in is loaded? [Note to self: can we easily get analytics on third party embeds?] I think Eddie is working on this, so I won’t embarrass myself further wittering on about things I don’t know anything about!;-)

Note that a similar problem arises when using a Javascript (<script> tag) based embed code: there is no explicit anchor link present. Script tags also have the additional problem that they are often sanitised (i.e. stripped out) of web pages in many institutional web publishing systems. (In some circumstances, a workaround for the institutional case may be possible. For example, if a variant of WTR/JISCPress was running as a white label solution in an institution, a shortcode plugin could be provided that allowed authors to embed paragraphs from documents in that environment within other documents in that environment. See the WordPress shortcode API for more details.)

As well as the straightforward embed code, we’ve also been considering other ways in which paragraph level content can be published so that third parties have convenient access to it in a format that is appropriate for their needs.

And this is what we came up with – an output switch that can be appended to the end of a paragraph URI that allows the paragraph level content to be published in a variety of formats:

&output=html

&output=rss

&output=txt

&output=js

&output=json

As and when these come on stream, we’ll publish use-case examples for each of them.

If you have any comments on our “paragraph republishing” strategy, please post a comment below.