.NET

BibPort: Creating Bibliographic References

Source Code Accompanies This Article. Download It Now.

BibPort extracts relevant information from a set of legacy documents to create a database of bibliographic references using Visual Studio Tools for Office in combination with Visual Studio 2008 and Word 2007.

Text Formatting Cues to Assist in Resolving Ambiguities

Properly interpreting a reference requires nontextual cues offered by text formatting. To obtain nontextual properties, BibPort requires a mechanism for understanding the document's underlying file format. A manuscript usually follows the standard format of the host wordprocessing tool, such as the structure of a Word document. BibPort understands the Word format and extracts the nontextual information in the citation.

Mining a reference through visual cues is not easy. Many common formats, including the Word document format, encode information in a binary format rather than plaintext. Furthermore, mining the visual cues is tied to a specific wordprocessor's output and is difficult to generalize due to differences in the file formats of wordprocessors. The situation has additional complexity because a parser may need to evolve with the host wordprocessor. For example, each new version of Word tends to change the standard document format in some way, requiring either a more complicated parser or several parsers to decode all formats.

BibPort includes a format-agnostic parser that processes the tokens of a document without understanding the specific underlying format. BibPort's own internal data structure can be extended with a wrapper to integrate with the host wordprocessor's data structure to deliver tokens to the parser. Many wordprocessors provide an API to access the internal structure of documents. For example, Word allows such interaction through VSTO and the Word Object Model (WOM).

Ambiguity Within Reference Types

Ambiguity within a single grammar rule can be resolved partially through nontextual information, which can serve as a guide to delimit the beginning and ending of certain elements (a journal name) of a reference list entry. However, additional challenges remain when parsing a full set of reference types.

The Typical Style

Creating the BibPort grammar for a citation format is not difficultthe process is similar to defining the grammar for a simple programming language. However, there are several citation formats that fail to distinguish between similar types of publications. In the example BibPort grammar for APA (Figure 2), you can see that the grammar productions distinguish between a book and a magazine. At a visual level, the title of the book is completely underlined, but the article title is not. Such visual cues can serve as attributes to assist the parser for a specific grammar. Therefore, there is no ambiguity between a book and magazine in the APA grammar when nontextual information is available.

[Click image to view at full size]

Figure 2: Subset of the APA grammar used in BibPort.

However, note that the grammar for the journal simply references the grammar for the magazine. This is because the APA style does not distinguish between magazines and journals. However, other citation formats may decide to treat magazines and articles differently, and many management tools distinguish between magazine articles and journal articles. Therefore, even though the grammar may be ambiguous, it's important that the parser have some mechanism for disambiguation so that integration between different styles is possible.

In a grammar specification, optional information is specified by square brackets; for instance, the volume and issue information for the magazine in Figure 2. It is difficult to analyze a citation grammar to produce a parser that's aware of all ambiguities. User-directed parsing needs to disambiguate some citation formats. Most citation formats permit the description of several dozen types of references. In terms of selecting candidates for ambiguous references, BibPort streams every reference list entry to the parser for each reference type. This way, parsers can remain simple, yet all ambiguities are always detected with candidate suggestions.

After an ambiguity is found, users must be involved. Of course, BibPort output should be reliable, so BibPort does not guess about the correct classification of an ambiguous reference without consulting users. Because BibPort is integrated with Microsoft Word, a pop-up dialog menu for each ambiguous reference list entry is displayed to users, who are asked to classify the reference from among the recommended candidates. A similar action can be taken for a reference for which the parser cannot find a suitable production match. An interface lets users populate a data structure with appropriate information, as well as indicate the reference type.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!