How to evaluate a #dh tool?

I am currently holding a seminar on “DH: tools, methods, epistemology” (Humanités numériques: outils, méthodes, épistémologie [in French]). Last week, we received Pierre Ratinaud who is developping the excellent text analysis tool IRaMuTeQ (a python-based visual interface to R). Next week, we’ll go a bit further on the text analysis road. We’ll try to use a corpus made of speeches from Roy Jenkins as head of the European Commission (1977-1980).

The aim will be to analyse this corpus with three different tools: IRaMuTeQ / Paper Machines / Voyant Tools in order to understand differencies and similitudes between those three tools and to emphasize how important it is to choose a tool, and how decisive it is in obtaining research results. Choosing a digital tool is fully part of your research methods.

So I created this Roy Jenkins corpus (whose quality is questionable, but it’s for an exercise, not for true research), 74 speeches, in three forms:

Results

Voyant Tools

Paper Machines

IRaMuTeQ

Dendogram (classification)

Similitude analysis

Wordcloud

Comparison

What I feel when I compare those results of a text analysis with three different tools (a part from the bad quality of the OCR of some documents):

Design is at the heart of interpretation: the way those results are displayed is influencing the interpretation of the data. It’s outrageous in the case of wordcloud – and that’s a reason why wordclouds are so questionable;

Those three tools are rather complementary. It seems to be obvious, but it is really important not to use only one tool;

It is easier to get back to the text and to alternate distant and close readings of your sources with IRaMuTeQ especially and Voyant Tools (a bit less than with IRaMuTeQ). It is sometimes possible with Paper Machines, but not that much. For a historian like me, it’s just strategic to be able a multi-scale reading of those sources.

Dernière publication

Digital technologies may impact the practices of historians at every stages of their research, from retrieving archives, building a corpus to writing and editing. Digital tools have influenced spaces for scientific dialogue and free flow of ideas within the historians' community. As such, Digital Humanities are challenging the shaping of history, but they could also become an object of history. Therefore, we will analyse such an hybridization that may enrich our field of study.