I must admit I don’t seem to be perfectly happy with what reference manager software have to offer. I am probably unjust and have high expectations. After all, they are ‘reference managers’ and in that sense they are already over-performing. Originally they were only intended to help scholars keep bibliographical data in one place (electronically) so that when they needed to add citations to their documents they be at hand rather than having to go through piles of print-outs to track down the sources. In that function they now fit very nicely (almost seamlessly) into the academic writing workflow. Is that simply their well-defined little niche? Or could they become more useful than that?

Well, over the years of the evolution of the ‘reference manager’ genre, many applications have started to expand their functionalty. Recognizing the benefits of storing and managing bibliographical data together with the electronic documents, many applications went on to become ‘library managers’. (Actually, Papers came the other way around. As far as I remember, their original idea was to store PDFs in one place and only later did they put more emphasis on citation features.)

One other important step was when these applications started to offer highlighting and note-taking features. After all, scholars don’t just want to administer these documents, but they work with them before they use them as cited sources. Okay, now they store store highlights and notes with each PDF. I am thankful for that, but that I still consider more an administrative function rather than something that lubricates the thinking (and writing) process.

By then end of my PhD, I had over 800 documents in my Sente library incuding journal articles and full books, many with highlights and notes. How am I supposed to find interesting bits related to one concept, idea or topic? My highlights and notes are there somewhere in those documents but there is no easy way of tracking them down and working with them. They are searchable or can be made searchable (see Jeff Pooley’s guide on Macademic here), but that is often not very helpful. I would for instance like to see them in one place organized according to some logic. My current practice is that I make the highlights in Sente for any potential future use and at the same time I copy the text (quote) to Scrivener with the citation info and keep these snippets organized there. I would for instance have a card for ‘innovation (def.)’ in which I would only collect various definitions of innovation from the sources I read.

If these papers were printed out, rather than making paper clippings and pasting them in one place, I would simply highlight the relevant portion of the text in neon yellow and I would also add a note in the margin saying something like “def.” That would be for quick reference, so that when I flip through the pages I could get an idea at a glance of why I highlighted something. This is basically good old fashioned tagging. And this is what is missing from all the reference software: being able to easily tag not the PDF itself, but the highlighted parts in them. Then ideally I could get the app to display all snippets of highlighted text in my whole document library associated with a specific tag (or combination of tags). I am dreaming of being able to enter <innovation> and <def.> and getting a list of snippets tagged ‘innovation’ and ‘def.’ Maybe I could even set up a smart collection named “Definitions of innovation” that would be automatically updated and whenever I highlight and tag something new it makes its way over to my snippet collection. Now, that would be a great step forward in not only helping scholars with administering sources but also facilitating the thinking part of the research process.

Until these features are implemented in our favorite now-way-more-than-just-reference managers, you can see how something very similar is done in the popular content analysis software Nvivo. So far, I have only thought of Nvivo (or the like) as useful in analyzing (qualitative) data such as interview transcripts, reports etc., but the same principles can be applied to processing academic sources during literature reviews. Check out this official video from QSR International, makers of Nvivo, on how to do a literature review with Nvivo.

You may also want to watch this other piece on how to use Nvivo for a PhD.

The bad news is that Nvivo is only available for Windows. The good news, however, is that it is coming to Mac very soon.

Like this:

LikeLoading...

Related

About Csaba Pusztai

I graduated from CEU with a Ph.D. in Environmental Science and Policy. I taught undergrad and graduate economics and business for 16 years. After spending two years at UNU-IAS in Japan, I am now working in forestry research at Scion in New Zealand.
I converted in 2004 purchasing a 12-inch PowerBook which I used until 2012, when it finally gave in. I find pleasure in reading, thinking about and discussing science and technology.

35 Responses to How reference managers could help is in our thinking process rather than just hold our libraries…

Have you tried Docear? I was looking for a mind-mapping solution to organize all aspects of a sales territory,(I happen to be in technical sales) when I came across Docear. It automatically imports highlighted sections and annotations into a mind-map like structure.

From their website:
Docear is a unique solution to academic literature management, i.e. it helps you organizing, creating, and discovering academic literature. Among others, Docear offers:

A single-section user-interface that allows the most comprehensive organization of your literature. With Docear, you can sort documents into categories; you can sort annotations (comments, bookmarks, and highlighted text from PDFs) into categories; you can sort annotations within PDFs; and you can view multiple annotations of multiple documents, in multiple categories – at once.
A ‘literature suite concept‘ that combines several tools in a single application (pdf management, reference management, mind mapping, …). This allows you to draft your own papers, assignments, thesis, etc. directly in Docear and copy annotations and references from your collection directly into your draft.
A recommender system that helps you to discover new literature: Docear recommends papers which are free, in full-text, instantly to download, and tailored to your information needs.
And did we mention that Docear is free, open source, available for Windows, Linux, and Mac OS X, and not evil?

oh yes, couldn’t agree more… The thing is that all this software is great and of course would sound like a dream 15 years ago, but it is still just translating scholar’s standard activities to digital interface. It’s just a new, faster way of using old methods. It’s maybe time to think about features that have no “analog” counterpart. Tagging is one such thing. Of course an index of a book is a kind of a list of tags. But once you can filter by tags, it becomes something else, it’s not even comparable toan index anymore. And I really think that Sente, Papers, Zotero, Scrivener, Devonthink and the like are just a few steps away from a real revolution in how we work with data. Tags are a great concept and its potential has not been properly exploited.

Thanks for suggesting Docear. That’s new to me. I only had 10 minutes in the morning to check it out, will take it for a longer testride later on. I am not much fond of mindmaps, maybe because the way they are visualized is not my style of thinking. I don’t like the tree metaphor that much and the corresponding hierarchical arrangement of ‘objects’ in the map. But apart from that, yes, it seems that Docear is trying to implement what I was saying in my post. I will try to neglect my aversions, and will seriously try it. Although, I usually try to avoid Java apps and go for something that is natively implemented on a particular OS. I have been using Java software for very specific quantitative tasks and visualization, because the functionality they have is not available in platform specific counterparts, but they are sometimes difficult to deal with, slower than other apps. Of course if there is nothing else out there, it is good to have at least something in Java.

I decided to store the important information outside of the reference manager and use Zettelkasten for this purpose (see here: http://zettelkasten.danielluedecke.de/en/index.php). This German software is also available in English and Spanish. Besides taking notes, it stores links to the original PDFs, allows keywords, and you can link pieces of information. The software find knowledge clusters and one can extract information for publication purpose.

I have been thinking about exactly the same problem for a longtime: in somer circles, we dubbed the problem “Tagging to the bullet point level”. As you correctly stated, the point is make the smallest pieces of information visible for latter processes using “tags”. I have been trying to address the problem using Sente and Tinderbox: by surrounding my key words (phrases) with special characters as [[KeyWord]] while commenting inside Sente. Well, it turn out that Tinderbox has no straightforward ways for collecting the KeyWords and converting them into its attributes. I then decide to drop the whole process.

From the above videos, NVIVO looks powerful. I will give it a try. Thank you for the wonderful post.

Yes, that is quite pricey. The reason may be is that while reference managers are more targeted at the individual academic and hence priced accordingly, Nvivo and its competition are positioned as qualitative software suites: research tools for groups and institutions, and that leads to a hefty price tag. I guess it may only be worth it if one gets a license through his/her institution. But you could check out Zettelkasten suggested by Frank above. That is completely free and may address much of your need in terms of handling text snippets (quotes, ideas).

Thanks for everyone for their hints and feedback. Frank, I will give Zettelkasten a try. Judging by the screenshots it seems promising, more so than Docear, which may be a great app, but the GUI is quite cluttered.

I think Nvivio is far too expensive for academics. Zettelkasten interesting, but it does not recognize the url from DEVONthink and Sente (I do link to the source in each note). In addition, it is developed by a single programmer.
My approach is as follows. Highlights and notes prepared in Sente or Skim are exported to DEVONthink using these scripts: https://github.com/RobTrew/tree-tools/tree/master/DevonThink% 20scripts . Then in DEVONthink I add tags to each note, and sometimes create links between them. The result is similar to Zettelkasten, but with cross-links and power of DEVONthink’ artificial intelligence (searching using a variety of Boolean operators, see and also, etc).

Funny that you should say this right now as I am coincidentally on github checking out exactly the thing you just suggested. I spent the past couple of days exploring options, and I figured that there is no single-app solution. Use a ref manager and some snippet manager connected through scripts.

I have not been much involved in computer programming since high-school, so I cannot accurately judge the size of the challenge it would pose, but I was thinking that it would be nice to develop a little DevonThink extension that is similar to what gets embedded in Safari. After highlighting a piece of text, I would like to hit a key (combination) while in Sente or Papers (whichever allows for such extensions), get a pop-up in which I could specify the DevonThink tags and instead of the title (is in Safari) it would grab the citekey and page number combo into a field. Switching to DevonThink to tag up snippets would not be such a burden, but if this little extension could be implemented, things would be even more streamlined.

Currently, if you highlight your note in Sente and hit the DevonThink take note hotkey (default is Shift-Command-2), you can then copy and paste the text into a new DT note, select format, add tags and title, specify which database to add it to, and even create a new group for it.

Unfortunately, yes. But if you have your Sente gestures configured, you can use Command C and Command V to add whatever citation information you prefer (Sente URL, Chicago AD, etc.) to the DT takenote pane. There may still be some unnecessary navigation involved — you have to be in the library tab in order to select the entry (Sente doesn’t seem to permit the copy-page gesture from any tab except the library tab). Still, it may still be the easiest approach right now — until you can develop your proposed extension.

I can see how an application which does what you describe in your post and in the comments would be more useful than the plain reference managers of today. I once was about to get the source of Skim.app (PDF reader) and add some features to it so I could attach my thoughts and notes to original files in a more useful way. The interface is not encouraging this a lot at the moment, although it’s nice to have a list of annotations in the sidebar.

Yet, I don’t think this is a good way out of the problem, though. Notes on texts are my knowledge and belong in a vessel where I can work with the things I’ve thought; it belongs into a note archive. Marking definitions for example is just the first step: after you highlight a passage, you need to get the information out unless you don’t really care about having a way to contrast definitions from various authors.

Since you, too are an academic, I assume our needs are very similar: read and annotate texts quickly; get the information out of the text into an archive so you can search it easily, work with it, comment and expand on it; connect pieces from texts to prepare what you want to say in your own writing; write a text yourself, insert fitting quotations, cite sources and point out similarities and differences. The point is that you can’t do a lot with the inspiring passages unless you get it out of the PDF. When you work with notes in Scrivener that way, it’s on a per-project basis only. I suggest you try and fill a separate note archive where you keep all your knowledge. Then you can draw interesting connections between pieces over long periods of time.

TL;DR: “Fixing” reference managers would be nice, but we still need to work with our annotations later and get text passages out so we can work with quotations later without needing the original PDF any more. When we open a book, look at a text or read a PDF a second time this is admittance of failure, as MK states:

I rarely consult secondary sources again. If I have to do so, it means that I did not do the job right the first time.

Essentially, what I was saying–in contrast to you–is that drawing connections between pieces of text should be part of the software we today as reference managers (integrated view), whereas you say that it needs to be done in a separate application (fragmented approach). As for me, I like to have to rely on as few apps as possible. Also, I don’t believe that reading the original source again is a sign of failure.
My understanding of the problems I work on evolves over time, so the value I “extracted” from one paper (“pdf”) at an earlier point in time may not be the same I extract at a later point.

There was always room for outlining and note archives in the past. I seem to know something about paper-based reference managers, about the way scholars worked the past few hundred years. Your desire to unify the process of managing references and storing notes sounds similar to a usual approach involving index cards. People put everything on cards, so the medium was the same for detailled references, quotations and unique thougts, wheras the content differed. You only had to write stuff on paper cards and that was it. Say you had alphabetical filing, then you could use one unified box for all types of cards. (Cf. “Zettelkasten”, http://zettelkasten.danielluedecke.de/en/index.php) Workflows always differed. Some had a file to keep track of books they read. Some only noted pages of interest on the backside of these cards instead of copying the contents into notes. Others put everything on cards for a writing project so they could rearrange the movable parts before compiling a first draft. That’s what outliners are good for, today. (Johann Jacob Moser wrote about this technique in 1773, only in German, though.)

Some indeed relied on the availability of the texts they had read. There’s nothing wrong with it in principle, especially if you consider there’re texts which are worth picking up time and again in every field you can study. It’s useful to extract everything you deem useful at once, though, to keep up the pace of the quickly moving community. There’s just so much input it’s costly to hesitate. This is a question of efficiency.

From today’s perspective I would rather separate notes from citations to keep both digital representations of information clean. Why use a proprietary format to store data just to attach notes to reference items? The lock-in would be higher than using an open reference format like BibTeX and, say, MultiMarkdown-based plain text notes which re-use BibTeX’s “citekey”s.

Thinking about the paper-based solutions again, do you experience friction only because you have to resort to copy & paste all too often, while on paper you wouldn’t need to switch from writing to finding a reference?

I don’t get NVivo by watching the videos. Maybe you could expand on the friction you perceive some more or draw a sketch to visualize the desired outcome?

I was thinking about something “similar more-than-just-reference-collection”-like but with an graphical approach as my brain prefers graphical (but not mind maps…they are too simple) approaches. I usually scribble pathways (nodes connected by arrows (activation) and blunt-ended lines (inhibition)) on paper and then note for the different links of the nodes the reference. But on paper this gets really messy. A software solution would be grate as it would allow filtering. Anyone interested in programming this idea? Please get into contact – find me in LinkedIN or ResearchGate. Cheers, Christoph.

Csaba Pusztai says:
December 7, 2013 at 7:12 pm
Yes, but I would have to add the citation information manually, wouldn’t I?
————-
Rob Trew scripts (https://github.com/RobTrew/tree-tools/tree/master/DevonThink%20scripts) capture the Sente note AND the reference in a format Sente can scan. The notes related to a reference are copied in a Devonthink group, and tagged with Sente metadata. The note and reference can later be copied to Page or Scrivener, then scanned by Sente.

I think Masterfile may do some of what you want with its “extract” function and “report” function. You highlight, cut and paste the highlight to an extract, add comments, and tag it. Then you can generate a report (a brief) with it organized by tag or by chronology.

I researched Masterfile a couple of years ago, but am now implementing Devonthink – and desperately searching for the Extract adn Report functions that I assumed would be there…

I’ve used Nvivo for my PhD literature review. It has some great features, but tends to force me into assembling and organizing short codes or snippets, rather than helping me consider longer passages of text and notes in comparison with each other. Also, I found that the workflow of exporting from Papers2 and importing to Nvivo was cumbersome, although this is partially because I am using a Mac and have to use Nvivo via Parallels which means a tricky conversion of file names via Endnote in Parallels.

I’m about to try MAXQDA, which I think handles longer passages better than Nvivo as it allows you to tabulate them, make notes on the tables or cells, than tabulate your second set of notes. However, whilst it has RIS import, it won’t import the PDF at the same time.

I think a better model would be for MAXQDA to add a bibliographic manager plugin so that there is no need to move between software, which can be quite disruptive.

This is still a key need for many of us. Many people are building workflows to accomplish this.
I found something (from 2011!) about creating a flow between Sente and Devonthink for this. Papers2 also seems to have some of this capability, e.g. see http://wippp.com/blog/capturing-annotations-to-enhance-writing-and-knowledge-retrieval-advanced-scientific-writing-workflow/. (Oct. 2012) For example: “As I highlight or make notes in Papers , a separate layer is created in the PDF that keeps track of annotation information. As I annotate, each highlight and note appears under the “Notes” tab in Papers.” He also uses Devonthink.
Personally, I hope that Bookends will provide what I need, because it beats Sente in several respects. Bookends already builds a stream of notes and highlights for any PDF that you annotate.
I’ve signed up to get follow-up comments. Does anyone else have suggestions? thanks