Pages

Sunday, September 22, 2013

A tongue in a cheek, thought experiment or perhaps precautionary tale of the ultimate fate of library discovery services in 2035.

With a sigh, Frank Murphy, head of library systems of Cambridge-Yale University made a quick gesture at his computing device and the system began to shut down the library discovery service for the last time. Or at least that was what he intended but the gesture based device - a distant descendant of the Kinect device refused to comply.

With another sigh, Frank tried again using the more reliable voice recognition system, "Shut down Library discovery service server" he commanded, and this time was rewarded with a request for confirmation.

"Yes, do it".

It was done. Head of library outreach Annie Watson observing the proceedings, chimed in, "Oh well, it was inevitable, but at least we tried."

"Besides this way we don't need to worry about the year 2038 problem", she added perhaps a little insensitively.

Frank didn't respond. Where had it all gone wrong? It began so promisingly, in the 2010s. After years of declining usage of library catalogues, libraries fought back with web scale discovery tools.

Summon, EDS (god rest their souls) and many others that followed seem to be the answer to the riddle "Why start at the library home page". Spotting modern interfaces light years ahead of existing catalogues, designed for users and not librarians, and most importantly the ability to break down silos in library content, libraries were finally able to compete with web search engines on a even footing using technologies and architecture that mimicked the then dominant web search giant Google.

Back then, Frank a fresh new systems librarian in Yale University (before the MOOC wars of 2020s forced the extinction and consolidation of all but the top universities in the world) was so impressed he enthusiastically blogged that "after false dawn of Federated Searches and Next generation catalogues, web scale discovery services were the answer".

To be fair, Frank had reason to be optimistic. Users were raving, usage of eresources was up substantially and the future seemed bright. The rot seemed to have stopped.

But this didn't last. As libraries caught up with existing commercial web technology, the web companies pulled ahead again and usage started to fall again. In retrospect, it seems sheer hubris to think that library technology companies most of whom were squabbling with each other could compete with the likes of Apple, Google, Amazon who had a larger budget on their entertainment fund alone than all the revenues of the library technology vendors combined.

While library technology companies were figuring out how to make interfaces that weren't hellish to use in the 2010s, commercial companies were figuring out voice recognition technologies, intelligent agents, semantic web technologies and more.

The last stung in particular. Many visionary prominent librarians had opined that one way the library could have a place at the table would be as part of the linked data movement, exposing the vast store of rich metadata libraries had.

It didn't work out that way. Squabbles over MARC, RDA , inertia and the inability to collaborate meant that as usual the library missed the boat.

By the time, libraries and library vendors realised the danger it was too late. Even a desperate final last ditch consolidation of Ebsco -Proquest - Gale to create a super search that boasted the combined content and metadata of practically all library content providers did nothing but delay the end.

The logic of pushing your content to where the eyeballs are was too strong. While this initially benefited Web scale discovery searches, where content providers were forced to add their content there or see declining usage. In the long run, this logic was even stronger for putting content on Google, Mendeley and other truly web scale services. Divisions and squabbles among content providers and discovery vendors led to the irony where Google and their cousins had *MORE* content you could discover than on the library discovery systems.

Today, most libraries just publish holding data online that are tapped upon by commercial web systems around the world (based on a improved variant of the openurl protocol) and give up on the pretense of offering a superior discovery experience.

While there were still holdouts who could point to extreme specialised uses where the web scale discovery service was a little better and a few libraries still retained such obscure systems, in 99% of cases the web search engines were superior with the ability to almost know what you wanted before you looked for it. Libraries hampered by the need to respect privacy had no way to compete even if they had the technology, though they did give it a go with learning analytics and gamification. Sadly, the facebook generation did not appreciate the value of privacy.

The response of information literacy librarians was surprising. By the 2020s, most recognised that information literacy was not about teaching the tool, or even finding sources, and most opted out of the debate, that didn't save them in the long run, but that's another story for the future....

While most academic libraries in the 2010s bought into the discovery meme, a few others saw the writing on the wall sooner and their rallying cry was "delivery not discovery" and decided to opt out of the discovery wars which they saw (correctly as history will record) libraries had already lost and ceded discovery mechanics to outside the library but focused on delivering content once discovery occurred elsewhere.

The majority of commenters branded such moves as "Defeatist" , deriding such moves as turning the library back to a warehouse (albeit a digital one with libraries focusing on digitizing their special collections and having in available everywhere).

In the end though the final seeds of the defeat of the library discovery movement was sowed by the librarians themselves in a completely different direction. By 2030, to their immense surprise, Open Access became the norm (the story of how this came to past is too long and complicated to detail here), hitting over 75% of all published literature on average and close to 100% in areas like life sciences.

This all but annihilated the reason discovery systems existed - aka the need to track which of the articles your institution had access to.

Frank was suddenly broken out of his introspection by a chime.

"Frank, you may want to know your next appointment is in 30 minutes and you may need to prepare for the meeting as I notice your slide deck isn't complete", chimed in Annie.

"Also you may want to put in more content about why you need such a big budget for human staff when most of the library functions can be run a lot more cheaply by intelligent agents like me", she added a bit too smugly.

Frank sighed again. He almost forgot , Annie wasn't really human. These intelligent agents were so human-like these days and getting more capable by the day. As it is , he was only one of a dozen humans on staff in the whole library and he knew it would only get worse.

God, he was looking forward for his retirement.

The end..............

In many ways, the place of search & discovery in the library cuts into the heart of the debate of what a library is for. This piece is to provoke discussion not an act of prognostication. I was inspired to write by the following essays.

Responses have ranged from "this is radical" to "this isn't radical enough and will obviously happen way before 2035". Another comment was that this is the death of library "search" not "discovery". Suggestions to avoid this future (see this and this as well as comment below), recommend that we leverage our understanding of our community users to create a “personalized” slice of the discovery pie". I fully agree, but I wonder if individual libraries are up to the task as traditionally the ones provided personalised online experiences have not been libraries.

A tongue in a cheek, thought experiment or perhaps precautionary tale of the ultimate fate of library discovery services in 2035.

With a sigh, Frank Murphy, head of library systems of Cambridge-Yale University made a quick gesture at his computing device and the system began to shut down the library discovery service for the last time. Or at least that was what he intended but the gesture based device - a distant descendant of the Kinect device refused to comply.

With another sigh, Frank tried again using the more reliable voice recognition system, "Shut down Library discovery service server" he commanded, and this time was rewarded with a request for confirmation.

"Yes, do it".

It was done. Head of library outreach Annie Watson observing the proceedings, chimed in, "Oh well, it was inevitable, but at least we tried."

"Besides this way we don't need to worry about the year 2038 problem", she added perhaps a little insensitively.

Frank didn't respond. Where had it all gone wrong? It began so promisingly, in the 2010s. After years of declining usage of library catalogues, libraries fought back with web scale discovery tools.

Summon, EDS (god rest their souls) and many others that followed seem to be the answer to the riddle "Why start at the library home page". Spotting modern interfaces light years ahead of existing catalogues, designed for users and not librarians, and most importantly the ability to break down silos in library content, libraries were finally able to compete with web search engines on a even footing using technologies and architecture that mimicked the then dominant web search giant Google.

Back then, Frank a fresh new systems librarian in Yale University (before the MOOC wars of 2020s forced the extinction and consolidation of all but the top universities in the world) was so impressed he enthusiastically blogged that "after false dawn of Federated Searches and Next generation catalogues, web scale discovery services were the answer".

To be fair, Frank had reason to be optimistic. Users were raving, usage of eresources was up substantially and the future seemed bright. The rot seemed to have stopped.

But this didn't last. As libraries caught up with existing commercial web technology, the web companies pulled ahead again and usage started to fall again. In retrospect, it seems sheer hubris to think that library technology companies most of whom were squabbling with each other could compete with the likes of Apple, Google, Amazon who had a larger budget on their entertainment fund alone than all the revenues of the library technology vendors combined.

While library technology companies were figuring out how to make interfaces that weren't hellish to use in the 2010s, commercial companies were figuring out voice recognition technologies, intelligent agents, semantic web technologies and more.

The last stung in particular. Many visionary prominent librarians had opined that one way the library could have a place at the table would be as part of the linked data movement, exposing the vast store of rich metadata libraries had.

It didn't work out that way. Squabbles over MARC, RDA , inertia and the inability to collaborate meant that as usual the library missed the boat.

By the time, libraries and library vendors realised the danger it was too late. Even a desperate final last ditch consolidation of Ebsco -Proquest - Gale to create a super search that boasted the combined content and metadata of practically all library content providers did nothing but delay the end.

The logic of pushing your content to where the eyeballs are was too strong. While this initially benefited Web scale discovery searches, where content providers were forced to add their content there or see declining usage. In the long run, this logic was even stronger for putting content on Google, Mendeley and other truly web scale services. Divisions and squabbles among content providers and discovery vendors led to the irony where Google and their cousins had *MORE* content you could discover than on the library discovery systems.

Today, most libraries just publish holding data online that are tapped upon by commercial web systems around the world (based on a improved variant of the openurl protocol) and give up on the pretense of offering a superior discovery experience.

While there were still holdouts who could point to extreme specialised uses where the web scale discovery service was a little better and a few libraries still retained such obscure systems, in 99% of cases the web search engines were superior with the ability to almost know what you wanted before you looked for it. Libraries hampered by the need to respect privacy had no way to compete even if they had the technology, though they did give it a go with learning analytics and gamification. Sadly, the facebook generation did not appreciate the value of privacy.

The response of information literacy librarians was surprising. By the 2020s, most recognised that information literacy was not about teaching the tool, or even finding sources, and most opted out of the debate, that didn't save them in the long run, but that's another story for the future....

While most academic libraries in the 2010s bought into the discovery meme, a few others saw the writing on the wall sooner and their rallying cry was "delivery not discovery" and decided to opt out of the discovery wars which they saw (correctly as history will record) libraries had already lost and ceded discovery mechanics to outside the library but focused on delivering content once discovery occurred elsewhere.

The majority of commenters branded such moves as "Defeatist" , deriding such moves as turning the library back to a warehouse (albeit a digital one with libraries focusing on digitizing their special collections and having in available everywhere).

In the end though the final seeds of the defeat of the library discovery movement was sowed by the librarians themselves in a completely different direction. By 2030, to their immense surprise, Open Access became the norm (the story of how this came to past is too long and complicated to detail here), hitting over 75% of all published literature on average and close to 100% in areas like life sciences.

This all but annihilated the reason discovery systems existed - aka the need to track which of the articles your institution had access to.

Frank was suddenly broken out of his introspection by a chime.

"Frank, you may want to know your next appointment is in 30 minutes and you may need to prepare for the meeting as I notice your slide deck isn't complete", chimed in Annie.

"Also you may want to put in more content about why you need such a big budget for human staff when most of the library functions can be run a lot more cheaply by intelligent agents like me", she added a bit too smugly.

Frank sighed again. He almost forgot , Annie wasn't really human. These intelligent agents were so human-like these days and getting more capable by the day. As it is , he was only one of a dozen humans on staff in the whole library and he knew it would only get worse.

God, he was looking forward for his retirement.

The end..............

In many ways, the place of search & discovery in the library cuts into the heart of the debate of what a library is for. This piece is to provoke discussion not an act of prognostication. I was inspired to write by the following essays.

Responses have ranged from "this is radical" to "this isn't radical enough and will obviously happen way before 2035". Another comment was that this is the death of library "search" not "discovery". Suggestions to avoid this future (see this and this as well as comment below), recommend that we leverage our understanding of our community users to create a “personalized” slice of the discovery pie". I fully agree, but I wonder if individual libraries are up to the task as traditionally the ones provided personalised online experiences have not been libraries.