Posts Tagged ‘Search Sites’

Although the internet is overflowing with deep data most of it is trapped in old APIs that make working with the data complicated. Wikipedia, one of the largest repositories of contributed information in several languages, still relies on an old MediaWiki API that is almost useless at any meaningful search. Sites have sprung up to try to deal with these shortcoming such as DBPedia that tries to wrangle all this information into a standard meaningful API. But how great would it be instead if Wikipedia developers actually created and maintained a powerful API. It would mean giant leaps to the integration and usefulness of Wikipedia. All of the sudden any app would be able to take advantage of all the crowd sourcing done in Wikipedia in a reliable way.

As much as I wish the above for Wikipedia, this blog post is about the developers who have done just that. The Discogs developers have been constantly maintaining their API. Although they had gone down a path I did not approve of recently with adding limits to the number of images that could be downloaded per account as well as requiring OAuth verification, that was complex and cumbersome to implement. They are now fixing those issues while maintaing backwards compatibility. So the good news is that our search plugin for Discogs will continue to work without update. We have already used the Google OAuth framework to implement authentication with our own private account, to make it as easy as possible for users. But new developers wishing to integrate Discogs will have a much easier time using a simple key/secret pair. The 1000 image limit has also been removed and the signed URL provided directly in the downloaded information. This is the kind of simplicity one wishes in all APIs.

I mention Discogs by name only because they are the latest site in adding simple new features to make accessing their data by their users easier. But the truth is many developers have been doing an extraordinary job sharing their data via an API. The TMDB API always comes to the top when I am asked about good API designs, it has been easy to work with and 100% reliable for DVDpedia. The TV Rage and TheTVDB are more complex APIs but work well all the same. In the music world MusicBrainz has been solid. MovieMeter has been moving backwards in features since maintaining the original API required a lot of work for a single developer. BoardGameGeek has an API that is embedded into their system that works but could do with some 2015 technology improvements. But speaking of improvements the API that could really do with some love is the Library of Congress. Although now using a new SRU standard, it’s still based on a system that is decades old. But it seems updating the way libraries share their catalog is a gigantic undertaking. The above list is not extensive as there are many other search sites with APIs that we are thankful exist: OFDB, Open Library, Google Books, Freebase (Bought by Google), Amazon AWS and AbeBooks.

We look forward to a world where standards improve, for example JSON format has been marvelous at making integrating APIs easy and consistent. Some day information on the internet will be truly available to our future computer program overlords that will do all the communicating for us and present the information we want on command. Hopefully they will be able to even update themselves as data improves. In the meantime we hope developers continue the hard work of maintaining all these endpoints and adding more where needed.