Announcing Webrecorder and DAT Integration

Aug 2, 2018 • Ilya Kreymer

After the major UI overhaul in June, which introduced new narrative and curatorial capabilities to Webrecorder, the Webrecorder team has been exploring how to improve the sharing and distribution of web archives.

In addition to high-fidelity capture and replay, sharing and distribution of web archives in a decentralized way has always been a core goal of Webrecorder.

A few years ago, we’ve launched the Webrecorder Player, a desktop tool that allows users to browse any web archive they have saved locally through a desktop application.

To further expand the distribution and preservation capabilities of Webrecorder, we’ve decided to integrate support for the
Dat Protocol into Webrecorder and Webrecorder Player. The DAT protocol provides a peer-to-peer
network that we are excited to use for sharing web archive collections.

Sharing Web Archives on the DAT Peer-to-Peer Network

With this feature, users will be able to share any Webrecorder collection as a Dat and receive a dat:// url for the collection.

The next version of Webrecorder Player (in addition to loading WARC files) will then allow users to enter
a corresponding dat:// url obtained from Webrecorder and sync the collection locally, allowing users to browse it on the desktop (and even offline!).

While Webrecorder Player is running, it will also seed the collection on the peer-to-peer Dat network, allowing others to download it to their copy of Webrecorder Player, without requiring access to https://webrecorder.io/

When a collection is updated on webrecorder.io, users will be able to load the updated collection via the same Dat url, and an updated collection will be synced automatically.

Here’s a sneak preview of how this functionality will look like:

Your browser does not support the video tag.

Using Dat will augment the existing WARC download/upload features, and provide users a way to automatically sync collections from webrecorder.io to a local machine (or several) and share them with a private Dat url.

While we’re not quite ready to release this feature yet on webrecorder.io (although all the work is on Github and can be run locally), please do tell us if you have any feedback or feature requests. Feel free to contact us via GitHub, via comments here or via support [at] webrecorder.io