Nutch 2.X Tutorial

Introduction

This document describes how to get Nutch 2.X to use HBase as a storage backend for Gora. It is assumed that you have a working knowledge of configuring Nutch 1.X, as currently configuration in 2.X is more complex. It is important to take this in to consideration before progressing any further. We therefore strongly advise that you check out the Nutch 1.X tutorial.

Obtaining Software and Configuration

Grab the latest distribution of Nutch 2.X from here. Do NOT build the source yet. From now on we will refer to the directory where the Nutch code resides as $NUTCH_HOME.

Download and configure HBase 0.98.8-hadoop2. You can get it here (N.B. Each version of Gora is tied to a particular version of HBase, we therefore suggest you use this version if possible. If you decide to use another version of HBase please do not be surprised if the stack does not work. You should also obtain current documentation for HBase however please again take into consideration that the version of HBase we recommend you use may not correlate to the current documentation. Please keep this in mind and use your initiative.

Specify the GORA backend in $NUTCH_HOME/conf/nutch-site.xml along with all of the other Configuration options suggested within the Nutch 1.x tutorial.