NCFS - Network-Coding-Based Distributed File System

Introduction

NCFS is a proof-of-concept prototype of a Network-Coding-based Distributed
File System. NCFS is a proxy-based file system that interconnects multiple
storage nodes. It relays regular read/write operations between user
applications and storage nodes, and relays data among storage nodes during the
data repair process. NCFS is built on FUSE, an open-source, programmable
user-space file system that provides application programmable interfaces
(APIs) for file system operations. From the point of view of user
applications, NCFS presents a file system layer that transparently stripes
data across physical storage nodes.

NCFS supports a specific regenerating coding scheme called Exact Minimum
Bandwidth Regenerating (E-MBR) codes [Rashmi et al.; 2009], which seek to
minimize repair bandwidth. One key property of E-MBR is that it does not
require any encoding/decoding intelligence on the storage nodes, as long as
the storage nodes provide the standard I/O interfaces. NCFS also supports
RAID-based erasure coding schemes, so as to enable us to conduct a
comprehensive empirical study of different classes of data recovery for
distributed storage under real network settings. To the best of our
knowledge, NCFS is the first work that realizes regenerating codes in a
practical distributed storage system.