hadoop-common-user mailing list archives

To be precise, the projected data is around 1 PB.
But the publishing rate is also around 1GBPS.
Please suggest.
________________________________
From: jagaran das <jagaran_das@yahoo.co.in>
To: "common-user@hadoop.apache.org" <common-user@hadoop.apache.org>
Sent: Wednesday, 10 August 2011 12:58 AM
Subject: Namenode Scalability
In my current project we are planning to streams of data to Namenode (20 Node Cluster).
Data Volume would be around 1 PB per day.
But there are application which can publish data at 1GBPS.
Few queries:
1. Can a single Namenode handle such high speed writes? Or it becomes unresponsive when GC
cycle kicks in.
2. Can we have multiple federated Name nodes sharing the same slaves and then we can
distribute the writes accordingly.
3. Can multiple region servers of HBase help us ??
Please suggest how we can design the streaming part to handle such scale of data.
Regards,
Jagaran Das