On 03/29/2013 01:09 AM, David Parks wrote:
> Hmm, seems intriguing. I’m still not totally clear on bigtop here. It
> seems like they’re creating and maintain basically an installer for Hadoop?
>
> I tried following their docs for Ubuntu, but just get a 404 error on the
> first step, so it makes me wonder how reliable that project is.
>
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
>
> Has anyone actually used bigtop to deploy Hadoop in a production
> environment?
>
Hi David,
You may want to send an email to Apache Bigtop mailing lists with your
questions, but the goal of Apache Bigtop is three folds:
1/ Provide top notch packages for Apache Hadoop related projects for the
most popular GNU/Linux distributions
2/ Provide a point of integration and testing for all these projects
3/ Provide means to reliably deploy a complete stack.
Apache Bigtop was donated by Cloudera to the Apache Foundation and is
now used as the base for CDH (Cloudera's distribution).
All the projects supported by Apache Bigtop have packages for
Debian/Ubuntu/SLES11/Fedora/CentOS.
They also have tests to exercise integration points between all of them
(ex: Hive can use HBase which sits on top of HDFS). And in order to run
these tests, we also have a test framework.
Also before we can test for integration, we also have to ensure they can
be properly installed/upgraded/removed, with the right users, ulimits,
rights and so forth. So to that end, we also have a large chunk of the
tests and testing framework dedicated to testing the packages themselves.
And finally, regarding the deployment, we have the following:
* Boxgrinder recipe people can use and modify to suit their need. This
is useful if you want to create your own virtual machine (kvm, vmware,
virtualbox) but also want to create images for ec2.
* kickstart file to build a live fedora image
* puppet recipes to deploy all these services.
Regarding your issue with the instructions on the wiki, this is because
they have not been updated since Apache Bigtop became a top level
project, and the location of the artefacts has changed. When reading the
instructions, please use the following as the base url:
http://www.apache.org/dist/bigtop/bigtop-0.5.0/
Please, let me know if you still have some issues.
Thanks,
Bruno