From user-return-14281-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Mon Mar 07 19:32:40 2011
Return-Path:
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: (qmail 31296 invoked from network); 7 Mar 2011 19:32:40 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
by minotaur.apache.org with SMTP; 7 Mar 2011 19:32:40 -0000
Received: (qmail 81227 invoked by uid 500); 7 Mar 2011 19:32:38 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 81199 invoked by uid 500); 7 Mar 2011 19:32:38 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help:
List-Unsubscribe:
List-Post:
List-Id:
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 81176 invoked by uid 99); 7 Mar 2011 19:32:38 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Mar 2011 19:32:38 +0000
X-ASF-Spam-Status: No, hits=-0.7 required=5.0
tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of lewilists@gmail.com designates 209.85.212.44 as permitted sender)
Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44)
by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Mar 2011 19:32:29 +0000
Received: by vws6 with SMTP id 6so4709556vws.31
for ; Mon, 07 Mar 2011 11:32:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;
h=domainkey-signature:content-type:mime-version:subject:from
:in-reply-to:date:content-transfer-encoding:message-id:references:to
:x-mailer;
bh=hn12xhZMcuDViFG0VKOgtuhpmT698pp2cVyC7F+EB0A=;
b=rjU9ACaOsacUr7Imf9N5Kl+d3JTZfLbz7cqFIW7OZNT7Pd+AVa6BqvFe0HZ6iq7o8t
66Hlf3cCK8lZCzWGDasdPZ6yUnBeuNA9Txv/1K6GGMPdn/z1yKwCLnOVLjugNvfS+GHJ
vnrEkURpSWWHYIcKLLOv1VWTJp13YPsoNzqRA=
DomainKey-Signature: a=rsa-sha1; c=nofws;
d=gmail.com; s=gamma;
h=content-type:mime-version:subject:from:in-reply-to:date
:content-transfer-encoding:message-id:references:to:x-mailer;
b=bMJDiuvgFcPedbTDiQE51z0ij90xPnoIbiHXYXEBCIDvEawRx5mxV5oaEyL3toXzAD
6lD59QxfaHRVJq8/7Rcg9wzVqiF75YbBmTRB/oTfU9efJwHtcypG04DP9wVkk/MhP3C3
HGbxYhPC5TsRTILvLOE7ekd2o0lcxpa8OpBgI=
Received: by 10.52.0.43 with SMTP id 11mr5924682vdb.283.1299526327570;
Mon, 07 Mar 2011 11:32:07 -0800 (PST)
Received: from [192.168.0.197] (173-13-177-162-sfba.hfc.comcastbusiness.net [173.13.177.162])
by mx.google.com with ESMTPS id a11sm320414vdu.25.2011.03.07.11.32.06
(version=TLSv1/SSLv3 cipher=OTHER);
Mon, 07 Mar 2011 11:32:07 -0800 (PST)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1082)
Subject: Re: how large can a cluster over the WAN be?
From: John Lewis
In-Reply-To:
Date: Mon, 7 Mar 2011 11:32:05 -0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <23A30D2D-AB34-4CA0-8B59-79BD6D4637CA@gmail.com>
References:
To: user@cassandra.apache.org
X-Mailer: Apple Mail (2.1082)
X-Virus-Checked: Checked by ClamAV on apache.org
When you say decent latency and throughput what numbers do you consider =
decent? I know throughput would be highly dependent on the quantity of =
kb shoved through the pipe so I would expect throughput needs would be =
highly dependent on the data actually in cassandra.=20
Thanks for the info, I am getting ready for a multisite deployment and =
any bit of information will help us qualify or disqualify network =
vendors with regards to what we anticipate our needs to be.
Lewis
On Mar 7, 2011, at 10:25 AM, Robert Coli wrote:
> On Sun, Mar 6, 2011 at 1:39 PM, Mimi Aluminium =
wrote:
>> Are you familiar with Cassandra cluster that is installed in =
datacenters
>> that are spread across the WAN? can you comment on the perfromance of =
such
>> installation?
>> What is the largest size of of such a cluster you are aware of?
>=20
> Digg operated a 40 node cassandra 0.6.1 to 0.6.6+1072 cluster in two
> datacenters, one on each coast of the US. It worked fine, with the
> usual caveats that come with that sort of network latency. We used a
> custom snitch and rackaware, but the implementation in 0.6.x was
> insufficiently robust and ended up disabled in favor of simple snitch
> and rack unaware with the nodes simply alternating data centers on the
> ring. As we only read from one half of the cluster, this meant that we
> often only had one local replica of our data. We recently moved this
> cluster from two physical DCs to one. Other than the network trickery
> involved in keeping IP addresses the same, the only negative aspect of
> the WAN in this case was the bottleneck while copying the data.
>=20
> As long as you have decent latency and throughput across the WAN link,
> cassandra should be fine. This should be especially true in 0.7 and
> with the DynamicEndpointSnitch enabled. I have clearly used a number
> of weasel words in this summary, you should of course test for your
> case, especially if that case involves more than two datacenters.
>=20
> =3DRob