From user-return-27392-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Wed Jul 4 01:56:53 2012
Return-Path:
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
by minotaur.apache.org (Postfix) with SMTP id A81CA90A3
for ; Wed, 4 Jul 2012 01:56:53 +0000 (UTC)
Received: (qmail 81391 invoked by uid 500); 4 Jul 2012 01:56:50 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 81357 invoked by uid 500); 4 Jul 2012 01:56:50 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help:
List-Unsubscribe:
List-Post:
List-Id:
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 81346 invoked by uid 99); 4 Jul 2012 01:56:50 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jul 2012 01:56:50 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of dan.foody@gmail.com designates 209.85.216.172 as permitted sender)
Received: from [209.85.216.172] (HELO mail-qc0-f172.google.com) (209.85.216.172)
by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jul 2012 01:56:42 +0000
Received: by qcac10 with SMTP id c10so3393535qca.31
for ; Tue, 03 Jul 2012 18:56:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=from:mime-version:content-type:subject:date:in-reply-to:to
:references:message-id:x-mailer;
bh=d7b+SP8/eGUMkqhKCI3GYIvb+uqtO3f0ic/HDY5K/7s=;
b=PacOd3d3yS8VHfn5clYdRvSM809dV1EMOwiBS4Ad/weLF26s1GujWDqEnPjXGvF2pQ
QAnzWUd0/r/7L7Z+mbFg2OZgrfjfRTdN+2jFdDqFe93MjSdnRBe6N39uN+MUkrLECHKV
uRAPWVGL7HQ52VGsXVpTHrc2eRLZDhQ33yJ7N0X7gLN+Qmj+Wflr4Vm24H+jcXhR0/4c
4e2c5GtEaoO4xPJPtqzipTnn7ukDjx6zZZzG00oLGgaRRRkz6ddwHReEynvogYTrDsHu
QuFmWolJnio7ybbitehPtLNKwJs6QpEGoF3OzMA9Ob4CKwYdpkMvVe+iHT28TToAI1Eh
sqrQ==
Received: by 10.229.135.9 with SMTP id l9mr9738165qct.38.1341366981955;
Tue, 03 Jul 2012 18:56:21 -0700 (PDT)
Received: from dan-foodys-air.home (pool-173-76-21-132.bstnma.fios.verizon.net. [173.76.21.132])
by mx.google.com with ESMTPS id he6sm40380422qab.13.2012.07.03.18.56.19
(version=TLSv1/SSLv3 cipher=OTHER);
Tue, 03 Jul 2012 18:56:20 -0700 (PDT)
From: Dan Foody
Mime-Version: 1.0 (Apple Message framework v1278)
Content-Type: multipart/alternative; boundary="Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195"
Subject: Re: Expanding Cassandra on EC2 with consistency
Date: Tue, 3 Jul 2012 21:56:32 -0400
In-Reply-To:
To: user@cassandra.apache.org
References:
Message-Id: <47D9EE5E-2AA3-4B79-88B6-A5D8C8390B0A@gmail.com>
X-Mailer: Apple Mail (2.1278)
X-Virus-Checked: Checked by ClamAV on apache.org
--Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=iso-8859-1
Hi Alex,
Can you share what replication factor you're running?
And, are you using ephemeral disks or EBS volumes?
Thanks!
- Dan
On Jul 3, 2012, at 5:52 PM, Alex Major wrote:
> Hi Mike,
>=20
> We've run a small (4 node) cluster in the EU region since September =
last year. We run across all 3 availability zones in the EU region, with =
2 nodes in one AZ and then a further node in each AZ. The latency =
difference between running inside of and between AZ's has been minimal =
in our experience.=20
>=20
> It's only when we've gone cross-region that there's been latency =
problem. We temporarily ran a 9 node cluster across 3 regions, however =
even then using local quoram the latency was better than the standard =
datacenter - datacenter latency we're used to.
>=20
> EC2Snitch is definitely the way to go in favour of NTS in my opinion. =
NTS was a pain to get setup with the internal (private) IP address =
setup, so much so that we never got it safely replicating the data as we =
wanted.
>=20
> Alex.
>=20
> On Tue, Jul 3, 2012 at 2:16 PM, Michael Theroux =
wrote:
> Hello,
>=20
> We are currently running a web application utilizing Cassandra on EC2. =
Given the recent outages experienced with Amazon, we want to consider =
expanding Cassandra across availability zones sooner rather than later.
>=20
> We are trying to determine the optimal way to deploy Cassandra in this =
deployment. We are researching the NetworkTopologyStrategy, and the =
EC2Snitch. We are also interested in providing a high level of read or =
write consistency,
>=20
> My understanding is that the EC2Snitch recognizes availability zones =
as racks, and regions as data-centers. This seems to be a common =
configuration. However, if we were to want to utilize queries with a =
READ or WRITE consistency of QUORUM, would there be a high possibility =
that the communication necessary to establish a quorum, across =
availability zones?
>=20
> My understanding is that the NetworkTopologyStrategy attempts to =
prefer replicas be stored on other racks within the datacenter, which =
would equate to other availability zones in EC2. This implies to me =
that in order to have the quorum of nodes necessary to achieve =
consistency, that Cassandra will communicate with nodes across =
availability zones.
>=20
> First, is my understanding correct? Second, given the high latency =
that can sometimes exists between availability zones, is this a problem, =
and instead we should treat availability zones as data centers?
>=20
> Ideally, we would be able to setup a situation where we could store =
replicas across availability zones in case of failure, but establish a =
high level of read or write consistency within a single availability =
zone.
>=20
> I appreciate your responses,
> Thanks,
> -Mike
>=20
>=20
>=20
>=20
--Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=iso-8859-1
Hi =
Alex,

Can you share what replication factor you're =
running?

And, are you using ephemeral disks or EBS =
volumes?

Thanks!

- =
Dan

On Jul 3, 2012, at 5:52 PM, Alex Major wrote:

Hi =
Mike,

We've run a small (4 node) cluster in the EU =
region since September last year. We run across all =
3 availability zones in the EU region, with 2 nodes in one AZ =
and then a further node in each AZ. The latency difference between =
running inside of and between AZ's has been minimal in our =
experience.

It's only when we've gone cross-region that there's =
been latency problem. We temporarily ran a 9 node cluster across 3 =
regions, however even then using local quoram the latency was better =
than the standard datacenter - datacenter latency we're used to.

EC2Snitch is definitely the way to go in =
favour of NTS in my opinion. NTS was a pain to get setup with the =
internal (private) IP address setup, so much so that we never got it =
safely replicating the data as we wanted.

We are currently running a web application utilizing Cassandra on EC2. =
Given the recent outages experienced with Amazon, we want to =
consider expanding Cassandra across availability zones sooner rather =
than later.

We are trying to determine the optimal way to deploy Cassandra in this =
deployment. We are researching the NetworkTopologyStrategy, and =
the EC2Snitch. We are also interested in providing a high level of =
read or write consistency,

My understanding is that the EC2Snitch recognizes availability zones as =
racks, and regions as data-centers. This seems to be a common =
configuration. However, if we were to want to utilize queries with =
a READ or WRITE consistency of QUORUM, would there be a high possibility =
that the communication necessary to establish a quorum, across =
availability zones?

My understanding is that the NetworkTopologyStrategy attempts to prefer =
replicas be stored on other racks within the datacenter, which would =
equate to other availability zones in EC2. This implies to me that =
in order to have the quorum of nodes necessary to achieve consistency, =
that Cassandra will communicate with nodes across availability =
zones.

First, is my understanding correct? Second, given the high latency =
that can sometimes exists between availability zones, is this a problem, =
and instead we should treat availability zones as data centers?

Ideally, we would be able to setup a situation where we could store =
replicas across availability zones in case of failure, but establish a =
high level of read or write consistency within a single availability =
zone.