From user-return-16368-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Tue May 3 13:25:10 2011
Return-Path:
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
by minotaur.apache.org (Postfix) with SMTP id C06C6A6C
for ; Tue, 3 May 2011 13:25:10 +0000 (UTC)
Received: (qmail 80867 invoked by uid 500); 3 May 2011 13:25:08 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 80846 invoked by uid 500); 3 May 2011 13:25:08 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help:
List-Unsubscribe:
List-Post:
List-Id:
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 80838 invoked by uid 99); 3 May 2011 13:25:08 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 May 2011 13:25:08 +0000
X-ASF-Spam-Status: No, hits=3.7 required=5.0
tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of stevenpsmith123@gmail.com designates 209.85.160.172 as permitted sender)
Received: from [209.85.160.172] (HELO mail-gy0-f172.google.com) (209.85.160.172)
by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 May 2011 13:25:00 +0000
Received: by gyf3 with SMTP id 3so27395gyf.31
for ; Tue, 03 May 2011 06:24:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;
h=domainkey-signature:mime-version:from:date:message-id:subject:to
:content-type;
bh=UwoVJGAsjqAWdid2fJfWn4THHd0sd/5KKRX4WUQLrTQ=;
b=raTegwluhwN1ah8kdwM6kbR+NxxUaHsxqzV1cGKaJhgS4mcxzbcmXLmMQINW+RCe0H
XOakg2tYtZY42OefEwqQIiCj/zpU0jZUs2tzCIirFHZVUJLqA0RvzBoTtaKMhVwvKcf4
kKPQvfGr9mH6AsUVtu6Bj6MGmSAHoqPnfhqt4=
DomainKey-Signature: a=rsa-sha1; c=nofws;
d=gmail.com; s=gamma;
h=mime-version:from:date:message-id:subject:to:content-type;
b=vO27q1s0/g0mHpB976zj9cCoQLx99gzdJb03S33VMU5faiielSKU04UI68x5+wYB8m
i39nPAaS+msQBU8E3ZpcCkPgQUSDq4Pjep60GPmQ+cmfJFpLOh+JX65nLs3dGN9CSK13
SK4tyAg3r5/gdmjEjVEC3hX+peMM6LSIBI++Y=
Received: by 10.236.73.162 with SMTP id v22mr2222385yhd.247.1304429080079;
Tue, 03 May 2011 06:24:40 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.147.39.4 with HTTP; Tue, 3 May 2011 06:24:20 -0700 (PDT)
From: Steve Smith
Date: Tue, 3 May 2011 09:24:20 -0400
Message-ID:
Subject: Write performance help needed
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=20cf3005139cf3eeee04a25f0f27
X-Virus-Checked: Checked by ClamAV on apache.org
--20cf3005139cf3eeee04a25f0f27
Content-Type: text/plain; charset=ISO-8859-1
I am working for client that needs to persist 100K-200K records per second
for later querying. As a proof of concept, we are looking at several
options including nosql (Cassandra and MongoDB).
I have been running some tests on my laptop (MacBook Pro, 4GB RAM, 2.66 GHz,
Dual Core/4 logical cores) and have not been happy with the results.
The best I have been able to accomplish is 100K records in approximately 30
seconds. Each record has 30 columns, mostly made up of integers. I have
tried both the Hector and Pelops APIs, and have tried writing in batches
versus one at a time. The times have not varied much.
I am using the out of the box configuration for Cassandra, and while I know
using 1 disk will have an impact on performance, I would expect to see
better write numbers than I am.
As a point of reference, the same test using MongoDB I was able to
accomplish 100K records in 3.5 seconds.
Any tips would be appreciated.
- Steve
--20cf3005139cf3eeee04a25f0f27
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
I am working for client that needs to persist 100K-200K records per second =
for later querying. =A0As a proof of concept, we are looking at several opt=
ions including nosql (Cassandra and MongoDB).

I have bee=
n running some tests on my laptop (MacBook Pro, 4GB RAM, 2.66 GHz, Dual Cor=
e/4 logical cores) and have not been happy with the results.

The best I have been able to accomplish is 100K records=
in approximately 30 seconds. =A0Each record has 30 columns, mostly made up=
of integers. =A0I have tried both the Hector and Pelops APIs, and have tri=
ed writing in batches versus one at a time. =A0The times have not varied mu=
ch.

I am using the out of the box configuration for Cassand=
ra, and while I know using 1 disk will have an impact on performance, I wou=
ld expect to see better write numbers than I am. =A0

As a point of reference, the same test using MongoDB I was able to acc=
omplish 100K records in 3.5 seconds.