We are looking to use zookeeper for optimistic concurrency. Basically when the user saves data on a screen, we need to lock, read to ensure that no one else has changed the row while user is editing data, persist data and unlock znode.

If the app/thread does not get a lock, we may set a watch so that polling is avoided.

Our application is write intensive certain times of the day. We may get about 100k requests per second. Can zookeeper handle this volume?

This sounds highly error prone to me regardless of whether or not zookeepercan handle the load-. Why not just use a standard transaction model with avector clock or other timing device to detect conflicts so you don't haveto worry about a second server to talk to (zookeeper) to do an update?On Jul 31, 2013 7:17 AM, "Baskar Duraikannu" <[EMAIL PROTECTED]>wrote:

> Hello>> We are looking to use zookeeper for optimistic concurrency. Basically when> the user saves data on a screen, we need to lock, read to ensure that no> one else has changed the row while user is editing data, persist data and> unlock znode.>> If the app/thread does not get a lock, we may set a watch so that polling> is avoided.>> Our application is write intensive certain times of the day. We may get> about 100k requests per second. Can zookeeper handle this volume?

We cannot always resolve conflicts ourselves. For example, let us say that a) user1 changed the name from 'Kathy' to Katherineb) user2 changes the name from 'Kathy' to 'Kat' Both read 'Kathy' as input; user1's update succeeded. If we need to let user2 know that something has changed as this may result in the user not changing 'Kathy' to 'Kat' (as an example).Hope this explains

> Date: Wed, 31 Jul 2013 07:49:39 -0400> Subject: Re: Zookeeper performance> From: [EMAIL PROTECTED]> To: [EMAIL PROTECTED]> > This sounds highly error prone to me regardless of whether or not zookeeper> can handle the load-. Why not just use a standard transaction model with a> vector clock or other timing device to detect conflicts so you don't have> to worry about a second server to talk to (zookeeper) to do an update?> On Jul 31, 2013 7:17 AM, "Baskar Duraikannu" <[EMAIL PROTECTED]>> wrote:> > > Hello> >> > We are looking to use zookeeper for optimistic concurrency. Basically when> > the user saves data on a screen, we need to lock, read to ensure that no> > one else has changed the row while user is editing data, persist data and> > unlock znode.> >> > If the app/thread does not get a lock, we may set a watch so that polling> > is avoided.> >> > Our application is write intensive certain times of the day. We may get> > about 100k requests per second. Can zookeeper handle this volume?

1. Read the current version of the database (stored in a znode's versionmetadata). If it is even, wait and try again; even numbers mean someone iscommitting and the DB might be in an inconsistent state. Then read thestate from the database your update will rely upon (user1.name, in thisinstance). You must also be able to atomically read the current versionfrom the database as well as zookeeper, to ensure that the data is from theversion you think it is. If the DB version does not match the ZK version,restart.2. Once an update is ready to commit, test-and-increment the currentversion in ZK to an even number, write your update to the DB, along withthe eventual version of the data (the next odd number).3. Increment the current version in ZK to an odd number.

The even / odd distinction means that you can detect when someone else isupdating the database, since otherwise there's no way to do so atomicallywith an update to ZK (so another transaction can't tell if you've finishedyour update or not, and so doesn't know when to wait until).

The problem is failure - what happens if a client fails while it's writinga transaction? Eventually someone can increment the transaction number, andif you provide an 'undo' log before you make any changes, that client canpossibly recover from a partial commit. But at this point you need tounderstand your application's requirements in much more detail than we doto make recommendations.

In particular, your storage layer may offer sufficiently powerfulprimitives such that you don't need ZK; although if it's a filesystem thenthat probably isn't true.

> We cannot always resolve conflicts ourselves. For example, let us say that> a) user1 changed the name from 'Kathy' to Katherineb) user2 changes the> name from 'Kathy' to 'Kat'> Both read 'Kathy' as input; user1's update succeeded. If we need to let> user2 know that something has changed as this may result in the user not> changing 'Kathy' to 'Kat' (as an example).> Hope this explains>> > Date: Wed, 31 Jul 2013 07:49:39 -0400> > Subject: Re: Zookeeper performance> > From: [EMAIL PROTECTED]> > To: [EMAIL PROTECTED]> >> > This sounds highly error prone to me regardless of whether or not> zookeeper> > can handle the load-. Why not just use a standard transaction model with> a> > vector clock or other timing device to detect conflicts so you don't have> > to worry about a second server to talk to (zookeeper) to do an update?> > On Jul 31, 2013 7:17 AM, "Baskar Duraikannu" <> [EMAIL PROTECTED]>> > wrote:> >> > > Hello> > >> > > We are looking to use zookeeper for optimistic concurrency. Basically> when> > > the user saves data on a screen, we need to lock, read to ensure that> no> > > one else has changed the row while user is editing data, persist data> and> > > unlock znode.> > >> > > If the app/thread does not get a lock, we may set a watch so that> polling> > > is avoided.> > >> > > Our application is write intensive certain times of the day. We may get> > > about 100k requests per second. Can zookeeper handle this volume?>>

Starting with an expected transaction load well above the normal limits ofoperation is not a grand idea.

Much better to do something simpler like have ZK coordinate shard mastersthat each use conventional methods for handling transactions (see voltdbfor one approach to sharding well to allow each transaction to never spanshards).

Similarly, you can also shard and maintain version numbers, transactionid's and an in-memory transaction table. This allows multi-shard MVCCcommit semantics but can be a bit tricky to deal with transactions stalledby dead nodes.

> So how about the following optimistic approach:>> 1. Read the current version of the database (stored in a znode's version> metadata). If it is even, wait and try again; even numbers mean someone is> committing and the DB might be in an inconsistent state. Then read the> state from the database your update will rely upon (user1.name, in this> instance). You must also be able to atomically read the current version> from the database as well as zookeeper, to ensure that the data is from the> version you think it is. If the DB version does not match the ZK version,> restart.> 2. Once an update is ready to commit, test-and-increment the current> version in ZK to an even number, write your update to the DB, along with> the eventual version of the data (the next odd number).> 3. Increment the current version in ZK to an odd number.>> The even / odd distinction means that you can detect when someone else is> updating the database, since otherwise there's no way to do so atomically> with an update to ZK (so another transaction can't tell if you've finished> your update or not, and so doesn't know when to wait until).>> The problem is failure - what happens if a client fails while it's writing> a transaction? Eventually someone can increment the transaction number, and> if you provide an 'undo' log before you make any changes, that client can> possibly recover from a partial commit. But at this point you need to> understand your application's requirements in much more detail than we do> to make recommendations.>> In particular, your storage layer may offer sufficiently powerful> primitives such that you don't need ZK; although if it's a filesystem then> that probably isn't true.>> Henry>>> On 31 July 2013 15:51, Baskar Duraikannu <[EMAIL PROTECTED]> >wrote:>> > We cannot always resolve conflicts ourselves. For example, let us say> that> > a) user1 changed the name from 'Kathy' to Katherineb) user2 changes the> > name from 'Kathy' to 'Kat'> > Both read 'Kathy' as input; user1's update succeeded. If we need to let> > user2 know that something has changed as this may result in the user not> > changing 'Kathy' to 'Kat' (as an example).> > Hope this explains> >> > > Date: Wed, 31 Jul 2013 07:49:39 -0400> > > Subject: Re: Zookeeper performance> > > From: [EMAIL PROTECTED]> > > To: [EMAIL PROTECTED]> > >> > > This sounds highly error prone to me regardless of whether or not> > zookeeper> > > can handle the load-. Why not just use a standard transaction model> with> > a> > > vector clock or other timing device to detect conflicts so you don't> have> > > to worry about a second server to talk to (zookeeper) to do an update?> > > On Jul 31, 2013 7:17 AM, "Baskar Duraikannu" <> > [EMAIL PROTECTED]>> > > wrote:> > >> > > > Hello> > > >> > > > We are looking to use zookeeper for optimistic concurrency. Basically> > when> > > > the user saves data on a screen, we need to lock, read to ensure> that> > no> > > > one else has changed the row while user is editing data, persist data> > and> > > > unlock znode.> > > >> > > > If the app/thread does not get a lock, we may set a watch so that

On Wed, Jul 31, 2013 at 4:16 AM, Baskar Duraikannu<[EMAIL PROTECTED]> wrote:>> We are looking to use zookeeper for optimistic concurrency. Basically when the user saves data on a screen, we need to lock, read to ensure that no one else has changed the row while user is editing data, persist data and unlock znode.>

> If the app/thread does not get a lock, we may set a watch so that polling is avoided.>> Our application is write intensive certain times of the day. We may get about 100k requests per second. Can zookeeper handle this volume?

We are using optimistic concurrency similar to one explained in http://en.wikipedia.org/wiki/Optimistic_concurrency_controlI went through the "version" concept in zookeeper. I do not understand how that could be used for "concurrency control". Are you suggesting that all writes go to zookeeper first before making it to the database. We are trying to do it differently... For example, let us say we are trying to update rowKey 1 of Person table,1. Create a ephemeral node /locks/Person/rowkey12. Zookeeper may create a node called /locks/Person/rowKey1-0000 (This means that we got lock)3. If we got, /locks/Person/rowKey1-0001 , we decrement and keep checking child nodes of /locks/Person until /locks/Person/rowKey1-0001 becomes the first one in the list. Pls help/

> We may get about 100k requests per second. Can zookeeper handle this volume?

Depends on your r/w ratio. ZK isn't always ideal for 'user-scale' write heavy workload (lots of things being tracked and updated) . My experience is that you'll have memory pressure (due to the size of the tree) and disk pressure (due to the frequency of snapshotting). You'll need to look at ssds or ram disks for the latter and expect to invest time tuning the JVM, ZK parameters for the former.

I agree that it is likely to be error prone without a lot of careful engineering; a design that avoids/reduces locking or a technology that has locking built-in might be a better fit.

BillOn Wednesday 31 July 2013 at 12:16, Baskar Duraikannu wrote:

> Hello> > We are looking to use zookeeper for optimistic concurrency. Basically when the user saves data on a screen, we need to lock, read to ensure that no one else has changed the row while user is editing data, persist data and unlock znode. > > If the app/thread does not get a lock, we may set a watch so that polling is avoided.> > Our application is write intensive certain times of the day. We may get about 100k requests per second. Can zookeeper handle this volume?

> Date: Wed, 31 Jul 2013 20:09:59 +0100> From: [EMAIL PROTECTED]> To: [EMAIL PROTECTED]> Subject: Re: Zookeeper performance> > > > We may get about 100k requests per second. Can zookeeper handle this volume?> > Depends on your r/w ratio. ZK isn't always ideal for 'user-scale' write heavy workload (lots of things being tracked and updated) . My experience is that you'll have memory pressure (due to the size of the tree) and disk pressure (due to the frequency of snapshotting). You'll need to look at ssds or ram disks for the latter and expect to invest time tuning the JVM, ZK parameters for the former.> > I agree that it is likely to be error prone without a lot of careful engineering; a design that avoids/reduces locking or a technology that has locking built-in might be a better fit.> > Bill> > > > > On Wednesday 31 July 2013 at 12:16, Baskar Duraikannu wrote:> > > Hello> > > > We are looking to use zookeeper for optimistic concurrency. Basically when the user saves data on a screen, we need to lock, read to ensure that no one else has changed the row while user is editing data, persist data and unlock znode. > > > > If the app/thread does not get a lock, we may set a watch so that polling is avoided.> > > > Our application is write intensive certain times of the day. We may get about 100k requests per second. Can zookeeper handle this volume? > >

Starting with an expected transaction load well above the normal limits ofoperation is not a grand idea.

Much better to do something simpler like have ZK coordinate shard mastersthat each use conventional methods for handling transactions (see voltdbfor one approach to sharding well to allow each transaction to never spanshards).

Similarly, you can also shard and maintain version numbers, transactionid's and an in-memory transaction table. This allows multi-shard MVCCcommit semantics but can be a bit tricky to deal with transactions stalledby dead nodes.

> So how about the following optimistic approach:>> 1. Read the current version of the database (stored in a znode's version> metadata). If it is even, wait and try again; even numbers mean someone is> committing and the DB might be in an inconsistent state. Then read the> state from the database your update will rely upon (user1.name, in this> instance). You must also be able to atomically read the current version> from the database as well as zookeeper, to ensure that the data is from the> version you think it is. If the DB version does not match the ZK version,> restart.> 2. Once an update is ready to commit, test-and-increment the current> version in ZK to an even number, write your update to the DB, along with> the eventual version of the data (the next odd number).> 3. Increment the current version in ZK to an odd number.>> The even / odd distinction means that you can detect when someone else is> updating the database, since otherwise there's no way to do so atomically> with an update to ZK (so another transaction can't tell if you've finished> your update or not, and so doesn't know when to wait until).>> The problem is failure - what happens if a client fails while it's writing> a transaction? Eventually someone can increment the transaction number, and> if you provide an 'undo' log before you make any changes, that client can> possibly recover from a partial commit. But at this point you need to> understand your application's requirements in much more detail than we do> to make recommendations.>> In particular, your storage layer may offer sufficiently powerful> primitives such that you don't need ZK; although if it's a filesystem then> that probably isn't true.>> Henry>>> On 31 July 2013 15:51, Baskar Duraikannu <[EMAIL PROTECTED]> >wrote:>> > We cannot always resolve conflicts ourselves. For example, let us say> that> > a) user1 changed the name from 'Kathy' to Katherineb) user2 changes the> > name from 'Kathy' to 'Kat'> > Both read 'Kathy' as input; user1's update succeeded. If we need to let> > user2 know that something has changed as this may result in the user not> > changing 'Kathy' to 'Kat' (as an example).> > Hope this explains> >> > > Date: Wed, 31 Jul 2013 07:49:39 -0400> > > Subject: Re: Zookeeper performance> > > From: [EMAIL PROTECTED]> > > To: [EMAIL PROTECTED]> > >> > > This sounds highly error prone to me regardless of whether or not> > zookeeper> > > can handle the load-. Why not just use a standard transaction model> with> > a> > > vector clock or other timing device to detect conflicts so you don't> have> > > to worry about a second server to talk to (zookeeper) to do an update?> > > On Jul 31, 2013 7:17 AM, "Baskar Duraikannu" <> > [EMAIL PROTECTED]>> > > wrote:> > >> > > > Hello> > > >> > > > We are looking to use zookeeper for optimistic concurrency. Basically

+

Baskar Duraikannu 2013-07-31, 23:57

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext