In a small relaxed interview with JavaFree, Klaus Wuestefeld speaks about one of the most known and controversial brazilian projects, the Prevayler. Klaus is the founder and developer of this project that makes a persistence layer, or better, a prevalence layer.

Klaus talk about Prevayler X Relational databases, prevalence and OO concepts, the use of ORM frameworks, Tiger features in Prevayler, JNode and Prevayler 3.

Until each computer has 100Gb RAM each it would probably be a good idea to mark apps using Prevayler with "Powered by Prevayler". That way we don't have to make the mistake of trying to run two such apps on the same computer, as each one will try to consume the RAM resources fully and will, most likely, sooner or later actually succeed in doing so.

It's unfortunate that the attitude of the Prevayler guys still hasn't changed a bit...

A Prevayler is the cure for a cancer that the people not even know they have.

The main problem with Prevayler is not so much the "all-in-memory" aspect but the fact that it doesn't support SQL. And as long as these guys don't understand this (and also drop the attitude), Prevayler will most likely remain a niche product.

And obviously, Karl still doesn't have a clue why people are still using RDBMS:

7. Do you still use databases?Unfortunally, yes. And I'll be forced to use databases every time there isn't enough RAM to any of my applications.

He also forgets that while the price of RAM keeps going down, the needs of today's applications in RAM keeps going up, and probably even faster than the prices are dropping, so one might even wonder if Prevayler's premise will ever hold true.

Holding data in memory - what a revolutionary concept :-) Real innovation that !What happens if the machine with copious memory crashes ? You loose all your data right ?

It depends on how it is implemented. In case of Prevaylerthis is not a case.

A relational database still seems a much more reliable solution to me.Greg.

You are confusing two things: the way in which database is structured or what query language it provides (if it is relational etc ) has nothing to do with the location where the data is kept. You can have relational in-memory databases.

Rickard :>Until each computer has 100Gb RAM each it would probably>be a good idea to mark apps using Prevayler with "Powered>by Prevayler". That way we don't have to make the mistake>of trying to run two such apps on the same computer

>Cedric:>The main problem with Prevayler is ... the fact that it>doesn't support SQL.

If you insist on using SQL to query in-memory objects, you could add this on top of prevayler. I would rather use another query language.

>Prevayler will most likely remain a niche product.

That's ok with me. Let everybody spend their money on expensive database maintenance - this will only give a market vantage for those who are able to develop their applications cheaper and faster.

>He also forgets that while the price of RAM keeps going>down, the needs of today's applications in RAM keeps>going up, and probably even faster than the prices are>dropping, so one might even wonder if Prevayler's premise>will ever hold true.

RAM needs keep going up, yes. But which part of today's applications needs more RAM compared to applications 10 years ago? The persistence layer? Are you serious?

>Greg:>Holding data in memory - what a revolutionary concept :-)>What happens if the machine with copious memory crashes?>You loose all your data right ?

Wrong, you should really free you mind and try to understand the concept of Prevayler.

If you insist on using SQL to query in-memory objects, you could add this on top of prevayler. I would rather use another query language.

So, what do you suggest? Seriously. Even I don't like SQL syntax.

Or is your ideal query language the new foreach loop from J2SE 5.0?

Prevayler will most likely remain a niche product.That's ok with me. Let everybody spend their money on expensive database maintenance - this will only give a market vantage for those who are able to develop their applications cheaper and faster.

From my experience, the most expensive part of database maintenance (nitpick: you are using a database even with prevalyer, it's just that it isn't relational) is schema migration and data cleansing. What's it like with Prevayler? Any special techniques for generating the Prevayler snapshot? Or did you rerun all the commands in the new code base?

But which part of today's applications needs more RAM compared to applications 10 years ago? The persistence layer? Are you serious?

Keeping all "historic data" (and not just "current data") in the on-line transactional database is getting more and more common these days...

Wrong, you should really free you mind and try to understand the concept of Prevayler.

Hmm, such words usually come from members of fringe religious cults, especially when faced with questions they can't answer. Don't tell me you have a glass of Kool-Aid waiting for me...

From my experience, the most expensive part of database maintenance is schema migration and data cleansing. What's it like with Prevayler? Any special techniques for generating the Prevayler snapshot? Or did you rerun all the commands in the new code base?

From the Prevayler Wiki (which is currently closed due to "spamming", i.e. questions that Klaus cannot answer) I gathered that the persistent Prevayler state consists of two parts: A snapshot containing all serialized objects (Java serialization, or better XML serialization, which might be more amenable to transformation), plus a log of all actions since the last snapshot. When restarting, the server takes the most recent snapshot, and re-executes all actions logged since.Note that these two parts reside on different conceptual levels: One, data representation, and two, business logic.Now when you change your business logic and restart the server, the replay will happen with a modified interpretation of the logged actions; you may end up in a different state than you did before. Oops, where did my money transfer go?Conversely, if you change your data representation and convert the snapshot, you still need to be able to interpret all actions logged since the snapshot was taken, exactly as before (can you say isomorphism?). It's hard to get this right on the first attempt.

From the process point of view, the only safe way to do schema migration with Prevalyer is: take your system offline, create one consistent snapshot (with empty redo log), convert the snapshot, and restart from there.

Give guy a break: he at least tries to push something different and does not come up with YetAnotheJavaSQLDatabase. Zealot attitude is almost inevitable …

Why? Just because you come up with a creative idea doesn't give you the right to insult everyone who disagrees with you.

The concept of Prevayler is neat. It is, I really mean it, especially when the first version came out in just two Java classes. Hat's off, it's hard to be innovative.

But it's incredible that such a long time after this first version, the Prevayler team still hasn't learned from the reaction of the community to their tool and keep acting as if they were going to take the world by storm any day now.

Prevalent system logs your system state every N milli seconds (or nano or watever). It also logs each transaction/activity before it performs the activity. Nice.

So, you can _always_ recover, as long as your activitites are deterministic in nature. Meaning they leave the system in the same state everytime they are run. Isn't this a huge limitation? Can anybody exemplify a non-deterministic process vs a deterministic process... ?

Wrong, you should really free you mind and try to understand the concept of Prevayler.

This kind of constant pontification from Prevayler advocates is really insulting and condescending.

Look, *we get it*, okay? You have a host of people intervening on this thread and other forums expressing doubts on Prevayler, whose documentation and principles are extremely simple. Can you accept that we did read the said documentation and we do understand not only what Prevayler works but how it's implemented?

And instead of hearing us, you just hand wave and talk to us like some cult leader telling skeptics that "you don't get it".

Did it occur to you, just for one second, that maybe, just maybe, there is something behind these critics, and that Prevayler is much less applicable to the software world than you and your team think?

Just because you come up with a creative idea doesn't give you the right to insult everyone who disagrees with you.

Hi Cedric. I believe people are not insulted by the way I address them or by my "attitude" or by my tone. They are simply insulted by the technical beliefs I express!

It should not be a matter of being or not insluted. It should just be a matter of proving me wrong or not.

How could someone in the dark ages say "the world is round" in a way that was non-insulting to the dominating dogmatic authorities?

But it's incredible that such a long time after this first version, the Prevayler team still hasn't learned from the reaction of the community to their tool and keep acting as if they were going to take the world by storm any day now.

No we don't. I don't, at least.

I believe the adoption will be very slow and gradual. Inevitable none the less.

Hi Cedric. I believe people are not insulted by the way I address them or by my "attitude" or by my tone. They are simply insulted by the technical beliefs I express!It should not be a matter of being or not insluted. It should just be a matter of proving me wrong or not.

Is it even theoretically possible to "prove" "technical beliefs" to be "wrong"? By very definition "beliefs" are not based on objective reasoning, and hence cannot be "proven wrong".

I don't think anyone who has objected to your "technical beliefs" have any hope of ever being able to "prove you wrong", simply because within your own subjective view of the world it would be impossible. Within your own Matrix reality you are the only prophet who can ever be right, Klaus.

It is much less time-consuming and realistic to instead simply try to contain your "beliefs" where they can do no harm, e.g. by marking Prevayler apps with "Powered By Prevayler". You should like that as well, but for the opposite reason. ;-)

If you had understood the Matrix properly you would understand that the "prison" that Morpheus rules is only just that: a handy containment zone endorsed by the Architect for the purpose of knowing where they are, in order to make their inevitable desctruction swift and efficient. This is explained by the Architect in a fairly lengthy scene, but you seem to have missed it. Or at least, the point that was made. Hence, that you identify with Morpheus, as the leader of your following, is ironic.

Overall, you seem to have missed one of the main points of the Matrix movie: it doesn't matter where you are ("which prison you are in"), what matters is who you are and what you do. What you are doing, in promoting Prevayler the way you do, is just to lead people from one "prison" to another.

You are not helping people to think, to reason, to be free, to make objective choices. You are asking them to believe your religion. "Meet the new ruler, same as the old".

How could someone in the dark ages say "the world is round" in a way that was non-insulting to the dominating dogmatic authorities?

It would be impossible, just as it is (seemingly) impossible for you to understand that you are promoting yet another "prison". The problem is that you don't understand that you are just yet another dominating dogmatic authority.

Is it even theoretically possible to "prove" "technical beliefs" to be "wrong"? By very definition "beliefs" are not based on objective reasoning, and hence cannot be "proven wrong".

If you think a certain technical belief of mine is not based on objective reasoning, since it is just a technical issue, you can perfectly well come along with your own objective reasoning and show me my belief is wrong.

Simply saying "you are wrong but I don't care to show you why" doesn't help much. :(

I don't think anyone who has objected to your "technical beliefs" have any hope of ever being able to "prove you wrong", simply because within your own subjective view of the world it would be impossible. Within your own Matrix reality you are the only prophet who can ever be right, Klaus.

Not true. I have already been convinced that some of the things I believed about system prevalence were wrong.

It is really hard to convince anyone, though, just by saying "You are wrong" and providing no argument. :(

It is much less time-consuming and realistic to instead simply try to contain your "beliefs" where they can do no harm, e.g. by marking Prevayler apps with "Powered By Prevayler".

Yes. I read your other post above. You are afraid these apps will take RAM that might be needed by other apps.

But that problem can be avoided by a really revolutionary and innovative idea: having apps state their minimum system requirements. ;)

If you had understood the Matrix properly you would understand that the "prison" that Morpheus rules is only just that: a handy containment zone endorsed by the Architect for the purpose of knowing where they are, in order to make their inevitable desctruction swift and efficient.

Have you also noticed there is no parallel to the "Woman in the Red Dress" in my Database<->Matrix metaphor? I really wish there was, but unfortunately there isn't. ;)

That is a problem with metaphors. They tend to break when stretched beyond a certain point. If you want to take the Matrix metaphor beyond where I take it, that is perfectly OK, but you have to show the coherence of your new analogies.

Do you really want to insinuate Prevayler was devised by an Architect-like mastermind to make the destruction of the software development community swift and efficient (in which case it would be really nice of you to present a backing argument), or can we safely say the metaphor simply cannot be stretched to accomodate for the lame sequels to The Matrix?

What you are doing, in promoting Prevayler the way you do, is just to lead people from one "prison" to another. You are not helping people to think, to reason, to be free, to make objective choices.

Rickard, if you don't provide examples of "what I am doing the way I do it", you are just making vain, vague accusations. :(

You are asking them to believe your religion.

1) You are calling it a religion without any explanation. :(

2) It is perfectly OK for people not to believe me. "My beliefs don't require them to." ;) It is also OK for people to say I am wrong without saying why (it is just not useful). It is NOT OK, though, for people to accuse me without presenting an argument for it.

How could someone in the dark ages say "the world is round" in a way that was non-insulting to the dominating dogmatic authorities?

It would be impossible

Suppose that is what is happening. Suppose the world is actually round, not only in my head.

it is (seemingly) impossible for you to understand that you are promoting yet another "prison".

"Seemingly" based on what?

Have you ever presented an argument that I have ended unreasonably?

Have you ever seen anyone present an argument that I have ended unreasonably?

The problem is that you don't understand that you are just yet another dominating dogmatic authority.

Have I been promoted from a nut in a subjective universe of his own, whose "harmful beliefs" have to be "contained", to a "dominating authority"?

Anyway, why do you accuse me of being dogmatic? Please back that up, unless you are just slandering. :(

Holding data in memory - what a revolutionary concept :-) Real innovation that !What happens if the machine with copious memory crashes ? You loose all your data right ?A relational database still seems a much more reliable solution to me.Greg.

In-memory databases are hardly revolutionary. TimesTen has been around for a long time and has an impressive customer list. And it provides transparent data replication, transaction support and a SQL interface.

Mike wrote "single-process" and you wrote "single-threaded". Pay attention. Those are completely different things and you know it (should know, at least).

If you are really interested in denouncing "frauds" and promoting "the plain truth", Christian, you should stop doing this kind of thing.

Anyway, how do you know Mike Spille's account (which I comment on a posting below) isn't completely distorted? Have you spent the time to understand Prevayler objectively and have an opinion of your own?

I wouldn't have the patience for this.

Oh. So you simply choose to slander what you haven't objectively evaluated...

:~(

---

Anyway, it is not enough to point fingers at Prevayler.

If you are going to accuse me of selling "snake oil", you actually have to quote a claim of mine that is not delivered by Prevayler.

In-memory databases are hardly revolutionary. TimesTen has been around for a long time ..

TimesTen is a database.

Prevayler is not.

Once you understand that Prevayler is not a database, it's really easy to use. Just don't swallow all the marketing nonsense on the Prevayler site. Here's what Prevayler does:

1) You tell Prevayler what the "root" of your data is. All your data is belong to us from this point forward.

2) You can access or modify your data directly. This will screw you. Do not do it.

3) Instead, write "service oriented architecture" ;-) which is basically a series of Java stored procedures to manipulate your data. For example, if you have a banking system, you would have a root object which is your BankData, and it contains a collection (or map or whatever) of Account objects, and you would write a Java "stored proc" to do a transfer from one account to another.

4) Now, to do a bank transfer, you instantiate your stored proc and give it to Prevayler and say "I want to run this."

5) Prevayler runs the actions in serial (it's single threaded for all practical purposes) and also logs the actions (think Hypersonic but it's log is now a series of stored procs.) (I'm still a bit fuzzy on this one .. I looked through all the code recently, but it's commented very poorly in the two or three places that there are any comments at all.)

6) It has code in to be "recoverable" but I wouldn't use it for anything that you don't actually want to lose. In other words, if a VM dies, you probably won't lose your data if you followed the instructions on how to use Prevayler, but on the other hand if want to be certain that your data won't get lost, then Prevayler is definitely not for you.

7) Regarding GOTCHAS, I would not suggest the use of Prevayler unless you have ALREADY been bitten repeatedly in the past by Java serialization errors when your application version changed (i.e. your classes changed), AND you ALREADY figured out how to avoid this from EVER biting you again.

Anyway, remember that Prevayler is not a database, and you should be OK. It's just a way to keep some of your live Java objects backed up in a file on disk for the next time you restart your application.

My opinion is that it is definitely cute, often has the potential to be useful, and certainly should only be ignored if you are looking for a database ;-)

I got an anonymous comment on this topic emailed to me:<br>You did miss one thing, which is arguably the biggest Prevayler killerof them all: Prevayler is not multiprocess. 2 or more processes can'taccess the same "Prevayled" data (to coin a phrase).

This may have been a problem in 1.x days, but as of Prevayler 2.0 I don't believe this is true. In 1.x, Prevayler did serialise all access to the "prevayled" data, however in 2.0 it differentiates read-only "queries" (which can be run in parallel) from read/write "commands" (which must have exclusive access to the "prevayled" data). It's basically exactly the same problem Java synchronisation had before Doug Lea's util.concurrent read/write locks came along.

Of course allowing concurrent reads won't improve performance much unless you have a great deal more read activity than write activity, but the point is that your anonymous contributor may not be abreast of the latest Prevayler functionality.

<off topic observation>It's interesting to see some of the OO principles that Bertrand Meyer has been espousing for decades (ie. command / query separation) might finally be starting to prove their worth in Java land. With any luck Design by Contract will be next (but I won't be holding my breath!). ;-)</oto>

This is pretty much a good summary. Prevayler is awesome for what it does. The tension between object and relational models has been around for some time and given that the typical RDBMSs like Oracle also do a lot of what increasingly middleware does, it makes sense to finesse the problem by getting rid of the relational piece and relying on a simple persistence model based on the file system.

Note that with Prevayler one has to manage the state of the objects with these "command pattern" objects, which themselves serve as the transaction logs. They are serialized to disk. It's actually quite robust. However since Prevayler needs to serialize these objects they can contain no object references internally. Otherwise the logging itself can destroy in memory state like transient references ( The Prevayler folks call this the "Baptism Problem" ). Essentially one ends up creating a hierarchy of these command objects that more or less models the underlying object graph. In certain apps it can be very useful and indeed the query language is Java itself so it's a good approach. But it does require more objects and it requires one be aware of what is persistent and what is not. I think this is an improvement given all the O-R crap I've seen out there.

A better approach would be persistence at the JVM level, what I call orthogonal persistence. This is much harder and I'm not sure why SUN never pursued the PJAMA effort which was basically this level of persistence. Various Lisp and Scheme systems have done this in the past with "images" which dump the heap to disk.

Others like Sleepycat, have a simple model, a persistent hash, assuming no complex SQL like querying over the DB is needed and the user can maintain the object marshalling/unmarshalling.

Systems like db40, imho, have a fairly orthogonal approach, they handle schema evolution, have a nice approach to an object based query language with this SODA stuff. From the documentation I gather that "activation levels" enable partial object graphs to be in memory and I suspect weak references are used to page the rest in, but I'm speculating. Given it's been recently GPLed I'd highly recommend folks take a hard look. From what I've seen so far it's excellent code.

Still, at the end of the day, if I'm managing a large bank and all I need do is batch processing and balancing of accounts, in other words simple schemas over which at most I'll want some reporting of totals, I'll take SQL any day. It's based on some solid mathematical foundations and has years of research on transaction processing and query optimization by folks like IBM backing it up. I hope the OO persistence folks don't end up re-inventing all.

6) It has code in to be "recoverable" but I wouldn't use it for anything that you don't actually want to lose. In other words, if a VM dies, you probably won't lose your data if you followed the instructions on how to use Prevayler, but on the other hand if want to be certain that your data won't get lost, then Prevayler is definitely not for you.

6) It has code in to be "recoverable" but I wouldn't use it for anything that you don't actually want to lose. In other words, if a VM dies, you probably won't lose your data if you followed the instructions on how to use Prevayler, but on the other hand if want to be certain that your data won't get lost, then Prevayler is definitely not for you.

No explanation, no argument. Just fear, uncertainty and doubt.Ridiculous.

Recoverable does not mean "might be able to recover if everything went well." Rather it means that there are no edge conditions where data could be lost, regardless of what instant in time someone could have shut off the power, pulled the plug, etc. I spent the time read through the source code, and I don't believe you spent any time considering edge conditions.

Thus, there are two possibilities here:

1) That someone with little or no experience in implementing recoverable data stores just happened to accidentally "do it right" the first time, or

1) You PRESUME I have no experience in writing recoverable data stores.

2) You READ my code.

3) You DON'T FIND anything wrong with it and are unable of pointing out any concrete problem.

4) Still, you proceed to cast DOUBT in public and incite FEAR.

What am I missing?

1) I only presumed that you had no experience in writing recoverable systems after ...

2) I read your code.

3) It would be more correct to say that I did not find anything right: Prevayler does not even begin to attempt to implement a recoverable system.

4) Klaus, it is sad that you cannot participate in a reasoned discussion, and your random spewing of accusations is completely self-defeating. (Translation: You will gain some modicrum of respect only when you stop blindly bashing others.)

everything is stored in memory, but there's a transaction log. anybody here that uses hypersql for anything is helping to make his case. hypersql writes out its entire database as a sequence of sql statements that get iterated over on restart. "the data" is not stored out to disk. it's good been enough for JBoss. hypersql doesn't page anything out to memory, so it's wasting a lot of memory in order to provide a SQL interface to the data. if you are using hibernate with hypersql, then you are wasting a lot of memory on OO to relational round trips. stated another way, if you are using hypersql to do OR mapping then maybe you could increase capacity and performance greatly by using prevayler instead.

however, prevayler not supporting SQL makes it simple for object persistence needs that are known to stay below a certain size threshhold. when you are paying people to develop and maintain a solution, there are clearly cases where this is very sensible. if you really need to support SQL and have unbounded data storage requirements, then continue to use JDBC instead...it's great for that.

There are many different scenarios where nowadays typically a rdbms is used.

1.) A software that needs transactional persistence just to save its complex state. The persisted data has only an internal meaning, no other source is interested in it. Why do I need a heavyweight rdbms for this. Refactoring, development and unit testing consumes so much more time compared to using prevayler, without gaining anything.

2.) A text mining software that gathers gigabytes of data every day to categorize, extract and index. The data is very important also for other applications. It's very datacentric and SQL can play out its strength. Rdbms is a very good solution here.

So, surpise, surprise, there is set of applications where prevayler is a fantastic solution. This set is much bigger than the type described by example 1. But there are sets where prevayler is either not possible to use or at least not suitable.

You can combine these approaches as well. For example you can use prevayler to persist your application and then write and xslt-job to export the prevayler xml-data in a data warehouse.

The main problem with Prevayler is not so much the "all-in-memory" aspect but the fact that it doesn't support SQL. And as long as these guys don't understand this (and also drop the attitude), Prevayler will most likely remain a niche product.And obviously, Karl still doesn't have a clue why people are still using RDBMS:

Cedric, it's always nice to pay attentions to the names of the people. Anyway, it would be interesting if you could give details on your point.

It's unfortunate that the attitude of the Prevayler guys still hasn't changed a bit...

A Prevayler is the cure for a cancer that the people not even know they have.

Hi Cedric,

I believe:1. People's software development efforts suffer from a certain cancer called "RDBMS", preventing them from producing decent software.2. They don't know it.3. System prevalence is the cure.

You have provided us with an example of 2 and this is why I believe in 1 and 3:

Decent software design can be represented, for example, by design patterns. Many patterns cannot be applied to objects mapped to a relational database.

Consider the Observer pattern. Since database records cannot "notify" other database records and it would be too expensive to activate observers just so that they could ignore the notification 99% of the time, people's tendency, at best, is to make the observed object interpret the notification and only call the relevant observers. So now the observed has intelligence about the observers. You instantly get a spurious cyclic dependency.

If spurious cyclic dependencies are not cancer, I don't know what is.

Why is system prevalence the cure? Because it simply gets persistence out of the way and allows you to produce any design you want, including good design, including all patterns, including the Observer pattern.

On the other hand, if you think you are really hot with your stored procedures + SQL, and you don't believe in design patterns, I cannot convince you, and I will perfectly understand you getting pissed at my "attitude". :(

The main problem with Prevayler is not so much the "all-in-memory" aspect but the fact that it doesn't support SQL.

The approach will make more sense to me is when we can have the data portions of an application in a JVM run in battery backed or static RAM. Then you don't have to do all this disk crap and you can use normal locking mechanisms for protection.

No it is not. You saw there are comments below. You don't even have to click to open them. They are just there. You just chose to ignore them.Why?See you, Klaus.

Klaus, can you please point us exactly where in that page linked above is explained how Pralayler supports SQL? Mabe I have overlooked something, but after reading the whole page, I've found nothing stating how can someone use SQL with Prevayler, despite your statement that it does: http://www.theserverside.com/news/thread.tss?thread_id=30735#150608. Thanks.

No it is not. You saw there are comments below. You don't even have to click to open them. They are just there. You just chose to ignore them.

Klaus, can you please point us exactly where in that page linked above is explained how Pralayler supports SQL?

Henrique, Prevayler "supports" SQL by not preventing you from using SQL, just like MS-DOS "supports" SQL and x86 assembly language "supports" SQL. Similarly, you could say that Microsoft Solitaire does not "support" SQL, because you can't use SQL from within Solitaire.

There are a couple ways you could use SQL with Prevayler, including:

1) Your "prevaylent system" (that's a fancy term for the object that Prevayler writes to an ObjectOutputStream) could be an in-memory Java SQL database.

2) You could write or adapt a SQL engine to provide a relational view of your Java data structures that make up your "prevaylent system".

Henrique, Prevayler "supports" SQL by not preventing you from using SQL, just like MS-DOS "supports" SQL and x86 assembly language "supports" SQL. Similarly, you could say that Microsoft Solitaire does not "support" SQL, because you can't use SQL from within Solitaire.There are a couple ways you could use SQL with Prevayler, including:1) Your "prevaylent system" (that's a fancy term for the object that Prevayler writes to an ObjectOutputStream) could be an in-memory Java SQL database.2) You could write or adapt a SQL engine to provide a relational view of your Java data structures that make up your "prevaylent system".Peace,Cameron PurdyTangosol, Inc.Coherence: Shared Memories for J2EE Clusters

Well, not even _that_ is explained clearly in that page...

I suggest Prevayler people cut the metaphoric, religious and propaganda crap off of their site, and keep to the technical and factual only in order to get users and supporters much faster. We know what throwing BS around did to Jboss' respectability, people should learn from that.

Henrique, Prevayler "supports" SQL by not preventing you from using SQL, just like MS-DOS "supports" SQL and x86 assembly language "supports" SQL. Similarly, you could say that Microsoft Solitaire does not "support" SQL, because you can't use SQL from within Solitaire.

Exactly. Saying "Prevayler does not SUPPORT SQL" is incorrect and would bring it closer to Solitaire.

Instead, saying "Prevayler does not PROVIDE SQL" is correct and brings it closer to assembly language.

Henrique, Prevayler "supports" SQL by not preventing you from using SQL, just like MS-DOS "supports" SQL and x86 assembly language "supports" SQL. Similarly, you could say that Microsoft Solitaire does not "support" SQL, because you can't use SQL from within Solitaire.

Exactly. Saying "Prevayler does not SUPPORT SQL" is incorrect and would bring it closer to Solitaire.Instead, saying "Prevayler does not PROVIDE SQL" is correct and brings it closer to assembly language.See you, Klaus.

Henrique, Prevayler "supports" SQL by not preventing you from using SQL, just like MS-DOS "supports" SQL and x86 assembly language "supports" SQL. Similarly, you could say that Microsoft Solitaire does not "support" SQL, because you can't use SQL from within Solitaire.

Exactly. Saying "Prevayler does not SUPPORT SQL" is incorrect and would bring it closer to Solitaire.Instead, saying "Prevayler does not PROVIDE SQL" is correct and brings it closer to assembly language.See you, Klaus.

Edmund Blackadder: Baldrick, have you no idea what irony is?Baldrick: Yeah, it's like goldy and bronzy, only it's made of iron.And Cameron even sprayed quote marks all over his reply...

This is indeed a strange new usage of the word 'supports' of which I've been previously unaware.

In future, to understand what Prevayler is *truly* capable, replace all occurrances of the word 'supports' with the phrase 'does not necessarily prevent you from adding such a feature from scratch yourself'. Suddenly all of the bluster you read in forums like this will make perfect sense. How can you imagine anybody taking this project seriously with that kind of attitude? How can anybody judge what its capabilities are, and what it does and does not do without resorting to walking the code? I have a 'hello world' sitting on a floppy disk somewhere that 'supports' load-balanced, clustered operation and failover - if you write that bit yourself.

This is why discussions like this end up with comments about snake-oil - by saying that it 'supports' SQL you would lead someone with only a cursary knowledge of the software to imagine the you could query the system's objects using sql, possibly even through some sort of console. What you actually provide is nothing of the sort, only the promise that some day such a thing could exist, if somebody somewhere writes it. This, frankly, is snake-oil - the selling of something based on properties that it simply does not possess.

For the record, I think Object Prevalance is a good idea, but that making misleading claims like this do nothing but harm to Prevayler.

Someone mentioned on this forum that "Prevayler does not support SQL", which is simply false and had to be corrected.

RDMSs do not provide and DO NOT SUPPORT anything other than SQL and some sort of PL. So saying "Prevayler SUPPORTS SQL but does not provide it" is actually saying something in this context.

It would be deceitful, though, I agree, if the Prevayler site would simply state "Prevayler supports SQL" out of the blue, because it would have people believe Prevayler PROVIDED SQL.

But it doesn't.

What you read on the Prevayler site is: "We are finally free to use whatever query languages, algorithms and data-structures we wish" and "you are finally free to eat whatever you like, including porridge, if you are that desperate, but you have to cook it yourself".

RDMSs do not provide and DO NOT SUPPORT anything other than SQL and some sort of PL.

By following your logic, RDBMS can too support anything other than SQL: just plug a ORM framework on top of it, and there you have it. You'll be able to use other query languages besides SQL to search and update your database. By using your logic, by the way, one can say that almost anything is possible...

So saying "Prevayler SUPPORTS SQL but does not provide it" is actually saying something in this context.It would be deceitful, though, I agree, if the Prevayler site would simply state "Prevayler supports SQL" out of the blue, because it would have people believe Prevayler PROVIDED SQL.But it doesn't.What you read on the Prevayler site is: "We are finally free to use whatever query languages, algorithms and data-structures we wish" and "you are finally free to eat whatever you like, including porridge, if you are that desperate, but you have to cook it yourself".How snake-oily is that?See you, Klaus.

I'd rather see stated clearly what is done and readily useful with no large extra coding, and of what it can do, but some major coding has to be done before it. Yes, I am free to use any query language I want, as long as I implement it! It's like: yes, you can fly, as long as you develop wings... These porridge, prison, matrix, blah blah metaphoric talk is rather confusing, and doesn't transmit clearly the correct and complete message about prevailer's capabilities, uses and limitations.

Sorry Klaus, but prevailer wiki site is barely readable: full of anonymous comentaries, it is unable to tell which are yours and which are someone elses, many valid questions are left unanswered, many explanations are either too short, incomplete, or resort to dubious analogies, uneeded religious fight or philosophic blatter.A good example is the discussion around Prevailer's ability to provide ACID transactions: it is impossible to come to any conclusion by reading the wiki page about it, the impression I got is that prevailer by itself doesn't provide it, the programmer has to code it in their systems, and this would be a big hinder to prevailer acceptance.

- Prevayler provides your transactions with Atomicity and Durability. That is pretty evident to most people.

- Prevayler also provides Isolation through strict sequencing of all your transactions and, optionally, of all your queries. You are free, though, to use the isolation semantics you want for queries. Isolation is an issue of essential complexity and I believe it is wrong to provide a single one-size-fits-all solution. Prevayler provides a coarse-grained option, which is more than enough for most apps, but does not force you into it.

- Prevayler does not provide your transactions with Consistency nor does any database server. It is always up to the application to make its transactions Consistent, even with database servers.

Since we are in the middle of a discussion, anything we write doesn't have to be a complete, comprehensive treaty on the matter.

When in the middle of a discussion, there are many things - like the "Prevayler supports SQL" thing - that I leave incomplete on purpose, because we can always follow them up if anyone is interested.

I don't expect people to accept what I say blindly. I just hope people will follow up with questions, comments AND COUNTER-POINTS, in a respectful, productive and non-accusing manner, as these lasts posts have.

In this industry, this is called "multi version concurrency control" (short MVCC). It is supported (as in implemented) in most RDBMSes. Is most prominently marketed for Oracle RDBMS (mantra "readers dont block writers and writers dont block readers").

Klaus, could you put that into the wiki, as it is read-only for the outside users?

Prevayler does not provide your transactions with Consistency nor does any database server. It is always up to the application to make its transactions Consistent, even with database servers.

Oh really? Even when using a "database server"?

Let's see:"Consistency is the ACID property that ensures that any changes to values in an instance are consistent with changes to other values in the same instance. A consistency constraint is a predicate on data which server as a precondition, post-condition, and transformation condition on any transaction."

There are two kinds of consistency: internal (consistent according to the rules implementor of the application or database defined) and external (consistent with business specification or with "what the customer wants").

What are those mysterious "predicates"? In SQL, you have unique constraints, foreign key constraints, check constraints, table constraints (widely unimplemented) and so on. As a fallback, predicates can be *implemented* as triggers, but that hardy counts as a proper consistency support.

On the other hand Prevayler provides no capabilities for defining predicates, so the data is always internally consistent as far as Prevayler is concerned. So consistency must be completely managed by the application.

In this industry, this is called "multi version concurrency control" (short MVCC). It is supported (as in implemented) in most RDBMSes. Is most prominently marketed for Oracle RDBMS (mantra "readers dont block writers and writers dont block readers").

I know. Oracle uses the rollback segment to do that.

MVCC is equivalent only to this part of the most scalable architecture (MSA), though: "You just ask the system: 'What time is it?' and issue a historical query at that time [MVCC does this for you]. Even if the query takes a few seconds to run, it will produce a consistent result without blocking the rest of the system."

To acheive the MSA, though, I believe you also need instantaneous transactions.

Klaus, could you put that into the wiki, as it is read-only for the outside users?

It is read-only to me too. I would have to FTP it in. I am doing that only for pretty major stuff, sorry. :(

Please do contribute though, whenever the new site comes out.

Prevayler does not provide your transactions with Consistency nor does any database server. It is always up to the application to make its transactions Consistent, even with database servers.

Oh really? Even when using a "database server"? Let's see: ... In SQL, you have unique constraints, foreign key constraints, check constraints, table constraints (widely unimplemented) and so on.Of course, but who is responsible for creating them?

Will the database server guess a certain field is not supposed to be null, for example, and create the "not null" constraint itself?

No.

So, database servers do a lot to "support" transaction consistency, but >>YOU<< still have to PROVIDE your database schema with the proper constraints.

In other words, the definitive "Consistency" in ACID is still up to the application.

Do you agree, or am I just twisting words around again?

On the other hand Prevayler provides no capabilities for defining predicates...

And does not have to.

So consistency must be completely managed by the application.

No. You are free to use things like JContract (invariants), for example.

There are many differences between the ACID properties that an RDBMS gives to you vs. what Prevayler does.

First, a database provides things such as constraints (not null, FK, PK, etc) in a centralized and declarative manner which is easy to see and understand. You can very easily query your system and see what must not be NULL, what the relationships are, etc.

In Prevayler, you have to do the same thing in you Java code. And application logic is mixed in with the constraints, so you cannot easily see what they are - you might say that you can't see the constraints for the trees :-)

That's one layer of consistency and atomicity. Another layer is concurrency control. In a database, this is again mostly declarative. Depending on the framework you use for accessing the database, the code side can also be declarative e.g. EJBs transaction code and isolation controls, similar things in Spring, Hibernate, etc. In a nutshell, if you wish to mutate "N" items in your database, the database will work very hard for you to guarantee that those mutations are atomic and consistent.

Prevayler doesn't give you any tools for doing this beyond a) ensuring that your Command object is executed atomically b) optionally giving you rollback capability at the cost of triple the memory requirements and double-execution of your code. If you choose not to use the rollback capability, in order to mutate "N" items in your Prevayled tree consistently and correctly requires that you manually write Java code to ensure this is so. If you do use the rollback capability, you can code willy nilly - but this extracts the triple memory and double execution cost. And you are still running your command objects in a single threaded manner.

The last part is more wide-open concurrency. Prevayler side steps this in two manners. First, only one process can ever access the Prevayled tree, so Prevayler doesn't have to worry about multiprocess synchronization or locking. Secondly, it synchronizes on the Prevayled tree for each command that's executed.

Concurrency will always be a prickly pear for Prevayler, for a very simple reason. The Prevayled object tree is completely opaque to Prevayler. It has zero knowledge of what's going on inside that tree. It has zero knowledge of what mutations a given command performs on the tree. This means that Prevayler is severely constrained in what it can do, because it literally does not know what's going on.

Compare this to a database, where the databse knows _exactly_ what has changed, by whom, and when. It can track the pages, columns, tables, rows, etc and intelligently manage them. Prevayler cannot do this, because it has no knowledge of your tree's internal structure or how it's mutated.

When Klaus talks about "Instantaneous Transactions", mostly he's taking advantage of the fact that Prevayler disallows concurrency and does zero management of your internal tree at a fine-grained level. To allow concurrency Prevayler would need internal knowledge of tree mutations, which it can't do (no, you can't do it any other way because they would be way, way too slow).

On that note - Klaus' idea of the "Most Scalable Architecture" is unachievable in Prevayler, again because it has no knowledge of mutations. To achieve this concept in Prevayler, you'd need to snapshot the tree for _each and every command_. Here's an example:

- Commands 1, 2, and 3 run at times N, N+1, N+2 - To support this, Prevayler would need to have the entire prevayled state at N-1, N, N+1, and N+2 e.g. 4 copies of the entire system state. Assuming you could do this, queries would run at time required "N+1" or whatever and be shunted off against the proper snapshot.

In fact, Klaus realizes that this is unachievable - you'd very quickly run out of memory. And Prevayler cannot reach into the Prevayled tree's internals and track only deltas, as an RDBMS does. The answer then is to only make copies as a transaction is mutating state, and collapse copies at convenient times when no one is running a query or mutation against the tree. This is possible - but still has enormous memory penalties and execution penalties. You're still holding many copies of the prevayled tree and paying many copying costs.

An alternative to this was suggested by Stephen Kestle on the wiki. His suggestion is that you wrap all of your business classes and keep a historical list of objects as they are mutated. However, this has a very deep problem - now the app developer is coding all of the history mechanism themselves! Ack, who wants to write that?

You will find that most of the stuff on the Wiki boils down to a few Prevayler characteristics:

- only one command can run at a time - Prevayler has no knowledge of how your persistable data is structured, or when it mutates - Prevayler writes your Command object to disk first, unsynchronized with the execution stream, _then_ executes the transaction - The former idea (always write the "transaction" first) only works if use the expensive rollback functionality or manually hand-code all your business methods to pre-validate conditions before ever mutating anything.

If you want to remember only one thing about Prevayler, it's this: Prevayler has no knowledge of app structure or how the app mutates data/objects. All Prevayler _ever_ sees is:

- Object. This is the business tree. - byte[] - The serialized form of the above - Transaction - an interface with one method, execute().

That's it. Prevayler hasn't the slightest idea how your object relationships work, how data relates, how/when mutations happen, any sorts of constraints, or how a Transaction may mutate your state.

In other words, the definitive "Consistency" in ACID is still up to the application.Do you agree, or am I just twisting words around again?

I'm not sure which definition of ACID or consistency you're talking about. when i think of ACID, I think of the terms defined by TPC in their benchmark specifications. In the case of TPC, if I understand it correctly, consistency is not up to the application. Since TPC is all about the database and it's ability to reliably maintain data, external applications have nothing to do with it.

From an application perspective, consistency is a different issue altogether depending on the application and architecture. If we're talking about distributed objects, then the container/framework/middleware would be responsible for maintaining the consistency of the data.

So lets say I have 10 instances of prevayler setup in a partition scheme on 10 physical systems. Now, if each partition is absolutely isolated from the others and no data is replicated, consistency is fairly straight forward. Let's say I want full failover capability and distribute the load, how would one achieve that with prevayler? Could it do what C-JDBC does? or something equivalent?

I'm not sure which definition of ACID or consistency you're talking about.The issue of Consistency is a simple one, I believe:

Transactions must be Consistent.

So that transactions can be Consistent in an RDBMS, there has to be integrity constraints.

The RDBMS will NOT create integrity constraints on its own, so just by using an RDBMS, your transactions can be Atomic, Isolated and Durable, but there is no guarantee your transactions will be Consistent.

YOU have to create the integrity constraints, so YOU are responsible for guaranteeing your transaction Consistency. The RDBMS only helps you in providing means for you to do so.

Using JContract is a way to do so with Prevayler, but with Prevayler you are free to use whatever Consistency scheme you want.

Klaus, you're only talking about half of the consistency equation of an RDBMS, which you could call enforcing higher level constraints.

There is a lower level of consisistency which is just as important. In a nutshell, it states that if a transaction fails somewhere, all work done up until the failure point is rolled back.

The original Prevayler model, which you still seem to favor, is that that lower level consistency should be completely programmed by the developer. This is done by having all application commands preverify state before doing any mutations to the tree, and aborting if any of the verifications fail. In that model, Prevayler provides 0 consistency guarantees - it is entirely up to the developer to write all of their code to always preverify, and to never, ever throw an exception once any state has been changed within a command.

The somewhat newer model (which is optional) is to use the StrictTransactionCensor. This gives you low-level consistency similar to an RDBMS. But it comes at a very high cost - your commands are all executed twice, and you need 3 times the memory, and on failure the entire prevayled tree needs to be copied.

I think where you're going wrong, Klaus, is that you're looking at the ACID properties individually and assigning just one trait to each. In fact, they're all heavily interrelated to each other. Atomicity, consistency, isolation, and durability all tightly related. For example:

- Your level of isolation has a large impact on the visible transient consistency of the system. - Atomicity and consistency are very tightly intertwined - in many ways you can consider them different viewpoints on the same concept. - Durability is yet another aspect of consistency, in how the system appears over time and in the face of faults.

In an ideal world stripped of performance considerations, everyone would probably want "perfect" ACID - all transactions are 100% durable, completely isolated, entirely consistent all of the time, and completely atomic. However - all databases and most applications choose looser levels of ACIDity, because perfectly ACID transactions are dog-slow. This is where various isolation levels come into play, different levels of durability (precisely when is a transaction 100% durably recorded), row locking vs. page locking, etc. Other products with transactional support also have their own variations on this theme, such as various clustered caching solutions.

Unlike databases, Prevayler doesn't really given you very many options in how close to or far from "perfectly ACID" you wish to be. Without the StrictTransactionCensor, Prevayler on its own only gives you isolation and durability and partial atomicity and very weak consistency. With the StrictTransactionCensor turned on, you get much higher levels of atomicyt and consistency, but it still does not approach what you can choose to achieve in a database. I say it doesn't approach it because a developer still much code within the constraints of a command, and only one such command is allowed to run at a time. The developer has to work harder to achieve consistency, and has fewer guarantees from the "container" that they can rely on.

Klaus, let me suggest again that you study transactional systems a bit more in-depth before you try to replace them.Before you start debating the merits of ACID properties, it will help enormously if you understand them in full first.

I'm not sure which definition of ACID or consistency you're talking about.The issue of Consistency is a simple one, I believe: Transactions must be Consistent. So that transactions can be Consistent in an RDBMS, there has to be integrity constraints. The RDBMS will NOT create integrity constraints on its own, so just by using an RDBMS, your transactions can be Atomic, Isolated and Durable, but there is no guarantee your transactions will be Consistent. YOU have to create the integrity constraints, so YOU are responsible for guaranteeing your transaction Consistency. The RDBMS only helps you in providing means for you to do so. Using JContract is a way to do so with Prevayler, but with Prevayler you are free to use whatever Consistency scheme you want.I'm going to have to disagree here. From the description you provided, I think there is a mis-understanding. I've never used JContract and don't know what it is. From my limited experience with Oracle, Sybase and Sql Server over the last 7 years, consistency is much more complicated that your description.

There's consistency in terms of concurrent transactions, dirty reads, dropped connections, rollback and snapshots. Here is a direct question. Let's say I have auto-commit on and I'm doing a mix of inserts and updates. If my process does the inserts first, updates second and my connection dies right before the updates, how is that hanlded in prevayler?

So lets say I have 10 instances of prevayler setup in a partition scheme on 10 physical systems. Now, if each partition is absolutely isolated from the others and no data is replicated, consistency is fairly straight forward. Let's say I want full failover capability and distribute the load, how would one achieve that with prevayler? Could it do what C-JDBC does? or something equivalent?

Prevayler does not provide for transparent partitioning/distribution of your system.

It will provide for replication in version 3 (you can already experiment it in version 2) but not partitioning.

- Atomicity: since prevailer just calls the Transaction object, it is the coder's job to guarantee that either all or none operations inside a Transaction executes. and since it is in the coder's hand to code this correctly (either by try/catching and correcting any errors, or leveraging prevailer's "rollback" feature), an effort much more complex compared to simply issuing a "begin", "commit" and "rollback" around a RDBM transaction, IMO prevailer lags behind databases in this issue, making it more complex.

- Consistency: as you have stated, prevailer does not provide it, it is (again) up to the coder. RDBMS, OTOH, do provide some mechanisms to prevent this (triggers, pks), even if the coder makes some mistakes. It may not prevent every possible error, but what it does provide again puts RDBMS in advantage over prevailer, IMO.

- Isolation: as you have explained, it is up to the coder to guarantee it, while RDBMS can do a great part of the job for us, again helping us prevent some errors in this regard.

- Durability: prevailer provides this, althou one should be reminded about problems with disk caches, as cited in prevailer's wiki pages. In RDBMs case, this would not be a coder's job to guarantee it, but a DBA's. Anyway in this regard, both are similar.

In all, for me it seems that prevailer puts too big a burden and responsability on the coder's shoulder, in order to have a system that behaves correctly, making the chances of mistakes much higher than coding systems that access RDBMs. This is my opinion, from what I have read about prevailer up to now.

Being prevailer such a radically different approach to persistence from the norm, I think it is rather normal that so many doubts, questions and challenges are raised against it.

I like wiki, but seriously it's not a substitute for good documentation. I'll be the first to admit writing documentation is much harder than writing code and often I struggle with it. With my own attempts, the first 4 drafts are usually horrible and I totally suck at proof reading and revisions.

For me, providing good documentation is a long and tedious process, but it is very important. Writing great documentation is really hard and really takes a ton of experience. Perhaps prevayler should try to recruit a technical writer to help improve the situation. This way, users and those doing research don't have to go through a ton of wiki pages only to leave more confused.

In the past yes, but not recently. As much as i love javadocs, I strongly believe in having well written documentation that isn't just a description of each object/method.

That is one of the biggest weaknesses of OSS, in my bias opinion. Even though I prefer to dive into the code, it's not always possible when I am evaluating a product. I should be more clear with what I mean. By documentation I mean step-by-step tutorials, factual description of exactly what the product does and doesn't do in bullet points, pros/cons of the product and real-world whitepapers.

In the past yes, but not recently. As much as i love javadocs, I strongly believe in having well written documentation that isn't just a description of each object/method.That is one of the biggest weaknesses of OSS, in my bias opinion. Even though I prefer to dive into the code, it's not always possible when I am evaluating a product. I should be more clear with what I mean. By documentation I mean step-by-step tutorials, factual description of exactly what the product does and doesn't do in bullet points, pros/cons of the product and real-world whitepapers.

I've always been curious about this phrase the "real-world". The closest I can figure is that it normally refers to where you are not. Strange --

my definition of real-world is this. person/company A actually uses the product to build a product and deploys it. the result of that process are written up with details showing how, why, what, when, and all the little details that provide a convincing picture of how technology X should be used.

Someone mentioned on this forum that "Prevayler does not support SQL", which is simply false and had to be corrected.RDMSs do not provide and DO NOT SUPPORT anything other than SQL and some sort of PL.

I completely disagree with you that the statement "Prevayler does not support SQL" is "simply false" and "had to be corrected". Until someone adds the facility to use SQL to query your object model to Prevayler, claiming SQL support exists is not true.

But lets just assume for a moment you are correct in your interpretation of what it means to "support" SQL. By extension, you are spreading falsehoods when you say:

RDMSs do not provide and DO NOT SUPPORT anything other than SQL and some sort of PL.

...because all RDBMSs would "support" any querying mechanism under the sun. SQL Server would "support" the facility to traverse a graph of java objects in memory simply because there is nothing to stop Microsoft from rewriting the damn thing to add that facility in future. Or if your "supports" logic is restricted to open source, since anybody can get the HSQLDB source and hack it about any way they like its plainly obvious that it "supports" xpath and OQL, and all of the scalability and fault-tolerance of Oracle.

Whichever way you cut it, what you say is not true - either you don't support SQL, or by some stretch of logic you do and the statement that RDBMSs only support SQL is false, thereby removing one supposed benefit of Prevayler.

Prevayler SUPPORTS SQL but does not provide it

This is quibbling over semantics, and frankly I think you are making a distinction between the two words that is very hard to credit. This is much less clear than simply saying "Prevayler supports the facility for the developer to add an arbitrary querying mechanism, which could conceivably be SQL". It is, however, an improvement your statement earlier in this thread:

Many people don't get this. Prevayler does support SQL.

When describing your project, please be realistic and state clearly and honestly, preferably without resorting to analogies, what it actually does out of the box rather than trying to fudge the issue. If I have to write a feature for Prevayler myself, then it does not support it.

Since TPC org specifications are available, why not produce results with a 100Gb database?

I have run the Prevayler scalability test with 15M objects on a 10GB RAM Sun box 2 years ago but, yes, a TPC.org benchmark would be much cooler.

It is quite an amount of work, though, and I don't have access to the kind of last-generation hardware they normally use.

If anyone knows of any group out there responsible for the TPC benchmarks for any such hardware vendor (Bull, Dell, Fujitsu Siemens, HP, IBM, Itautec Philco, RackSaver, Sun, Unisys), I will be glad to help them blow the competition out of the water with Prevayler.

Since TPC org specifications are available, why not produce results with a 100Gb database?

I have run the Prevayler scalability test with 15M objects on a 10GB RAM Sun box 2 years ago but, yes, a TPC.org benchmark would be much cooler.It is quite an amount of work, though, and I don't have access to the kind of last-generation hardware they normally use.If anyone knows of any group out there responsible for the TPC benchmarks for any such hardware vendor (Bull, Dell, Fujitsu Siemens, HP, IBM, Itautec Philco, RackSaver, Sun, Unisys), I will be glad to help them blow the competition out of the water with Prevayler.See you, Klaus.

that's a considered as low end to many of the systems i am aware of. I know several systems that get a constant stream of 1-5mb of data 24/7. In those situations, the burdon of proof is going to be on prevaylers. you're absolutely right that large enterprises work different and have their own way of doing things. Some of the bigger systems I know of range in the 50-100TB range.

I see value in in-memory database for smallish applications that don't need to store millions of rows of data. I've read the faq on prevayler in the past and I read it again. In my mind, it is far from conclusive or persuasive. The reason is this, if i want to tell a CTO or lead architect "we can use this for the datastore", I better have rock solid base numbers. If not, I won't even get a chance to build a prototype to run more realistic stress tests. If prevayler is unable to provide that basic information, how are developers suppose to present a credible proposal?

To quote the movie Jerry Maguire, you have to "help developers help prevayler" gain wider acceptance. The term "late adopter" is rather negative in tone and probably turns many people off. I realize not everyone has the resources to run TPC benchmark, but without it, is it realistic to expect developers to accept it on faith? You're asking others for concrete proof, but I see very little concrete proof on prevayler's site on actual performance characteristics.

Sarcasm aside, there are plenty of cases where something like prevayler is appropriate. Just not Gb-Tb databases that have complex models. Any attempt to pretend it is appropriate is rather foolish on the part of prevayler developers. Given the small business market is huge in the US, there is a market for in-memory databases. Though I'm totally bias. I prefer to work on large systems, so tools like prevayler aren't very useful for what I do.

I think many of the criticisms in the thread were reasonable and valid. Whether the responses have proven the value of prevayler will likely depend on each person's requirements and needs.

I'd say Prevalent people are asking for trouble. Just to quote one phrase from their manifesto: "We no longer have to license, install, configure and maintain a database and application server every single time we want to develop, demonstrate or deploy our systems for any of our clients. Give us a Java VM and we are good to go". Preaching in such bold statements is just asking for trouble. The world is simply not that black and white.

A bit of modesty sometimes helps you get along with others. If they had another posture since the start, I doubt they would get that much criticizm from almost everyone else, and people would focus more on what such a different solution prevalent really is, what it means, its uses, its concepts, instead of just trying to refute everything prevalent supporters say or do.

there's no such thing as bad publicity. Though I would hope the IT world is a little different than hollywood. Acting like a diva might work and result in a role in a movie, but with software I find it less acceptable. Though, it's really quite a shame prevayler doesn't find a balanced way of advertising the software. Look at all the apache software that succeeds without a ton of publicity or flame wars.

It's true that memory gets cheaper but Java has a big problem using it. It's the garbage collector that has a complexity of O(x^2), where x is the number of objects you have. All of the SUN garbage collectors stop all threads for some time to get their job done. The result is that the system stops respondig for half a second with only half a GB of memory. I read about an IBM system stopping for 8 minutes. This problem increases with larger memories.

The other problem of in memory databases is the problem that queries are more complex than in an RDBMS. You have to define how to find the data not only what to find. Nobody really cares how an RDBMS organizes its memory to quickly find the data. With in memory databases you have to design the memory structure yourself and it's hard coded into your application. If you add queries which needs different data access you have to change your application.

If you think of an lifecycle of 15 or 20 years for your data do you really think Java is still the programming language of choice? I'm sure you will have access to an RDBMS with the language of choice in the year 2020.

If you think of a web shop Prevayler might be a good choice. If you think of a industry application grown by dozends of sub projects and add ons over the past 5 years you are in trouble with Prevayler.

Many patterns cannot be applied to objects mapped to a relational database.Consider the Observer pattern.

Implementing the Observer pattern using database triggers is possible. The more recent Oracle versions integrate a messsage-oriented middleware, allowing you to send a notification from inside your database transaction. Thanks to 2PC, the notification is sent if and only if your transaction succeeds.Using prevayler, I understand how to use the observer pattern between objects inside a prevalent system (the business object tree). This corresponds directly to database triggers that work inside the database.Interacting with other parts of the Java process seems more difficult. For example, when using the StrictTransactionCensor, notifications may get sent twice, or from the shadow copy; also, care must be taken to make all listener references volatile (so they do not end up in the snapshot), and to not let the listener's behaviour influence the interpretation of the command object (so the p-system remains deterministic); are listeners allowed to call back into the system, and if so, is that a separate transaction to be logged and replayed? What does single-threaded notification handling do to performance? It would be helpful if some user of Prevayler could comment on some of these points.

If spurious cyclic dependencies are not cancer, I don't know what is.

See www.cancer.org for details. This is not an especially accurate, let alone sensitive metaphor.

Implementing the Observer pattern using database triggers is possible.

How do database triggers dynamically and anonymously register to be called, without explicit knowledge by the caller, such as the observers in the Observer pattern do?

Using prevayler, I understand how to use the observer pattern between objects inside a prevalent system (the business object tree). This corresponds directly to database triggers that work inside the database.

Regular procedural code running inside a prevalent system corresponds to triggers in the database. The Observer pattern is quite different from any old procedural code.

Interacting with other parts of the Java process seems more difficult. For example, when using the StrictTransactionCensor, notifications may get sent twice, or from the shadow copy;

No they cannot. Not if your system is deterministic. Do you have a specific question about that?

also, care must be taken to make all listener references volatile (so they do not end up in the snapshot),

Yes. And thanks to OO, that can easily be done in a single point in the code.

and to not let the listener's behaviour influence the interpretation of the command object (so the prevalent system remains deterministic);

Yes, of course. Is determinism not a good thing?

are listeners allowed to call back into the system, and if so, is that a separate transaction to be logged and replayed?

Yes and yes. You would do that if you have a non-deterministic listener. If it were deterministic, it could simply be part of the prevalent system and not have to use transactions.

What does single-threaded notification handling do to performance?

Probably kill it, why? Why would you use single-threaded notification?

If spurious cyclic dependencies are not cancer, I don't know what is.

See www.cancer.org for details. This is not an especially accurate, let alone sensitive metaphor.

No metaphor I can use to describe the damage database atrophy causes to the software development community can be "sensitive". I am sorry. :(

See you, Klaus."And many of them are so inured, so hopelessly dependent on the system that they will fight to protect it." --Morpheus in The Matrix

Implementing the Observer pattern using database triggers is possible.

How do database triggers dynamically and anonymously register to be called, without explicit knowledge by the caller, such as the observers in the Observer pattern do?

You define a trigger that reacts to insert/update/delete on a specific table. The trigger contains PL/SQL code, which is run every time the specified event occurs. The (part of the) application modifying the table can be completely unaware of the (other part of the) application reacting to changes.Registering a trigger is a DDL statement. Multiple listeners can be registered for the same event.This is an implementation of the Observer pattern.In addition, the code in the trigger has the option of aborting the transaction (like a vetoable listener in Java), with a guaranteed clean rollback.

Regular procedural code running inside a prevalent system corresponds to triggers in the database.

Shouldn't that be: ...corresponds to stored procedures in the database?

The Observer pattern is quite different from any old procedural code.

The difference being that when adhering to the procedural paradigm, all options for the flow of control are fixed at program development time. The observer pattern is a special case of late binding, which allows one call site to dynamically invoke different targets. By dynamically plugging is additional code, the calling code can remain unchanged, while being adapted to new functionality.In RDBMS, one means of dynamically extending an application is by defining additional triggers.

also, care must be taken to make all listener references volatile (so they do not end up in the snapshot),

Yes. And thanks to OO, that can easily be done in a single point in the code.

I just checked the example code (org.prevayler.demos.demo2.business from prevayler 2.02.005). I find class Account defines a "private transient Set listeners", and class Bank also defines a "private transient BankListener bankListener". Seems like every Observable needs to manage its own set of attached listeners, which will be reimplemented for every Observable. After all, Observer is a pattern. Implementing it in exactly one point in the code is difficult because of varying interfaces (or did you mean "thanks to AOP" ?).

and to not let the listener's behaviour influence the interpretation of the command object (so the prevalent system remains deterministic);

Yes, of course. Is determinism not a good thing?

The problem is that there are so many things that each programmer has to be aware of, and if those rules are not followed (due to ignorance or oversight or over-cleverness), the system will fail in non-obvious ways.By the way, if you can't have vetoable events, then you don't have the full flexibility of Java, right?

What does single-threaded notification handling do to performance?

Probably kill it, why? Why would you use single-threaded notification?

As I understand, the business logic (everything inside the "prevalent system") is completely single threaded, because all "Transactions" are executed serially. The standard Java listener pattern works with synchronous invocations. Hence, a straight-forward Java implementation will use single-threaded notification. The banking example cited above even does Swing callbacks from within a listener. (see AllAcountsFrame#accountCreated)

Of course you could implement asynchronous notification. But in the examples, you didn't, and I doubt many users of prevayler will; otherwise, where's the simplicity they came looking for?

How do database triggers dynamically and anonymously register to be called, without explicit knowledge by the caller, such as the observers in the Observer pattern do?

You define a trigger that reacts to insert/update/delete on a specific table. The trigger contains PL/SQL code, which is run every time the specified event occurs.

To me that qualifies as something like a Smalltalk secondary method (a method that is "attached" to a class by another file, without knowledge of the original class) not the observer pattern, in which observers can observe a single instance rather than be called for all changes in a given table.

The issue is only one of OO: "per instance and procedure" (OO) versus only "per procedure" (triggers).

You are right, in that triggers do not necessarily produce spurious dependencies.

also, care must be taken to make all listener references volatile (so they do not end up in the snapshot),

Yes. And thanks to OO, that can easily be done in a single point in the code.

I just checked the example code (org.prevayler.demos.demo2.business from prevayler 2.02.005)...

I knew we were going there. :)

The goal of the Prevayler bank demo is not to teach OO. It is not complex enough to justify many abstractions, such a volatile listener abstraction. Do you agree?

People must know they can resort to OO techniques way beyond what they see in the "Prevayler bank demo". Is that reasonable?

It is designed simply to teach people the workings of Prevayler, not as a template for any OO system, regardless of how complex.

The kind of abstractions that are justified in a 100 class system are quite different than the ones that emerge in a 2 class system.

Maybe I should make that clear in the javadoc or something. What do you think? Is that necessary?

After all, Observer is a pattern. Implementing it in exactly one point in the code is difficult because of varying interfaces.

In the case of the volatile listeners it can be done, no problem.

The problem is that there are so many things that each programmer has to be aware of, and if those rules are not followed (due to ignorance or oversight or over-cleverness), the system will fail in non-obvious ways.

Yes. Prevayler is about giving you freedom. Prevayler is not about taking you by the hand. I understand your fear.

"With great power comes great responsibility". :)

But, in the end, it all boils down to writing a deterministic business system.

In addition, the code in the trigger has the option of aborting the transaction (like a vetoable listener in Java), with a guaranteed clean rollback. ... if you can't have vetoable events, then you don't have the full flexibility of Java, right?

You can have Java vetoable events with Prevayler.

Java does not automatically rollback the changes some object does in the middle of a vetoable event processing chain. Java also leaves it up to the object to do so.

I consider vetoable events a very bad design practice, though. They are a complete disaster from a design-by-contract point-of-view.

As I understand, the business logic (everything inside the "prevalent system") is completely single threaded, because all "Transactions" are executed serially. The standard Java listener pattern works with synchronous invocations. Hence, a straight-forward Java implementation will use single-threaded notification.

I hereby declare you free to use whichever implementation you want, straight-forward or otherwise. :)

The banking example cited above even does Swing callbacks from within a listener.

Do you think that is a bad thing, in the context of the demo?

Does the demo pretend to be a template for the best OO design patterns for arbitrarily complex systems?

Of course you could implement asynchronous notification.

Woohoo! I am free! :)

But in the examples, you didn't, and I doubt many users of prevayler will;

If they are not good OO programmers, they should not use Prevayler. The Prevayler site says so.

Some people read this as "if you don't use Prevayler, you are a bad programmer" and call me arrogant because of that. :~(

Those people should probably also not be programmers, let alone use Prevayler, because they have serious problems with basic logic.

otherwise, where's the simplicity they came looking for?

You don't have to write your own VolatileListener. You can use mine if you like. :)

The simplicity is in that people are finally free to use each other's OO components, without being stuck to the restrictions of a given RDBMS.

This looks like another Prevalence implementation? Forgive me for a quick perhaps stupid question: From the example it looks like you allow the object references to be passed into the command (log) objects? In Prevayler this can lead to what they call the Baptism Problem (see http://prevayler.codehaus.org/The+Baptism+Problem). Do you address this?

Another somewhat related issue is that Prevaler serializes these command objects out to disk so if there are object references in them they also are serialized (This is called the FoodTaster in the code, a cute name). This is needed to ensure that the commands executed upon recovery are identical. However if the objects have transient instance members in them they can be destroyed by this process. In other words the act of executing a "command" against an in memory object changes the object itself. This is not mentioned in the critiques above but I consider it a shortcoming of Prevayler as it greatly increases the amount of code needed for persistence. I call this "the furnace woman comes over to fix my furnace and rearranges my living room furniture on the way out problem" :^)

By itself, opening up new possibilities for representing persistent business objects is not enough. In order to compete with the well-established traditional database world, some guidelines on how to use those possibilities in a sustainable way are required.

Firstly, persistent data generally lives longer than the business process that exist when your application is first designed. A layered design therefore makes sense independent of technological questions.

Secondly, the simpler the persistent data structures, the easier it gets to access them using a different client, a different technology, or to migrate them to a different schema (see Klaus' remark on "shortcomings" in Java's schema migration framework - could these be the effects of a too powerful representation language?). An inexperienced programmer using the "full power of java" for long-lived business objects will soon encounter a maintenance problem.

Thirdly, traditional databases allow to monitor operation, add and remove indices at runtime, and use elaborate query optimizers to make an unchanged query run faster. How to simulate that in Java? Which access accelerating data structures should you use, when and how do you update them? Do you have to write explicit code for accessing them, or do you use some kind of in-memory database framework? In the former case, assumptions about the data stored in your system and its access patterns are scattered all through your code. In the latter case, you're back to a multi-language solution not far better than SQL.

Main-memory databases definitely have their uses, and SQL as a technology is far from optimal. I fear that through his uncomprimising stance (just read the Wiki!), Klaus will rather further the use of SQL databases, by discrediting his alternative.

Firstly, persistent data generally lives longer than the business process that exist when your application is first designed. A layered design therefore makes sense independent of technological questions.

In regard to your point I can't see any disadvantages or shortcomings when using prevayler

Secondly, the simpler the persistent data structures, the easier it gets to access them using a different client, a different technology, or to migrate them to a different schema (see Klaus' remark on "shortcomings" in Java's schema migration framework - could these be the effects of a too powerful representation language?). An inexperienced programmer using the "full power of java" for long-lived business objects will soon encounter a maintenance problem.

The maintenance problem you now have with prevayler is far easier to solve compared to the maintenance problem when using a rdbms (for example when doing a big refactoring).

Thirdly, traditional databases allow to monitor operation, add and remove indices at runtime, and use elaborate query optimizers to make an unchanged query run faster. How to simulate that in Java?

When using prevayler, I think the hot spot JVM is your build-in query optimizer.

In regard to your point I can't see any disadvantages or shortcomings when using prevayler

The point is, some of the complexity of a real-world app stems from multiple layers with different representations, and the conversions needed at layer boundaries. This complexity won't go away if you use prevayler instead of an RDBMS. It's not a shortcoming of prevayler, but neither is it an advantage (as is sometimes suggested by its advocates).

The maintenance problem you now have with prevayler is far easier to solve compared to the maintenance problem when using a rdbms (for example when doing a big refactoring)

Could you add some evidence, or a reason for that?In my experience, many changes to my business logic do not affect the database at all. This is exactly because I have made up my mind about what is the valuable data to keep persistent, and what are the auxiliary structures needed for my current application. If you mix up the two, you run into trouble.

Thirdly, traditional databases allow to monitor operation, add and remove indices at runtime, and use elaborate query optimizers to make an unchanged query run faster. How to simulate that in Java?

When using prevayler, I think the hot spot JVM is your built-in query optimizer.

The JVM optimizer knows which code is executed how often, but it will not examine the statistical distribution of my data, and will not switch from an ArrayList to a HashMap representation if necessary.

Initially, they will feel like Neo, floating on that water container, completely atrophic, for living their entire life inside a database bubble. Slowly, they will exercise this new freedom, and their muscles will develop. Then, when they say: "I know OO", I'll anxiously answer: "Show me".

Funny, this happend to me in the reverse. The rush you feel when you do your first UPDATE ... SET ... WHERE <some complex SELECT expression>, the hours of programming replaced with mere minutes it takes to write, fully test and execute an SQL command, the switch from "My precious! My precious objects!" to "like stored procedures, but on the server"... The power of the objects is insignificant compared to the power of the RDBMS.

Having used object databases like LDAP, they are powerful, but not very flexible. One could argue that's either a blessing or a curse. By the very fact that you can easily add a column, create a dynamic view, or a materialized view means developers will do it. After a while, the database model looks like swiss cheese.

A few years back, I tried to use LDAP for storing customer data and found it rather inflexible. When ever we need to make minor or major changes like adding new columns or chaning the relationship between the user and the number of addressbooks, we had to rebuild LDAP. Needless to say with a small dataset, it wasn't a problem. Once we loaded up LDAP with 2 million accounts for functional and stress testing, rebuilding LDAP became a huge headache. It was rather unproductive at that point. We quickly switched back to Oracle at that point and never looked back.

Object database are great, if your model is pretty static and isn't going to change more than once a year.

Having used object databases like LDAP, they are powerful, but not very flexible ...

Maybe someday someone will get it right.

Unfortunately RDMBSs aren't 100% flexible either. Especially in the wrong hands (gotta guess who I think that is). So until the perfect tool shows up ... RDBMS it is. But it doesn't mean I will shove my head in the sand. Keep an eye to the sky and the future.

Having used object databases like LDAP, they are powerful, but not very flexible. One could argue that's either a blessing or a curse. By the very fact that you can easily add a column, create a dynamic view, or a materialized view means developers will do it. After a while, the database model looks like swiss cheese.A few years back, I tried to use LDAP for storing customer data and found it rather inflexible. When ever we need to make minor or major changes like adding new columns or chaning the relationship between the user and the number of addressbooks, we had to rebuild LDAP. Needless to say with a small dataset, it wasn't a problem. Once we loaded up LDAP with 2 million accounts for functional and stress testing, rebuilding LDAP became a huge headache. It was rather unproductive at that point. We quickly switched back to Oracle at that point and never looked back.Object database are great, if your model is pretty static and isn't going to change more than once a year.

Object database are great, if your model is pretty static and isn't going to change more than once a year.

First of all, prevayler is not an OODBMS. About your statement, my team and I developed the CMS for the 3rd largest TV station in Brazil for 2 year and half, XP-style with weekly releases. The object model changed every other week and we had near-to-zero downtime during the whole period. It's no magic and we are no wizards. It's just simple and good technology put to work.

Object database are great, if your model is pretty static and isn't going to change more than once a year.

First of all, prevayler is not an OODBMS. About your statement, my team and I developed the CMS for the 3rd largest TV station in Brazil for 2 year and half, XP-style with weekly releases. The object model changed every other week and we had near-to-zero downtime during the whole period. It's no magic and we are no wizards. It's just simple and good technology put to work.

Honestly I haven't used Prevayler, so my comment about OODBMS may not apply to prevayler. I'm curious, how did you handle rebuilding the database when the model changed?

from my experience, it was rather painful and took 2-4 hours in the beginning. Once the database grew in size, it took upwards of a day. I ask because I couldn't find a reasonable way to simply add/remove columns or rename a column without having to rebuild the database. Using RDBMS, we could modify the table and create a view so that the existing stored procedures worked with a minor change. It didn't require completely rebuilding the database in many cases.

The other thing OODBMS doesn't handle as well is many-to-many relationships. I probably don't understand OODBMS well enough to know how to do this elegantly compared to the standard RDBMS approach.

I should have mentioned on my post that we didn't use prevayler for java but Bamboo.Prevalence for .net. bp includes a schema migration tool. Really hard migrations (lots of "extract class" refactorings over the week) would take us at most an afternoon of work to both write and test. Most of them were between 1-2 hours of work.

The actual migration was really fast as we had only 5-8MB of data most of the time.

Hi Peter,I should have mentioned on my post that we didn't use prevayler for java but Bamboo.Prevalence for .net. bp includes a schema migration tool. Really hard migrations (lots of "extract class" refactorings over the week) would take us at most an afternoon of work to both write and test. Most of them were between 1-2 hours of work.The actual migration was really fast as we had only 5-8MB of data most of the time.cheers,Rodrigo

Ok, that explains it. We had a sample database of 200K user accounts to begin with and something like 30-50 address entries per user. That was the first prototype. As we progressed, the sample data went up to 1million accounts with 100-200 addresses + personalization data. In my case, once we hit 500K user accounts and 100 addresses per account, migrating the database became a huge pain. After we went through several releases, our test database had anywhere from 1-5million accounts. Luckily by then we had changed back to Oracle, so changes to the tables, views and stored procedures was easy. the oodbms was LDAP and boy it was a pain. even though there was migration tools, it still took a long time. For small projects, I would consider it an option, but for large databases with 20-30Gb of working data, it is not a good fit in my bias opinion. That's not to say it can't be done, but from my experience using LDAP as an OODBMS, it was more work. for things like single-sign-on where the data model is static, LDAP works well.

Same here! As a high-school student I had to program Clipper (a dBase clone), explicitly writing all loops, filters, navigation etc., all the time wondering: which loop should I put outside, which one inside? Which collection will be larger at runtime, which condition has the higher selectivity, what's going to be more efficient? Statically, as a programmer, you just can't tell.When I first learned about declarative database query languages, it felt like *wow*, the machine does it all for me, and it can even do it *better*.

Note that "declarative" does not have to mean "relational" or "SQL", but it certainly means "not Java", so some kind of language barrier is probably unavoidable.

Does anyone knoe if I can get Perbaler to work in a Web Start Security sandbox? What I want to do is cache some data (that I get from SQL) on my Swing application, but it is JNLP.This would increase my performance.

I _think_ you should be able to, but you'll need file access locally.

If you're just looking for a temp cache, consider a cache solution (like Coherence .. just kidding, you don't need clustering for Swing :-) .. you might use Prevayler if you want some data to be stored locally for when the Swing app isn't running.

... This kind of bashing is ugly and leads to nowhere. I respect a lot some of the foks on this thread, follow their blogs, and I do not want to read stuff like that. It is Christmas time, could you be a bit more polite to each other, please.

To Prevayler skeptics: Please, make sure your comments reflect the fact that you have actually examined Prevayler's code and you *get it, but do not like it, because ...*. Thoughts in general are not very interesting (at least not to me). Also, I think it is only fair if people that work for ORM/RDBMS-related projects/companies disclose that fact up-front.

Oh, and please do not call Prevayler enthusiasts "Kool aid drinkers”. We understand (and work with) RDBMSes, whether we like it or not, okay? Prevayler is actively discussed in a very serious manner by Martin Fowler (bliki), Ted Neward (his latest EEJ book), Scott Ambler ... and look who's at number 2 on the Prevayler manifesto (already pretty lengthy) list.

Rickard-,You seem to have a problem with Klaus' personality and/or the aggressive way he promotes Prevayler. That is understandable, but please - say so (your “technical” rebuttal sounds very light to me).

Cedric-,In addition to the above, you seem to have examined the code and do *get the idea*. Indeed, not having a query API is deterrence for certain types of projects/developers. However, now that Carl (db4o) and Klaus (Prevayler) work together, it is highly likely that a S.O.D.A implementation for Prevayler, similar to the one for db4o, will appear soon. Java's default serialization is another problem that Klaus admits in the interview, but some folks successfully replaced it with XStream. Prevayler is not perfect nor it is the “universal golden hammer”, but is one of the best ideas lately, tackling a significant software problem for all of us. Give Klaus some slack, please:-)

Cameron -,You make the most sense to me here. Indeed, understand the Prevayler idea/code, figure out when/if Prevayler works for you and if so - try to use it or ... make money out of it! Prevayler is exactly the kind of disruptive technology that could give competitive advantage to small companies over big shops, where RDBMSes have entrenched deeply.

Thanks for the comments, Hristo. I'll point out again that the biggest problem with Prevayler is attempting to position it as a database replacement, which it is understandably abysmal at. It's just a way to keep some Java objects in memory, and have a somewhat recoverable backup mechanism on disk for them if you have to restart. It can be handy for that purpose, and (I would suggest) for that purpose alone.

Regarding Rickard's terse comment, I think you'll find lots of musings from him on the Java object persistence topic; check his blog entries for this past year.

Cameron-,"...It's just a way to keep some Java objects in memory, and have a somewhat recoverable backup mechanism on disk for them if you have to restart". Indeed, and in fact, many, many application need ONLY that for their core functionality (and this is where Prevayler excels), but end up paying the full price for using RDBMS (meaning poor performance, software/license fee/support costs, configuration pain, db administrator salaries and extra soft. developers costs, etc.).

Conversely, if you need to data-mine your historic logs (there is no need to grow Prevayler’s memory consumption with finished data), offload it from the Prevayler engine into ... Excel (or Oracle) and use the tools/algorithms there to write separate set of specialized analytical applications!

Oh, and there is a whole class of problems that Prevayler is not an option for - memory-constrained J2ME devices, shrink-wrapped applications, which will run on an UNKNOWN hardware/JVM, etc.. In such cases, consider using PERST, DB4O or even DERBY.

I do not understand why people are looking for “golden storage hammer” or do not apply a pragmatic “mix-and-match” approach.

I learned about Prevayler a good while back. The sales pitch sounded like a bad joke and I still think someone is snickering behind the scenes over all this. But you know, if the claim is outrageous enough, it can be passed as truth if you stay on the message and keep on banging.

wow! I am not sure if your associations are correct but I know other people matching with these profiles (kung-fu-master, diplomat, lurker, therapist...). very cool and fun way of describing people. I remember the who's who of TSS. we should have Flame Warriors of TSS(or Java :).

I’m not sure how many people actually looked at the code & design of Prevayler. Mike Spille actually did: http://www.pyrasun.com/mike/mt (which is actually way too much attention this garbage deserves, IMHO, of course).

Hat’s off to Mike. I mean just look at it and make your own opinion. Isn’t it symptomatic this thread runs so long massaging this "revolutionary" idea with equally "revolutionary" implementation…?

How long we should be pretending to be nice and polite to people that have significantly sub-standard abilities and skills just because they open sources their latest drool?

I would go on record calling this idea simple dumb (as it was more than a year ago when I saw it for the first time) and implementation is worth of creating a black list for the authors like that to be used by all serious employers.

But in a spirit of Christmas, let all pretend this is just one of those "revolutionary, slightly misguided and misunderstood open source ideas"...

2) False statements ("Prevayler doesn't have transactions", "You have to code middleware stuff yourself, over and over again", etc) which might not be lies but could be honest misunderstandings.

3) Appeals to "common sense" along the lines of "Klaus says human beings have evolved from apes! How absurd!" without any argument about why I am wrong.

4) Prevayler limitations which are evident from the Prevayler site, and which are presented as great surprises.

It is a waste of time, therefore, to comment that post as a whole but, if anyone thinks there is anything useful there, and would like to discuss it in a respectful manner here, I would be more than glad to. :)

See you, Klaus."First they ignore you, then they laugh at you, then they fight you, then you win." Gandhi

Klaus, it appears you've gotten yourself a bit mixed up. The points you're making don't seem to apply at all to the post you're referring to. Perhaps you're thinking of the wrong post? Well, don't worry, we all make mistakes.

For your reference, you got the link right, and the title of that blog post is "Prevayler Revisited". The opening paragraphs state:

I see from the blog comments that some people are convinced that Prevayler is indeed wortwhile for certain applications. I concede that there are some such applications out there, but their population is almost vanishingly small.

To support this thesis of mine, let's look at the actual Prevayler code. Let's strip away the rhetoric and just focus on what the Prevayler code actually does.

I humbly put forth that the blog entry does exactly what the opening states - it looks at the actual Prevayler code and exposes what it does in some detail under the covers. Please read the blog entry for the full details, including references to the source code. But, to rather grossly summarize the post, it says:

- snapshots ultimately are embodied by the code "ObjectOutputStream stream = new ObjectOutputStream(out); stream.writeObject(prevalentSystem);". This describes the entire "file format" for snapshots using serialization.

- snapshots are only taken when the _application code_ requests it. No automatic snapshots ever occur. It's all up to the application to ensure that this happens in a timely manner.

- So called "Transactions" really aren't. They're command classes that have been renamed to Transaction to make them easier to swallow. A transaction optionally "approves" of a transaction, logs it to a log binary log file, and then executes it while holding a lock on the entire business object tree. Hence it single-threads all mutations to said tree. If you don't use the approval mechanism, you have to carefully write all of your classes to pre-verify state and ensure that everything in the transaction is correct before mutating state. If you fail to do this, you can get inconsistent results.

- Again there's no file format in the binary log - just ObjectOutputStream.writeObject() again.

- On the approval stuff, you can optionally use the StrictTransactionCensor. This removes the need for checking pre-conditions, but requires triple the RAM requirements, a re-read from the snapshot and binary log files if an exception occurs, and double-execution of all "Transactions". If you have several million objects than this code can literally take minutes to recover from an exception - for each and every exception that occurs.

- In total, the "transactional" capability which Prevayler addresses is saving transactions to a log and single-threading on the root of the business object tree. Everything else is up to the developer. You can get Prevayler to help with consistency by using the StrictTransactionCensor, but at a huge cost in resources.

- You must always code with the expectation that your transactions _will_ be run multiple times, due to how recovery works (this is true even if you're not using the StrictTransactionCensor). This means your code must always be "determinstic". This means that you should not use code that works in terms of deltas, but only in absolutes, and that you should be highly wary of using any external resources.

- Query support consists of another Command object, which is executed while the business object tree is locked. That's Prevayler's _total_ support for transactions. Anything else is up to the developer.

- In several places Klaus indicates that replication is available in the current production version and ready for use. For example, in the interview Klaus states "The most amazing new feature of Prevayler 3 will be the failure-tolerant replication. The applications will be able to run in 'cluster' mode, without changing even one line of code. The regular replication may already be tried on Prevayler 2". In fact, the "regular replication" referred to is implemented in 4 classes which are quite buggy. This is the code which swallows exceptions and goes into infinite loops on exception. Despite what Klaus claims in the interview and on the wiki, the replication features do not really exist. There's just a few highly experimental and very buggy classes that make a rather poor attempt at replication, but I don't think anyone would seriously consider using this in any sort of production application, enterprise or not.

Believe it or not, the above long-winded verbiage is indeed a summary of the blog entry. See:

On the subject of bugs, the version I looked at (2.02.005, the latest available on sourceforge) is indeed quite buggy. Klaus, your comments about bug-freeness may have conceivably applied when Prevayler was 2 classes. It certainly does not in the 37 classes of the releases 2.02.005 version. The replication code alone has scores of bugs. Yes, yes, on this forum you state its experimental, but you've lead people to believe it's a serious part of the release in other forums. The disk-writing and recovery parts also some serious issues. As Cameron states, you really can't enumerate them, because funamentally the design around recovery is just plain fragile. If you want to see what I mean, compare your recovery code to that of, say, HOWL (Highspeed Object Web Logger), a binary logger developed as a transaction log for projects like JOTM, Geronimo, and ActiveMQ. Comparing HOWL to Prevayler's implementation shows some large deficiencies in the Prevayler code.

Indeed, looking at the work on "Prevayler 3.0" and comparing it to 2.02.005 you can see that a great deal of code has changed, and a number of obvious and glaring bugs have been addressed. But, at the same time, the new code is an order of magnitude larger and more complex than the 37 classes of 2.02.005, so in quashing some things from the 2.x line many more bugs have been introduced, which may be part of the reason why the 3.x line has been under development for over a year and a half. On that note, the code under development is devilishly hard to find - it's rather obscurely buried over at codehaus.org these days.

However, the most insidious bug I've seen is one I mentioned on my blog comments - the StrictTransactionCensor and regular "transaction" processing do not guarantee order between them. More specifically, the STC may see transactions in the order, say, A-B-C-D, while the regular transaction processor sees ADCB. This means that the StrictTransactionCensor is verifying execution against a different state then the real state. This can lead to a number of bad outcomes. In some situations, invalid exceptions may occur within the STC that should not (since it is looking at the wrong state). Or, worse, the STC may "pass" invalid code because it works in the state that the STC sees, but then fails with an exception in the real object tree, leading to inconsistent state.

Hopefully this post passes your criteria for "respectful" and "useful". If it does not, please inform me of the pieces that do not pass muster and I will try reword them to conform to your rigorous criteria.

Klaus, it appears you've gotten yourself a bit mixed up. The points you're making don't seem to apply at all to the post you're referring to. Perhaps you're thinking of the wrong post?

I might have gotten the link wrong but you know perfectly well what post I am referring to. You are just being deceitful. :(

Why?

Well, don't worry, we all make mistakes.

Yes, specially when our motives are questionable. It is clear you are not out to "determine whether or not Prevayer is a good thing". Your goal is to "make as many people as possible think Prevayler is a bad thing, regardless of whether that is the case". :(

To make that clear to you, I put a [*] sign below, every time that is happening.

I humbly put forth that the blog entry does exactly what the opening states - it looks at the actual Prevayler code and exposes what it does in some detail under the covers. Please read the blog entry for the full details, including references to the source code.

Cool. That is quite an attitude change from the first post, which started:

"You know, you just can't keep a total ass down. No matter what [*] slings and arrows are sent zinging their way, they always bounce back ready for more."

I am glad you adopted a respectful posture now. :)

Dave Orme did a good job of showing you a few misconceptions you had about Prevayler there on your new post. When proven wrong, though, instead of simply standing corrected, you rebut [*] with the ever-so-clever: "Yes, but Klaus might change that in Prevayler 3.". :P

How can you expect to be taken seriously after that? :(

Why would you do something like that? What would make a person go to such lengths, making a fool of himself in public, simply to bash another project? :~(

You bring a lot of attention to Prevayler that way (you "triple dog dared" people to visit the Prevayler site, for example). I could yank your chain forever to get that effect but I am worried about you.

What are you afraid of?

But, to rather grossly summarize the post, it says:- snapshots ultimately are embodied by the code "ObjectOutputStream stream = new ObjectOutputStream(out); stream.writeObject(prevalentSystem);". This describes the entire "file format" for snapshots using serialization.

Cool, hey?

snapshots are only taken when the _application code_ requests it. No automatic snapshots ever occur. It's all up to the application to ensure that this happens in a timely manner.

Nobody has ever pretended Prevayler did that.

Again there's no file format in the binary log - just ObjectOutputStream.writeObject() again.

You really don't get it, do you? Simple is good.

Up to now, you have only agreed with the opening statements on the Prevayler home page: "Ridiculously simple, ...", have you not?

If you want to properly bash Prevayler and not only make a fool of yourself in public, it is not enough to say it uses "just serialization", you actually have to show why that means Prevayler does not deliver on what it promises[*]. Prevayler has never pretended to use anything other that plain Java serialization or XML serialization, has it? Even the PrevalenceSkepticalFAQ says that and you know it.

But you don't care [*]. You will just present "Prevayler uses plain serialization" as if it were an "exposed" finding "under the hood".

Why?

So called "Transactions" really aren't.

But, of course, you don't care to explain why not [*].

They're command classes that have been renamed to Transaction to make them easier to swallow.

Actually I would prefer Prevayler not to require transactions (you can see that in the original ObjectPrevalenceSkepticalFAQ), but I was convinced by a reasonable argument that what Prevayler had with its commands were actually transactions.

A transaction optionally "approves" of a transaction, logs it to a log binary log file, and then executes it while holding a lock on the entire business object tree. Hence it single-threads all mutations to said tree.

If you don't use the approval mechanism, you have to carefully write all of your classes to pre-verify state and ensure that everything in the transaction is correct before mutating state. If you fail to do this, you can get inconsistent results.

In other words: "If your code sucks, don't use Prevayler". :P

If you are going to denounce the "fraud" I am, at some point you actually have to stop agreeing with the Prevayler site.

"Stop trying to hit me and hit me."

On the approval stuff, you can optionally use the StrictTransactionCensor. This removes the need for checking pre-conditions, but requires triple the RAM requirements

Just so you know, Prevayler development code on CVS requires only double the RAM already.

a re-read from the snapshot and binary log files if an exception occurs,

Absolutely not true.[*] :(

and double-execution of all "Transactions". If you have several million objects than this code can literally take minutes to recover from an exception - for each and every exception that occurs.

You have people believe [*] they cannot use exceptions at all or their systems will take minutes to recover. Once again you know (should know at least) that is not true. Only uncaught RuntimeExceptions and Errors will cause that (yes, bad code again). TransactionsWithQuery can very well throw Exceptions, but you chose to ignore that [*].

Klaus, thanks! You've done an amazing job of twisting my words, selectively quoting, using inaccurate characterizations, throwing in the occasional dig, and of course probably having the highest percentage of ':(' in any TSS post ever.

Taking a page from your book, your entire set of 3 "Show Me" posts can be summarized by looking at your tired Matrix quote embedded in the first one:

"Stop trying to hit me and hit me."

What you've shown here, Klaus, is that it is indeed pointless for anyone to try to hit you - because you do such a good job of hitting yourself. No critic could possibly damage your reputation more than you damaged it yourself with these sorts of posts.

"Klaus, thanks! You've done an amazing job of twisting my words, selectively quoting, using inaccurate characterizations..."Yet another load of vain accusations without examples.Never mind. I had high hopes but low expectations.---I have to stop doing this. It drains a lot out of me emotionally.See you, Klaus.

wouldn't it be easy to prove everyone wrong by simply building a real application with a 1TB database? If you can prove Prevayler can handle a large complex database with multiple database models and provide 99.999% reliability and fault tolerance, I'm sure plenty of people will shut up and give it a shot. I would for one would love to see it done, because it would give me more options. not only that, you probably would be able to get venture funding to take prevayler to the next level. I don't know prevayler and haven't used it, so I can say with any confidence whether it can or can't.

Having used other object databases like LDAP, using OODBMS as a general purpose database for large enterprise applications is very difficult. I've only seen LDAP used for single sign-on and basic directory services. I realize prevayler is not an OODBMS, but as the developer of Prevayler, the burden of proof is clearly on your shoulders klaus. Proposing a new approach isn't easy. If you're unable or unwilling to explain it clearly so the public can understand it, than what is the value proposition of prevayler? A tool is only useful if users can grasp it quickly and do the same tasks they are used to doing with RDBMS. I understand your frustration, and I've come across situations where I couldn't find the right words to explain an approach.

Perhaps the prevayler team should focus on examples and tutorials to explain the approach? If the percieved limitations of prevayler is purely lack of understanding, than solid examples would make a huge difference. If the percieved limitations are real, than i would say there's a lot of work ahead. In either case, you've chosen to promote an approach that is not popular and has a lot of negative history. I know people who tried ObjectStore and had a miserable time. The end result is they won't even consider OODMBS, even if it would be appropriate. Regardless of the merits of prevayler, one thing is clear. There are significant barriers for in-memory databases and OODBMS. I don't think Prevayler will be taken seriously by large enterprises without addressing these issues. Good luck.

I was simply asking that guy to present a minimum of evidence for some pretty direct and concrete accusations he made, which he failed to do.

We were not having a high level "Is Prevayler suitable for 1TB systems?" discussion.

But let's move on and address 1TB systems and your other suggestions.

wouldn't it be easy to prove everyone wrong by simply building a real application with a 1TB database?

It is not a matter of proving "everyone" wrong. There are many people who actually use Prevayler and people who don't for valid reasons but know Prevayler isn't a "fraud". :)

Also, if organizations have too much of a legacy RDBMS culture, or cannot afford enough RAM, Prevayler is clearly not for them right now. There is no point in trying to push it.

I, for one, cannot afford 1TB of RAM yet, so it is not a matter of "simply" building a real 1TB application. Remember I would also have to host it somewhere and make it notorious. That would be a huge task.

That would help, but I don't think it would be worth the effort.

If you're unable or unwilling to explain it clearly so the public can understand it, than what is the value proposition of prevayler?"

The first two answers of the SkepticalFAQ explain what Prevayler does: snapshots and transaction logs.

Some people think they cannot replace a database with that while some people think they can.

Some people see databases as the solution to many of their problems. They would be completely lost without databases.

Other people see databases as an obstacle. They realize they could do much more if they were relieved from database constraints.

Prevayler is clearly for the latter group.

Perhaps the prevayler team should focus on examples and tutorials to explain the approach?

Trying to TEACH late adopters how to go beyond what RDBMSs allow them to do today would be too much, I believe.

It is enough of a task already for the Prevayler project to actually FREE people to do that.

you've chosen to promote an approach that is not popular and has a lot of negative history. I know people who tried ObjectStore and had a miserable time. The end result is they won't even consider OODMBS, even if it would be appropriate.

Yes. Before open source approaches, OODBMS have traditionally been slow, heavy and expensive. But even this wasn't enough to stop the OO revolution:

- The next five years will be dominated by Java and dotNET. There is no serious non-OO contender in the industry at the moment, is there? That wasn't the case in the early 90s.

- People are starting to grok patterns. That wasn't the case in the early 90s.

- IDEs, that had traditionally focused on quick-and-dirty RAD development in the 90s, with "data-aware" components directly coupled to the database, are now investing in OO techniques with refactoring, testing and profiling.

So, although there is very little OO culture in enterprise software development, trends like the above give me the impression it is just a matter of time.

And I am not in a hurry.

I don't think Prevayler will be taken seriously by large enterprises without addressing these issues.

It will take years for large enterprises to consider using prevalence regardless of what I do. That is just the way they work.

I am not going to develop, host and publicize a 1TB system just so that some fearful IT manager can feel a bit less fearful of making the switch from an RDBMS or COBOL or something of the sort.

A few years ago, innovators started using Prevayler because they found it good and because they found it cool.

Early adopters have heard from the innovators and are using Prevayler when they find it appropriate.

Late adopters will only use Prevayler when they see what the early adopters are achieving, not when they see I have a fantastic system up.

You are finally free to use other people's code, without being restricted to what a given RDBMS provides.

You can get Prevayler to help with consistency by using the StrictTransactionCensor, but at a huge cost in resources.

Currently that cost is about US$ 100 per GigaByte and falling.

Huge or not, that is completely subjective, of course. If I had a biased point-of-view, I would obviously say that is a "huge" cost [*].

You must always code with the expectation that your transactions _will_ be run multiple times, due to how recovery works (this is true even if you're not using the StrictTransactionCensor). This means your code must always be "determinstic". This means that you should not use code that works in terms of deltas, but only in absolutes, and that you should be highly wary of using any external resources.

I don't know what you mean by "in terms of deltas" but the rest is just paraphrasing the Prevayler site and documentation.

Quite a "fraud" you are denouncing. :P

Query support consists of another Command object, which is executed while the business object tree is locked.

Again that is not true and again you know it [*]. Prevayler supports direct queries to the business objects using plain old Java method calls. Prevayler even supports business objects notifying observers.

But, of course [*], you chose to ignore that in your account.

That's Prevayler's _total_ support for transactions.

Once again no argument [*] as to why that is not enough.

Anything else is up to the developer.

Once again the fallacy I mentioned above. [*]

In several places Klaus indicates that replication is available in the current production version and ready for use.

Absolutely not true. Please quote a few of those "several places".

Let's see your example:

For example, in the interview Klaus states "The most amazing new feature of Prevayler 3 will be the failure-tolerant replication. The applications will be able to run in 'cluster' mode, without changing even one line of code. The regular replication may already be tried on Prevayler 2".

I write "will be able" and you write "is available".

I write "may already be tried" and you write "ready for use".

Simply ridiculous. :(

In fact, the "regular replication" referred to is implemented in 4 classes which are quite buggy.

1) You are bashing code which is not ready to be used, which is documented in the API as being not ready to be used and which produces a message like "Replication is still being implemented" when run. But you know all that, of course. Do you care? No[*]. You just go on. :(

2) You write "quite buggy" but produce only one example [*], and a bad one at that:

This is the code which swallows exceptions and goes into infinite loops on exception.

A bug is a piece of code that does not comply to its requirements.

Would you like to know what the requirements are for that code, before you go on bashing it, or are you going to insist on the fantastically arrogant stance [*] of "something like that could never be right" that other guy above also took? :(

Despite what Klaus claims in the interview and on the wiki, the replication features do not really exist.

"Claims"? :(

There's just a few highly experimental

Stop quoting Prevayler javadoc as if it were great news. :(

and very buggy classes that make a rather poor attempt at replication,

Maybe if you try saying that yet again, even without presenting proof, I might start believing that myself. :(

but I don't think anyone would seriously consider using this in any sort of production application, enterprise or not.

3% of the New York stock exchange transactions go through Prevayler.

On the subject of bugs, the version I looked at (2.02.005, the latest available on sourceforge) is indeed quite buggy.

Oh, really? Care to show some evidence? [*]

Klaus, your comments about bug-freeness may have conceivably applied when Prevayler was 2 classes. It certainly does not in the 37 classes of the releases 2.02.005 version.

Hey, you know what, that is the third time [*] you say that IN THE SAME "SUMMARY" POST!

It must be true! :P

Yes, yes, on this forum you state its experimental, but you've lead people to believe it's a serious part of the release in other forums.

Hey, are you having a déjà vu [*], or is it just me?

The disk-writing and recovery parts also some serious issues. As Cameron states, you really can't enumerate them, because funamentally the design around recovery is just plain fragile.

X^))

HAHHAHHAHAHHAHAH AAAHHAHHAHAHH AHAHAHHA AHAHAH!!!

I apologize. I could not resist. :~)

That is about the lamest excuse I ever read.

"you really can't enumerate them" ???

Oh please! Stop making a fool of yourself. :(

If there are so many, produce at least one!!!

If you want to see what I mean, compare your recovery code to that of, say, HOWL (Highspeed Object Web Logger), a binary logger developed as a transaction log for projects like JOTM, Geronimo, and ActiveMQ. Comparing HOWL to Prevayler's implementation shows some large deficiencies in the Prevayler code.

Please quit your idle slandering :( and describe at least one of these "large deficiencies" [*].

Indeed, looking at the work on "Prevayler 3.0" and comparing it to 2.02.005 you can see that a great deal of code has changed,

True.

and a number of obvious and glaring bugs have been addressed.

Present proof or stop lying [*].

But, at the same time, the new code is an order of magnitude larger and more complex than the 37 classes of 2.02.005

Really? Prevayler is 370 classes now? That is an outright lie. :(

so in quashing some things from the 2.x line many more bugs have been introduced

Again no proof is presented [*], just vain speculation:

which may be part of the reason why the 3.x line has been under development for over a year and a half.

On that note, the code under development is devilishly hard to find - it's rather obscurely buried over at codehaus.org these days.

The Prevayler site contains precise instructions of how to access Prevayler's CVS on Codehaus.

one I mentioned on my blog comments - the StrictTransactionCensor and regular "transaction" processing do not guarantee order between them. More specifically, the STC may see transactions in the order, say, A-B-C-D, while the regular transaction processor sees ADCB. ... leading to inconsistent state.

False alarm. That is not a bug. :(

Access to the StrictTransactionCensor is synchronized by the CentralPublisher using the Turn class.

Normally people will ask "is there a bug here...?" or will say "I believe there might be a bug here..." but not you. You categorically assert there is a bug and "risk" making people wrongly believe Prevayler is buggy [*] and making a fool of yourself. :(

Why?

Hopefully this post passes your criteria for "respectful" and "useful". If it does not, please inform me of the pieces that do not pass muster and I will try reword them to conform to your rigorous criteria.

I am posting a link to this on your blog too.

I can obviously be wrong but, from a minimally serious person, for every accusation I made AND backed up above, I expect either an apology or an ARGUMENT as to why I am wrong.

But, of course, you are free to simply evade my points, continue promoting Prevayler FUD and making a fool of yourself in public. :(

I posted this comment at Mike Spille's blog, but I realized that it probably belongs here:

Having both Prevayler and Hibernate apps in production, I think the strong tone against Prevayler is a little overdone here. I agree that there are many circumstances in which it doesn't make sense to use Prevayler, and I agree that the Prevayler code is a little scruffy, but saying the number of places where it can be used is "vanishingly small" seems like hyperbole to me.

The discussion here has been interesting, and I appreciate both Mike Spille and Dave Orme's thoughtful posts. There are two positive things about Prevayler, though, that I think were missed.

Mike writes "Prevayler may require slightly less code, but not enough to make a difference in my opinion."

I think the number of lines of code is the wrong thing to focus on. The biggest development benefit of Prevayler comes from the reduced system complexity and the resulting ease of design and refactoring. There's no duplication of information between the relational database schema and the object model, no mysterious mapping layer, no worries about stale caches, no need to do a lot of mocking in your unit tests, and no worries about operations failing because of concurrent activity.

There's also no need to hire a database expert to solve a performance problem, no need to involve a DBA in a schema change, and no need to give each developer a copy of the database. It's all just Java, it's all in RAM, and it's really, really fast. Put all this together, and in my experience you get a substantial speedup in development.

Of course, Prevayler provides none of the amenities that an RDBMS does, so there's a cost to this. But for projects where agility is important and you have smart, expert Java developers, it can be a good choice. For both of the Prevayler-based apps I was involved in building, we were happy with our choice.

The other big positive point to Prevayler is that it serves to shock a lot of developers who seem unable to imagine building a serious application without an SQL database involved. Our profession needs that shock. I've seen a number of shops waste thousands of hours and millions of dollars because of their inability to think of Java as anything other than a language for generating SQL commands and displaying SQL results.

Every serious developer should understand the Prevalent System design pattern well enough to be able to build a system using it. Whether or not one choses to use the Prevayler code base or roll your own while doing that is an entirely different question.

William, I agree with you on the positive aspects of using something like Prevayler. However, those strengths are also the greatest weakness of the approach.

In particular in an IT sort of environment, the biggest issue is the complete lack of multiprocess or multi-language access. For anything built with Prevayler, one application is going to be able to access one set of data. There can't be any scripts, can't be any reporting, can't be any quick fixes for data problems. Your single Java process is it. All access to the data has to go through one application, everything has to be coded in Java. This is a gigantic limitation.

The second limitation of course is the JVM size limitation. Realistically speaking, your total Java object size under Prevayler control should be significantly less than 1GB.

So if you can live with only a single process ever accessing your data, and <1GB of data, OK. I think you'd be better off using anything _but_ Prevyaler, given its problems, but to each its own. But the combination of two constraints is a knock-out punch in most IT environments. I'd think a simple caching API with a switchable backing store would be far superior - it would give you almost all of Prevayler's benefits and none of its drawbacks.

I think is just plain mean to bash someone's code this way, especially in public. I've read the Prevayler code, extensively, a couple of different versions of it. As many have commented here for what it does Prevayler is pretty good.

My major beef has been the coding restrictions that require one to be very aware and careful of what is persistent and what is not. The beauty of Prevayler is that it does what it does with a small amount of code. So it's fairly easy for users to study it and patch up the bits they aren't completely happy with.

All code has bugs in it, this is a conseqence of Rice's theorem. Programming is a craft that we all work hard at to improve. In fact we had a recent thread here on TSS on "Why your code sucks". So please, let's keep the discussion technical, we all know Java's approach of checked exceptions encourages us to swallow them now and then, I'd rather see a segue into that topic than a posting of Kaus' code and a smart alec remark.

I think is just plain mean to bash someone's code this way, especially in public.

My comments were mean and on purpose because, unfortunately, it is beyond being anecdotal comparing the flamboyant rhetoric of Prevayler author with actual design/coding of this framework

Most have been said already here so I will refrain from repeating.

The beauty of Prevayler is that it does what it does with a small amount of code. So it's fairly easy for users to study it and patch up the bits they aren't completely happy with.

No offense here, but this and the comment before is a typical “wounded innocence syndrome” so often diagnosed in OSS community.

All code has bugs in it, this is a conseqence of Rice's theorem.

True. But all of us have very different tolerance for bugs. In a miniature project like Prevayler any of the coding bugs (leaving alone design decisions for a moment) pointed, for example, by Mike Spille are either testament to plain ignorance to quality as a concept or lack of basic skills.

Maybe yes, maybe not- the code does not block execution of other threads( althought slows them dowd);

"Slows them down" is a nice way to put it. How can you possibly defend code like "while (true) Thread.yield();"? What purpose can this code possibly serve? And no, it does not qualify as a bug, it qualifies as plain lack of knoweledge or professionalism!

- thread might be killed outside by stop() or interrupt();

Tell me what you smoke, I want that too :-) The code is just plain wrong. Period!

How can you possibly defend code like "while (true) Thread.yield();"? What purpose can this code possibly serve? And no, it does not qualify as a bug, it qualifies as plain lack of knoweledge or professionalism! Dmitriy.

If you continue looking really closely, you might find an actual Prevayler problem, and join the Hall of Fame.That would be great.Thanks, Klaus.

Dear Klaus,

You might want to do what Knuth did with Tex, offer a nickel more with each bug that people discover. Most programmers would prefer money to membership in the Hall of Fame and if the code is as bug free as claimed it will provide a good measure of that. People who have checks from Knuth cherish them for obvious reasons and I don't think he's even over $3.00 yet which is very impressive given what Tex does. I must say I find this claim of yours to be a bit over the top and beyond what I would consider appropriate marketing. It's one thing to compare query performance between in memory versus on disk and claim revolutionary performance, it's another to claim your code is bug free or even fairly close. All software has bugs. I like Prevayler a lot and consider the small code size a big plus. In my app I need to have it all in RAM so it's a good fit. Having all these command objects is awkward but I can hide all that without sacrificing too much. I'm using it in conjunction with JISP for storage of large data that needs indexing. JISP is also small and well written and architected so that one can replace the default B-Tree indexing if needed. If I can get the performance I need from DB4O I will use that as it's more of a full OODBMS but in the interim I'm sticking with Prevaler/JISP.

3 production bugs were found in Prevayler so far, so I would have lost 15 cents till now. :)

and beyond what I would consider appropriate marketing.

Of course having a Hall of Fame is absurd. You should take it with the good humor it was intended.

Don't you see I was pulling the poor guy's leg? :)

It's one thing to compare query performance between in memory versus on disk and claim revolutionary performance,

No. It is RAM against RAM. Please pay attention:

"These results are obtained even with the Oracle and MySQL databases FULLY CACHED IN RAM and on the same physical machine as the test VM. See Scalability Test Results.

How is this possible?

Every single String, Date or BigDecimal of every column, of every row, of every ResultSet, for every JDBC query you perform is an object and has to be instantiated. That is EXTREMELY costly. With Prevayler, your objects are simply there, ready to be used."

My apologies, you are right. I had forgotten you were comparing Prevayler to databases that had all data in cache. Of course the application path lengths are considerably longer so the numbers are not that surprising. Even running in memory and factoring out JDBC there's a lot happening in the database code to execute a query.

Prevayler is neat, I'm still using it for some simple prototypes that actually handle large object graphs. Large enough graphs that I'm forced to over ride the Serializable methods and break them down somewhat. I'm not so interested in replication and other middleware issues and so I prefer the simpler earlier implementations. I would like to see this Baptism problem handled better (see my other post today ) and if I have time will try to resolve that in Prevayler and offer you any ideas I have about it.

To repeat mself I'm looking also for something more orthogonal, .ie. persistence for POJOs that doesn't require any additional layers. This is why I'm looking also at DB4O. It's closer to that goal. Perhaps one day we'll have a persistent JVM ( like Squeak :^) and the Java community will ressurected yet another cool Lisp feature.

I'll hasten to add though that SQL is not going away. For large general purpose aggregation type queries it's perfect and it's founded on solid principles. I think the key point here is that if one doesn't need all that, then a simple approach like Prevayler provides a cleaner implementation and takes Oracle/MySQL completely out of the loop. It solves the need for all this O-R stuff. Of course one gives up the power of a general purpose query language like SQL. The SODA approach is cool but it is not a formal query language that I'm aware of, something like FQL (http://hopl.murdoch.edu.au/showlanguage.prx?exp=993&language=FQL)

What does happen in Prevalayer when you run out of memory in the middle of one of your transactions? Does Prevalayer back out any changes to the object graph occurring prior to the OutOfMemoryError? If not, couldn't that allow the object graph to be in an inconsistent state? Is there a chance that the snapshot could be taken while the object graph is in this inconsistent state? Also, is your application pretty much hosed (for transactions, at least) until you shut it down, install more memory, and restart it?

Note: I did actually try to figure this out for myself, but got confused by the "food taster" references in the code, as they didn't make any sense to me. Am I missing something?

Most of these comments are intelligent but fail to address the bigger truth of "there is no perfect solution" for data management, at least not a universal one. And there never will be one. Every data management mechanism (persistence/access) can find its own niche and some can be more global than others. It all depends on individual needs of the applications (including time/money/people/knowledge and the list goes on!). So keep encouraging all the "novel" or "segway" kind of ideas and take them all with a pinch of salt! :-)For small applications one of the solutions (Pure OR Mapping or pure Object DB or plain JDBC/ODBC for relational data sources or LDAP or prevalent model etc...) might be sufficient and bigger (true enterprise) apps might need more than one solution and/or modifications there of!From whatever little I have seen of Prevayler, I can.. if I want to.. always find a niche for that kind of data management for "sections" of "data" that I want to manage in any give large application!And please dont ridicule anything in public!(code/idea/person/tool...) It would only hurt the community in the long run.. (either people not willing to come up into public with their ideas or worse.. never make them open source!) :-)Just my two cents! :-) Keep up the good work!

Hi,I found the ODBMS solution more flexible than Prevayler. In most of ODBMSs much work is in the cache side. You can tune the client cache to handle in memory how many objects you want, but if memory lacks the ODBMS use the storage in transparently way.

The same features you can find in most of RDBMS mapping tools and generally in JDO but performances cannot be comparable with Prevayler.

For this reason I see an ODBMS a more flexible solution to Prevayler.

No flame please ;-)

bye,Luca GarulliOrienTechnologies.com - Light ODBMS, All in one JDO solutionwww.Pro-Netics.com (member of Orixo.com - The XML business alliance)

I've just found out that BerkeleyDB Java Edition (BDBJE) provides implementations of Map, SortedMap and Iterator backed by a BDB database (which has ACID transactions and such).Prevayler uses a jva.util.HashMap to store all of its data in memory, am I right ?The question is: given a choice of using BDBJE and Prevayler, what would be the reasons to choose the latter ?Can Prevayler prevail over BDBJE ?Regards,Elifarley

Not really a answer to your question but i think another option would be to use "Persisted Object Tree" pot.forgeahead.hu

I've just found out that BerkeleyDB Java Edition (BDBJE) provides implementations of Map, SortedMap and Iterator backed by a BDB database (which has ACID transactions and such).Prevayler uses a jva.util.HashMap to store all of its data in memory, am I right ?The question is: given a choice of using BDBJE and Prevayler, what would be the reasons to choose the latter ?Can Prevayler prevail over BDBJE ?Regards,Elifarley

Not really a answer to your question but i think another option would be to use "Persisted Object Tree" pot.forgeahead.hu

TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations technology projects - with its network of technology-specific websites, events and online magazines.