Fog Creek Software
Discussion Board




Prevayler

Prevayler - "Do you still use a database" ( http://www.prevayler.org/ and http://sourceforge.net/projects/prevayler ) seems to be a persistence layer for Java Applications, based on the principle that you would keep all the objects in memory, and, instead of persisting them to a database, they would be periodically serialized to disk files. According to the site, a disk-based message log allows allow reconstitution of the in-memory state in case of a system crash.

Has anyone tried this ? It is obvious that you must have lots of RAM (all your data will be in-memory), at least until some pooling schema is implemented (or is it already there ?).

I am a bit skeptical, but my continous search for a simple and scalable solution for the persistence layer in Java applications made me ask your opinons.

Chester
Wednesday, July 30, 2003

Keeping all your data in memory will lead to using lots of memory, which may lead to poor scalability and performance.

But, that's what .NET is doing, anyway.

John K.
Wednesday, July 30, 2003

I have read their web site, and their system seems to be mostly bullshit.

I mean, it may work, but they won't honestly admit there are limitations.

John K.
Wednesday, July 30, 2003

"But, that's what .NET is doing, anyway."

Huh? There's no analog to Prevayler in the .NET Framework. What, pray tell, are you thinking of?

Brad Wilson (dotnetguy.techieswithcats.com)
Wednesday, July 30, 2003

Reminds me of a shared property manager in MTS.

Rick Watson
Wednesday, July 30, 2003

My company has an MFC app that can load and save it's data either in a database or text files, and text files are much faster.  That is only because it does just load everything into memory, and then dump it back out to file when you save.  DB's strengths are in filtering, manipulating, and joining data.  If you don't use those, you just get a remarkable amount of overhead.  And there are many times when working with the data in memory, rather than through the DB, is much much faster.

Prevayler makes great sense if prevayler is all you need.  I don't think anyone behind Prevayler would suggest that you never need a database layer in a program, rather that many applications using databases are paying for a lot of features that they don't use.  If all your program ever does is a bunch of unfiltered table loads, then your database is probably hurting the performance of your application.

Keith Wright
Wednesday, July 30, 2003

Prevayler is a very interesting approach.... databases are often total overkill, memory costs a whopping 140 dollars for a gig of ram... I have seen many databases serving up 50 megs of data, what a waste.  Plus, storing your data in the same jvm leads to massive (order of magnitude) increases in speed.  Plus, their website readily admits if the size of you data will exceed that of ram, it's not suitable.

mph
Wednesday, July 30, 2003

If you use mmap/MapViewOfFile, and it makes sense to keep your data below 2GB, then it's probably the best way to go.

That's NOT, however, what prevayler is doing; It also doesn't play well with modern OO implementations (it has no conflict with OO concepts).

He-who-would-not-be-named
Wednesday, July 30, 2003

Brad,

I think John was refering to .net's disconnected datasets, which do store all requested data in local memory and allow operations on said data before persisting to the database, so they use more local memory but less bandwidth / server side cpu and memory.

Andrew Murray
Wednesday, July 30, 2003

With cheap RAM, large address spaces and 'faster is better' requirements, the use of in-memory databases has been growing. Whtether it's cached relational data like ADO.NET or various forms of object persistence, it will just keep on growing.

"Disks are a hack, not a design feature". So says Alan Cooper in About Face 2.0:The Essentials of Interaction Design. "They are a compromise, a dilution of the solid-state architecture of digital computers" he argues.

Teacher: "Back at the turn of the centruy, people stored data on mechanical devices!"

Class: "No way!"

fool for python
Wednesday, July 30, 2003

There's still an order-of-magnitude difference between the cost of a gig of RAM and a gig of disk.  Well, that, and limits as to how much RAM you can plug into an individual machine.

And I know of plenty of data sets that are well beyond 2 gig per-process limits under most 32 bit OSes.

Flamebait Sr.
Wednesday, July 30, 2003

"I think John was refering to .net's disconnected datasets, which do store all requested data in local memory and allow operations on said data before persisting to the database, so they use more local memory but less bandwidth / server side cpu and memory."

That's quite a stretch, to say that "data from a database is temporarily in memory while you use it" is equivalent to "here's a system of serialized objects that's always in memory".

Brad Wilson (dotnetguy.techieswithcats.com)
Wednesday, July 30, 2003

Flamebait,

yes, it is a temporal displacement problem.

fool for python
Wednesday, July 30, 2003

"There's still an order-of-magnitude difference between the cost of a gig of RAM and a gig of disk.  Well, that, and limits as to how much RAM you can plug into an individual machine."

There's also an order of magnitude different between the performance of a gig of ram and a gig of disk.  And a gig of storage on a db stores much less data than a gig of ram, due to the storage overhead of a db (indexes, which are primarily used to overcome slow disk performance)  Both are cheap, who cares?

"And I know of plenty of data sets that are well beyond 2 gig per-process limits under most 32 bit OSes. "

And I know plenty of data sets that are well within 2 gigs.  It's a moot point anyways... no one (including the most die hard prev supporters) is suggesting this approach when you have more data than ram.

mph
Wednesday, July 30, 2003

No, I'm mostly suggesting that the memory-based system works great for things... until you run out of address space, DIMM sockets, or money.  And then you are in what I like to call a world of pain.

Flamebait Sr.
Wednesday, July 30, 2003

We use it. We have a cms product that has multiple data retrieval strategies (eg, database, prevayler, ldap, filesystem, etc) configured through xml.

Prevayler is stupidly fast and very reliable. You do want to make sure your serialisation knowledge is up to scratch to make it work well, in terms of object schema changes, etc. And the default serialisation form gives a larger file size than XML!

There is also the problem of reporting and data migration - although we can solve this reasonably easily through our cms xml export and import (we can migrate the data to oracle with a few clicks).

Rhys Keepence
Wednesday, July 30, 2003

Oh, that's okay, if there isn't enough physical RAM it can always use virtual memory.

I'll get my coat
Thursday, July 31, 2003

Yes,

When you run out of resources there are problems. But there will come a day when we have enough RAM for most applications. Until then, we have to use mechanical devices to store data.

fool for python
Thursday, July 31, 2003

Nonsense, the more resources we have, the bigger problems we will try to tackle.  Some projects are always going to have a small finite set of data to work on that can easily fit on a computer, and some projects are always going to be working on as massive a set of data as is doable.

Keith Wright
Thursday, July 31, 2003

There was a pretty thorough discussion of Prevayler on Slashdot a few months back.

My take is:

* It's a great approach for prototyping, demos and simple applications

* It's a good approach for applications that are uncertain whether they need a database or not; you can start with Prevayler and use a "real database" when or if you need to

* It's a reasonable approach if certain other conditions are met:
  + Size of data set less than RAM
  + The Prevayler application server owns the data: no concurrent access to the files which contain the application data

Peter Breton
Thursday, July 31, 2003

Joel,

Some framworks are complex and hard to quickly test and get an opinion upon. This is not the case with Prevayler however. I tested it by writing a simple domain model consisting of three releated entities (Person, Address, Company). I hooked up Prevaylers command pattern by delegation from store methods on the entities and ended up with a simple but pretty OO:ish persistable domain model. This took me 3 hours and has given me enough own (un-biased) knowledge about Prevayler to know when to use it and when to not.

So what did I think?
I am really impressed with its simplicity and beautiful (yet simple) idea behind it.

But hey, don't take my word for it. My suggestion is, try it yourself.

Tobias Hill
Tuesday, March 23, 2004

I largely agree with Tobias, I have tried it also with good results. The essential idea is object persistence via serialization. This part is cool as one more or less gets the objects by reachibility for free. The downside is that one has to use the command pattern to basically capture state changes which by wrapping in "command object" can also be peristed as logs. This makes it robust in terms of handling crashes. However what's ugly is that as one builds larger systems these command objects end up forming a hierarchy that models the application paths.

I've been investigating a more orthogonal persistence that would remove the need for these command objects and make for a simpler and cleaner api for the progammer. I think this can be done using AOP or other reflective techniques. I'm told this is what the Nanning project is about but I'm not sure I agree with that.

When one considers the enormous amount of time/money spent on this whole O-R mapping problem it's easy to see there are great opportunities in finessing the whole problem via object persistence. There's a lot of middleware in conventional database systems that is largely overlap with what app servers are about.. JDO doesn't really address this, it's more or less JDBC for objects and still encourages a persitences model that is bolted on the back end.

A true orthogonal persistence is probably best had at the JVM level. Sun did pursue that once with a project called PJama but abandoned it. I think something very neat and simple can be done with a combination of technologies like Prevayler, JISP, and Javassist.

Setting aside the marketing hype on the Prevayler site, it is very cool and a very small piece of code. For what it does it is well worth the few hours needed to read the code and understand it. It's a fine piece of work.

Robert Dionne
Saturday, May 01, 2004

*  Recent Topics

*  Fog Creek Home