How do PIM's handle data?

Since someone mentionned PIM's again the other day, that got me wondering how those tools (Ecco, Notes, etc.) manage this kind of free-form data efficiently like notes or outlines, since I assume they don't store them in neat rows and columns?

FredF
Tuesday, January 27, 2004

Treepad's format is open and plaintext. Just go to www.treepad.com and browse around. It resembles XML.

www.MarkTAW.com
Tuesday, January 27, 2004

What's wrong with storing them in rows and columns?  Data is data, and most SQL DBMS manufacturers (I know Sybase ASA makes up most of the market) make embedded products which work on Palm-type products.  Why reinvent the wheel?

MR
Tuesday, January 27, 2004

Right, but I was wondering how you can make efficient random access if the data don't fit rows and columns, eg. one big note with no predetermined size?

FredF
Tuesday, January 27, 2004

Define efficient :) The quick little note app I just whipped up for myself just has all data in a table with one big text field. The search works by just using LIKE.

It's efficient enough for N < a few thousand medium-sized records. If I need more, I'll upgrade it then. It would be absolutely trivial to simply store more data. You can always just store more in the DB. Of course, you might triple the database size, too. But you'll get lightning fast searches.

Mike Swieton
Tuesday, January 27, 2004

Fred:

I see what you mean now.  Well, there are numerous ways; if you look at 'big' (tens or hundreds of millions of rows of text data to search) installations of enterprise DBMS use they run into the same problems (constrained memory, CPU, etc.).

I’ve not a ton of experience in the ultra-lite environment but usually the enterprise market the text itself is either stored as-is and indexes are built up around it to facilitate full-text search or the text is broken up and inserted into component parts which are each searchable (and reassembled to display to the user).

MR
Tuesday, January 27, 2004

You could always define an overflow table, where the remainder of text for large entries can go.  Then reference a row in there from the original.

H. Lally Singh
Tuesday, January 27, 2004

Not sure where Chandler is on how they are storing data now, but it is similar to Agenda. I think thery are using RDF as the model and maybe RDF/XML  (Yuck!) as storage.

http://www.osafoundation.org/

Notes stores data in documents with arbitrary numbers of fields (and data type enforced in the UI).  Indexed views can be built on any item in the document.  Works reasonably well for application that don't push the intended scalability limits (should have been in an RDBMS to begin with).

fool for python
Wednesday, January 28, 2004

Thanks everyone for the input :-) What I was curious about, is how to maintain good performance even though all data is saved in a single file, and data are pretty much free-form.

For instance, I'd be curious to understand how they save notes, since you can only set a maximum allowed size, but a note can have any size within that limit, and its size changes every time the user updates the note.

Would the file hold some kind of structure similar to a file system, ie. linked sectors/clusters?

In other words, would it be a good idea to write a PIM using SQLite to hold all the data?

FredF
Wednesday, January 28, 2004

How do messages boards and Wiki's store text? How is a Wiki different from a PIM?

www.MarkTAW.com
Wednesday, January 28, 2004

Precisely, do they just define a big, variable text column, and let the DBMS handle the issue?

FredF
Wednesday, January 28, 2004

I don't know, why not download one and find out?

www.MarkTAW.com
Wednesday, January 28, 2004

  Recent Topics

  Fog Creek Home