Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv13389

Modified Files: NOTES Log Message: more changes than i can count

Date: Fri Aug 27 10:31:08 2004 Author: blee

Index: elephant/NOTES diff -u elephant/NOTES:1.1 elephant/NOTES:1.2 --- elephant/NOTES:1.1 Thu Aug 26 19:59:46 2004 +++ elephant/NOTES Fri Aug 27 10:31:08 2004 @@ -6,7 +6,12 @@ this has been optimized for use with CMUCL. it has been tested and somewhat optimized for allegro. SBCL and OpenMCL are definitely also desired targets. Lispworks is a target -as well but less so; i can't test it. +as well but less so: i don't have access to it. + +Theoretically one can port this to any lisp with a decent +FFI and MOP. However since those are two of the less +standardized bits of Lisp, in practice this might be +difficult.

From top to bottom, here are the implementation layers:

@@ -24,6 +29,52 @@ database / serializer, specials are needed. The store controller (see below) is also a special.

+----------------------- +CLASSES AND METACLASSES +----------------------- + +Persistent classes which the user defines are declared and +instrumented by using the persistent-metaclass. + +----------- +COLLECTIONS +----------- + +While we support serializing and persisting a wide class of +Lisp data types, there are problems with aggregate types +(conses, lists, arrays, objects, hash-tables...) + +1) not automatic: there's no way for elephant to know when +you've changed a value in an aggregate object, so you have +to manually restore it back into the slot to get it saved. + +slot-1 of obj A contains a cons. you change the car of the +cons. this is not reflected into the database unless you +re-set slot-1 of obj A with the cons. + +2) merge-conflicts: changing one value and saving an +aggregate will write out the whole aggregate, possibly +blowing away changes other threads have made behind your +back. this is not protected by transactions! + +3) consing, non-lazy and expensive (de)serialization: you +have to serialize/deserialize the entire aggregate every +time you save it or restore it. This is pretty fast all +things considered, but it's probably better to use +persistent collections. + +4) you have to store the entire collection in memory, +whereas one of the points of the database to store large +collections of objects..... + +For these and other reasons, we provide a hash-table-like +interface to Berkeley BTrees. These have many advantages +over ordinary hash-tables from the point of view of +persistence. + +TODO: programmatic way to create secondary indicies +(probably Lisp-level, since FFI callbacks are nasty.) + ---------- CONTROLLER ---------- @@ -57,6 +108,8 @@ determines liveness. The root object is a btree collection with a special OID 0.

+TODO: write the garbage collector. + ------------ TABLE LAYOUT ------------ @@ -157,32 +210,6 @@ SERIALIZER: AGGREGATE TYPES ---------------------------

-problems, or Do You REALLY Want to Persist Aggregates? - -0) not automatic: there's no way for elephant to know when -you've changed a value in an aggregate object, so you have -to manually restore it back into the slot to get it saved. - -1) merge-conflicts: changing one value in the aggregate will -write out the whole aggregate, possibly blowing away changes -other threads have made behind your back. this is not -protected by transactions! - -2) object identity / circularity: what are the right -semantics for things not tagged with OIDs? certainly can't -maintain identity across slots. - -obj A has slots slot-1 and slot-2. puts same cons in each -slot. no way to keep track that they are supposed to be the -same cons! - -3) consing, non-lazy and expensive (de)serialization - -"solution": in the FEW BIG TABLES schema, it is cheap to -create and use persistent collection types, which will -preserve identity et al. as with all persistent objects, -persistent collections are lazy-loading. - if you really must store non-persistent aggregates, we support:

conses (lists): 1 + car + cdr @@ -238,6 +265,8 @@ default accessed through a special variable *current-transaction*. The special *auto-commit* (which defaults to nil!) is also a special. + +TODO: do transactions memory leak?

UFFI is used as much as possible. Because some implementations can't call function pointers, and because we