Update of /project/elephant/cvsroot/elephant/doc In directory clnet:/tmp/cvs-serv3920/doc

Modified Files: elephant-design.texinfo elephant.texinfo installation.texinfo make-ref.lisp reference.texinfo scenarios.texinfo tutorial.texinfo user-guide.texinfo Log Message: Check for cross-store loading errors in multi-store operation; more documentation; backend language to data store language

--- /project/elephant/cvsroot/elephant/doc/elephant-design.texinfo 2007/04/01 14:33:29 1.3 +++ /project/elephant/cvsroot/elephant/doc/elephant-design.texinfo 2007/04/21 17:22:35 1.4 @@ -1,8 +1,8 @@ @c -*-texinfo-*-

-@node Elephant Design +@node Elephant Architecture @comment node-name, next, previous, up -@chapter Elephant Design +@chapter Elephant Architecture @cindex design

Elephant's early architecture was tightly coupled to the Berkeley DB --- /project/elephant/cvsroot/elephant/doc/elephant.texinfo 2007/04/12 02:47:23 1.8 +++ /project/elephant/cvsroot/elephant/doc/elephant.texinfo 2007/04/21 17:22:35 1.9 @@ -20,8 +20,8 @@

@titlepage @title Elephant User Manual -@subtitle Elephant version 0.6.1 -@author Ben Lee and Ian Eslick +@subtitle Elephant version 0.9 +@author Ian Eslick and Ben Lee

@c The following two commands @c start the copyright page. @@ -50,9 +50,9 @@ * Tutorial:: A basic ``getting started'' tutorial. * Installation:: Installation and test-suite procedures. * User Guide:: In depth discussion of all Elephant facilities and features. -* Usage Scenarios:: Design scenarios for Elephant applications. * User API Reference:: Function and class documentation of the user API. -* Elephant Design:: An overview of elephant's internal architecture. +* Design Patterns:: Design scenarios for Elephant applications. +* Elephant Architecture:: An overview of elephant's internal architecture. * Data Store API Reference:: Function level documentation for data store implementors. * Copyright and License:: Your rights and freedoms. @end menu @@ -78,8 +78,8 @@ @include tutorial.texinfo @include installation.texinfo @include user-guide.texinfo -@include scenarios.texinfo @include reference.texinfo +@include scenarios.texinfo @include elephant-design.texinfo @include data-store-reference.texinfo @include copying.texinfo --- /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/04/12 02:47:23 1.9 +++ /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/04/21 17:22:35 1.10 @@ -9,9 +9,9 @@ * Requirements:: Supported lisps and required libraries. * Configuring Elephant:: Setting up Elephant and the configuration file. * Loading Elephant:: Loading Elephant and the data store loading protocol. -* Berkeley DB Data Store:: Installing support for the Berkeley DB data store +* Berkeley DB:: Installing support for the Berkeley DB data store * Berkeley DB Example:: An example of installing and running the Berkeley DB data store. -* CL-SQL Data Store:: Install and connecting to the CL-SQL data store +* CL-SQL:: Install and connecting to the CL-SQL data store * CL-SQL Example:: An example of using the CL-SQL data store. * Elephant on Windows:: More details about running Elephant on Windows * Test Suites:: How to run and interpret the output of the regression test suite @@ -88,8 +88,8 @@ of you with asdf-install on your system)

In addition to these libraries, each data store has their own -dependencies as discussed in @ref{Berkeley DB Data Store} and -@ref{CL-SQL Data Store}. +dependencies as discussed in @ref{Berkeley DB} and +@ref{CL-SQL}.

@node Configuring Elephant @comment node-name, next, previous, up @@ -224,9 +224,9 @@ database and return the @code{store-controller} subclass instance for that backend.

-@node Berkeley DB Data Store +@node Berkeley DB @comment node-name, next, previous, up -@section Berkeley DB Data Store +@section Berkeley DB

The Berkeley DB Data Store started out as a very simple data dictionary in the Berkeley Unix operating system. There are many ``Xdb'' systems that use the @@ -275,9 +275,9 @@ in your application.

-@node CL-SQL Data Store +@node CL-SQL @comment node-name, next, previous, up -@section CL-SQL Data Store +@section CL-SQL

Although originally designed as an interface to the BerkeleyDB system, the original Elephant system has been experimenetally extended to --- /project/elephant/cvsroot/elephant/doc/make-ref.lisp 2007/04/12 02:47:23 1.8 +++ /project/elephant/cvsroot/elephant/doc/make-ref.lisp 2007/04/21 17:22:35 1.9 @@ -1,5 +1,7 @@ (require 'asdf) (asdf:operate 'asdf:load-op 'elephant :force t) +(in-package :elephant) + (load (merge-pathnames #p"src/elephant/query" (asdf:component-pathname (asdf:find-system 'elephant)))) @@ -19,8 +21,6 @@ (sb-posix:chdir include-dir-path) (load docstrings-path)

-(in-package :elephant) - (defclass simple-store-controller (store-controller) ())

--- /project/elephant/cvsroot/elephant/doc/reference.texinfo 2007/04/12 02:47:23 1.10 +++ /project/elephant/cvsroot/elephant/doc/reference.texinfo 2007/04/21 17:22:35 1.11 @@ -8,8 +8,8 @@

@menu * Store Controllers:: Connecting to a data store. -* Persistent Class:: Defining persistent classes and creating and manipulating persistent indices. -* Class Indexing:: Convenient indexing of persistent classes. +* Persistent Objects:: Defining persistent classes and creating and manipulating persistent indices. +* Persistent Object Indexing:: Convenient indexing of persistent classes. * Persistent Sets:: Maintaining persistent collections the easy way. @c * Query Interfaces:: Finding instances. * BTrees:: BTrees and indices, a low level persistent data structure. --- /project/elephant/cvsroot/elephant/doc/scenarios.texinfo 2007/04/19 22:25:51 1.6 +++ /project/elephant/cvsroot/elephant/doc/scenarios.texinfo 2007/04/21 17:22:35 1.7 @@ -11,7 +11,7 @@ * Persistent System Objects:: Making persistent objects a natural part of your system * Elephant as Database:: Using Elephant as a database for records and user data instead of using a SQL relational Database * Multithreaded Web Applications:: Elephant is a natural match for web applications -* Graph-oriented Applications:: Elephant is good, but not optimized, for graph-oriented applications. +@c * Graph-oriented Applications:: Elephant is good, but not optimized, for graph-oriented applications. * Real-World Application Examples:: See some real-world applications Elephant has been used for and a brief discussion of how it was used and any novel uses of Elephant. @end menu

@@ -32,6 +32,92 @@ that they will be motivated to contribute design patterns of their own.

+@node Persistent System Objects +@comment node-name, next, previous, up +@section Persistent System Objects + +The simplest design pattern supported by Elephant is the use of +persistent objects in the place of standard objects. Typically you +can just modify the old class definition to inherit the +@code{persistent-metaclass}. Depending on your application, objects +may need to have transient slots for performance reasons. We'll +create a dummy class to illustrate: + +@lisp +(defclass system-object () + ((appname :accessor system-appname :initarg :name) + (url :accessor system-url :initarg :url) + (state :accessor system-state :initarg :state :initform 'idle)) + (:metaclass persistent-metaclass)) +@end lisp + +When starting up your application you need to recover references to +any persistent objects that were created in a prior session or +initialize a new one. + +If you are storing system objects in parameters, you can just call +an initialization function on startup: + +@lisp +(defparameter *system* nil) + +(defun initialize-system (appname) + (let ((system-object (get-from-root '*system*))) + (setf *system + (if system-object system-object + (make-instance 'system-object :name appname))))) + +*system* +=> #<SYSTEM-OBJECT ...> +@end lisp + +And now you can use your parameter as you did before. If you want to +avoid calling initialization functions, you can just accesss system +objects through functions instead of parameters. + +@lisp +(defparameter *system* nil) + +(defun sys-object () + (unless *system + (let ((appname (get-application-name)) + (url (get-system-url))) + (setf *system* (make-instance 'system-object :name appname :url url)))) + *system*) + +(sys-object) +=> #<SYSTEM-OBJECT ...> +@end lisp + +One constraint to keep in mind is that slot access will be slower as +it has to synchronize to disk. This is usually not noticable for +objects that are accessed on the order of seconds instead of +milliseconds. For objects read constantly, but where you want to save +any written values it helps to have a transient slot to cache values. +You can override some methods to ensure that the persistent value is +always updated, but that reads happen from the cached value and that +the cached value is restored whenever the object is loaded. + +@lisp +(defclass system-object () + ((appname :accessor system-appname :initarg :name) + (url :accessor system-url :initarg :url) + (laststate :accessor system-laststate :initarg :state :initform 'idle) + (state :accessor system-state :initarg :state :transient t) + (:metaclass persistent-metaclass)) + +(defmethod (setf system-state) :after (state (sys system-state)) + (setf (system-laststate sys) state)) + +(defmethod initialize-instance :after ((sys system-state) &rest rest) + (declare (ignore rest)) + (when (slot-boundp sys 'laststate) + (setf (system-state sys) (system-laststate sys)))) +@end lisp + +And now you have an instant read cache for a slot value. This pattern +is used several times within the Elephant implementation. + @node File System Replacement @comment node-name, next, previous, up @section File System Replacement @@ -43,11 +129,14 @@ Because Elephant's serializer supports most lisp types, Elephant can greatly simplify ease these concerns and allow you to work directly with your natural in-memory representations with almost no work to -encode/decode formats or manage files in the file system. +encode/decode formats or manage files in the file +system@footnote{Example provided by Ian Eslick, April 2007}.

The simplest way to accomplish this is to simply open a store -controller and use the root btree as a key-value store instead of a -file system directory. You might hide some of the details like this: +controller and initialize a key-value pair in the root btree as a +instead of a filename and file data in some system directory. Like +the initialization process described for standard objects, you can +hide some of the details like this:

@lisp (defvar *resources* (make-hash-table)) @@ -97,7 +186,7 @@ conditions that allowed restarts that initialized values or allowed users to update the hash in the background and continue computation.

-@footnote{Example provided by Ian Eslick, April 2007} +

@node Checkpointing Conventional Program State @comment node-name, next, previous, up @@ -123,29 +212,27 @@

@subsection Assumptions

-To get speed, we want all our objects to be standard lisp objects that -are in memory and have no special harnesses that would interfere with -using the full power of lisp. At some point in execution, we want to -store the current state of a bunch of objects to disk, but make it -easy to reproduce the exact state at a later point in time. For -simplicity, we'll assume that we are talking about collections of CLOS -objects. - -An additional complication is that many programs have sets of -interdependant objects. These could be complex program graphs, the -state of an active search process or a standard OO system that uses a -bunch of program objects to function. This means that we need to -persist not just object state, but also references and any object that -is referred to. +To maintain processing speed and convenience we would like all our +objects to be standard lisp objects without special harnesses that +would interfere with applying the full power of lisp. At some point +during execution, we want to store the current state of a set of +objects to disk and yet make it easy to reproduce the original state +at a later point in time. For simplicity, we'll limit ourselves +to collections of CLOS objects. + +A complication is that many programs have sets of interdependant +objects. These could be complex program graphs, the state of an +ongoing search process or a standard OO system that uses a bunch of +different program object types to run. This means that we need to +persist not just object state, but also references to other objects.

Using CLOS reflection we can provide a general solution to capturing objects, slot values and references. However to reproduce references, -we'll need to be able to find the object referenced and the only -general way to do that is to store it as well. Thus a snapshot is a -closed set of self-referential objects. +we'll need to be able to find the object referenced and the only way +to do that is to store it as well. Thus we want to create a snapshot +of a closed set of self-referential objects.

-The assumptions required to implement the simple checkpointing -implemented here is: +The assumptions underlying the snapshot mechanism is:

@itemize @item @strong{Use standard CLOS objects and references to other CLOS objects.} @@ -166,7 +253,149 @@ fine. @end itemize

-@subsection Implementation: The Snapshot Set +@subsection Snapshot Set + +The snapshot implementation is called a @code{snapshot-set}. The next +section will go into detail, but a walkthrough will help make it +clearer@footnote{Example provided by Ian Eslick, April 2007}. + +A snapshot set is quite easy to use. Load the complete code and play +with this simple walk through. The code can be located in the +Elephant source tree under @code{src/conrib/eslick/snapshot-set.lisp}. + +The first step is to create a @code{snapshot-set} object, + +@lisp +(setf my-set (make-instance 'snapshot-set)) +@end lisp + +and add it to the root so we don't lose track of it. + +@lisp +(add-to-root 'my-set my-set) +@end lisp + +Then we need some objects to play with. + +@lisp +(defclass my-test-class () + ((value :accessor test-value :initarg :value) + (reference :accessor test-reference :initarg :reference))) + +(setf obj1 (make-instance 'my-test-class :value 1 :reference nil)) +(setf obj2 (make-instance 'my-test-class :value 2 :reference obj1)) +(setf obj3 (make-instance 'my-test-class :value 3 :reference obj2)) + +(register-object obj3 my-set) +(snapshot my-set) +@end lisp + +Now your set should have persistent versions of all three classes that +are reachable from @code{obj3}. + +@lisp +(map-set (lambda (x) (print (test-value x))) my-set) +=> +3 +2 +1 +@end lisp + +Of course such fully connected objects are not always common, so we'll +demonstrate using hash tables to create root indexes into our objects +and sidestep registration calls entirely. We'll create a fresh set to +work with. + +@lisp +(setf my-set (make-instance 'snapshot-set)) +(add-to-root 'my-set my-set) + +(setf obj4 (make-instance 'my-test-class :value 4 :reference obj1)) +(setf obj5 (make-instance 'my-test-class :value 5 :reference nil)) + +(setf hash (make-hash-table)) +(setf (snapshot-root my-set) hash) + +(setf (gethash 'obj3 hash) obj3) +(setf (gethash 'obj4 hash) obj4) +(setf (gethash 'obj5 hash) obj5) + +(snapshot my-set) +@end lisp + +To properly simulate restoring objects, we need to drop our old hash +table as well as clear the persistent object cache so the snapshot set +transient object is reset. + +@lisp +(setf my-set nil) +(setf hash nil) +(elephant::flush-instance-cache *store-controller*) +@end lisp + +Now we'll pretend we're startup up a new session. + +@lisp +(setf my-set (get-from-root 'my-set)) +(setf hash (snapshot-root my-set)) +@end lisp + +The cache is automatically populated by the implicit @code{restore} +call during snapshot-set initialization, and our hash table should now +have all the proper references. We'll pull out a few. + +@lisp +(setf o4 (gethash 'obj4 hash)) +(setf o3 (gethash 'obj3 hash)) +(setf o2 (test-reference o3)) + +(not (or (eq o4 obj4) + (eq o3 obj3) + (eq o2 obj2))) +=> t +@end lisp + +The new objects should not be eq the old ones as we have restored +fresh copies from the disk. + +If you review the setup above, @code{obj3} references @code{obj2} +which references @code{obj1} and @code{obj4} also references +@code{obj1}. So if the objects were properly restored, these +references should be @code{eq}. + +@lisp +(eq (test-reference o2) (test-reference o4)) +=> t +@end lisp + +And finally we can demonstrate the restorative power of snapshot sets. + +@lisp +(remhash 'obj5 hash) + +(gethash 'obj5 hash) +=> nil nil + +(restore my-set) +(setf hash (snapshot-root my-set)) + +(gethash 'obj5 hash) +=> #<MY-TEST-CLASS ..> t + +(test-value *) +=> 5 +@end lisp + +This means that while our set object was not reset, the restore +operation properly restored the old reference structure of our root +hash object. Unfortunately, in this implementation you have to reset +your lisp pointers to get access to the restored objects. + +A future version could traverse the existing object cache, dropping +new references and restoring old ones so that in-memory lisp pointers +were still valid. + +@subsection Snapshot Set Implementation

In this section we walk through the implementation of the snapshot set in detail as it provides: @@ -323,7 +552,7 @@ standard object would get an immediately serialized version rather than a reference. This of course makes it impossible to restore multiple references to a single object. The approach taken here is to -instantiate a @emphasize{proxy} object which is a copy of the original +instantiate a @emph{proxy} object which is a copy of the original class and stores references to normal values in its slots. Any references to hashes or standard classes are replaced with a reference object that records the unique id of the object so it can be properly @@ -391,144 +620,6 @@ obj))) @end lisp

-A full set of source code for @code{snapshot-sets} can be found in the -Elephant source tree under @code{src/conrib/eslick/snapshot-set.lisp}. - -@subsection Using Snapshot Sets - -A snapshot set is quite easy to use. Load the complete code and play -with this simple walk through. First we need to create a set object, - -@lisp -(setf my-set (make-instance 'snapshot-set)) -@end lisp - -and add it to the root so we don't lose track of it. - -@lisp -(add-to-root 'my-set my-set) -@end lisp - -Then we need some objects to play with. - -@lisp -(defclass my-test-class () - ((value :accessor test-value :initarg :value) - (reference :accessor test-reference :initarg :reference))) - -(setf obj1 (make-instance 'my-test-class :value 1 :reference nil)) -(setf obj2 (make-instance 'my-test-class :value 2 :reference obj1)) -(setf obj3 (make-instance 'my-test-class :value 3 :reference obj2)) - -(register-object obj3 my-set) -(snapshot my-set) -@end lisp - -Now your set should have persistent versions of all three classes that -are reachable from @code{obj3}. - -@lisp -(map-set (lambda (x) (print (test-value x))) my-set) -=> -3 -2 -1 -@end lisp - -Of course such fully connected objects are not always common, so we'll -demonstrate using hash tables to create root indexes into our objects -and sidestep registration calls entirely. We'll create a fresh set to -work with. - -@lisp -(setf my-set (make-instance 'snapshot-set)) -(add-to-root 'my-set my-set) - -(setf obj4 (make-instance 'my-test-class :value 4 :reference obj1)) -(setf obj5 (make-instance 'my-test-class :value 5 :reference nil)) - -(setf hash (make-hash-table)) -(setf (snapshot-root my-set) hash)

[286 lines skipped] --- /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2007/04/12 02:47:23 1.16 +++ /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2007/04/21 17:22:35 1.17 @@ -1048,7 +1048,7 @@ more than two levels of transactional accesses with the top using with-transaction and the bottom using ensure-transaction.

-See @ref{Transaction Details} for more details and @ref{Usage Scenarios} +See @ref{Transaction Details} for more details and @ref{Design Patterns} for examples of how systems can be designed and tuned using transactions.

@node Advanced Topics @@ -1114,4 +1114,4 @@ lisps. @end itemize

-Further, @pxref{Usage Scenarios} for information about Elephant design patterns, solutions to common problems and other scenarios with multiple possible solutions. +Further, @pxref{Design Patterns} for information about Elephant design patterns, solutions to common problems and other scenarios with multiple possible solutions. --- /project/elephant/cvsroot/elephant/doc/user-guide.texinfo 2007/04/12 02:47:23 1.11 +++ /project/elephant/cvsroot/elephant/doc/user-guide.texinfo 2007/04/21 17:22:35 1.12 @@ -12,8 +12,8 @@ * Class Indices:: In-depth discussion about indexing persistent indices. @c * Querying persistent instances:: Retrieving instances of classes. * Using BTrees:: Using the native btree. -* Secondary Indices:: Alternative ways to index collections. * Using Cursors:: Low-level access to BTrees. +* BTree Indices:: Alternative ways to index collections. * Transaction Details:: Develop a deeper understanding of transactions and avoid the pitfalls. * Multi-repository Operation:: Specifying repositories. * Multi-threaded Applications:: What considerations are required for safe multi-threading @@ -23,6 +23,8 @@ * Performance Tuning:: How to get the most from Elephant. * Berkeley DB Data Store:: Commands and concerns specific to the :BDB data store * CL-SQL Data Store:: Commands and concerns specific to the :CLSQL data store +* Postmodern Data Store:: +* Native Lisp Data Store:: @end menu

@node The Store Controller @@ -280,7 +282,7 @@ to other objects can copy every reachable object! This is a direct and dire consequence of the ``store-by-value'' restriction. (@pxref{Persistent Classes and Objects} for how to design around the -store-by-value restriction}). +store-by-value restriction).

This list describes how aggregates are handled by the serializer.

@@ -332,19 +334,26 @@ @section Persistent Classes and Objects

Persistent classes are instances of the @code{persistent-metaclass} -metaclass. All instances of this persistent class have a unique ID -and a pointer to the @code{store-controller} specification they are -associated with. Accesses to slot values become direct reads and -writes to the data store and are thus always persisted. When the -instance itself is writtent to the database, for example as a key or -value in a @code{btree}, only the unique ID is stored. - -Thus serialization of persistent objects is exceedingly cheap compared -to standard objects, but slot access can be much more expensive. -Persistent objects are excellent at, for example, storing the link -structure of a graph. A graph of persistent objects can itself be -persisted by storing the head node of a fully-connected graph to the -store controller root. +metaclass. All persistent classes keep track of which slots are +@code{:persistent}, @code{:transient} and/or @code{:indexed} and are +used as specializers in the persistence meta-object protocols +(initialization of slots, slot-access, etc). + +All persistent classes create objects that inherit from the +@code{persistent} class. The persistent class has two slots that +contain a unique object identifier (oid) and a reference to the +@code{store-controller} specification they are associated with. +Persistent slots do not take up any storage space in memory, instead +the slot access protocol is changed into calls to the configured data +store via the store controller. Typically, data stores then perform +reads and writes to disk. + +When a reference to a @code{persistent} instance itself is written to +the database, for example as a key or value in a @code{btree}, only +the unique ID and class of the instance is stored. When read, a +persistent object instance is re-created (see below). This means that +serialization of persistent objects is exceedingly cheap compared to +standard objects, but slot access time can be much more expensive.

@subsection Persistent Class Definition

@@ -352,10 +361,12 @@ @code{persistent-metaclass} to the class initarg @code{:metaclass}. The only differences between standard and persistent class definitions is the specification of a slot storage policy and an index policy. +The @code{defpclass} macro is provided as a convenience to hide the +@code{:metaclass} slot option.

-Storage policies are specified by a boolean argument to the slot -initargs @code{:persistent} or @code{:transient}. Slots are -@code{:persistent} by default +Slot value storage policies are specified by a boolean argument to the +new slot initargs @code{:persistent}, @code{:transient} and +@code{indexed}. Slots are @code{:persistent} by default.

@lisp (defclass my-pclass () @@ -365,15 +376,23 @@ (:metaclass persistent-metaclass)) @end lisp

+In the definition above, @code{pslot1} and @code{pslot2} are +persistent while @code{tslot1} is transient and stored in memory. + Slot storage class implications are straightforward. Persistent slot writes are durably stored to disk. Transient slots are initialized on -instance creation according to initforms or to initargs. Transient -slot values are never stored to nor loaded from the database. +instance creation according to initforms or initargs. Transient slot +values are never stored to nor loaded from the database.

-During a given lisp session transient values will be cached as long as -they are not collected by the Lisps GC. After GC, if you retrieve an -object from the store its transient slots will be reset to the slot -initforms from the class definition. +During a given lisp session transient values will be cached regardless +of how often you retrieve an object from the store as long as there is +a reference to it stored in memory or tere are zero references but the +object has not yet been collected by the lisp garbage collector. +After collection, if you retrieve an object from the store its +transient slots will be reset to the slot initforms from the class +definition. You can only use @code{:initargs} to initialize transient +or persistent slots during the initial call to @code{make-instance} or +by manually creating the instance from an oid (see below).

@lisp (setf pobj1 (make-instance 'my-pclass :pslot1 1 :tslot3 3)) @@ -403,27 +422,31 @@ @end lisp

The implications of this behavior is that you need to think carefully -about using transient values. Primarily you cannot make assumptions -about the state of transient values in objects loaded from the store -unless you know that they were loaded and cannot be GC'ed. - -Finally, the index policy tells the data store whether to maintain an -inverted index that maps slot values back to their parent objects. -The :index options and behaviors of persistent classes are discussed in -depth in @ref{Class Indices}. +about how to use transient values. Essentially you cannot make +assumptions about the state of transient values in objects loaded from +the store unless you know that they were loaded and cannot be GC'ed. + +The @code{:index} option tells Elephant whether to maintain an +inverted index that maps slot values to their parent objects. The +behavior of indexed classes and class slots are discussed in depth in +@ref{Class Indices}.

@subsection Instance Creation

-Persistent objects are instances of the persistent classes defined -above. All persistent objects inherit from the class -@code{persistent} and share two properties: a unique ID and a -reference to the specification of the @code{store-controller} in which -they reside. This is ensured by the instance creation protocol -implemented by @code{persistent-metaclass}. +To the user, persistent objects are created as usual with a call to +@code{make-instance} with the desired slot initargs as illustrated in +the last section. However, this call will fail unless there is a +default @code{store-controller} instance in the variable +@code{*store-controller*} or the @code{:sc} keyword argument is +provided with a valid store controller object. This tells the +metaobject protocol implementing persistent slot creation where to +allocate storage for this instance's slots and

-Instances are created as normal, with a call to make-instance and -appropriate initargs. +* Default store controller & instance creation +* What happens to persistent objects when store-controller is closed?

+The best policy for initializing transient slots is an @code{:after} +method on @code{initialize-instance}.

The two properties of @code{persistent} can be specified explicitly during instance creation: @@ -440,12 +463,60 @@ precedence over values in the database, which take precedences over any @code{initforms} defined in the class.

-* Default store controller & instance creation -* What happens to persistent objects when store-controller is closed? +A good policy is to initialize transient values using an @code{:after} +method on @code{initialize-instance} to initialize transient values +using system defaults or from persistent slot values. That way +whether the slots are valid when the object is pulled from + +Also currently there is a bug where @code{initforms} are always +evaluated, so beware. (What is the current model here?) + +@subsection Class Redefinition + +This section discusses the appropriate model a user should employ in +thinking about the behavior of persistent object when their clases are +redefined. + +- What happens when you redefine a class online? +- Drop & add slots? Change slot status? +- What if you connect to an old database with a new class specification? + (ref to class indicies behavior) + +@subsection Synchronizing Code and Database + + +@subsection Persistent objects and the MOP

+This section provides increased detail on the use of the MOP to create +and support persistent objects.

+new slot types +- transient-slot +- persistent-slot

-:: User-defined persistent objects +slot instantiation protocols +- effective-slot-definition +- direct-slot-definition-class +- compute-effective-slot-definition +- compute-effective-slot-definition-initargs + +class initialization protocols +- shared-initialize :around (class) +- finalize-inheritance :around (class) +- reinitialize-instance :around (class) + +instance initialization protocols +- initialize-instance :before (instance) +- shared-initialize :around +- update-instance-for-redefinied-class +- update-instance-for-different-class + +slot access protocol +- slot-value-using-class (setf) +- slot-boundp-using-class +- slot-makunbound-using-clas + +@subsection Overriding methods on persistent objects

* slot types * caching @@ -458,20 +529,6 @@ the specification to work properly with persistent slots. However the proper behavior has been verified on SBCL, Allegro and Lispworks.

-:: Initialization - -Also currently there is a bug where @code{initforms} are always -evaluated, so beware. (What is the current model here?) - -:: Class Redefinition and Evolution - -* What happens when you redefine a class online? -* Drop & add slots? Change slot status? -* What if you connect to an old database with a new class specification? - (ref to class indicies behavior) - -:: Storage and Performance Considerations - @node Class Indices @comment node-name, next, previous, up @section Class Indices @@ -642,15 +699,15 @@ blocks of data is relatively inexpensive after a seek and comparisons on objects that are stored in memory is cheap.

-@node Secondary Indices +@node Using Cursors @comment node-name, next, previous, up -@section Secondary Indices +@section Using Cursors

Empty.

-@node Using Cursors +@node BTree Indicies @comment node-name, next, previous, up -@section Using Cursors +@section BTree Indicies

Empty.

@@ -665,7 +722,6 @@ throwing a condition when @code{with-transaction} forms are nested dynamically.

- ;; Transaction architecture: ;; ;; User and designer considerations: @@ -847,28 +903,20 @@ @code{*inhibit-slot-writes*} in your user method using @code{with-inhibited-slot-copy} a convenience macro.

-@node Garbage Collection -@comment node-name, next, previous, up -@section Garbage Collection - -GC is not implemented, but migration (@pxref{Repository Migration and -Upgrade}) will consolidate storage and recover OIDs which emulates GC. -No online solution is currently supported. - @node Performance Tuning @comment node-name, next, previous, up -@section Performance +@section Performance Tuning

Performance is usually measured in transactions per second. Database reads are cheap. To get more transactions throughput, consider setting

@lisp -* (db-env-set-flags (controller-environment *store-controller*) 1 - :txn-nosync t) +(db-env-set-flags (controller-environment *store-controller*) + 1 :txn-nosync t) @end lisp

-or look at other flags in the sleepycat docs. This will greatly +or look at other flags in the Berkeley DB docs. This will greatly increase your throughput at the cost of some durability; I get around a 100x improvement. Durability can be recovered with judicious use of checkpointing and replication, though this is currently not supported @@ -906,13 +954,29 @@ mode is not suitable for use in web servers or other typically multi-threaded applications.

+@node Garbage Collection +@comment node-name, next, previous, up +@section Garbage Collection + +GC is not implemented, but migration (@pxref{Repository Migration and +Upgrade}) will consolidate storage and recover OIDs which emulates GC. +No online solution is currently supported. + + @node Berkeley DB Data Store @comment node-name, next, previous, up @section Berkeley DB Data Store

- @node CL-SQL Data Store @comment node-name, next, previous, up @section CL-SQL Data Store

+ +@node Postmodern Data Store +@comment node-name, next, previous, up +@section Postmodern Data Store + +@node Native Lisp Data Store +@comment node-name, next, previous, up +@section Native Lisp Data Store