Update of /project/elephant/cvsroot/elephant/doc In directory clnet:/tmp/cvs-serv31032/doc
Modified Files: copying.texinfo data-store-reference.texinfo elephant-design.texinfo elephant.texinfo installation.texinfo make-ref.lisp reference.texinfo tutorial.texinfo user-guide.texinfo Log Message: Significant documentation string and documentation edits towards 0.6.1 manual. Clean up packages so elephant exports user visible symbols and backend exports backend-relevant symbols. Change required fix in serializer packages also. Added :elephant-user package.
--- /project/elephant/cvsroot/elephant/doc/copying.texinfo 2007/03/24 12:16:02 1.3 +++ /project/elephant/cvsroot/elephant/doc/copying.texinfo 2007/03/30 14:34:34 1.4 @@ -1,38 +1,39 @@ @c -*-texinfo-*-
-@node Copying +@node Copyright and License @comment node-name, next, previous, up -@chapter Copying -@cindex Copying +@chapter Copyright and License +@cindex Copyright and License @cindex License
-@quotation -@b{Elephant}: an object-oriented database for Common Lisp. +@section Elephant Licensing + +@b{Elephant}: a persistent metaprotocol and object-oriented database +for Common Lisp.
Homepage: @uref{http://www.common-lisp.net/project/elephant%7D
+@quotation Elephant users are granted the rights to distribute and use this software as governed by the terms of the Lisp Lesser GNU Public License @uref{http://opensource.franz.com/preamble.html%7D, also known as the LLGPL. +@end quotation
Copyrights include:
+@quotation Copyright (c) 2004 by Andrew Blumberg and Ben Lee
-Copyright (c) 2006-2007 by Ian Eslick - Copyright (c) 2005-2007 by Robert L. Read
+Copyright (c) 2006-2007 by Ian Eslick +@end quotation
-Portions of this program (namely the C unicode string -sorter) are derived from IBM's @b{ICU}: - -@uref{http://oss.software.ibm.com/icu/%7D - -whose copyright and license follows the GPL below. - - +Portions of this program (namely the C unicode string sorter) are +derived from IBM's @b{ICU}: @uref{http://oss.software.ibm.com/icu/, +ICU Website} whose copyright and license follows below.
+@quotation ICU License - ICU 1.8.1 and later COPYRIGHT AND PERMISSION NOTICE
@@ -72,5 +73,32 @@
All trademarks and registered trademarks mentioned herein are the property of their respective owners. - @end quotation + +@section Data Store Licensing Considerations + +The Berkeley DB data store is based on the Berkeley DB C library, now +owned by Oracle, but available as GPL'ed software. It is important to +understand that applications using Berkeley DB must also be GPL'ed +unless you negotiate a commercial license from Oracle. In most +interpretations of the license, this includes a requirement to make +code available for the entirety of any publicly visible website that +is based on Berkeley DB. See + +@uref{http://www.oracle.com/@/technology/@/software/@/products/@/berkeley-db/@/htd.... + +The CL-SQL backend, depending on which SQL engine you use, may not +carry this restriction and you can easily migrate data between the +two. Since the Berkeley DB store is 4-5x faster than SQL, it may make +sense to develop under BDB and transition to SQL after you've tuned +the performance of the application. Licenses for various SQL engines +can be found at: + +@itemize +@item SQLite: Public Domain, see @uref{http://www.sqlite.org/copyright.html, the SQLite license page} +@item Postgresql: BSD License, see @uref{http://www.postgresql.org/about/licence, the Postgresql license page} +@item MySQL: Dual licensing (similar to BDB), see @uref{http://www.mysql.com/company/legal/licensing/, the MySQL license page} +@end itemize + + + --- /project/elephant/cvsroot/elephant/doc/data-store-reference.texinfo 2007/03/24 13:55:15 1.1 +++ /project/elephant/cvsroot/elephant/doc/data-store-reference.texinfo 2007/03/30 14:34:34 1.2 @@ -7,24 +7,34 @@ @cindex Data Store @cindex API Reference
-These are the functions that need to be overridden to implement -support for a data store backend. Included are the exported elephant -functions that need methods defined on them. Some functions here are -utilities from the main elephant package that support store -implementations. Migration, class indices and query interfaces are +This reference includes functions that need to be overridden, classes +inherited from or other action taken to implement support for a new +data store backend. Included are the exported elephant functions that +need methods defined on them as well as the backend-only functions +exported in backends.lisp. Some functions here are utilities from the +main elephant package that support store implementations, but are not +required. Migration, class indices and query interfaces are implemented on top of the store API and require no special support by implementors.
+Because the number of backend implementors is small, this is a minimal +documentation set intended to serve as an initial guide and a +reference. However, it is anticipated that some interaction will be +needed with the developers to properly harden a datastore for release. + +The sections each contain a short guide and a list of functions +relevant to them. + @menu -* Registration:: Register the backend to parse controller specifications +* Registration:: Register the backend to parse controller specifications. * Store Controllers:: Subclassing the store controller. +* Handling Serialization:: Available facilities for serializing objects. +* C Utilities:: Writing primitive C types. * Slot access:: Support for metaprotocol slot access. * Collections:: BTrees and indices. * Cursors:: Traversing BTrees. * Transactions:: Transaction implementation. * Multithreading:: Multithreading considerations. -* Serialization:: Facilities for serializing objects. -* C Utilities:: Writing primitive C types. * Foreign libraries:: Using UFFI and ASDF to build or link foreign libraries @end menu
@@ -33,8 +43,17 @@ @section Registration @cindex Registration
-@include includes/fun-elephant-register-backend-con-init.texinfo -@include includes/fun-elephant-lookup-backend-con-init.texinfo +Elephant looks at the first element of the specification list to +determine which backend code base to use. The master table for this +information is @code{*elephant-backends*} in elephant/controller.lisp. +This will need to be augmented for every backend with the +specification keyword tag to be used (such as @code{:BDB} or +@code{:CLSQL}) and the required asdf dependencies. + +In addition, the backend source should use an eval-when statement to +call the following function: + +@include includes/fun-elephant-backend-register-backend-con-init.texinfo
@node Store Controllers @comment node-name, next, previous, up @@ -44,25 +63,22 @@ Subclass store-controller and implement store and close controller which are called by open-store and close-store respectively.
-@include includes/fun-elephant-store-controller.texinfo +@include includes/class-elephant-backend-store-controller.texinfo @include includes/fun-elephant-backend-open-controller.texinfo @include includes/fun-elephant-backend-close-controller.texinfo
-The slots for these accessors must be initialized. +
@include includes/fun-elephant-backend-database-version.texinfo -@include includes/fun-elephant-backend-controller-serialize.texinfo -@include includes/fun-elephant-backend-controller-deserialize.texinfo -@include includes/fun-elephant-backend-root.texinfo -@include includes/fun-elephant-backend-class-root.texinfo
These functions are important utilities for implementing store-controllers.
-@include includes/fun-elephant-backend-oid.texinfo @include includes/fun-elephant-backend-get-con.texinfo +@include includes/fun-elephant-backend-oid.texinfo @include includes/fun-elephant-backend-next-oid.texinfo @include includes/fun-elephant-backend-connection-is-indeed-open.texinfo +@include includes/fun-elephant-get-user-configuration-parameter.texinfo
@node Slot Access @comment node-name, next, previous, up @@ -82,17 +98,37 @@ @section Collections @cindex Collections
-@c #:btree #:btree-index #:indexed-btree -@c #:build-indexed-btree #:build-btree #:existsp -@c #:map-indices +To support collections, the data store must subclass the following +classes.
+@include includes/class-elephant-btree.texinfo.texinfo +@include includes/class-elephant-btree-index.texinfo +@include includes/class-elephant-indexed-btree.texinfo + +To create the backend-appropriate type of btree, the backend +implements these methods aginst their store-controller. + +@include includes/fun-elephant-build-btree.texinfo +@include includes/fun-elephant-build-indexed-btree.texinfo + +And every btree needs accessors, these must be implemented for btree, +indexed-btree and btree-index. + +@include includes/fun-elephant-get-value.texinfo +@include includes/fun-elephant-setf-get-value.texinfo +@include includes/fun-elephant-existsp.texinfo +@include includes/fun-elephant-remove-kv.texinfo + +@include includes/fun-elephant-map-indices.texinfo +@include includes/fun-elephant-get-index.texinfo +@include includes/fun-elephant-remove-index.texinfo
@node Cursors @comment node-name, next, previous, up @section Cursors @cindex Cursors
-@c #:cursor +@include includes/class-cursor.texinfo @c #:cursor-btree @c #:cursor-oid @c #:cursor-initialized-p @@ -106,19 +142,23 @@ @c #:make-transaction-record @c #:transaction-store @c #:transaction-object + @c #:execute-transaction @c #:controller-start-transaction @c #:controller-commit-transaction @c #:controller-abort-transaction
-@node Multithreading +@node Multithreading Considerations @comment node-name, next, previous, up -@section Multithreading +@section Multithreading Considerations @cindex Multithreading
-@node Serialization +@c utils locks +@c utils thread-vars + +@node Handling Serialization @comment node-name, next, previous, up -@section Serialization +@section Handling Serialization @cindex Serialization
@c #:deserialize #:serialize --- /project/elephant/cvsroot/elephant/doc/elephant-design.texinfo 2007/03/24 13:55:15 1.1 +++ /project/elephant/cvsroot/elephant/doc/elephant-design.texinfo 2007/03/30 14:34:34 1.2 @@ -1,5 +1,16 @@ -Debugger entered--Lisp error: (void-variable Design) - eval(Design) - eval-last-sexp-1(nil) - eval-last-sexp(nil) - call-interactively(eval-last-sexp) + +@node Elephant Design +@comment node-name, next, previous, up +@section Elephant Design +@cindex design + +When the main elephant @code{open-store} function is called, it calls +@code{get-controller} which grabs an existing store controller if the +spec is identical, or builds a new controller. Building the +controller requires loading any dependencies via asdf, calling a +backend initialization function (if it is the first instance of that +backend being created), and then calling an initialization function +that returns a @code{store-controller} subclass instance specific to +that backend. + +Elephant than calls open-controller --- /project/elephant/cvsroot/elephant/doc/elephant.texinfo 2007/03/24 12:16:02 1.5 +++ /project/elephant/cvsroot/elephant/doc/elephant.texinfo 2007/03/30 14:34:34 1.6 @@ -5,8 +5,9 @@ @c %**end of header
@copying -Copyright @copyright{} 2004 Ben Lee and Andrew Blumberg. -Copyright @copyright{} 2006-2007 Robert L. Read and Ian Eslick +Original Version, Copyright @copyright{} 2004 Ben Lee and Andrew Blumberg. +2006 SQL Backend, Copyright @copyright{} 2006 Robert L. Read. +2007 Rewrite, Copyright @copyright{} 2007 Ian Eslick.
@quotation Permission is granted to copy, distribute and/or modify this document @@ -52,7 +53,7 @@ * User API Reference:: Function and class documentation of the user API. * Elephant Design:: An overview of elephant's internal architecture. * Data Store API Reference:: Function level documentation for data store implementors. -* Copying:: Your rights and freedoms. +* Copyright and License:: Your rights and freedoms. @end menu
@chapheading Appendices @@ -66,6 +67,7 @@ @end menu
@node Table of Contents +@unnumbered @comment node-name, next, previous, up @contents
--- /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/03/24 12:16:02 1.4 +++ /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/03/30 14:34:34 1.5 @@ -6,157 +6,232 @@ @cindex Installation
@menu -* Installation Basics:: Basic installation -* Test-Suites:: Running the test suites -* Berkeley DB Introduction:: The Berkeley DB backend -* SQL Data Store:: The design and status of the SQL back-end extension. -* Lisp Data Store:: A native lisp-based repository. -* Multi-repository Operation:: Specifying repositories. -* Setting up PostGres:: An example. +* Requirements:: Supported lisps and required libraries. +* Configuring Elephant:: Setting up Elephant and the configuration file. +* Loading Elephant:: Loading Elephant and the data store loading protocol. +* Berkeley DB Data Store:: Installing support for the Berkeley DB data store +* Berkeley DB Examples:: An example of installing and running the Berkeley DB data store. +* CL-SQL Data Store:: Install and connecting to the CL-SQL data store +* CL-SQL Examples:: An example of using the CL-SQL data store. +* Elephant on Windows:: More details about running Elephant on Windows +* Test Suites:: How to run and interpret the output of the regression test suite +* Documentation:: Building documentation from texinfo sources. @end menu
-@node Installation Basics +@node Requirements @comment node-name, next, previous, up -@section Installation +@section Requirements
-Please see the file ``INSTALL'' in the source distribution for -more precise information; this is an overview. +Elephant is a multi-platform, multi-lisp and multi-backend system. As +such there is a great deal of complexity in testing. The system has +tried to minimize external dependencies as much as possible to ease +installation, but it still requires some patience and care to bring +Elephant up on any given platform. This section attempts to simplify +this for new users as much as possible. Patches and suggestions will +be gladly accepted. + +@subsection Supported Lisp, Platform and Data store combinations + +Elephant supports SBCL, Allegro, Lispworks, OpenMCL and CMUCL. Each +lisp is supported on each of the platforms it runs on: Mac OS X, Linux +and Windows. As of release 0.6.1, both 32-bit and 64-bit systems +should be supported. Elephant has a small developer base and as of +the writing of this manual, there are:
-Installation of Elephant itself is easy because of the asdf system. -Just execute: -@lisp -(asdf:operate 'asdf:load-op :elephant) -@end lisp +@enumerate +@item Five lisp environments +@item Three Operating System platforms +@item 32-bit or 64-bit OS/compilation configuration +@item Three data store configurations: Berkeley DB, SQLite3 and Postgresql +@end enumerate
-However, Elephant cannot function without a back-end repository. -Elephant presents exactly the same API no matter what you choose -as a repository. In most cases Elephant will automatically load -the backend you refer to with your controller spec when you call -open store. However, you may have to use asdf to load the -code that interfaces to particular repository system. +This means that the total number of combinations that should be tested +comes to:
-The basic choices are to use the BerkeleyDB system or -a SQL based system. You must perform one of these: -@lisp -(asdf:operate 'asdf:load-op :ele-clsql) -(asdf:operate 'asdf:load-op :ele-bdb) -@end lisp +@math{lisps * os * radix * dstore = 5 * 3 * 2 * 3 = 90 configurations}
-If you choose a SQL based system, you may have to -load a specific package for that system, such as: +Of course not all of these combinations are valid, but the +implications of these combinatorics is that not every combination will +be tested in any given release. The developers and active user base +currently cover all three data store configurations on the following +platforms: + +@itemize +@item 32/64-bit SBCL on Linux and Mac OS X +@item 32-bit Lispworks on Windows and Mac OS X +@item 32-bit Allegro on Mac OS X +@end itemize + +The developers will do their best to accomodate users who are keen to +test other combinations, but practically these configurations will be +the most stable and reliable. Elephant is becoming quite stable in +general, so don't be afriad to try an unemphasized combination - +chances are it is just a little more work to bring it up.
-@lisp -(asdf:operate 'asdf:load-op :ele-sqlite3) -@end lisp -or, for Postgres, -@lisp -(asdf:oos 'asdf:load-op :clsql-postgresql-socket) -@end lisp +@subsection Library dependencies
-You will have to have the CL-SQL package installed. Following the -documentation for CL-SQL under the section ``How CLSQL finds and loads foreign -libraries'' you may need to do something like: -@lisp -(clsql:push-library-path "/usr/lib/") -@end lisp +The Elephant core system requires:
-before doing -@lisp -(asdf:oos 'asdf:load-op :clsql-postgresql-socket) -@end lisp +@enumerate +@item asdf -- @uref{http://www.cliki.net/asdf%7D +@item uffi -- version 1.5.17 or later, @uref{http://uffi.b9.com/%7D or @uref{http://www.cliki.net/UFFI%7D +@item cl-base64 -- @uref{http://www.cliki.net/cl-base64%7D +@item gcc -- Your system needs GCC (or Cygwin) to build the Elephant C-based serializer library. (Precompiled DLL's are available for Windows platforms on the @uref{http://www.common-lisp.net/project/elephant/downloads.html, download page}. +@item rt -- The RT regression test sytem is required to run the test suite: @uref{http://www.cliki.net/RT%7D +@end enumerate + +Follow the instructions at these URLs to download and setup the +libraries. (Note: uffi and cl-base64 are +@uref{http://www.cliki.net/ASDF-Install, asdf-installable} for those +of you with asdf-install on your system) + +In addition to these libraries, each data store has their own +dependencies as discussed in @ref{Berkeley DB Data Store} and +@ref{CL-SQL Data Store}. + +@node Configuring Elephant +@comment node-name, next, previous, up +@section Configuring Elephant
-in order for clsql to find the PostGres library libpq.so, for example. +Before you can load the elephant packages into your running lisp, you +need to setup the configuration file. First, copy the reference file +config.sexp from the root directory to my-config.sexp. my-config.sexp +contains a lisp reader-formatted list of key-value pairs that tells +elephant where to find various libraries, how to build libraries, etc. + +For example:
-Without modifcation, Elephant uses this as it's lib path: @lisp -/usr/local/share/common-lisp/elephant-0.3/ -@end lisp +#+(and (or sbcl allegro) macosx) +((:berkeley-db-include-dir . "/opt/local/include/db45/") + (:berkeley-db-lib-dir . "/opt/local/lib/db45/") + (:berkeley-db-lib . "/opt/local/lib/db45/libdb-4.5.dylib") + (:berkeley-db-deadlock . "/opt/local/bin/db45_deadlock") + (:pthread-lib . nil) + (:clsql-lib . nil) + (:compiler . :gcc)) +@end lisp + +The following is a guide to the various parameters. For simplicity, +we include all the parameters here, although we will go into more +detail in each of the data store sections. + +@itemize +@item @strong{:compiler} -- This tells Elephant which compiler to use to build any C libraries. The only options currently are :gcc on Unix platforms and :cygwin for the Windows platform. +@item @strong{:berkeley-db-include-dir} -- The pathname for the Berkeley DB include files (db.h) +@item @strong{:berkeley-db-lib-dir} -- The pathname for all the Berkeley DB library files +@item @strong{:berkeley-db-lib} -- The full pathname for the specific Berkeley DB library (libdb45.so) +@item @strong{:berkeley-db-deadlock} -- The full pathname to the BDB utility function db_deadlock +@item @strong{:pthread-lib} -- Not needed for SBCL 9.17+ +@item @strong{:clsql-lib} -- Currently unused, adds paths to the CL-SQL library search function +@end itemize + +The config.sexp file contains a set of example configurations to start +from, but you will most likely need to modify it for your system. + +Elephant has one small C library that it uses for binary serialization +which means that you need to have gcc in your path (@pxref{Elephant on +Windows} for exceptions on the Windows platform). + +@node Loading Elephant +@comment node-name, next, previous, up +@section Loading Elephant
-So you could put a symbolic link to libpq.so there, where libmemutil.so and -libsleepycat.so will also reside. +@subsection Loading Elephant via ASDF
-Elephant is designed to allow multi-repository operation; -so you could concievably use two or more repositories at the -same time. More particularly, you can seamlessly migrate your -data from one repository to a different one at a later date. -In a long duration project, this might occur because of a licensing -or performance issue with a particular respository. Migrating to -a new repository of the same type is a cheap form of GC although -migration is limited to the total size of main memory to store -a hash table that tracks all copied object ID's. - -@node Test-Suites -@comment node-name, next, previous, up -@section Test-Suites - -Elephant is moderately mature. Hopefully, it will work out-of-the-box -for you. - -However, if you are using an LISP implementation different than the ones -on which it is developed and maintained (currently OpenMCL, SBCL, and ACL), -or as the repositories evolve, or just because of mistakes, you may need -to run the test suites. If you report a bug, we will ask you -to run these tests and report the output. Running them when you -first install things may give you a sense of confidence and understanding -that makes it worth the trouble. +Now that you have loaded all the dependencies and created your +configuration file you can load the Elephant packages and +definitions:
-There are three files that execute the tests. You should choose -one as a starting point based on what backend(s) you are using. -If using BerekleyDB, use @lisp -BerkeleyDB-tests.lisp +(asdf:operate 'asdf:load-op :elephant) @end lisp
-If using both, use both of the above and also use: +This will load the cl-base64 and uffi libraries. It will also +automatically compile and load the C library. The build process no +longer depends on a Makefile. This build process has been verified on +most platforms, but if you have a problem please report it, and any +output you can capture, to the developers at +@email{elephant-devel@@common-lisp.net}. We will update the FAQ at +@uref{http://trac.common-lisp.net/elephant%7D with common problems users +run into. + +@subsection Two-Phase Load Process + +Elephant uses a two-phase load process. The core code is loaded and +the code for a given data store is loaded on demand when you call +@code{open-store} with a specification referencing that data store. +The second phase of the load process requires ASDF to be installed on +your system. + +(NOTE: There are some good reasons and not so good reasons for this +process. One reason you cannot load ele-bdb.asd directly as it +depends on lisp code defined in elephant.asd. We decided not to fix +this in this release although later releases may avoid the oddity of +the two phase loading) + +@subsection Packages + +Now that Elephant has been loaded, you can call @code{use-package} in +the cl-user package or create a new package that imports the symbols +exported from package :elephant. + @lisp -MigrationTests.lisp +CL-USER> (use-package :elephant) +=> T + +OR + +(defpackage :elephant-user + (:use :common-lisp :elephant)) @end lisp
-The text of this file is included here to give the -casual reader an idea of how elepant test can be run in general: -@lisp -;; This file is an example of how to perform the -;; migration tests. You will have to modify it -;; slightly depending on the systems that want to test... -;; You can test migration even between two BDB respositories if you wish -(asdf:operate 'asdf:load-op :elephant) -(asdf:operate 'asdf:load-op :ele-clsql) -(asdf:operate 'asdf:load-op :clsql-postgresql-socket) -(asdf:operate 'asdf:load-op :ele-bdb) -(asdf:operate 'asdf:load-op :elephant-tests) -;; For sqlite-3.. -;; (asdf:operate 'asdf:load-op :ele-sqlite3) +Beginners can skip to the end of this section.
-(in-package "ELEPHANT-TESTS") +Elephant has a common package called elephant that exports a set of +generic functions. It also contains a dispatcher based on the first +element of a specification list that calls the relevant backend +version of @code{open-controller}, the internal method that creates a +@code{store-controller}. Each backend has it's own subclass +implementing the abstract interface of @code{store-controller}.
-;; The primary and secondary test-paths are -;; use for the migration tests. +@subsection Opening a Store
-;; This this configuration for testing between BDB and SQL.... -(setq *test-path-primary* *testpg-path*) -;; (setq *test-path-primary* *testsqlite3-path*) -(setq *test-path-secondary* *testdb-path*) +As discussed in the tutoral, you can now open a store to begin using +Elephant:
-;; This this configuration for testing from one BDB repository to another... -(setq *test-path-primary* *testdb-path*) -;; (setq *test-path-primary* *testsqlite3-path*) -(setq *test-path-secondary* *testdb-path2*) +@lisp +(open-store '(:BDB "/Users/owner/db/my-bdb/")) +... +ASDF loading messages +... +=> #<BDB-STORE-CONTROLLER>
-(do-migrate-test-spec *test-path-primary*) +(open-store '(:CLSQL (:POSTGRESQL "localhost.localdomain" "mydb" "myuser" "")))) +... +ASDF loading messages +... +=> #<SQL-STORE-CONTROLLER> @end lisp
-The appropriate test should execute for you with no errors. -If you get errors, you may wish to report it the -@code{ elephant-devel at common-lisp.net} email list. +The first time you load a specific data store, Elephant will call ASDF +to load all the specified data store's dependencies, connect to a +database and return the @code{store-controller} subclass instance for +that backend.
-@node Berkeley DB Repository +@node Berkeley DB Data Store @comment node-name, next, previous, up -@section Berkeley DB Repository +@section Berkeley DB Data Store +
-@node SQL Repository +@node Berkeley DB Example +@comment node-name, next, previous, up +@section Setting up Berkeley DB + +@node CL-SQL Data Store @comment node-name, next, previous, up -@section SQL Repository +@section CL-SQL Data Store
Although originally designed as an interface to the BerkeleyDB system, the original Elephant system has been experimenetally extended to @@ -227,44 +302,14 @@ the multi-repository version somewhat complicates the underlying persistent object management.
-@node Multi-repository Operation +@node PostGres Examples @comment node-name, next, previous, up -@section Multi-repository Operation - -Elephant now keeps a small hashtables that maps ``database specifications'' into -actual database connections. - -If a database spec is a string, it is assumed to be a BerkeleyDB path. -If it is a list, it is a assumed to be a CL-SQL connection specification. -For example: -@lisp -ELE-TESTS> *testdb-path* -"/home/read/projects/elephant/elephant/tests/testdb/" -ELE-TESTS> *testpg-path* -(:postgresql "localhost.localdomain" "test" "postgres" "") -ELE-TESTS> -@end lisp - -The tests now have a function @code{do-all-tests-spec} that take a spec and -based on its type attempt to open the correct kind of store controller and -perform the tests. - -The routine @code{get-controller} takes this specifiation. - -The basic strategy is that the ``database specification'' object is stored in -every persistent object and collection so that the repository can be found. - -In this way, objects that reside in different repositories can coexist within -the LISP object space, allowing data migration. - - - +@section Setting up PostGres
@node Setting up PostGres @comment node-name, next, previous, up @section Setting up PostGres
- To set up a PostGres based back end, you should:
@enumerate @@ -294,7 +339,6 @@ Before you attempt to connect with Elephant.
[114 lines skipped] --- /project/elephant/cvsroot/elephant/doc/make-ref.lisp 2007/03/24 13:55:15 1.4 +++ /project/elephant/cvsroot/elephant/doc/make-ref.lisp 2007/03/30 14:34:34 1.5 @@ -16,11 +16,14 @@ (load docstrings-path)
(defun make-docs () - (when t - (elephant:open-store elephant-tests::*testbdb-spec*) - (make-instance 'elephant::persistent-collection) - (make-instance 'elephant::secondary-cursor) - (make-instance 'elephant::indexed-btree) - (sb-texinfo:generate-includes #p"/Users/eslick/Work/fsrc/elephant-cvs/doc/includes/" (find-package :elephant) (find-package :elephant-backend) (find-package 'elephant-memutil) (find-package 'elephant-system)))) + (elephant:open-store elephant-tests::*testbdb-spec*) + (make-instance 'elephant::persistent-collection) + (make-instance 'elephant::secondary-cursor) + (make-instance 'elephant::indexed-btree) + (sb-texinfo:generate-includes #p"/Users/eslick/Work/fsrc/elephant-cvs/doc/includes/" + (find-package :elephant) + (find-package :elephant-backend) + (find-package :elephant-memutil) + (find-package :elephant-system)))
(make-docs) --- /project/elephant/cvsroot/elephant/doc/reference.texinfo 2007/03/24 12:16:02 1.6 +++ /project/elephant/cvsroot/elephant/doc/reference.texinfo 2007/03/30 14:34:34 1.7 @@ -14,7 +14,6 @@ * Collections:: BTrees and indices. * Cursors:: Traversing BTrees. * Transactions:: Transactions. -* Multithreading:: Multithreading. * Migration and Upgrading:: Migration and upgrading. @end menu
@@ -56,23 +55,48 @@ @section Persistent Object Indexing @cindex Persistent Object Indexing
-@include includes/fun-get-instances-by-class.texinfo -@include includes/fun-get-instance-by-value.texinfo -@include includes/fun-get-instances-by-value.texinfo -@include includes/fun-get-instances-by-range.texinfo +@subsection Indexed Object Accessors + +@include includes/fun-elephant-map-class.texinfo +@include includes/fun-elephant-map-class-index.texinfo + +@include includes/fun-elephant-get-instances-by-class.texinfo +@include includes/fun-elephant-get-instance-by-value.texinfo +@include includes/fun-elephant-get-instances-by-value.texinfo +@include includes/fun-elephant-get-instances-by-range.texinfo + +@include includes/fun-elephant-drop-instances.texinfo + +@subsection Direct Class Index Manipulation + +@include includes/fun-elephant-find-class-index.texinfo +@include includes/fun-elephant-find-inverted-index.texinfo +@include includes/fun-elephant-make-class-cursor.texinfo +@include includes/macro-elephant-with-class-cursor.texinfo +@include includes/fun-elephant-make-inverted-cursor.texinfo +@include includes/macro-elephant-with-inverted-cursor.texinfo + +@subsection Dynamic Indexing API
@include includes/fun-elephant-enable-class-indexing.texinfo @include includes/fun-elephant-disable-class-indexing.texinfo -@include includes/fun-add-class-slot-index.texinfo -@include includes/fun-remove-class-slot-index.texinfo -@include includes/fun-add-class-derived-index.texinfo -@include includes/fun-remove-class-derived-index.texinfo +@include includes/fun-elephant-add-class-slot-index.texinfo +@include includes/fun-elephant-remove-class-slot-index.texinfo +@include includes/fun-elephant-add-class-derived-index.texinfo +@include includes/fun-elephant-remove-class-derived-index.texinfo
@node Query Interfaces @comment node-name, next, previous, up @section Query Interfaces @cindex Query Interfaces
+Query interfaces are currently unimplemented. An example query +interface is provided for reference only, a new system is under +development for the 0.7 release. + +@include includes/fun-elephant-get-query-results.texinfo +@include includes/fun-elephant-map-class-query.texinfo + @node Collections @comment node-name, next, previous, up @section Collections @@ -145,12 +169,9 @@ @include includes/fun-elephant-commit-transaction.texinfo @include includes/fun-elephant-abort-transaction.texinfo
-@node Multithreading -@comment node-name, next, previous, up -@section Multithreading -@cindex Multithreading - @node Migration and Upgrading @comment node-name, next, previous, up @section Migration and Upgrading @cindex Migration and Upgrading + +@include includes/fun-elephant-migrate.texinfo --- /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2007/03/26 03:37:27 1.9 +++ /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2007/03/30 14:34:34 1.10 @@ -914,8 +914,8 @@ complicated. The best strategy at the beginning is a conservative one, break things up into the smallest logical sets of primitive operations and only wrap higher level functions in transactions when -they absolutely have to commit together. @xref{Transaction details} -for all the gory detail of transactions and @pxref{Usage scenarios} -for more examples of how systems can be designed using transactions. +they absolutely have to commit together. See @ref{Transaction details} +for the full details and @pxref{Usage scenarios} for more examples of +how systems can be designed and tuned using transactions.
--- /project/elephant/cvsroot/elephant/doc/user-guide.texinfo 2007/03/26 03:37:27 1.3 +++ /project/elephant/cvsroot/elephant/doc/user-guide.texinfo 2007/03/30 14:34:34 1.4 @@ -16,6 +16,7 @@ * Secondary Indices:: Alternative ways to index collections. * Using Cursors:: Low-level access to BTrees. * Transaction details:: Develop a deeper understanding of transactions and avoid the pitfalls. +* Multi-repository Operation:: Specifying repositories. * Repository Migration and Upgrade:: How to move objects from one repository to another. * Garbage collection:: How to recover storage and OIDs in long-lived repositories. * Performance tuning:: How to get the most from Elephant. @@ -82,10 +83,9 @@ @code{with-open-controller} macro. Opening and closing a controller is very expensive.
- -@node{Class indices} +@node Class Indices @comment node-name, next, previous, up -@section Class indicies +@section Class Indicies
You can enable/disable class indexing for an entire class. When you disable indexing all references to instances of that class are lost. If you re-enable @@ -130,7 +130,7 @@ somewhat user customizable; documentation for this exists in the source file referenced above.
-@node{Using BTrees} +@node Using BTrees @comment node-name, next, previous, up @section Using BTrees
@@ -225,7 +225,7 @@ to the target repository which you can then overwrite. To avoid the default persistent slot copying, bind the dynamic variable @code{*inhibit-slot-writes*} in your user method using -@code(with-inhibited-slot-copy () ...)} a convenience macro. +@code{with-inhibited-slot-copy} a convenience macro.
@node Threading @@ -318,3 +318,33 @@ ensure that multiple threads do not interleave access so single user mode is not suitable for use in web servers or other typically multi-threaded applications. + +@node Multi-repository Operation +@comment node-name, next, previous, up +@section Multi-repository Operation + +Elephant now keeps a small hashtables that maps ``database specifications'' into +actual database connections. + +If a database spec is a string, it is assumed to be a BerkeleyDB path. +If it is a list, it is a assumed to be a CL-SQL connection specification. +For example: +@lisp +ELE-TESTS> *testdb-path* +"/home/read/projects/elephant/elephant/tests/testdb/" +ELE-TESTS> *testpg-path* +(:postgresql "localhost.localdomain" "test" "postgres" "") +ELE-TESTS> +@end lisp + +The tests now have a function @code{do-all-tests-spec} that take a spec and +based on its type attempt to open the correct kind of store controller and +perform the tests. + +The routine @code{get-controller} takes this specifiation. + +The basic strategy is that the ``database specification'' object is stored in +every persistent object and collection so that the repository can be found. + +In this way, objects that reside in different repositories can coexist within +the LISP object space, allowing data migration.