Update of /project/elephant/cvsroot/elephant/doc In directory clnet:/tmp/cvs-serv13016

Modified Files: data-store-reference.texinfo installation.texinfo intro.texinfo make-ref.lisp tutorial.texinfo user-guide.texinfo Log Message: Sprucing up Chapter 3.

--- /project/elephant/cvsroot/elephant/doc/data-store-reference.texinfo 2007/04/01 14:33:24 1.5 +++ /project/elephant/cvsroot/elephant/doc/data-store-reference.texinfo 2007/04/02 00:51:06 1.6 @@ -138,10 +138,8 @@ @include includes/class-elephant-indexed-btree.texinfo

To create the backend-appropriate type of btree, the backend -implements these methods aginst their store-controller. - -@include includes/fun-elephant-build-btree.texinfo -@include includes/fun-elephant-build-indexed-btree.texinfo +implements this method (and possibly related methods) aginst their store-controller. +@include includes/fun-elephant-backend-build-btree.texinfo

Most of the user-visible operations over BTrees must be implemented. Class indexing functions such as @code{map-class} and --- /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/04/01 14:33:29 1.6 +++ /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/04/02 00:51:06 1.7 @@ -66,7 +66,9 @@ test other combinations, but practically these configurations will be the most stable and reliable. Elephant is becoming quite stable in general, so don't be afriad to try an unemphasized combination - -chances are it is just a little more work to bring it up. +chances are it is just a little more work to bring it up. In particular, +Elephant can probably work with MySQL or Oracle with just a little work, +but nobody has asked for this yet.

@subsection Library dependencies

@@ -223,12 +225,53 @@ @comment node-name, next, previous, up @section Berkeley DB Data Store

+The Berkeley DB Data Store started out as a very simple data dictionary in the +Berkeley Unix operating system. There are many ``Xdb'' systems that use the +same API, or a similarly one. A commercial version of the BDB was provided by +Sleepycat Software, and this product has since been purchased by Oracle corporation, +and is currently distributed under a similar licensing arrangement, with commercial +support also available. Please follow the download and installation procedures +defined here:

+http://www.oracle.com/technology/products/berkeley-db/db/index.html + +Elephant only works with version 4.5 of BerkeleyDB.

@node Berkeley DB Example @comment node-name, next, previous, up @section Setting up Berkeley DB

+Beyond ensuring that the file ``my-config.sexp'' points correctly +to your BDB installation, nothing else should be required to configure +the example that uses a local ``testdb'' directory as a dabase (under ``tests'') +in the top-level Elephant directory. + +On one Fedora based system, the ``my-config.sexp'' file looked like this: + +@lisp +((:berkeley-db-include-dir . "/usr/local/BerkeleyDB.4.5/include") + (:berkeley-db-lib-dir . "/usr/local/BerkeleyDB.4.5/lib") + (:berkeley-db-lib . "/usr/local/BerkeleyDB.4.5/lib/libdb.so") + (:berkeley-db-deadlock . "/usr/local/BerkeleyDB.4.5/bin/db_deadlock") + (:pthread-lib . nil) + (:clsql-lib . "/usr/local/share/common-lisp/") + (:compiler . :gcc)) +@end lisp + +The @ref{Test Suites} give a nice example of using BDB by running the test using +the specification: +@lisp +'(:BDB "/home/me/db/testdb/")) +@end lisp + +Once you start working on an application, you will want to change this path +to a directory that make sense for you application, and use that as +the specification passed to +@lisp open-store +@end lisp +in your application. + + @node CL-SQL Data Store @comment node-name, next, previous, up @section CL-SQL Data Store @@ -237,7 +280,7 @@ the original Elephant system has been experimenetally extended to support the use of relational database management systems as the implementation of the persistent store. This relies on Kevin Rosenberg's -CL-SQL interface to relational systems. +CL-SQL interface to a large number of relational systems.

Although the BerkeleyDB system is an ideal object store for LISP objects, one might prefer the licensing of a different system. For example, at @@ -246,13 +289,10 @@ http://www.sleepycat.com/download/licensinginfo.shtml#redistribute unless one releases the entire web application as open source.

-Neither the PostGres DBMS nor SQLite 3 has any such restriction. Elephant itself is released -under the GPL. It is somewhat debatable if the GPL allows one to construct -to construct a non-open-source web application but the preponderance of -opinion appears to be that it does. Thefore using Elephant and the other -GPLed software that it depends upon allows one to host a a non open-source -web application. This might be a reason to use Elephant on PostGres of SQLite rather -than Elephant on BerkeleyDB. +Neither the PostGres DBMS nor SQLite 3, nor Elephant itself, imposes +any such restriction. (Elephant is released under the LLGPL +(http://opensource.franz.com/preamble.html .) Older versions were +released under the GPL.)

Other reasons to use a relational database system might include: familiarity with those systems, the fact that some part of your application @@ -260,27 +300,22 @@ the tools associated with those systems, etc.

The SQL back-end extention of Elephant provides a function for migrating -data seamlessly between repositories. That is, one can quite easily move +data seamlessly between repositories. One can quite easily move data from a BerkeleyDB repository to a PostGres repository, and vice versa. -In fact, one of the most important aspects of the extention is that it -makes Elephant a multi-repository system, rather than a single repository -system, as addition to allowing different implementation strategies for -those repositories. This offers at least the possiblity than once -can develop using one backend, for example BerkeleyDB, and then later -move to MySQL. +This offers at least the possiblity than once can develop using one backend, +for example BerkeleyDB, and then later move to Postgres, or vice versa. +One could even operate simultaneously out of multiple repositories, if +there were a good reason to do so.

At the time of this writing, the basic strategy for the SQL implementation is quite simple. The same serializer used for the Sleepycat implementation is employed, the byte-string is base64 encoded, and placed in a single table which is managed by Elephant.

-As of Elephant 0.3, Elephant has been tested to work with both Postgres, and -SQLite 3, thanks do Dan Knapp. - -As far as is known at this writing, all functionality except nested transaction -support and cursor-puts supported by the BerkeleyDB backend is supported by the -CL-SQL back-end. Concurrency and transaction atomicity have not been stress tested -well for the CL-SQL based system. +All functionality except nested transaction support and cursor-puts +supported by the BerkeleyDB backend is supported by the CL-SQL +back-end. Concurrency and transaction atomicity have not been stress +tested well for the CL-SQL based system.

Additionally, it is NOT the case that the Elephant system currently provides transaction support across multiple repositories; it provides the transaction @@ -290,13 +325,7 @@ The PostGres backend is as currently employed is about 5 times slower than the BerkeleyDB backend. This could probably change with continued development.

-CL-SQL supports a lot of DBMS systems, but only PostGres has been tested. - -The SQL back-end extention has only been tested under SBCL 0.8.18. - -The SQL back-end is as easy to use as the BerkeleyDB back-end. However, -the multi-repository version somewhat complicates the underlying -persistent object management. +CL-SQL supports a lot of DBMS systems, but only PostGres and SqlLite 3 have been tested.

@node CL-SQL Example @comment node-name, next, previous, up @@ -306,6 +335,9 @@

@enumerate @item Install postgres and make sure postmaster is running. +Postgres may be installed on your system; you may be able to use a package manager to +install it, or you can install it quite easily from the PostgresSQL site directly +(http://www.postgresql.org/).

@item Create a database called ``test'' and set its permissions to be reached by whatever connection specification you intend to use. The @@ -377,34 +409,39 @@ The text of this file is included here to give the casual reader an idea of how elepant test can be run in general: @lisp -;; This file is an example of how to perform the -;; migration tests. You will have to modify it -;; slightly depending on the systems that want to test... -;; You can test migration even between two BDB respositories if you wish +;; If you are only using one back-end, you may prefer: +;; SQLDB-test.lisp or BerkeleyDB-tests.lisp (asdf:operate 'asdf:load-op :elephant) (asdf:operate 'asdf:load-op :ele-clsql) -(asdf:operate 'asdf:load-op :clsql-postgresql-socket) (asdf:operate 'asdf:load-op :ele-bdb) +(asdf:operate 'asdf:load-op :ele-sqlite3) + (asdf:operate 'asdf:load-op :elephant-tests) -;; For sqlite-3.. -;; (asdf:operate 'asdf:load-op :ele-sqlite3)

(in-package "ELEPHANT-TESTS")

-;; The primary and secondary test-paths are -;; use for the migration tests. - -;; This this configuration for testing between BDB and SQL.... -(setq *test-path-primary* *testpg-path*) -;; (setq *test-path-primary* *testsqlite3-path*) -(setq *test-path-secondary* *testdb-path*) - -;; This this configuration for testing from one BDB repository to another... -(setq *test-path-primary* *testdb-path*) -;; (setq *test-path-primary* *testsqlite3-path*) -(setq *test-path-secondary* *testdb-path2*) +;; Test Postgres backend +(setq *default-spec* *testpg-spec*) +(do-backend-tests) + +;; Test BDB backend +(setq *default-spec* *testbdb-spec*) +(do-backend-tests) + +;; Test SQLite 3 +(setq *default-spec* *testsqlite3-spec*) +(do-backend-tests) + +;; Test a Migration of data from BDB to postgres +(do-migration-tests *testbdb-spec* *testpg-spec*) + +;; An example usage. +(open-store *testpg-spec*) +(add-to-root "x1" "y1") +(get-from-root "x1")

-(do-migrate-test-spec *test-path-primary*) +(add-to-root "x2" '(a 4 "spud")) +(get-from-root "x2") @end lisp

The appropriate test should execute for you with no errors. @@ -440,5 +477,29 @@ @comment node-name, next, previous, up @section Documentation

+If you are getting the documentation as a released tar file, you will probably find +the documenation in .html or .pdf form in the release, or can find it at the +Elephant website. + +If you want to compile the documentation youself, for example, if you can +think of a way to improve this manual, then you will do something similar +to this in a shell or command-line prompt: +@code +cd doc +make +make pdf +@end code + +This process will populate the ``./includes'' directory with references +automatically extracted from the list code. It will then compile the +texinfo documenation source into both HTML and a PDF which will be left +in the ``doc/elephant'' directory. + +Don't edit anything in the ``doc/elephant'' directory or the +``doc/includes'' directories, as everything in these directories is +generated. Instead, edit the ``.texinfo'' files in the doc directory. + + +

--- /project/elephant/cvsroot/elephant/doc/intro.texinfo 2007/03/24 12:16:02 1.5 +++ /project/elephant/cvsroot/elephant/doc/intro.texinfo 2007/04/02 00:51:06 1.6 @@ -28,7 +28,8 @@

Elephant has been extended to provide support for multiple backends, specifically a relational database backend based on CL-SQL which has -been tested with Postgres and SQLite 3. It supports, with some care, +been tested with Postgres and SQLite 3, and probably support +other relational systems easily. It supports, with some care, multi-repository operation and enables convenient migration of data between repositories.

--- /project/elephant/cvsroot/elephant/doc/make-ref.lisp 2007/03/30 23:36:52 1.6 +++ /project/elephant/cvsroot/elephant/doc/make-ref.lisp 2007/04/02 00:51:06 1.7 @@ -30,7 +30,8 @@ (make-instance 'elephant::persistent-collection :sc sc :from-oid 10) (make-instance 'elephant::secondary-cursor) (make-instance 'elephant::indexed-btree :sc sc :from-oid 10) - (sb-texinfo:generate-includes #p"/Users/eslick/Work/fsrc/elephant-cvs/doc/includes/" +;; (sb-texinfo:generate-includes #p"/Users/eslick/Work/fsrc/elephant-cvs/doc/includes/" + (sb-texinfo:generate-includes #p"." (find-package :elephant) (find-package :elephant-backend) (find-package :elephant-memutil) --- /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2007/04/01 20:22:24 1.12 +++ /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2007/04/02 00:51:06 1.13 @@ -32,6 +32,11 @@ Elephant provides a persistent index which maintains an ordered collection of lisp values or persistent object references.

+The use of persistent objects makes coding concise, convenient, and +powerful, and makes persistence almost invisible to the programmer. +However, Elephant also allows the same basic data dictionary of +key/value retrieval that BerkeleyDB provides. + When someone says "database," most people think of SQL Relational Data Base Management Systems (e.g. Oracle, Postgresql, MySql). Those systems store data in statically typed tables with unique shared @@ -202,9 +207,11 @@ values and objects can be stored: numbers (except for complexes), symbols, strings, nil, characters, pathnames, conses, hash-tables, arrays, CLOS objects and structs. Nested and circular things are -allowed. You can store basically anything except lambdas, closures, -class objects, packages and streams. (These may eventually get -supported too.) +allowed. Nested and circular things are allowed. You can store +basically anything except compiled functions, closures, class objects, +packages and streams. Functions can be stored as uncompiled lambda +expressions. (Compiled functions and other kinds of objects may +eventually get supported too.)

Elephant needs to use a representation of data that is independant of a specific lisp or data store. Therefore all lisp values that are --- /project/elephant/cvsroot/elephant/doc/user-guide.texinfo 2007/04/01 20:56:19 1.7 +++ /project/elephant/cvsroot/elephant/doc/user-guide.texinfo 2007/04/02 00:51:06 1.8 @@ -155,8 +155,8 @@ suggest or add indices to facilitate better performance on common queries.

-There are two functions @ref{Function elephant:get-query-instances} -and @ref{Function elephant:map-class-query} which accept a set of +There are two functions @ref{Function elephant:get-query-instances,,,includes/fun-elephant-get-query-instance } +and @ref{Function elephant:map-class-query,,,includes/fun-elephant-map-class-query} which accept a set of constraints instead of the familiar value or range arguments.

We'll use the classes @code{person} and @code{department} to