Update of /project/elephant/cvsroot/elephant/doc In directory clnet:/tmp/cvs-serv3086/doc

Modified Files: installation.texinfo Log Message: Fix for lispworks deadlock detect (silently fails instead of asserting an error)

--- /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/04/28 14:24:57 1.13 +++ /project/elephant/cvsroot/elephant/doc/installation.texinfo 2007/04/29 21:54:51 1.14 @@ -36,8 +36,10 @@ Elephant supports SBCL, Allegro, Lispworks, OpenMCL and CMUCL. Each lisp is supported on each of the platforms it runs on: Mac OS X, Linux and Windows. As of release 0.6.1, both 32-bit and 64-bit systems -should be supported. Elephant has a small developer base and as of -the writing of this manual, there are: +should be supported. + +Due to the small number of developers and the large number of configurations +providing full test coverage is problematic. There are:

@enumerate @item Five lisp environments @@ -46,16 +48,14 @@ @item Three data store configurations: Berkeley DB, SQLite3 and Postgresql @end enumerate

-This means that the total number of combinations that should be tested -comes to: +which means that the total number of combinations to be tested could be as +much as:

@math{lisps * os * radix * dstore = 5 * 3 * 2 * 3 = 90 configurations}

-Of course not all of these combinations are valid, but the -implications of these combinatorics is that not every combination will -be tested in any given release. The developers and active user base -currently cover all three data store configurations on the following -platforms: +Not all of these combinations are valid, but the implication is that +not every combination will be tested in any given release. The +developers and user base regularly use the following platforms

@itemize @item 32/64-bit SBCL on Linux and Mac OS X @@ -63,13 +63,15 @@ @item 32-bit Allegro on Mac OS X @end itemize

-The developers will do their best to accomodate users who are keen to -test other combinations, but practically these configurations will be -the most stable and reliable. Elephant is becoming quite stable in -general, so don't be afriad to try an unemphasized combination - -chances are it is just a little more work to bring it up. In particular, -Elephant can probably work with MySQL or Oracle with just a little work, -but nobody has asked for this yet. +The CLSQL backend is used predominantly under SBCL on Linux and Mac OS +X at the time of writing. The developers will do their best to +accomodate users who are keen to test other combinations, but the +above configurations will be the most stable and reliable. + +Elephant is now quite stable in general, so don't be afraid to try an +unemphasized combination - chances are it is just a little more work +to bring it up. In particular, Elephant can probably work with MySQL +or Oracle with just a little work, but nobody has asked for this yet.

@subsection Library dependencies

@@ -77,7 +79,7 @@

@enumerate @item asdf -- @uref{http://www.cliki.net/asdf%7D -@item uffi -- version 1.5.17 or later, @uref{http://uffi.b9.com/%7D or @uref{http://www.cliki.net/UFFI%7D +@item uffi -- requires version 1.5.18 or later, @uref{http://uffi.b9.com/%7D or @uref{http://www.cliki.net/UFFI%7D @item cl-base64 -- @uref{http://www.cliki.net/cl-base64%7D @item gcc -- Your system needs GCC (or Cygwin) to build the Elephant C-based serializer library. (Precompiled DLL's are available for Windows platforms on the @uref{http://www.common-lisp.net/project/elephant/downloads.html, download page}. @item rt -- The RT regression test sytem is required to run the test suite: @uref{http://www.cliki.net/RT%7D @@ -86,7 +88,8 @@ Follow the instructions at these URLs to download and setup the libraries. (Note: uffi and cl-base64 are @uref{http://www.cliki.net/ASDF-Install, asdf-installable} for those -of you with asdf-install on your system) +of you with asdf-install on your system). Elephant, however, is not +asdf-installable today.

In addition to these libraries, each data store has their own dependencies as discussed in @ref{Berkeley DB} and @@ -97,10 +100,11 @@ @section Configuring Elephant

Before you can load the elephant packages into your running lisp, you -need to setup the configuration file. First, copy the reference file -config.sexp from the root directory to my-config.sexp. my-config.sexp -contains a lisp reader-formatted list of key-value pairs that tells -elephant where to find various libraries, how to build libraries, etc. +need to setup the configuration file. Copy the reference file +config.sexp from the root directory to my-config.sexp in the root +directory. my-config.sexp contains a lisp reader-formatted list of +key-value pairs that tells elephant where to find various libraries +and how to build them.

For example:

@@ -110,8 +114,6 @@ (:berkeley-db-lib-dir . "/opt/local/lib/db45/") (:berkeley-db-lib . "/opt/local/lib/db45/libdb-4.5.dylib") (:berkeley-db-deadlock . "/opt/local/bin/db45_deadlock") - (:pthread-lib . nil) - (:clsql-lib . nil) (:compiler . :gcc)) @end lisp

@@ -132,9 +134,9 @@ The config.sexp file contains a set of example configurations to start from, but you will most likely need to modify it for your system.

-Elephant has one small C library that it uses for binary serialization -which means that you need to have gcc in your path (@pxref{Elephant on -Windows} for exceptions on the Windows platform). +Elephant has one small C library that it uses for binary +serialization. This means that you need to have gcc in your path +(@pxref{Elephant on Windows} for exceptions on the Windows platform).

@node Loading Elephant @comment node-name, next, previous, up @@ -152,12 +154,11 @@

This will load the cl-base64 and uffi libraries. It will also automatically compile and load the C library. The build process no -longer depends on a Makefile. This build process has been verified on -most platforms, but if you have a problem please report it, and any -output you can capture, to the developers at -@email{elephant-devel@@common-lisp.net}. We will update the FAQ at -@uref{http://trac.common-lisp.net/elephant%7D with common problems users -run into. +longer depends on a Makefile and has been verified on most platforms, +but if you have a problem please report it, and any output you can +capture, to the developers at @email{elephant-devel@@common-lisp.net}. +We will update the FAQ at @uref{http://trac.common-lisp.net/elephant%7D +with common problems users run into.

@subsection Two-Phase Load Process

@@ -170,8 +171,7 @@ (NOTE: There are some good reasons and not so good reasons for this process. One reason you cannot load ele-bdb.asd directly as it depends on lisp code defined in elephant.asd. We decided not to fix -this in this release although later releases may avoid the oddity of -the two phase loading) +this in the 0.9 release although later releases may improve on this).

@subsection Packages

@@ -199,7 +199,8 @@ (:use :common-lisp :elephant)) @end lisp

-The symbols that are imported are captured in @ref{User API Reference} +The imported symbols are all that is needed to control Elephant +databases and are documented in detail in @ref{User API Reference}

@subsection Opening a Store

@@ -224,19 +225,18 @@ The first time you load a specific data store, Elephant will call ASDF to load all the specified data store's dependencies, connect to a database and return the @code{store-controller} subclass instance for -that backend. +that data store.

@node Berkeley DB @comment node-name, next, previous, up @section Berkeley DB

-The Berkeley DB Data Store started out as a very simple data dictionary in the -Berkeley Unix operating system. There are many ``Xdb'' systems that use the -same API, or a similarly one. A commercial version of the BDB was provided by -Sleepycat Software, and this product has since been purchased by Oracle corporation, -and is currently distributed under a similar licensing arrangement, with commercial -support also available. Please follow the download and installation procedures -defined here: +The Berkeley DB Data Store started out as a very simple data +dictionary in the Berkeley Unix operating system. There are many +``Xdb'' systems that use the same API, or a similarly one. A free for +non-commercial use version of Berkeley DB is provided by Oracle +corporation with commercial licenses available. Please follow the +download and installation procedures defined here:

http://www.oracle.com/technology/products/berkeley-db/db/index.html

@@ -246,68 +246,71 @@ @comment node-name, next, previous, up @section Setting up Berkeley DB

-Beyond ensuring that the file ``my-config.sexp'' points correctly -to your BDB installation, nothing else should be required to configure -the example that uses a local ``testdb'' directory as a dabase (under ``tests'') -in the top-level Elephant directory. +Beyond ensuring that the file ``my-config.sexp'' points to your BDB +installation directories and files, nothing else should be required to +configure the example that uses a local ``testdb'' directory as a +dabase (under ``tests'') in the top-level Elephant directory.

On one Fedora based system, the ``my-config.sexp'' file looked like this:

@lisp ((:berkeley-db-include-dir . "/usr/local/BerkeleyDB.4.5/include") - (:berkeley-db-lib-dir . "/usr/local/BerkeleyDB.4.5/lib") - (:berkeley-db-lib . "/usr/local/BerkeleyDB.4.5/lib/libdb.so") - (:berkeley-db-deadlock . "/usr/local/BerkeleyDB.4.5/bin/db_deadlock") + (:berkeley-db-lib-dir . "/usr/local/BerkeleyDB.4.5/lib") + (:berkeley-db-lib . "/usr/local/BerkeleyDB.4.5/lib/libdb.so") + (:berkeley-db-deadlock . "/usr/local/BerkeleyDB.4.5/bin/db_deadlock") (:pthread-lib . nil) - (:clsql-lib . "/usr/local/share/common-lisp/") - (:compiler . :gcc)) + (:clsql-lib . "/usr/local/share/common-lisp/") + (:compiler . :gcc)) @end lisp

The @ref{Test Suites} give a nice example of using BDB by running the test using the specification: @lisp -'(:BDB "/home/me/db/testdb/")) +'(:BDB "<elephant-root>/tests/testdb/") @end lisp

-Once you start working on an application, you will want to change this path -to a directory that make sense for you application, and use that as -the specification passed to -@lisp open-store -@end lisp -in your application. +Once you start working on an application, you will want to change the +path to a directory that is appropriate for your application, and use +that as the specification passed to @code{ open-store } on application +startup.

@node Upgrading Berkeley DB Databases @comment node-name, next, previous, up @section Upgrading Berkeley DB Databases

-At regular intervals, Elephant will require a specific version of Berkeley DB -because it does not parse the header files which tend to change with each release. -The patches are usually minor, but sometimes Elephant also depends on new -features of Berkeley DB. In this case, you have to update both the Berkeley DB -database files as well as any data Elephant has built within it. That creates -some special constraints for upgrading databases. +When there is a new release of Elephant, it will depend on a new +version of Berkeley DB. If so, you must upgrade your BDB databases to +use the new version Elephant. This forced upgrade is a consequence of +Elephant not parsing the BDB header files which tend to change various +important constants with each release. These patches are usually +minor. Upgrading also happens because Elephant tries to leverage new +features of Berkeley DB. + +The rest of this section talks about how to upgrade your existing +Berkeley DB databases, opening them in the new Elephant version and +migrating them to a newly created Elephant database.

@subsection Upgrading to 0.9

-This section outlines how to upgrade from Elephant code base version 0.6.0 -using Berkeley DB 4.3. +This section outlines how to upgrade from Elephant version 0.6.0 and +Berkeley DB 4.3.

@enumerate @item Install BDB 4.5 (keep 4.3 around for now) -@item Update my-config.sexp to point to the appropriate BDB 4.5 directories -@item Upgrade your database directory to 4.5 +@item Setup my-config.sexp to point to the appropriate BDB 4.5 directories +@item Upgrade your existing database directory to 4.5 @itemize @item Run db43_recover in your 0.6 database @item Optional: run db43_archive -d to remove all logs not part of a checkpoint This will make catastrophic recovery impossible, but reduces the amount of data you have to backup. @item Backup your db files and remaining logs - @item Run db45_checkpoint -1 in your 0.6 database directory + @item Run db45_checkpoint -1 in the database directory @end itemize -@item Migrate 0.6 data to a new 0.6.1 database +@item Upgrade 0.6 data to a fresh 0.9 database @itemize @item Open your old database: @code{(setf sc (open-store '(:BDB "/Users/me/db/ele060/")))} - @item Run upgrade: @code{(upgrade sc '(:BDB "/Users/me/db/ele061/"))} + @item Run upgrade: @code{(upgrade sc '(:BDB "/Users/me/db/ele090/"))} @end itemize @item Test your new application and report any bugs that arise to @email{elephant-devel@@common-lisp.net} @end enumerate @@ -315,14 +318,15 @@ @emph{(NOTE: close-store may fail when closing the old 0.6 database, this is OK.)}

@emph{(NOTE: 64-bit lisps will not successfully upgrade 32-bit 0.6 databases. Use a 32-bit - version of your lisp to update to 0.6.1 and then open that database in your 64-bit + version of your lisp to update to 0.9 and then open that database in your 64-bit lisp. There should be no compatibility problems. Best to test your application on a 32-bit lisp if you can, just to be sure.)}

@subsection Upgrade from Elephant 0.5

-Follow the upgrade procedures outlined in Elephant release 0.6.0 to migrate your database -from 0.5 to 0.6. Then follow the above procedures for upgrading from an 0.6 database. +Follow the upgrade procedures outlined in the Elephant 0.6.0 INSTALL +file to upgrade your database from 0.5 to 0.6.0. Then follow the above +procedures for upgrading to 0.9.

@emph{(NOTE: It may not take much work to make 0.9 upgrade directly from 0.5 However there are so few (none?) 0.5 users that it wasn't deemed worth @@ -333,55 +337,50 @@ @section CL-SQL

Although originally designed as an interface to the BerkeleyDB system, -the original Elephant system has been experimenetally extended to -support the use of relational database management systems as the -implementation of the persistent store. This relies on Kevin Rosenberg's -CL-SQL interface to a large number of relational systems. - -Although the BerkeleyDB system is an ideal object store for LISP objects, -one might prefer the licensing of a different system. For example, at -the time of this writing, it is my interpretation that one cannot use -the BerkeleyDB system behind a public website +the original Elephant system has been extended to support the use of +relational database management systems as the implementation of the +persistent store. This relies on Kevin Rosenberg's CL-SQL interface, +which provides access to a large number of relational systems. + +A major motivation of this extension is that one one might prefer the +licensing of a different system. For example, at the time of this +writing, it is our interpretation that one cannot use the BerkeleyDB +system behind a public website http://www.sleepycat.com/download/licensinginfo.shtml#redistribute unless one releases the entire web application as open source.

Neither the PostGres DBMS nor SQLite 3, nor Elephant itself, imposes -any such restriction. (Elephant is released under the LLGPL -(http://opensource.franz.com/preamble.html .) Older versions were -released under the GPL.) +any such restriction.

Other reasons to use a relational database system might include: familiarity with those systems, the fact that some part of your application needs to use the truly relational aspects of those systems, preference for the tools associated with those systems, etc.

-The SQL back-end extention of Elephant provides a function for migrating -data seamlessly between repositories. One can quite easily move -data from a BerkeleyDB repository to a PostGres repository, and vice versa. -This offers at least the possiblity than once can develop using one backend, -for example BerkeleyDB, and then later move to Postgres, or vice versa. -One could even operate simultaneously out of multiple repositories, if -there were a good reason to do so. - -At the time of this writing, the basic strategy for the SQL implementation -is quite simple. The same serializer used for the Sleepycat implementation -is employed, the byte-string is base64 encoded, and placed in a single -table which is managed by Elephant. - -All functionality except nested transaction support and cursor-puts -supported by the BerkeleyDB backend is supported by the CL-SQL -back-end. Concurrency and transaction atomicity have not been stress -tested well for the CL-SQL based system. - -Additionally, it is NOT the case that the Elephant system currently provides -transaction support across multiple repositories; it provides the transaction -support provided by the underlying repository to the user in a per-repository -basis. - -The PostGres backend is as currently employed is about 5 times slower than -the BerkeleyDB backend. This could probably change with continued development. - -CL-SQL supports a lot of DBMS systems, but only PostGres and SqlLite 3 have been tested. +Elephant provides functions for migrating data seamlessly between data +stores. One can quite easily move data from a BerkeleyDB repository +to a PostGres repository, and vice versa. This offers at least the +possibility than one can develop using one data store, for example +BerkeleyDB, and then later move to Postgres. One could even operate +simultaneously out of multiple repositories, if there were a good +reason to do so. + +The SQL implementation shares the serializer with the BDB data store, but +base64 encodes the resulting binary stream. This data is placed into +a single table in the SQL data store. + +All functionality except for nested transaction support and +cursor-puts supported by the BerkeleyDB data store is supported by the +CL-SQL data store. CL-SQL transaction integrity under concurrent +operation has not been extensively stress tested. + +Additionally, it is NOT the case that the Elephant system currently +provides transaction support across multiple repositories; it provides +transaction support on a per-repository basis. + +The PostGres backend is currently about 5 times slower than the +BerkeleyDB backend. As of the time of this writing, only PostGres and +SqlLite 3 have been tested as CL-SQL backends.

@node CL-SQL Example @comment node-name, next, previous, up @@ -391,20 +390,20 @@

@enumerate @item Install postgres and make sure postmaster is running. -Postgres may be installed on your system; you may be able to use a package manager to -install it, or you can install it quite easily from the PostgresSQL site directly -(http://www.postgresql.org/). +Postgres may be installed on your system; you may be able to use a

[104 lines skipped]