[elephant-cvs] CVS elephant

Update of /project/elephant/cvsroot/elephant In directory common-lisp:/tmp/cvs-serv25268

Modified Files: INSTALL Makefile ele-bdb.asd Log Message: New build process completed & tested for BDB

--- /project/elephant/cvsroot/elephant/INSTALL 2006/02/15 01:54:07 1.14 +++ /project/elephant/cvsroot/elephant/INSTALL 2006/02/19 17:25:52 1.15 @@ -26,9 +26,28 @@ A C compiler, probably gcc or Visual Studio. Presumably you have this if you installed Sleepycat.

------------- -Instructions ------------- +------------------ +Short Instructions +------------------ + +The new build system should work out of the box on most Un*x +platforms that have asdf, uffi and either clsql or Berkeley DB +installed in the usual places. + +Try: (asdf:operate 'asdf:load-op :elephant) +Then: (open-store '(<backend> <spec>)) + +Where <backend> = { :bdb | :clsql } + <spec> = { '(:sqlite3 "db path") | '(:postgresql "db path") | "bdb directory" } + +This should load all files, including compiling libraries, on +most systems. For Win32, see the instructions below. + +(We'll improve the build process for Win32 if there is demand) + +----------------- +Long Instructions +-----------------

I assume you have a supported lisp with asdf.

@@ -36,58 +55,69 @@

/usr/local/share/common-lisp/elephant-0.3/

+ 1) Install UFFI. If you're using 1.4.24 replace

-path-to-uffi/src/functions.lisp + path-to-uffi/src/functions.lisp + + with the provided file.

-with the provided file.

-2) Install a backend: Either Berkeley DB 4.3, PostGresql, or SQLite 3. +2) Install ASDF. + +Ensure that you have a recent version of ASDF installed as +the load process now depends upon it. + + +3) Install a backend: Either Berkeley DB 4.3, PostGresql, or SQLite 3. + +------- +SQL +-------

For relational database systems, refering the formal documentation other the heading "SQL-BACK-END".

-For Berkeley 4.3: -Under Un*x, you may actually -already have this installed, though it may be compiled with -funny options, so if things don't work you may want to try -to start from scratch. FreeBSD has a port for this, as I'm -sure do other BSDs (including Darwin/Fink.) Take note of -where libdb.so and db.h are installed (usually -/usr/local/BerekleyDB.4.3/lib/libdb.so and -/usr/local/BerekleyDB.4.3/include/db.h, or -/usr/local/lib/db42/libdb.so and -/usr/local/include/db42/db.h.) +------------- +Berkeley 4.3: +-------------

-Compile and install the libsleepycat shared library. +Under Un*x, you may actually already have this installed, though +it may be compiled with funny options, so if things don't work +you may want to try to start from scratch. FreeBSD has a port +for this, as I'm sure do other BSDs (including Darwin/Fink.) +Take note of where libdb.so and db.h are installed, usually:

-Under Un*x, edit Makefile and run (using GNU make, gmake on -BSD) + /usr/local/BerekleyDB.4.3/lib/libdb.so and + /usr/local/BerekleyDB.4.3/include/db.h, or

-make + /usr/local/lib/db42/libdb.so and + /usr/local/include/db42/db.h.)

-This compiles src/libsleepycat.c and installs it into +a) Edit Makefile variable DB43DIR and DB43INC, DB43LIB if necessary

-/usr/local/share/common-lisp/elephant-0.3/ +This makes sure that the build process can find your files. +You can test that the build works by calling:

-You probably have to make sure this directory exists before running -make install. +'make bdb'.

-If you need to change this path, you will change it in the Makefile -and also in controller.lisp on the line: +b) Also edit the variable *sleepycat-foreign-library-path* in

-(defvar *elephant-lib-path* "/usr/local/share/common-lisp/elephant-0.3/") + ele-bdb.asd

-or where you specified. On Darwin / OS X you need to have -the developer tools installed. In the Makefile and other places -there are commented-out lines showing settings that some users have used for -OS X; if you are using that I assume you will have to comment out the -appropriate lines and uncomment those examples. +to point to your local distribution of the Berkeley DB libraries

-For Win32 (directions courtesy of Bill Clementson): +Darwin / OS X +--------------- +You need to have the developer tools installed. In the Makefile and +ele-bdb.lib there are commented-out lines showing settings that some +users have used for OS X; if you are using that I assume you will +have to comment out the appropriate lines and uncomment those examples.

-Create an MSVC dll project and add src/libsleepycat.c, -src/libsleepycat.def and the Berkeley DB libdb43.lib files +For Win32 (directions courtesy of Bill Clementson): +--------------------------------------------------- +Create an MSVC dll project and add src/db-bdb/libsleepycat.c, +src/db-bdb/libsleepycat.def and the Berkeley DB libdb43.lib files to the project (should be in the build_win32/release folder)

Add the Berkeley DB dbinc include files directory and the @@ -101,15 +131,28 @@ libsleepycat.c, it may or may not be necessary to load libdb43.dll into Lisp (see below.)

-3) Compile and load Elephant:

-First, edit src/sleepycat.lisp so that it points to the -correct libraries. If you're using Un*x and ASDF, this is -probably automagic. - -Symlink elephant.asd to your asdf systems directory (mine is -/usr/local/share/common-lisp/systems). Fire up lisp and -depending on your database backend: +4) Compile and load Elephant: + +The new backend load process should work automatically on Un*x +systems but if there are problems with loading foreign libraries, +then you can test your C tools setup with 'make' in the elephant +root directory. This will build the common memutils library +in src/memutil/libmemutil.so/dylib that all backends require. + +There is a new two-phase load process. The first requires that +you use asdf to load the main elephant front-end: + +(asdf:operate 'asdf:load-op :elephant) + +This will load and compile Elephant. This will also automatically +load UFFI. + +When you call (open-store <spec>) inside lisp it will automatically +load the remaining dependencies for the specified backend via ASDF. + +To test the load process explicitly the following asdf files are +provided:

if you are using Sleepycat / Berkeley DB, type: (asdf:operate 'asdf:load-op :ele-bdb) @@ -120,16 +163,8 @@ if you are using SQLite3, type: (asdf:operate 'asdf:load-op :ele-sqlite3)

-This will load and compile Elephant. This will also -automatically load UFFI.

-I can't seem to make OpenMCL not intern default keyword -values of my macros -- something which doesn't happen on -other implementations. I can't reproduce the issue except -for in my code, but expect (use-package "ELE")'s to produce -conflicting symbol warnings. - -4) Make the documentation: +5) Make the documentation:

Execute:

@@ -137,6 +172,7 @@

In the doc directory should be build the HTML version of the texinfo files.

+ ------- Testing ------- @@ -145,16 +181,29 @@

http://www.cliki.net/RT

-Once RT is installed, edit tests/elephant-tests.lisp to make -*testdb-path* point to somewhere appropriate. Symlink -elephant-tests.asd to your asdf systems directory, then run +Once RT is installed

(asdf:operate 'asdf:load-op :elephant-tests) (in-package :ele-tests) -(do-all-tests) +(setf *default-spec* <backend>) + where <backend> = { *testsqlite3-spec* | *testpg-spec* | *testbdb-spec* } +(do-backend-tests) + +This will test the standalone API for your backend. Currently all tests are +passing on 0.6.0. There will be a set of migration tests that will be 'ignored' +but the final message should indicate no failing tests. + +This should take less than 5 minutes on decent hardware. + +Elephant allows migration between repositories. To test this: + +(do-migration-tests *default-spec* <backend>) + where <backend> is a different *testXXXXX-spec* variable to test migration + to that backend. + +This should take less than 2 minutes on decent hardware. + +A backend is considered "green" if it can pass both the backend tests and the +migration tests.

-this should take about 5 minutes on decent hardware. Note -that the "no-eval-initform" and "update-class" tests fail, -these are known bugs which will get fixed in a future -release.

--- /project/elephant/cvsroot/elephant/Makefile 2006/02/19 16:22:39 1.10 +++ /project/elephant/cvsroot/elephant/Makefile 2006/02/19 17:25:52 1.11 @@ -49,7 +49,7 @@

BDBSRC=src/db-bdb

-bdb: libsleepycat.$(EXT) +bdb: $(BDBSRC)/libsleepycat.$(EXT)

$(BDBSRC)/libsleepycat.$(EXT): $(BDBSRC)/libsleepycat.c gcc $(SHARED) -Wall -L$(DBLIBDIR) -I$(DBINCDIR) -fPIC -O3 -o $@ $< -ldb -lm --- /project/elephant/cvsroot/elephant/ele-bdb.asd 2006/02/19 04:52:58 1.6 +++ /project/elephant/cvsroot/elephant/ele-bdb.asd 2006/02/19 17:25:52 1.7 @@ -23,6 +23,82 @@

(in-package :ele-bdb-system)

+(eval-when (:compile-toplevel :load-toplevel :execute) + ;; We need this dependency satisfied to compute what to do for C files + (unless (find-package 'uffi) + (asdf:operate 'asdf:load-op 'uffi))) + +;; +;; EDIT ME FOR YOUR SYSTEM +;; +;; An attempt at good defaults is here. We should +;; later add a search function that users can add to +;; so they don't have to edit source +;; + +(defparameter *sleepycat-foreign-library-path* + ;; Sleepycat: this works on linux + #+linux +;; "/db/ben/lisp/db43/lib/libdb.so" + "/usr/local/BerkeleyDB.4.3/lib/libdb-4.3.so" + ;; this works on FreeBSD + #+(and (or bsd freebsd) (not (or darwin macosx))) + "/usr/local/lib/db43/libdb.so" + #+(or darwin macosx) + ;; for Fink (OS X) -- but I will assume Linux more common... +;; "/sw/lib/libdb-4.3.dylib" + ;; a possible manual install + "/usr/local/BerkeleyDB.4.3/lib/libdb.dylib") + +(defclass bdb-c-source (c-source-file) + ()) + +(defparameter *root-dir* (pathname-directory *load-truename*)) + +(defparameter *library-file-dir* (append (pathname-directory *load-truename*) + (list "src" "db-bdb"))) + +;; Compile foreign library on non-win32 platforms + +(defmethod output-files ((o compile-op) (c bdb-c-source)) + (let ((library-file-type + (funcall (intern (symbol-name '#:default-foreign-library-type) + (symbol-name '#:uffi))))) + (list (make-pathname :name (component-name c) + :type library-file-type + :directory *library-file-dir*)))) + +(defmethod perform ((o compile-op) (c bdb-c-source)) + (unless (operation-done-p o c) + #-(or win32 windows) + (unless (zerop (uffi:run-shell-command + (format nil + #-freebsd "cd ~A; make bdb" + #+freebds "cd ~A; gmake bdb" + (make-pathname :directory *root-dir*)))) + (format t "Couldn't build library from libsleepycat.c via 'make bdb'~%") + (error 'operation-error :component c :operation o)))) + +(defmethod operation-done-p ((o compile-op) (c bdb-c-source)) + (or (let ((lib (make-pathname :defaults (component-pathname c) + :type (uffi:default-foreign-library-type)))) + (and (probe-file lib) (probe-file (component-pathname c)) + (> (file-write-date lib) (file-write-date (component-pathname c))))))) + +;; Load op - ensure that foreign library is loaded + +(defmethod perform ((o load-op) (c bdb-c-source)) + "Nothing to do!" + t) + +(defmethod operation-done-p ((o load-op) (c bdb-c-source)) + "Operation is done when the foreign library is loaded which should + happen when we compile the interface lisp file" + (and (symbol-function (intern (symbol-name '#:%db-strerror) + (find-package '#:sleepycat))) + t)) + + (defsystem ele-bdb :name "elephant" :author "Ben Lee blee@common-lisp.net" @@ -37,6 +113,7 @@ ((:module :db-bdb :components ((:file "package") + (:bdb-c-source "libsleepycat") (:file "sleepycat") (:file "bdb-controller") (:file "bdb-transactions")