elephant-cvs

elephant-cvs@common-lisp.net

858 discussions

[elephant-cvs] CVS update: elephant/src/classes.lisp
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant/src In directory common-lisp.net:/tmp/cvs-serv13481/src Modified Files: classes.lisp Log Message: license, name changes Date: Fri Aug 27 10:31:30 2004 Author: blee Index: elephant/src/classes.lisp diff -u elephant/src/classes.lisp:1.2 elephant/src/classes.lisp:1.3 --- elephant/src/classes.lisp:1.2 Thu Aug 26 19:53:52 2004 +++ elephant/src/classes.lisp Fri Aug 27 10:31:30 2004 @@ -1,3 +1,41 @@ +;;; -*- Mode: Lisp; Syntax: ANSI-Common-Lisp; Base: 10 -*- +;;; +;;; classes.lisp -- persistent objects via metaobjects +;;; +;;; Initial version 8/26/2004 by Andrew Blumberg +;;; <ablumberg(a)common-lisp.net> +;;; +;;; part of +;;; +;;; Elephant: an object-oriented database for Common Lisp +;;; +;;; Copyright (c) 2004 by Andrew Blumberg and Ben Lee +;;; <ablumberg(a)common-lisp.net> <blee(a)common-lisp.net> +;;; +;;; This program is free software; you can redistribute it +;;; and/or modify it under the terms of the GNU General +;;; Public License as published by the Free Software +;;; Foundation; either version 2 of the License, or (at +;;; your option) any later version. +;;; +;;; This program is distributed in the hope that it will be +;;; useful, but WITHOUT ANY WARRANTY; without even the +;;; implied warranty of MERCHANTABILITY or FITNESS FOR A +;;; PARTICULAR PURPOSE. See the GNU General Public License +;;; for more details. +;;; +;;; The GNU General Public License can be found in the file +;;; LICENSE which should have been distributed with this +;;; code. It can also be found at +;;; +;;; http://www.opensource.org/licenses/gpl-license.php +;;; +;;; You should have received a copy of the GNU General +;;; Public License along with this program; if not, write +;;; to the Free Software Foundation, Inc., 59 Temple Place, +;;; Suite 330, Boston, MA 02111-1307 USA +;;; + ;; TODO: slot-bound-p (check the database) (in-package "ELEPHANT") @@ -78,9 +116,10 @@ (buffer-write-int (oid instance) *key-buf*) (let ((key-length (serialize ,name *key-buf*))) (handler-case - (deserialize (db-get-key-buffered (db *store-controller*) - (buffer-stream-buffer *key-buf*) - key-length)) + (deserialize (db-get-key-buffered + (controller-db *store-controller*) + (buffer-stream-buffer *key-buf*) + key-length)) (db-error (err) (if (= (db-error-errno err) DB_NOTFOUND) (error 'unbound-slot :instance instance :slot ,name) @@ -92,7 +131,7 @@ (buffer-write-int (oid instance) *key-buf*) (let ((key-length (serialize ,name *key-buf*)) (val-length (serialize new-value *out-buf*))) - (db-put-buffered (db *store-controller*) + (db-put-buffered (controller-db *store-controller*) (buffer-stream-buffer *key-buf*) key-length (buffer-stream-buffer *out-buf*) val-length :transaction *current-transaction*

1 0

[elephant-cvs] CVS update: elephant/NOTES
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv13389 Modified Files: NOTES Log Message: more changes than i can count Date: Fri Aug 27 10:31:08 2004 Author: blee Index: elephant/NOTES diff -u elephant/NOTES:1.1 elephant/NOTES:1.2 --- elephant/NOTES:1.1 Thu Aug 26 19:59:46 2004 +++ elephant/NOTES Fri Aug 27 10:31:08 2004 @@ -6,7 +6,12 @@ this has been optimized for use with CMUCL. it has been tested and somewhat optimized for allegro. SBCL and OpenMCL are definitely also desired targets. Lispworks is a target -as well but less so; i can't test it. +as well but less so: i don't have access to it. + +Theoretically one can port this to any lisp with a decent +FFI and MOP. However since those are two of the less +standardized bits of Lisp, in practice this might be +difficult. From top to bottom, here are the implementation layers: @@ -24,6 +29,52 @@ database / serializer, specials are needed. The store controller (see below) is also a special. +----------------------- +CLASSES AND METACLASSES +----------------------- + +Persistent classes which the user defines are declared and +instrumented by using the persistent-metaclass. + +----------- +COLLECTIONS +----------- + +While we support serializing and persisting a wide class of +Lisp data types, there are problems with aggregate types +(conses, lists, arrays, objects, hash-tables...) + +1) not automatic: there's no way for elephant to know when +you've changed a value in an aggregate object, so you have +to manually restore it back into the slot to get it saved. + +slot-1 of obj A contains a cons. you change the car of the +cons. this is not reflected into the database unless you +re-set slot-1 of obj A with the cons. + +2) merge-conflicts: changing one value and saving an +aggregate will write out the whole aggregate, possibly +blowing away changes other threads have made behind your +back. this is not protected by transactions! + +3) consing, non-lazy and expensive (de)serialization: you +have to serialize/deserialize the entire aggregate every +time you save it or restore it. This is pretty fast all +things considered, but it's probably better to use +persistent collections. + +4) you have to store the entire collection in memory, +whereas one of the points of the database to store large +collections of objects..... + +For these and other reasons, we provide a hash-table-like +interface to Berkeley BTrees. These have many advantages +over ordinary hash-tables from the point of view of +persistence. + +TODO: programmatic way to create secondary indicies +(probably Lisp-level, since FFI callbacks are nasty.) + ---------- CONTROLLER ---------- @@ -57,6 +108,8 @@ determines liveness. The root object is a btree collection with a special OID 0. +TODO: write the garbage collector. + ------------ TABLE LAYOUT ------------ @@ -157,32 +210,6 @@ SERIALIZER: AGGREGATE TYPES --------------------------- -problems, or Do You REALLY Want to Persist Aggregates? - -0) not automatic: there's no way for elephant to know when -you've changed a value in an aggregate object, so you have -to manually restore it back into the slot to get it saved. - -1) merge-conflicts: changing one value in the aggregate will -write out the whole aggregate, possibly blowing away changes -other threads have made behind your back. this is not -protected by transactions! - -2) object identity / circularity: what are the right -semantics for things not tagged with OIDs? certainly can't -maintain identity across slots. - -obj A has slots slot-1 and slot-2. puts same cons in each -slot. no way to keep track that they are supposed to be the -same cons! - -3) consing, non-lazy and expensive (de)serialization - -"solution": in the FEW BIG TABLES schema, it is cheap to -create and use persistent collection types, which will -preserve identity et al. as with all persistent objects, -persistent collections are lazy-loading. - if you really must store non-persistent aggregates, we support: conses (lists): 1 + car + cdr @@ -238,6 +265,8 @@ default accessed through a special variable *current-transaction*. The special *auto-commit* (which defaults to nil!) is also a special. + +TODO: do transactions memory leak? UFFI is used as much as possible. Because some implementations can't call function pointers, and because we

1 0

[elephant-cvs] CVS update: elephant/LICENSE
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv13213 Added Files: LICENSE Log Message: first version Date: Fri Aug 27 10:30:49 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/README
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv13081 Added Files: README Log Message: first version Date: Fri Aug 27 10:30:35 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/TUTORIAL
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv12993 Added Files: TUTORIAL Log Message: first version Date: Fri Aug 27 10:30:25 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/elephant.asd
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv12772 Added Files: elephant.asd Log Message: first version Date: Fri Aug 27 10:30:09 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/functions.lisp
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv12626 Added Files: functions.lisp Log Message: first version Date: Fri Aug 27 10:29:28 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/INSTALL
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv12575 Added Files: INSTALL Log Message: first version Date: Fri Aug 27 10:28:09 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/NOTES
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant In directory common-lisp.net:/tmp/cvs-serv23984 Added Files: NOTES Log Message: beginning of a developer doc Date: Thu Aug 26 19:59:46 2004 Author: blee

1 0

[elephant-cvs] CVS update: elephant/src/collections.lisp
by blee＠common-lisp.net 27 Aug '04

27 Aug '04

Update of /project/elephant/cvsroot/elephant/src In directory common-lisp.net:/tmp/cvs-serv23923/src Modified Files: collections.lisp Log Message: integrated with new serializer Date: Thu Aug 26 19:58:28 2004 Author: blee Index: elephant/src/collections.lisp diff -u elephant/src/collections.lisp:1.1.1.1 elephant/src/collections.lisp:1.2 --- elephant/src/collections.lisp:1.1.1.1 Thu Aug 19 10:05:14 2004 +++ elephant/src/collections.lisp Thu Aug 26 19:58:28 2004 @@ -3,93 +3,51 @@ ;;; collection types ;;; abstract hash-like collections ;;; equal hashing (except probably for array, hashe, instance keys!) -(defclass collection () - ((%db :accessor db)) - (:metaclass persistent-metaclass)) - -(defmethod initialize-instance :before ((instance collection) - &rest initargs - &key store-controller - &allow-other-keys) - (declare (ignore initargs)) - (setf (get-store-controller instance) store-controller)) - -(defmethod initialize-instance :after ((instance collection) - &rest initargs) - (declare (ignore initargs)) - (register-collection (get-store-controller instance) instance)) +(defclass collection (persistent) ()) (defgeneric get-value (key ht &rest args)) (defgeneric remove-kv (key ht &rest args)) -;;; auto-serialize keys, values -(defclass serial-hash-mixin () ()) - -(defmethod get-value (key (ht serial-hash-mixin) &rest args) - (deserialize (apply #'db-get (db ht) (serialize key) args) - (get-store-controller ht))) - -(defmethod (setf get-value) (value key (ht serial-hash-mixin) &rest args - &key (transaction *transaction*) - &allow-other-keys) - (apply #'%db-put (db ht) (serialize key) (serialize value) - :transaction transaction args)) - -(defmethod remove-kv (key (ht serial-hash-mixin) &rest args - &key (transaction *transaction*) &allow-other-keys) - (apply #'%db-remove (db ht) (serialize key) :transaction transaction args)) - -;;; string keys, values -(defclass string-hash-mixin () ()) - -(defmethod get-value (key (ht string-hash-mixin) &rest args) - (apply #'db-get (db ht) key args)) - -(defmethod (setf get-value) (value key (ht string-hash-mixin) &rest args - &key (transaction *transaction*) - &allow-other-keys) - (apply #'%db-put (db ht) key value :transaction transaction args)) - -(defmethod remove-kv (key (ht string-hash-mixin) &rest args - &key (transaction *transaction*) &allow-other-keys) - (apply #'%db-remove (db ht) key :transaction transaction args)) - ;;; btree access -(defclass %btree (collection) () - (:metaclass persistent-metaclass)) - -(defmethod initialize-instance :after ((instance %btree) &rest initargs) - (declare (ignore initargs)) - (setf (db instance) - (create-table (get-store-controller instance) - "p-btrees" - (prin1-to-string (oid instance)) - :type :btree))) - -;;; persistent serialized object btrees -(defclass p-btree (%btree serial-hash-mixin) () - (:metaclass persistent-metaclass)) - -;;; persistent string btree -(defclass p-string-btree (%btree string-hash-mixin) () - (:metaclass persistent-metaclass)) - -;;; hash-table access -(defclass %hash-table (collection) () - (:metaclass persistent-metaclass)) - -(defmethod initialize-instance :after ((instance %hash-table) &rest initargs) - (declare (ignore initargs)) - (setf (db instance) - (create-table (get-store-controller instance) - "p-hash-tables" - (prin1-to-string (oid instance)) - :type :hash))) - -;;; persistent serialized object hash-tables -(defclass p-hash-table (%hash-table serial-hash-mixin) () - (:metaclass persistent-metaclass)) +(defclass btree (collection) ()) -;;; persistent string hash-tables -(defclass p-string-hash-table (%hash-table string-hash-mixin) () - (:metaclass persistent-metaclass)) +(defmethod get-value (key (ht btree) &rest args) + (declare (ignore args)) + (buffer-write-int (oid ht) *key-buf*) + (let ((key-length (serialize key *key-buf*))) + (handler-case + (values + (deserialize (db-get-key-buffered (db *store-controller*) + (buffer-stream-buffer *key-buf*) + key-length)) + t) + (db-error (err) + (if (= (db-error-errno err) DB_NOTFOUND) + (values nil nil) + (error err)))))) + +(defmethod (setf get-value) (value key (ht btree) &rest args + &key (transaction *current-transaction*) + (auto-commit *auto-commit*) + &allow-other-keys) + (declare (ignore args)) + (buffer-write-int (oid ht) *key-buf*) + (let ((key-length (serialize key *key-buf*)) + (val-length (serialize value *out-buf*))) + (db-put-buffered (db *store-controller*) + (buffer-stream-buffer *key-buf*) key-length + (buffer-stream-buffer *out-buf*) val-length + :transaction transaction + :auto-commit auto-commit))) + +(defmethod remove-kv (key (ht btree) &rest args + &key (transaction *current-transaction*) + (auto-commit *auto-commit*) + &allow-other-keys) + (declare (ignore args)) + (buffer-write-int (oid ht) *key-buf*) + (let ((key-length (serialize key *key-buf*))) + (db-delete-buffered (db *store-controller*) + (buffer-stream-buffer *key-buf*) key-length + :transaction transaction + :auto-commit auto-commit)))

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

elephant-cvs