Hi,
I found a bug in cl-prevalence, where the objects that are linked are not properly reconstructed from the transaction log (restoring from a snapshot is fine).
I included minimal example code in the attached file and also on lisp paste [1] for easy reference.
Joe and Ma Dalton are two persons, and the slot 'PARENT of Joe is Ma. When the objects are reconstructed from the transaction log, three objects are created: Joe, Ma and another Ma that is the value of the slot 'PARENT of Joe, but not the same object as the first Ma.
A look at the transaction-log.xml file shows that in the transaction for the creation of Joe, Ma is referred to as a person and the values of its slots are indicated, instead of referring to the previously created instance.
Can anyone reproduce this? Any ideas to fix? I guess that the serialization doesn't know of the already existing objects.
I'm working on SBCL 1.0.6 on Ubuntu 07.10 and cl-prevalence is the last released version.
Thanks in advance,
Evan
Evan Monroig evan.monroig@gmail.com writes:
Hi,
I found a bug in cl-prevalence, where the objects that are linked are not properly reconstructed from the transaction log (restoring from a snapshot is fine).
I included minimal example code in the attached file and also on lisp paste [1] for easy reference.
Joe and Ma Dalton are two persons, and the slot 'PARENT of Joe is Ma. When the objects are reconstructed from the transaction log, three objects are created: Joe, Ma and another Ma that is the value of the slot 'PARENT of Joe, but not the same object as the first Ma.
A look at the transaction-log.xml file shows that in the transaction for the creation of Joe, Ma is referred to as a person and the values of its slots are indicated, instead of referring to the previously created instance.
Can anyone reproduce this? Any ideas to fix? I guess that the serialization doesn't know of the already existing objects.
I'm working on SBCL 1.0.6 on Ubuntu 07.10 and cl-prevalence is the last released version.
Thanks in advance,
Evan
Hi
I can duplicate your reported behaviour in LispWorks Pro 5.0.2 for Linux, but I thought this was normal behaviour. When passing a compound object as parameter to a data manipulation/creation transaction function, information is saved in the transaction log to recreate such an object. I suppose code can be added to check for OBJECT-WITH-ID descendants when the transaction log is replayed to just return the same (under EQ) object (instead of reconstructing it) if such an object already exists, but in the general case (i.e. non OBJECT-WITH-ID objects) there is no way to check for this type of equality and you'll end up with duplicates. Anyone please correct me if I am wrong.
So, 1) you'll either have to save Ma's ID in Joe's parent slot when using managed prevalence, or 2) don't use managed prevalence and write your own transaction functions, or wrap the managed prevalence transactions as helper-only functions in your own transactions. In this case you still pass Ma's ID to the transaction (and only Ma's ID is stored in the transaction log during the recording of the transaction in contrast to the all the info to reconstruct Ma's object). Then, inside the transaction, you can fetch Ma's object yourself and this way assign the Ma object to Joe's parent slot. I usually follow this approach. In general I don't pass compound objects to transactions, but pass IDs or the slots as parameters to construct the object inside the transaction. It's a bit tedious, though, and I am sure it can be automated with a more hefty managed prevalence implementation.
HTH. Nico
Evan, Nico,
I was just looking at the test code but Nico was faster, I guess I needed more time to get back into cl-prevalence. Indeed, my first reaction was to do something like this:
(defun tx-set-person-parent (system person-id parent-id) (let ((person (find-object-with-id system 'person person-id)) (parent (find-object-with-id system 'person parent-id))) (setf (parent person) parent)))
(defun make-joe-and-ma-1 () (let* ((ma (make-person :firstname "Ma" :lastname "Dalton")) (joe (make-person :firstname "Joe" :lastname "Dalton" :parent ma))) (setf *ma-id* (slot-value ma 'id)) (setf *joe-id* (slot-value joe 'id))))
(defun make-joe-and-ma-2 () (let* ((ma (make-person :firstname "Ma" :lastname "Dalton")) (joe (make-person :firstname "Joe" :lastname "Dalton"))) (setf *ma-id* (slot-value ma 'id)) (setf *joe-id* (slot-value joe 'id)) (tx-set-person-parent *test-system* *joe-id* *ma-id*)))
To set the link in a seperate transaction based on id's.
However, (1) I haven't fully tested this (2) it seems that Evan's test code suggest that the problem is not in replaying the transaction log but in recovering from the snapshot which is different. I will continue looking into this when I have more time today...
Sven
On 13 Mar 2008, at 10:08, Nico de Jager wrote:
Evan Monroig evan.monroig@gmail.com writes:
Hi,
I found a bug in cl-prevalence, where the objects that are linked are not properly reconstructed from the transaction log (restoring from a snapshot is fine).
I included minimal example code in the attached file and also on lisp paste [1] for easy reference.
Joe and Ma Dalton are two persons, and the slot 'PARENT of Joe is Ma. When the objects are reconstructed from the transaction log, three objects are created: Joe, Ma and another Ma that is the value of the slot 'PARENT of Joe, but not the same object as the first Ma.
A look at the transaction-log.xml file shows that in the transaction for the creation of Joe, Ma is referred to as a person and the values of its slots are indicated, instead of referring to the previously created instance.
Can anyone reproduce this? Any ideas to fix? I guess that the serialization doesn't know of the already existing objects.
I'm working on SBCL 1.0.6 on Ubuntu 07.10 and cl-prevalence is the last released version.
Thanks in advance,
Evan
Hi
I can duplicate your reported behaviour in LispWorks Pro 5.0.2 for Linux, but I thought this was normal behaviour. When passing a compound object as parameter to a data manipulation/creation transaction function, information is saved in the transaction log to recreate such an object. I suppose code can be added to check for OBJECT-WITH-ID descendants when the transaction log is replayed to just return the same (under EQ) object (instead of reconstructing it) if such an object already exists, but in the general case (i.e. non OBJECT-WITH-ID objects) there is no way to check for this type of equality and you'll end up with duplicates. Anyone please correct me if I am wrong.
So,
- you'll either have to save Ma's ID in Joe's parent slot when
using managed prevalence, or 2) don't use managed prevalence and write your own transaction functions, or wrap the managed prevalence transactions as helper- only functions in your own transactions. In this case you still pass Ma's ID to the transaction (and only Ma's ID is stored in the transaction log during the recording of the transaction in contrast to the all the info to reconstruct Ma's object). Then, inside the transaction, you can fetch Ma's object yourself and this way assign the Ma object to Joe's parent slot. I usually follow this approach. In general I don't pass compound objects to transactions, but pass IDs or the slots as parameters to construct the object inside the transaction. It's a bit tedious, though, and I am sure it can be automated with a more hefty managed prevalence implementation.
HTH. Nico _______________________________________________ cl-prevalence-devel site list cl-prevalence-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/cl-prevalence-devel
Sven Van Caekenberghe scaekenberghe@common-lisp.net writes:
Evan, Nico,
I was just looking at the test code but Nico was faster, I guess I needed more time to get back into cl-prevalence. Indeed, my first reaction was to do something like this:
(defun tx-set-person-parent (system person-id parent-id) (let ((person (find-object-with-id system 'person person-id)) (parent (find-object-with-id system 'person parent-id))) (setf (parent person) parent)))
(defun make-joe-and-ma-2 () (let* ((ma (make-person :firstname "Ma" :lastname "Dalton")) (joe (make-person :firstname "Joe" :lastname "Dalton"))) (setf *ma-id* (slot-value ma 'id)) (setf *joe-id* (slot-value joe 'id)) (tx-set-person-parent *test-system* *joe-id* *ma-id*)))
To set the link in a seperate transaction based on id's.
Nico, Sven,
Thanks for your fast answers. It seems that I asked for the moon :).
I understand that your approach to record only the IDs will work better, though I see that it can be tedious since you have to replace objects by their IDs when recording changes on them, and replace IDs by associated objects when retrieving them from the database.
Actually the example was a little contrived and the slot that I have is a tree containing any number of persisted objects (that all inherit the OBJECT-WITH-ID class), but I will try using this approach and see if that can be done without too much hassle.
On 13 Mar 2008, at 10:08, Nico de Jager wrote:
I can duplicate your reported behaviour in LispWorks Pro 5.0.2 for Linux, but I thought this was normal behaviour. When passing a compound object as parameter to a data manipulation/creation transaction function, information is saved in the transaction log to recreate such an object. I suppose code can be added to check for OBJECT-WITH-ID descendants when the transaction log is replayed to just return the same (under EQ) object (instead of reconstructing it) if such an object already exists, but in the general case (i.e. non OBJECT-WITH-ID objects) there is no way to check for this type of equality and you'll end up with duplicates. Anyone please correct me if I am wrong.
However, (1) I haven't fully tested this (2) it seems that Evan's test code suggest that the problem is not in replaying the transaction log but in recovering from the snapshot which is different. I will continue looking into this when I have more time today...
(pardon me for changing the order of your writings, I hope that the flow of ideas is respected this way)
As Nico wrote the behavior I'm having is not necessarily a bug, but anyway there still is a problem in that recovering from the transaction log or from the snapshot yields different results. I think that the "correct" way to go would be to check for OBJECT-WITH-ID descendants while making and/or replaying the transaction log.
Thanks again,
Evan
Evan,
On 13 Mar 2008, at 10:58, Evan Monroig wrote:
As Nico wrote the behavior I'm having is not necessarily a bug, but anyway there still is a problem in that recovering from the transaction log or from the snapshot yields different results. I think that the "correct" way to go would be to check for OBJECT-WITH-ID descendants while making and/or replaying the transaction log.
Each transaction is serialized independently from each other, using a fresh serialization-state. A snapshot is taken as a whole, correctly resolving identical objects.
It would sure be a nice experiment to somehow handle object-with-id descendants (or all known objects) differently here: it somehow feels like this should work, but I am not completely sure. I took a quick look at the code and it would require some hacking into the serialization code, specifically the serialization-state (known objects).
Good luck and keep us informed !
Sven
PS: I annotated you lisp paste.
On Fri, Mar 14, 2008 at 11:11 PM, Sven Van Caekenberghe scaekenberghe@common-lisp.net wrote:
Each transaction is serialized independently from each other, using a fresh serialization-state. A snapshot is taken as a whole, correctly resolving identical objects.
Yes this is what I understood, so changing this would require hacking around the serialization package.
It would sure be a nice experiment to somehow handle object-with-id descendants (or all known objects) differently here: it somehow feels like this should work, but I am not completely sure. I took a quick look at the code and it would require some hacking into the serialization code, specifically the serialization-state (known objects).
Thanks for all your suggestions. I finally worked around this by transforming the objects referred to to forms that look like (list 'find-object-by-id id) where id is the integer id of the idea. When I need the information I then just call funcall and everything works :). It also somehow feels faster (I didn't benchmark), since cl-prevalence doesn't have to recurse into the objects anymore and the transaction logs are shorter.
Evan
cl-prevalence-devel@common-lisp.net