Peter Hildebrandt wrote:
Ken,
thanks for the quick response. I'll try to illuminate things a bit.
Cool. I am curious what application requirement led to this.
I've been looking for a while for a way to store a (possibly large) number of model objects in a way that makes it easy and fast both to iterate over them and to access individual ones. In principle this can all be done with the fm-utilities, e.g. kids and fm-other. However, when things get big, I feel bad constantly traversing trees with fm-other and keeping tons of instances in the kids slot.
Understood. I have been thinking along the same lines lately, but I generally wait until I have a use case because (a) I am lazy and (b) I think without an actual use case one puts in too many bells and whistles or guesses wrong at what is needed and how it should work or all of the above.
Sounds like you have a Real Application to do. A nice extension, btw, would be an option for JIT creation should the thing not yet exist, but I gather you have Actual Work(tm) to do. When it comes up. I guess you would want the lookup then to be the key plus the argument list for make-instance.
btw, one of the things I sometimes think to point out is that Cells merely benefits (albeit greatly) from being used within a namespace of other objects to depend on, and even more when that namespace has Cells Inside(tm). I probably did Family only because I started with a GUI hierarchy, tho on second thought the tree is generally a powerful data structure.
All just a long way of saying "good idea". :)
There are three reasons I was looking for an alternative:
- My current research app models some type of human learning, which
involves huge numbers of interlinked nodes in a network. I was looking for a data structure which has O(1) access.
- fm-other traverses the tree at initialization time looking for the
target. I was looking for a way to reference instances to be created in the future. (For example, this is interesting when loading a network from file - either you have an acyclic graph and save it in a topological order, or you need some way to forward reference to-be-loaded nodes)
- I am afraid (correct me if this is wrong) that ruled cells using
fm-other are calculated more often than necessary -- e.g. you have a cell depending on (fm-other :something). Then every time a new kids is added somewhere in the tree, this rule should be run to check whether there is a :something now somewhere. My cells-store hash table makes sure that only those rules are run that are actually necessary.
The /potential/ for kicking off kids rules too often is there, but generally when rules are written sensibly the dependency on :something's existence ends up in the rule for an attribute, not in the rule that would make kids elsewhere get recalculated. In a sense we have the typical proble with Lisp: one false step and you can just hose your application. But as with Lisp, one quickly develops an instinct for things that are slow by mistake.
The immediate reason is that when debugging some cells-gtk stuff, I was confronted with a thick knot of fm-other initiated tree traversals, and I wished I had something more linear and predictable.
<g> Understood.
So I tried to use simple hash table look ups, but that did not work since the cells did not learn when new items were added to the table (i.e. forward references broke). Thus I needed a cells-aware hash table. And now I'll use that in my app and in cells-ode, too.
Not that it is not a good idea. Being able to reach inside other structures for dependency is a natural fleshing-out of Cells -- why stop at the slot, or force everything to /be/ a slot to depend on it?
Yep, there might be other cases -- even though I can't think of anything besides a hash table where I ever missed cells-awareness.
Meta-cool is that you are the first Cells user to do something like this, IIRC, altho there are several who took the more typical Lisper approach of dashing off to do their own versions of the entire package. :)
Thanks, I take it as a compliment.
The (c?-with-stored (var key store &optional default) &body body)
I have not looked at the code so maybe what I am about to suggest is already possible, but would this also be useful (just to confuse things I changed with-stored to bwhen-c-gethash:
(c? (if (^whatever) 42 (bwhen-c-gethash (user login-id *user-logins*) .....)))
Funny you picked that name. At first, I called it c?-bwhen-hash. Then I figured that with fit the semantics better. Maybe I was right the first time, tho.
As to you point, no, this is not possible right now since c?-with-stored expands into (c? ...). This is not necessary, though, because all my macro does is establish a dependency on an intermediate object which gets modified if the corresponding hash table entry is manipulated (i.e. an object added/removed). So indeed we could have this (w/o the c? part) as a standalone macro.
I changed it in CVS. The following works now, too:
(make-instance 'test-store-item :value (c? (if (value bypass-lookup?) 'no-lookup (bwhen-gethash (v :bar store 'nothing) (value v)))))I included it in the test func at the bottom of md-utilities. If you feel like it, you can integrate it with the test suite. (You're the maintainer, after all *g*)
One of my blog articles details my enthusiasm for testing. :)
I added a few macros for easier writing of unit tests, btw. There is assert-values, which takes instance-value pairs and with-assert-observers, which checks whether all and only the specified observers run.
The above might be easily done (just a tweak or two) by looking at how synapses work. They are like local and/or anonymous Cells. "Anonymous" because they do not mediate a slot, they mediate an arbitrary sub-form of a rule. "Local" because only the Cell envalued by the containing form depends on the synaptic Cell.
Ooops, I should read ahead. Haven't looked at synapses. Well, next time. ;-)
Historical note: synapses used to be a completely different data structure with distinct code supporting them, now we just have a "synaptic" attribute to guide the general Cells logic in a few places.
Anyway, then c?-with-stored is just the special case where this appears as the entire form:
(c? (bwhen-c-gethash (...) ...))
Yep, I'm with you. Just reimplemnented c?-with-stored in terms of bwhen-gethash.
I think you need a -c- in there because i can imagine a bwhen-gethash as pure CL, nothing cells-y.
btw, using bwhen-store vs gethash might be a good idea (not expose implementation) but sometimes I like to do exactly that just to make code more approachable. This one is a coin-toss for me.
Hmmm, you know, I almost never use synapses and yet I have always suspected that a lot of fun would be had with them in re expanding the semantics one could express with Cells. But this exchange has me wondering if I missed something: will new kinds of Cells tend to be synapses because the semantics will be useful within rules?
Ok, now you lost me. I'll have to look at synapses before I can tell. For now, I need to go back to my actual work and put the new store to work.
Or maybe I am just confused: should all cells be synapses now? I might be in the land of distinctions without differences now. :)
So I guess you are well prepared for Amsterdam :)
Well, I am off to Amsterdam shortly and will play with this when I get back. Not sure how much time I will be on-line, btw, until Monday.
Ok, have fun over here in old Europe :)
Thanks for a cool contrib.
You're welcome. Thanks for a cool library :)
Cheers, Peter