[slime-devel] weak hash tables and presentation
Hello, I've been looking over adding weak hashtable support to ECL, when I took a closer look at slime's use of them. Maybe they were used more aggressively in the past, but now they seem to only be used by the presentation contrib. I'm not sure they're necessary for that- as long as they're in the REPL shouldn't the presentation keep an extra reference to them? Perhaps a better way to deal with this is to have slime indicate what objects its holding and/or clear this buffer when it stops using them. This might better belong in a per-connection thing, and I wanted some advise on this before I started hacking on this. Are the weak hashtables legacy? Are they going to be used in preparation for something? Is this the only part that needs dealing with? Would it be better still to add weak hashtables to ECL anyway.
* Geo Carncross [2008-02-04 04:57+0100] writes:
Hello,
I've been looking over adding weak hashtable support to ECL, when I took a closer look at slime's use of them. Maybe they were used more aggressively in the past, but now they seem to only be used by the presentation contrib. I'm not sure they're necessary for that- as long as they're in the REPL shouldn't the presentation keep an extra reference to them?
Some people prefer weak references for this case. I guess weak references make it easier to "sell" presentations.
Perhaps a better way to deal with this is to have slime indicate what objects its holding and/or clear this buffer when it stops using them. This might better belong in a per-connection thing, and I wanted some advise on this before I started hacking on this.
That sounds like a quite a bit of work. Don't know if it's worth to bother. Presentations work also with normal (non-weak) hashtables and there is already a command to manually clear all presentations on the Lisp side.
Are the weak hashtables legacy? Are they going to be used in preparation for something? Is this the only part that needs dealing with?
Weak hashtables are only used for presentations and currently there are no plans to use them for something else.
Would it be better still to add weak hashtables to ECL anyway.
There are probably uses for weak hashtables outside of Slime. But it might be better to spend the time on other features like better source-location tracking etc. Helmut.
On Feb 4, 2008 10:17 AM, Helmut Eller <heller@common-lisp.net> wrote:
Perhaps a better way to deal with this is to have slime indicate what objects its holding and/or clear this buffer when it stops using them. This might better belong in a per-connection thing, and I wanted some advise on this before I started hacking on this.
That sounds like a quite a bit of work. Don't know if it's worth to bother. Presentations work also with normal (non-weak) hashtables and there is already a command to manually clear all presentations on the Lisp side.
Really? I guess it'd require saving *presentation-id-to-object* and *object-to-presentation-id* in the connection. Maybe instead a connection-closed-hook could look at the existing tables and clear off any entries that belonged to that connection. I don't know right now, I'd have to think about this more, but it doesn't seem to actually need to be that much work. It's just at that point, weak hash tables wouldn't contribute anything (that I can see), so it'd be a lot of code that could be removed.
Would it be better still to add weak hashtables to ECL anyway.
There are probably uses for weak hashtables outside of Slime. But it might be better to spend the time on other features like better source-location tracking etc.
This is true. I was hoping for some low-hanging fruit initially on the ECL-side since source-location tracking and proper arglist tracking would require deeper changes to ECL.
* Geo Carncross [2008-02-04 18:26+0100] writes:
On Feb 4, 2008 10:17 AM, Helmut Eller <heller@common-lisp.net> wrote:
Perhaps a better way to deal with this is to have slime indicate what objects its holding and/or clear this buffer when it stops using them. This might better belong in a per-connection thing, and I wanted some advise on this before I started hacking on this.
That sounds like a quite a bit of work. Don't know if it's worth to bother. Presentations work also with normal (non-weak) hashtables and there is already a command to manually clear all presentations on the Lisp side.
Really? I guess it'd require saving *presentation-id-to-object* and *object-to-presentation-id* in the connection. Maybe instead a connection-closed-hook could look at the existing tables and clear off any entries that belonged to that connection. I don't know right now, I'd have to think about this more, but it doesn't seem to actually need to be that much work.
Yes, that shouldn't be so hard. But I assumed you are aiming at something like a distributed garbage collector: track all references that Emacs has and when Emacs drops all references to an object also drop the reference to that object on the Lisp side. Such a scheme could release objects before the connection gets closed. And implementing it doesn't sound easy to me.
It's just at that point, weak hash tables wouldn't contribute anything (that I can see), so it'd be a lot of code that could be removed.
One could argue that SLIME shouldn't unnecessarily keep objects alive. With weak hashtables it's possible to release more objects at the price that some presentations no longer work. But the presentations for objects that are alive anyway (e.g. because some application needs them and usually the important objects) still work. I think that's a reasonable tradeoff. Helmut.
"Geo Carncross" <geocar@gmail.com> writes:
I've been looking over adding weak hashtable support to ECL, when I took a closer look at slime's use of them.
Weak hashtables are a useful feature, so if you want to add them to ECL, I would think this would be a good idea. They are not an essential feature for SLIME, though. We started using them in SLIME for presentations in implementations that have weak hashtables, because people complained that all the objects associated with presentations are protected from GC, which can be confusing. AFAIK, they are not used for anything else in SLIME.
as long as they're in the REPL shouldn't the presentation keep an extra reference to them?
This is an alternative model of using presentations, which could be a useful user option in my opinion. It has not frequently happened to me, though, that objects that I wanted to access were already garbage collected.
Perhaps a better way to deal with this is to have slime indicate what objects its holding and/or clear this buffer when it stops using them. This might better belong in a per-connection thing, and I wanted some advise on this before I started hacking on this.
In current SLIME, presentations can be cut and pasted arbitrarily, so checking which presentations are "alive" from the Emacs side can be complicated. A possible way would be using weak hashtables on the Emacs side. Then frequently this hashtable could be scanned, and dead references removed from the Lisp-side strong hashtable. Code that accomplishes this would be a welcome addition, which I would be happy to merge into the SLIME presentations contrib. As I said above, this would be an alternative model. We should support both models of operation, and the user should be able to pick one as a user option. Matthias -- Matthias Köppe -- http://www.math.uni-magdeburg.de/~mkoeppe
Matthias Koeppe write:
"Geo Carncross" <geocar@gmail.com> writes:
I've been looking over adding weak hashtable support to ECL, when I took a closer look at slime's use of them.
Weak hashtables are a useful feature, so if you want to add them to ECL, I would think this would be a good idea.
I word of warning: Weak hashtables (and weak pointers in general) have significant implications for the garbage collector. Even after many attempts to fix the CMUCL garbage collector, the use of weak hashtables still causes problems.
On Feb 4, 2008 5:53 PM, Lynn Quam <quam@ai.sri.com> wrote:
I word of warning:
Weak hashtables (and weak pointers in general) have significant implications for the garbage collector. Even after many attempts to fix the CMUCL garbage collector, the use of weak hashtables still causes problems.
ECL uses the bohem collector presently. Making weak hashtables basically involves adding a GC_register_disappearing_link() wipe out the cell when it finalizes and store a disguised pointer (say, xor ~0 or something) instead of the original pointer. If ECL ever got a precise collector (again) this would be a different story.
participants (4)
-
Geo Carncross -
Helmut Eller -
Lynn Quam -
Matthias Koeppe