Having used the clim listener, I wondered if it would be possible to make things in the SLIME repl clicky in the same way that they are there.
It turns out that this isn't too much of a problem, though the regexp horror was almost too much. I present herewith a minimal first example of use. Probably only works in SBCL, since that's what the regexp has been tested with, I don't know when the 'mouse-face text property was introduced in Emacs (or if it even exists in XEmacs), so, you might lose there, and I don't think that the 'keymap text property works in XEmacs.
How it works:
When the repl receives some output, it gets passed through SLIME-MAKE-CLICKY-THINGS, which looks for stuff of the form:
#<name something>
where something can have spaces in it. Then, depending on what NAME is, the relevant bit of the output is propertized, and inserted. Currently, it handles PACKAGE, FUNCTION, and foo-CLASS, the last is assumed to be a class of some kind.
A local keymap is installed, which is accessible via mouse-3, and allows you to do things like describe the thing, or see it's documentation, etc...
This requires one minor patch to swank.lisp, defining a DESCRIBE-PACKAGE function.
Index: swank.lisp =================================================================== RCS file: /project/slime/cvsroot/slime/swank.lisp,v retrieving revision 1.79 diff -u -r1.79 swank.lisp --- swank.lisp 14 Dec 2003 07:52:31 -0000 1.79 +++ swank.lisp 14 Dec 2003 21:44:31 -0000 @@ -724,6 +724,10 @@ (print-description-to-string (symbol-function (find-symbol-designator symbol-name))))
+(defslimefun describe-package (symbol-name) + (print-description-to-string + (find-package symbol-name))) + (defslimefun documentation-symbol (symbol-name &optional default) (let ((*package* *buffer-package*)) (let ((vdoc (documentation (symbol-from-string symbol-name) 'variable))
And the file itself:
lawrence mitchell wence@gmx.li writes:
Having used the clim listener, I wondered if it would be possible to make things in the SLIME repl clicky in the same way that they are there.
This is a very cool idea :-) I think a nice CLIM'alike could be built on Emacs/SLIME.
Do you know how CLIM associates types with outputs -- some kind of print-method specialization on magic streams or something? My feeling is that using regexps to match unreadable objects isn't a winning strategy.
-Luke
Luke Gorrie writes:
Do you know how CLIM associates types with outputs -- some kind of print-method specialization on magic streams or something? My feeling
I guess it uses presentations.
Paolo
Luke Gorrie wrote:
lawrence mitchell wence@gmx.li writes:
Having used the clim listener, I wondered if it would be possible to make things in the SLIME repl clicky in the same way that they are there.
This is a very cool idea :-) I think a nice CLIM'alike could be built on Emacs/SLIME.
Do you know how CLIM associates types with outputs -- some kind of print-method specialization on magic streams or something? My feeling is that using regexps to match unreadable objects isn't a winning strategy.
I don't think so either :P. However, it was the quickest way I could see of associating unreadable objects with descriptions, short of majorly hacking what slime sends back to Emacs.
I suppose one way of doing it would be for the Lisp to send a lot /more/ information back about its output, i.e., each time it prints an unreadable object, it could add some extra output, e.g. for classes, we could pass through CLASS-NAME.
I'm rather scrabbling in the dark here, since Emacs doesn't have as much introspection capability in the SLIME repl as CLIM does.
When I get time, probably not for two weeks or so, I shall try to look at the way CLIM does it, and see if it's possible to steal the idea and adapt it.
Meanwhile, I'll quote from Jamie Zawinski's gdb-highlight.el:
;; This code should be considered an example of how over-use of regular ;; expressions leads to code that is an unreadable, unmaintainable mess, ;; and why it's unfortunate that so much of emacs's speed depends on ;; their use, rather than on the use of more traditional parsers.
And try not to follow suite :P
lawrence mitchell wence@gmx.li writes:
Meanwhile, I'll quote from Jamie Zawinski's gdb-highlight.el:
;; This code should be considered an example of how over-use of regular ;; expressions leads to code that is an unreadable, unmaintainable mess, ;; and why it's unfortunate that so much of emacs's speed depends on ;; their use, rather than on the use of more traditional parsers.
And try not to follow suite :P
He's said it better:
Sometimes a programmer, when faced with a problem, thinks "I know, I'll solve it with a regular expression". Now he has two problems.
A wise man. But if I ever meet him, I'm gonna punch him in the nose for all the time I've spent porting stuff to XEmacs ;-)
Cheers, Luke
Luke Gorrie wrote:
[...] Clicky things
A wise man. But if I ever meet him, I'm gonna punch him in the nose for all the time I've spent porting stuff to XEmacs ;-)
Hmmm, yes....
I was thinking about how to not have to use regexps on the train. How about this:
Currently, slime get's a form "(:ok "#<FUNCTION FOO>")", and READs it on the Emacs side before parsing it and displaying it to the user.
I thought one might be able to, on the swank side, specialise PRINT-OBJECT, or PRINT-UNREADABLE-OBJECT to add some extra information, perhaps something of the form:
"(:ok (:unreadable "#<FUNCTION FOO>" :type 'function :name 'foo))"
Then, Emacs could have more information about said object, and, when parsing and inserting it in the REPL, could add text properties appropriately.
The way I envisage this happening is to have dispatching in the printer on object types to add extra info.
Something like:
(defgeneric object-info (object))
(defmethod object-info ((object function)) (do-stuff-to-extract-info))
(defmethod object-info ((object pathname)) ...)
This has the advantage that one doesn't need to use regexps on the Emacs side to parse stuff (when most of the introspection capabilities are already gone), and one can add object-info for arbitrary types, e.g. CONSes or something.
A very quick glance at the CLIM listener's presentations approach makes me think that it does it a similar way. Perhaps I should ask Andy Hefner.
Does this sound at all workable, it made sense on paper, whether or not it would work might be a different matter. Comments?
Lawrence
lawrence mitchell wence@gmx.li writes:
I thought one might be able to, on the swank side, specialise PRINT-OBJECT, or PRINT-UNREADABLE-OBJECT to add some extra information, perhaps something of the form:
"(:ok (:unreadable "#<FUNCTION FOO>" :type 'function :name 'foo))"
Just as an idea: you could use Emacs's syntax for strings with text-properties (which is similar to CL's syntax for arrays :-). The example would then look like
#("#<FUNCTION FOO>" 0 15 (:type function :name foo))
But I note that we use :ok messages only for the result of the evaluation and not for the redirected output to streams. For the redirected output we use either a gray stream that sends :read-output events or a dedicated socket connection with a simple process filter on the Emacs side. This can be configured with swank::*use-dedicated-output-stream*. A specialized gray stream with some extra events is probably what you need. But the dedicated socket connection is faster and Emacs remains responsive if there is a lot of output.
Helmut.
On Tue, Dec 16, 2003 at 09:51:52PM +0000, lawrence mitchell wrote:
This has the advantage that one doesn't need to use regexps on the Emacs side to parse stuff (when most of the introspection capabilities are already gone), and one can add object-info for arbitrary types, e.g. CONSes or something.
One question I would have is, how is the object accessed once it's clicked on? I imagine CLIM just holds a (weak?) pointer to the object in its presentation. From emacs, you may see "#<FOO {40541D01}>", and even know the address more concretely if its passed in some sort of object-info, but as soon as GC happens, you're likely screwed.
One probably wants some sort of mapping in SWANK of all things that have been sent with a "slime-presentation". Only problem is, that you probably want some sort of weak hash table so that you don't prevent the objects from being GC'd. Which of SLIME's lisps support weak keys/values in hash tables?
-bcd
Brian Downing bdowning@lavos.net writes:
One question I would have is, how is the object accessed once it's clicked on? I imagine CLIM just holds a (weak?) pointer to the object in its presentation. From emacs, you may see "#<FOO {40541D01}>", and even know the address more concretely if its passed in some sort of object-info, but as soon as GC happens, you're likely screwed.
If CLIM used weak references, it would have the same trouble. I would guess they use normal references and normal GC, but I don't know.
In Emacs it seems we get into distributed garbage collection, though probably a simple case if only Emacs can refer to Lisp and not vice-versa.
I guess Emacs would want to tag the textual representations of objects with a "type" property and an "object-reference" property. Text properties would do this nicely -- you could copy&paste objects while preserving the references. But then to find the "live set" of object-references in Emacs you would probably have to scan all buffers. Overlays could be simpler, since they are properties of a buffer region rather than text, and don't get copy&pasted or otherwise duplicated.
BUT,
How about integrating SLIME with McCLIM instead? For example, it would be nice to type "Edit Definition foo" in the McCLIM listener, and have an Emacs window pop up with the definition.
McCLIM is one of the coolest things happening. It would be great to join forces with them where possible.
Cheers, Luke
Luke Gorrie luke@bluetail.com writes:
If CLIM used weak references, it would have the same trouble. I would guess they use normal references and normal GC, but I don't know.
In Emacs it seems we get into distributed garbage collection, though probably a simple case if only Emacs can refer to Lisp and not vice-versa.
what you could do is resort to double indirection. i.e. have the Lisp side assign indices to the printed objects it gives to Emacs, and have Emacs use those indices when the objects are clicked on. this way you can simply decide to keep only the last two thousand printed objects or something, no weak hash tables needed. and when Emacs asks for an object whose index is too old, Lisp can unambiguously detect this and react politely.
How about integrating SLIME with McCLIM instead? For example, it would be nice to type "Edit Definition foo" in the McCLIM listener, and have an Emacs window pop up with the definition.
just as long as you don't slide into writing an Emacs backend for McCLIM, I guess. :)
just kibitzing, --m
On Wed, Dec 17, 2003 at 04:22:13PM +0100, Luke Gorrie wrote:
Brian Downing bdowning@lavos.net writes:
One question I would have is, how is the object accessed once it's clicked on? I imagine CLIM just holds a (weak?) pointer to the object in its presentation. From emacs, you may see "#<FOO {40541D01}>", and even know the address more concretely if its passed in some sort of object-info, but as soon as GC happens, you're likely screwed.
If CLIM used weak references, it would have the same trouble. I would guess they use normal references and normal GC, but I don't know.
Not the way I was thinking about it. What I meant is that the GC would probably move the object, not collect it, and if your reference to it was based on address, it would be broken even though the object still existed.
Weak references make sense to me, even for CLIM, because I really wouldn't want all the objects I've created in the REPL to persist just because their presentations still exist in my scrollback. That completely changes the semantics of working at the REPL. Maybe somebody who has worked on the LispM could explain what semantics presentations in its listener had with respect to garbage collection.
If I print a piece of data that is part of a larger structure, and not garbage, I'd want the presentation to still be able to access it after it has been moved by the GC. But I'd also like to be able to do (progn (print (make-instance 'two-hundred-megabyte-object)) nil) at the REPL and not saddle the GC with any limitations on how to collect this data that nobody else is using, like waiting for it to "time out" or having another 200 presentations created after it.
All IMHO, of course.
-bcd
I implemented something along these lines for MCL a while back and have been thinking of porting it to openmcl. I ran into a number of issues that you've been discussing so I thought I'd share my decisions of the time.
My implementation was text based, relying on a reader macro to be able to read "unreadable" objects. Since I wasn't using ascii i used #<= (where the <= was one character) so that the things printed looked almost the same as they would normally. I also had a mouse click that would copy a pointed to lisp object (as text) to the input line. So you could C-click on one of these then hit return in the listener and voila - you had your object. I was thinking of using #<^ as the prefix where I have to stick to ascii. So one of these objects looks like this: #<^buffer-mark #x77FF3E>. One of the advantages of working with just text as the representation is that the method still worked when people would print things to strings before subsequently printing them to their final destination stream (does pretty-printing do this).
I implemented this for any unreadable object that used :identity t, meaning that it printed out its address as part of the representation and hooked into the system by overriding the implementation of print-unreadable-object. In addition, I redefined a number of print-object methods to ensure that they used print-unreadable-object and :identity t (instead of (format "#<..." ...)).
The address was used as a key to the object in a weak hash table. If you used one of these things after its object had gced then it evaluated to :already-gced. Conceivably you could control this behaviour (hold on to printed objects strongly or weakly) with a special variable.
If an object moves during gc then referencing by the old address was fine. The next time it printed it would be saved using the new address as a key. In the case where an address of some object was the same as where some previously printed object was printed, I changed the printed representation of the new object so it was unique, e.g. #<^binary-stream #x77FF3E-1>
Because stack objects are likely to be invalid soon, I used a low level call to check if the object I was about to print was stack allocated and didn't use this scheme if so.
Because people sometimes write print methods that have unreadable objects as part of their printed representation, the reader had to make sure it skipped over such things, e.g. #<^buffer-mark for #<^editor #x55DEFF> #x7788EE>
Of course you can do much more with presentations, but they are more work to implement. On the other hand it is conceivable that you could take the code in McClim and write a different backend for emacs and so benefit from their effort.
This doesn't work for arrays, structures, etc, or if the user writes their own print-object method that doesn't used print-unreadable-object.
On the other hand it is fairly simple to implement and helped a debugging immensely.
---
Looking forward I was thinking about other strategies. Here's one:
Wrap print-object with a method that makes a structures that have as one of their slots the textual representation that print-object would normally have written. Give these objects unique serial numbers (incrementing counter whenever one is printed). Keep a weak hash table mapping these ids to the original objects. Have the default print method simply spit out the stored textual representation. If you do this then I think everything looks the same as it does before.
Now make the print method for these things be dependent on a dynamic variable *print-clickably* or somesuch. When that variable is t the print method prints out something that when read evaluates to the object, using some newly defined reader macro. This thing looks something like #<^127>, for instance.
The process filter looks for these things and requests back to the lisp for the textual representation (on a dedicated stream/process) of the object, and prints a propertized version of that, which now can respond to clicks. Before passing text back to the lisp it finds these regions and substitutes back the #<^127> representation so they can be read and used.
I think this lets you read many more objects that my previous scheme could read, and gives the user some control, via *print-clickably*, over whether to use the mechanism. There might be some issues with pprint, if it does printing to measure how wide some object would be (underestimating the width).
-Alan
On Dec 17, 2003, at 2:25 PM, Brian Downing wrote:
On Wed, Dec 17, 2003 at 04:22:13PM +0100, Luke Gorrie wrote:
Brian Downing bdowning@lavos.net writes:
One question I would have is, how is the object accessed once it's clicked on? I imagine CLIM just holds a (weak?) pointer to the object in its presentation. From emacs, you may see "#<FOO {40541D01}>", and even know the address more concretely if its passed in some sort of object-info, but as soon as GC happens, you're likely screwed.
If CLIM used weak references, it would have the same trouble. I would guess they use normal references and normal GC, but I don't know.
Not the way I was thinking about it. What I meant is that the GC would probably move the object, not collect it, and if your reference to it was based on address, it would be broken even though the object still existed.
Weak references make sense to me, even for CLIM, because I really wouldn't want all the objects I've created in the REPL to persist just because their presentations still exist in my scrollback. That completely changes the semantics of working at the REPL. Maybe somebody who has worked on the LispM could explain what semantics presentations in its listener had with respect to garbage collection.
If I print a piece of data that is part of a larger structure, and not garbage, I'd want the presentation to still be able to access it after it has been moved by the GC. But I'd also like to be able to do (progn (print (make-instance 'two-hundred-megabyte-object)) nil) at the REPL and not saddle the GC with any limitations on how to collect this data that nobody else is using, like waiting for it to "time out" or having another 200 presentations created after it.
All IMHO, of course.
-bcd
slime-devel site list slime-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/slime-devel
On Wed, Dec 17, 2003 at 01:25:53PM -0600, Brian Downing wrote:
Weak references make sense to me, even for CLIM, because I really wouldn't want all the objects I've created in the REPL to persist just because their presentations still exist in my scrollback. That completely changes the semantics of working at the REPL. Maybe somebody who has worked on the LispM could explain what semantics presentations in its listener had with respect to garbage collection.
There is a story of a user nuking the readtable and then recovering it because several weeks earlier he had printed the value of the readtable. It was simply a matter of "scrolling up".
Groetjes, Peter
Lawrence Mitchell writes:
A very quick glance at the CLIM listener's presentations approach makes me think that it does it a similar way. Perhaps I should
If you are interested in the theory behind presentations, you may check this technical report:
Presentation Based User Interfaces - MIT AI TR 794 ftp://publications.ai.mit.edu/ai-publications/pdf/AITR-794.pdf
Paolo
-
Alan Ruttenberg -
Brian Downing -
Helmut Eller -
lawrence mitchell -
Luke Gorrie -
Michael Livshin -
Paolo Amoroso -
Peter Van Eynde