Hi,
I made a quick hack that allows the Slime listener to use Lisp objects that were returned from the Common Lisp environment. We already can do this using *, **, and *** variables but, to me, is not enough. I have an hard time remembering the number of stars that I should write and, moreover, I prefer to 'point' to a value that is presented in the listener and I use it in the construction of next expression, just like in Symbolics Lisp machines. For those who never saw the Symbolics environment, let me give you a description of what I can do with this patch to Slime:
CL-USER> (cons 1 2) (1 . 2) ;;this is just like in good old Slime CL-USER> (cons (1 . 2) (1 . 2)) ;; something new: I wrote "(cons " ;;and then I moved the cursor to the previous line and pressed return. ;;The previous output is copied to the current prompt but, in fact, it's ;;not only text. It also represents the previous value returned by the ;;environment. In this example I did the same operation twice and then I wrote the closing ")". The result, as expected, is: ((1 . 2) 1 . 2) CL-USER> (eq (car ((1 . 2) 1 . 2)) (cdr ((1 . 2) 1 . 2))) ;;again, I wrote "(eq (car " and then I copied the previous output (using return). Then, I wrote ") (cdr " and I copied again the previous output. After finishing the form (with "))"), the environment returns: T ;;as expected.
You cannot see it in this email (I presume) but I also mark the reused values in a different color and I make then read-only (in Emacs) so that you can see that they are a different beast.
Another interesting extension is to directly inspect any returned value. I changed the inspect command so that, whenever the cursor is over a returned value, C-cC-I automatically inspects it.
There are lots of other things that we can do like discovering the types of the presented values and restrict their use for certain operations. For example, if there's a function that asks the user for a number, we can allow him to reuse old values as long as they are of type number. In short, this is just a quick hack to provide a stripped down version of presentation types that works with emacs. I guess that the edit-value feature of Slime could also be rewritten to take advantage of this stuff.
Now, it should be obvious that I am keeping all the returned values in the Common Lisp side for further use. This means that they are not GCed. So, from time to time, it might be wise to clean the Listener (just like before, with C-cC-t) or just the previous output (with C-cC-o). This also releases the corresponding objects from the Common Lisp side.
I suggest you try it out.
If the Slime team thinks this is useful then I suggest that you take a good look at the code (as you usually do :-) and refactor it radically because I did a lot of copy&paste just to test the idea. And it might also contains lots of bugs :-)
There are also lots of improvements that can be done like not storing the values that don't require identity preservation (like small numbers, and characters), like providing the functions that allow the type of a value to be returned (e.g., to provide a menu of operations for the object, etc), like allowing parts of lists to be reuse (this one seems easy but requires different conventions for the gestures needed for reusing previous values), etc, etc.
Best regards,
António Leitão.
Here's the patch:
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
I suggest you try it out.
yummy! (i have been meaning to do this for a looooong time. thank you very much for doing for me (and doing it better than i probably would have)).
there's one minor issue i ran into: i can't delete the text inserted by hitting RET on a previous result.
"Marco Baringer" mb@bese.it writes:
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
I suggest you try it out.
yummy! (i have been meaning to do this for a looooong time. thank you very much for doing for me (and doing it better than i probably would have)).
there's one minor issue i ran into: i can't delete the text inserted by hitting RET on a previous result.
Humm, my intention was to prevent changes on the copied text (so that we would know that it represented an object) but it's obvious we should allow for removing it. I'll look into Emacs documentation to see what kind of text property allows that.
Thanks,
António Leitão.
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
"Marco Baringer" mb@bese.it writes:
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
I suggest you try it out.
yummy! (i have been meaning to do this for a looooong time. thank you very much for doing for me (and doing it better than i probably would have)).
there's one minor issue i ran into: i can't delete the text inserted by hitting RET on a previous result.
Humm, my intention was to prevent changes on the copied text (so that we would know that it represented an object) but it's obvious we should allow for removing it. I'll look into Emacs documentation to see what kind of text property allows that.
Maybe an Emacs expert can help here: is there a simple way to mark a region so that we can't erase its contents but we can erase the entire region? I mean: either we have the entire region or we don't have it at all.
The intangible property looks interesting but it's not exactly the same thing (and it doesn't seem enough).
Thanks in advance,
António Leitão.
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
Hi,
I made a quick hack that allows the Slime listener to use Lisp objects that were returned from the Common Lisp environment. We already can do this using *, **, and *** variables but, to me, is not enough. I have an hard time remembering the number of stars that I should write and, moreover, I prefer to 'point' to a value that is presented in the listener and I use it in the construction of next expression, just like in Symbolics Lisp machines. For those who never saw the Symbolics environment, let me give you a description of what I can do with this patch to Slime:
ok, i just commited this with the following changes:
1) we don't attempt to add a slime-repl-result-face property if there's no result in slime-repl-insert-prompt.
2) renamed *current-id* to the (imho) more emacs-ish slime-current-output-id and added a defvar so repl shortucts don't break.
3) inserting a previous output now moves point past the inserted text (previously the pint was being left in the middle of the text).
4) inserted output is still undeleteable :( i tried messing with the intagible and modification-hooks properties but don't have enough emacs-foo to figure this out.
5) attempting (via M-p or similar) to access an output which has been cleared (via C-c C-t or similar) signals a error with a message explaining what's happened.
Well, this isn't exactly elegant, but I think it has the desired behavior of making the thing act as an unmodifiable unit. A command that deletes, executed in the middle of the unit deletes the whole unit as does forward delete at the beginning and backward delete at the end. Insertion commands in the middle don't do anything.
Might need to add some more 'action-type properties. Beware that if an error occurs in the command hook, it is removed. So if it stops working then that's what happened.
(defun slime-setup-command-hooks () "Setup a buffer-local `pre-command-hook' to call `slime-pre-command-hook'." (make-local-hook 'pre-command-hook) (make-local-hook 'post-command-hook) (add-hook 'pre-command-hook 'slime-pre-command-hook) (add-hook 'post-command-hook 'slime-post-command-hook) (add-hook 'pre-command-hook 'slime-presentation-command-hook)) ;; this is the new one
(defun slime-presentation-command-hook () (let* ((props-here (text-properties-at (point))) (props-before (text-properties-at (1- (point)))) (inside (and (getf props-here 'slime-repl-old-output))) (at-beginning (and inside (not (getf props-before 'slime-repl-old-output)))) (at-end (and (or (= (point) (point-max)) (not (getf props-here 'slime-repl-old-output))) (getf props-before 'slime-repl-old-output))) (start (cond (at-beginning (point)) (inside (previous-single-property-change (point) 'slime-repl-old-output)) (at-end (previous-single-property-change (1- (point)) 'slime-repl-old-output)))) (end (cond (at-beginning (or (next-single-property-change (point) 'slime-repl-old-output) (point-max))) (inside (or (next-single-property-change (point) 'slime-repl-old-output) (point-max))) (at-end (point))))) (when (and (or inside at-end) start end (> end start)) (let ((kind (get this-command 'action-type))) (cond ((and (eq kind 'inserts) inside (not at-beginning)) (setq this-command 'ignore-event)) ((and (eq kind 'deletes-forward) inside (not at-end)) (kill-region start end) (setq this-command 'ignore-event)) ((and (eq kind 'deletes-backward) (or inside at-end)) (kill-region start end) (setq this-command 'ignore-event)))))))
(put 'self-insert-command 'action-type 'inserts) (put 'yank 'action-type 'inserts) (put 'kill-word 'action-type 'deletes-forward) (put 'delete-char 'action-type 'deletes-forward) (put 'kill-sexp 'action-type 'deletes-forward) (put 'backward-kill-sexp 'action-type 'deletes-backward) (put 'backward-delete-char 'action-type 'deletes-backward) (put 'backward-kill-word 'action-type 'deletes-backward) (put 'backward-delete-char-untabify 'action-type 'deletes-backward) (put 'slime-repl-newline-and-indent 'action-type 'inserts)
On May 18, 2005, at 6:19 AM, Marco Baringer wrote:
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
Hi,
I made a quick hack that allows the Slime listener to use Lisp objects that were returned from the Common Lisp environment. We already can do this using *, **, and *** variables but, to me, is not enough. I have an hard time remembering the number of stars that I should write and, moreover, I prefer to 'point' to a value that is presented in the listener and I use it in the construction of next expression, just like in Symbolics Lisp machines. For those who never saw the Symbolics environment, let me give you a description of what I can do with this patch to Slime:
ok, i just commited this with the following changes:
we don't attempt to add a slime-repl-result-face property if there's no result in slime-repl-insert-prompt.
renamed *current-id* to the (imho) more emacs-ish slime-current-output-id and added a defvar so repl shortucts don't break.
inserting a previous output now moves point past the inserted text (previously the pint was being left in the middle of the text).
inserted output is still undeleteable :( i tried messing with the intagible and modification-hooks properties but don't have enough emacs-foo to figure this out.
attempting (via M-p or similar) to access an output which has been cleared (via C-c C-t or similar) signals a error with a message explaining what's happened.
-- -Marco Ring the bells that still can ring. Forget the perfect offering. There is a crack in everything. That's how the light gets in. -Leonard Cohen
slime-devel site list slime-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/slime-devel
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
I made a quick hack that allows the Slime listener to use Lisp objects that were returned from the Common Lisp environment.
Very nice :-)
Now, it should be obvious that I am keeping all the returned values in the Common Lisp side for further use. This means that they are not GCed. So, from time to time, it might be wise to clean the Listener (just like before, with C-cC-t) or just the previous output (with C-cC-o). This also releases the corresponding objects from the Common Lisp side.
I added a swank:*record-repl-results* variable so that people can turn this off if they want to. I'm a little worried about the potential for astonishment ("why isn't GC working?!") but not sure what the best solution is.
If the Slime team thinks this is useful then I suggest that you take a good look at the code (as you usually do :-) and refactor it radically because I did a lot of copy&paste just to test the idea. And it might also contains lots of bugs :-)
I get it: generate some code and then let the Emacs-monkeys worry about maintaining it. :-) We'll let you off this time because it's an irresistible feature. I hacked it a little bit after Marco.
Cheers, Luke
Luke Gorrie luke@synap.se writes:
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
I made a quick hack that allows the Slime listener to use Lisp objects that were returned from the Common Lisp environment.
Very nice :-)
Now, it should be obvious that I am keeping all the returned values in the Common Lisp side for further use. This means that they are not GCed. So, from time to time, it might be wise to clean the Listener (just like before, with C-cC-t) or just the previous output (with C-cC-o). This also releases the corresponding objects from the Common Lisp side.
I added a swank:*record-repl-results* variable so that people can turn this off if they want to. I'm a little worried about the potential for astonishment ("why isn't GC working?!") but not sure what the best solution is.
Tell people that everything that is visible on the screen is still reachable? Now, our eyes are part of the Common Lisp environment :-)
If the Slime team thinks this is useful then I suggest that you take a good look at the code (as you usually do :-) and refactor it radically because I did a lot of copy&paste just to test the idea. And it might also contains lots of bugs :-)
I get it: generate some code and then let the Emacs-monkeys worry about maintaining it. :-) ...
That was not my idea. I'll help maintaining it, obviously, but during its development I noticed that I was doing lots of things that I knew a Slime expert (and not an Emacs-monkey) would do much better. Just as an example, I'm reusing the continuation id to mark the saved result. I'm not sure this is the best solution. Also, this id is made available from the 'eval-for-emacs' function to the listener-eval function using a dynamic variable and this doesn't look very nice. The same problem happens in the Emacs side, where I left a comment:
(let ((slime-current-output-id id)) ;; this is not very ;; elegant but it avoids changing the protocol
Also, the slime-repl-grab-old-output is an almost exact copy of slime-repl-grab-old-input. Ugly, obviously.
I'll try to improve the code a bit.
Best regards,
António Leitão.
On Thu, 19 May 2005, Antonio Menezes Leitao wrote:
Tell people that everything that is visible on the screen is still reachable? Now, our eyes are part of the Common Lisp environment :-)
presentation-buffer-size?
nil for whole buffer, 0 ... N for a cyclic buffer of N elements.
Or wrap the objects in weak pointers on platforms that have them?
Cheers,
-- Nikodemus Schemer: "Buddha is small, clean, and serious." Lispnik: "Buddha is big, has hairy armpits, and laughs."
PS. This is probably familiar to most, but still seems appropriate:
=== ALL USERS PLEASE NOTE ========================
The garbage collector now works. In addition a new, experimental garbage collection algorithm has been installed. With SI:%DSK-GC-QLX-BITS set to 17, (NOT the default) the old garbage collection algorithm remains in force; when virtual storage is filled, the machine cold boots itself. With SI:%DSK-GC- QLX-BITS set to 23, the new garbage collector is enabled. Unlike most garbage collectors, the new gc starts its mark phase from the mind of the user, rather than from the obarray. This allows the garbage collection of significantly more Qs. As the garbage collector runs, it may ask you something like "Do you remember what SI:RDTBL-TRANS does?", and if you can't give a reasonable answer in thirty seconds, the symbol becomes a candidate for GCing. The variable SI:%GC-QLX-LUSER-TM governs how long the GC waits before timing out the user.
Cool. I added my command hook stuff so it is no longer read-only and made the thing mouse sensitive - mouse-2 copies it to point.
-Alan
On May 18, 2005, at 10:23 PM, Luke Gorrie wrote:
Antonio Menezes Leitao aml@gia.ist.utl.pt writes:
I made a quick hack that allows the Slime listener to use Lisp objects that were returned from the Common Lisp environment.
Very nice :-)
Now, it should be obvious that I am keeping all the returned values in the Common Lisp side for further use. This means that they are not GCed. So, from time to time, it might be wise to clean the Listener (just like before, with C-cC-t) or just the previous output (with C-cC-o). This also releases the corresponding objects from the Common Lisp side.
I added a swank:*record-repl-results* variable so that people can turn this off if they want to. I'm a little worried about the potential for astonishment ("why isn't GC working?!") but not sure what the best solution is.
If the Slime team thinks this is useful then I suggest that you take a good look at the code (as you usually do :-) and refactor it radically because I did a lot of copy&paste just to test the idea. And it might also contains lots of bugs :-)
I get it: generate some code and then let the Emacs-monkeys worry about maintaining it. :-) We'll let you off this time because it's an irresistible feature. I hacked it a little bit after Marco.
Cheers, Luke
slime-devel site list slime-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/slime-devel
I have committed some changes that I'm a little insecure about, so there is a bit of a hurdle to use them. Feedback desired. Essentially I've extended the recent repl output mechanism to arbitrary objects you choose to print that way.
e.g.
(swank::presenting-object object stream (print "This is really the object")).
This makes the string "This is really the object" behave like old repl input for the object.
As an example, I've hooked that into the printing of unreadable objects in openmcl and cmucl. So all unreadable objects are now mouse sensitive and can be copied and used in expressions in the repl.
To enable this stuff you need to explicitly load "present.lisp". Then try (describe 'class) and mouse over some of the #<things>.
This uses the ilisp bridge.el mechanism to embed messages in the output stream indicating the start and and of the objects so printed and associates an id with them. The downside is that if I've done things incorrectly you could see these characters when you don't want to.
To turn off the loading of the bridge, customize slime-repl-enable-presentations and turn it off.
-Alan
Alan Ruttenberg alanr-l@mumble.net writes:
I have committed some changes that I'm a little insecure about, so there is a bit of a hurdle to use them. Feedback desired.
At first glance this looks totally ultra-cool :-) I often wondered if we would end up at this point one day.
For now I have just hacked the code a bit to disable the feature on XEmacs (because it's not portable to there currently) and some 80-column'ification (sorry but it fits better with the rest :-)
Cheers, Luke
On Fri, 20 May 2005 21:23:36 +0200, Luke Gorrie wrote:
Alan Ruttenberg alanr-l@mumble.net writes:
I have committed some changes that I'm a little insecure about, so there is a bit of a hurdle to use them. Feedback desired.
At first glance this looks totally ultra-cool :-) I often wondered if we would end up at this point one day.
For now I have just hacked the code a bit to disable the feature on XEmacs (because it's not portable to there currently) and some 80-column'ification (sorry but it fits better with the rest :-)
Cheers, Luke
Hmm, since this latest update i get the following:
existing result (number -17). [Condition of type SIMPLE-ERROR]
Restarts: 0: [ABORT] Abort handling SLIME request. 1: [ABORT] Exit debugger, returning to top level.
Backtrace: 0: (SWANK:GET-REPL-RESULT -17) 1: (SB-INT:EVAL-IN-LEXENV (SWANK:GET-REPL-RESULT -17) #<NULL-LEXENV>) 2: (SB-IMPL::READ-MAYBE-NOTHING #<SB-IMPL::STRING-INPUT-STREAM {409BDA81}> ##)
Cheers Ralf Mattes
I think that's my mistake if you don't load present.lisp. Is that the case? I will fix later as I need to run out right now. -Alan (actually I will fix whether or not that is the problem :)
On May 20, 2005, at 7:13 PM, R. Mattes wrote:
On Fri, 20 May 2005 21:23:36 +0200, Luke Gorrie wrote:
Alan Ruttenberg alanr-l@mumble.net writes:
I have committed some changes that I'm a little insecure about, so there is a bit of a hurdle to use them. Feedback desired.
At first glance this looks totally ultra-cool :-) I often wondered if we would end up at this point one day.
For now I have just hacked the code a bit to disable the feature on XEmacs (because it's not portable to there currently) and some 80-column'ification (sorry but it fits better with the rest :-)
Cheers, Luke
Hmm, since this latest update i get the following:
existing result (number -17). [Condition of type SIMPLE-ERROR]
Restarts: 0: [ABORT] Abort handling SLIME request. 1: [ABORT] Exit debugger, returning to top level.
Backtrace: 0: (SWANK:GET-REPL-RESULT -17) 1: (SB-INT:EVAL-IN-LEXENV (SWANK:GET-REPL-RESULT -17) #<NULL-LEXENV>) 2: (SB-IMPL::READ-MAYBE-NOTHING #<SB-IMPL::STRING-INPUT-STREAM {409BDA81}> ##)
Cheers Ralf Mattes
slime-devel site list slime-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/slime-devel
On Fri, 20 May 2005 20:02:45 -0400, Alan Ruttenberg wrote:
I think that's my mistake if you don't load present.lisp. Is that the case? I will fix later as I need to run out right now. -Alan (actually I will fix whether or not that is the problem :)
Thanks, that worked!
This starts to be really impressive - and actually quite useful: i often do queries at the repl that return non-printable objects. Being able to point-and-click those to use them for further work helps a lot.
Thanks Ralf Mattes
Alan Ruttenberg alanr-l@mumble.net writes:
I have committed some changes that I'm a little insecure about, so there is a bit of a hurdle to use them. Feedback desired. Essentially I've extended the recent repl output mechanism to arbitrary objects you choose to print that way.
"In the long run every program becomes rococo - then rubble." -- Alan Perlis
(defun slime-open-stream-to-lisp (port) (let ((stream (open-network-stream "*lisp-output-stream*" (slime-with-connection-buffer () @@ -2539,9 +2574,19 @@ (when slime-kill-without-query-p (process-kill-without-query stream)) (set-process-filter stream 'slime-output-filter)
- (set-process-coding-system stream
slime-net-coding-systemslime-net-coding-system)
- (when slime-repl-enable-presentations
(require 'bridge)(defun bridge-insert (process output)(slime-output-filter process (or output "")))(install-bridge)(setq bridge-destination-insert nil)(setq bridge-source-insert nil)(setq bridge-handlers (list* '("<" . slime-mark-presentation-start)'(">" . slime-mark-presentation-end)bridge-handlers))(set-process-coding-system streamslime-net-coding-systemslime-net-coding-system))
This doesn't look like the right place to do it. slime-open-stream-to-lisp is only called if the dedicated output stream is used. If the dedicated stream is disabled the markers will not be scanned. You could do the scanning in slime-output-string.
The more fundamental question is why you use a special protocol. Why can't you use the existing infrastructure and send the string with the id as a list like we do in the inspector?
Helmut.
The more fundamental question is why you use a special protocol. Why can't you use the existing infrastructure and send the string with the id as a list like we do in the inspector?
Helmut.
Ah, you are the one I should be chicken of, not Luke :) (see my checkin comments)
I wish I could figure out a better way, but I haven't yet. Maybe you all could help me.
The reason that I think I can't is because rather than the in the case of the repl result, or the backtrace, we are expecting the thing being passed back to have some specific structure, at a specific time. On the other hand presentations are intermingled with arbitrary output, only a portion of which I have control over.
So take the case of hooking into print-unreadable-object. You don't get to choose when it is called. So when it is called and you want to make something sensitive you need to signal that somehow.
The issue here is synchronization. Let's suppose that we were writing to a stream straight to lisp output. Theoretically when we want to print something sensitive we could flush the stream, force the output event to be sent to emacs, then send a different kind of event for the sensitive object, and then resume output. But I don't know if that's possible when using the dedicated stream, and I didn't spend time looking at the non-dedicated stream case to see if it was possible there, since I didn't think it was an option to avoid use of the dedicated stream. (there might be performance issues as well...)
But there are other problems because during pprint you often are not writing directly to the final destination stream. As I understand it there is buffering in order to decide layout, so you don't really know, when you are about to start printing a sensitive object, how much text is buffered up. Thus you would have to be able to leave some synchronization mark, I think, which this strategy amounts to.
One could imagine intervening on the lisp side - catching the markers there at the stream-tyo level, stripping them, and then sending a separate message to slime about where the boundaries are. Here you seem to have a similar opportunity to handle the synchronization. But I don't think this makes it any cleaner, it just shifts the messiness to the lisp side.
I think the clean alternative is to essentially take over printing output in a more pervasive way and have it be aware of the fact that output is presentation based and not text based. But I also think that it is very intrusive and time consuming to do it. It is, as I understand it, what is done on lispms, and in Mcclim. It is possible that the Mcclim listener code might be adapted - I haven't looked into this. There are issues with the current implementation, like what to do about nested sensitive objects (a list presentation that contains a number of element presentations), that are better addressed in a broader framework.
So the bottom line is that I just can't figure out a better way to do it. And I'd rather do it messy than not do it. But I'd rather do it clean than messy, so maybe you and others can help by suggesting a nicer way of implementing this.
This doesn't look like the right place to do it. slime-open-stream-to-lisp is only called if the dedicated output stream is used. If the dedicated stream is disabled the markers will not be scanned. You could do the scanning in slime-output-string.
Yes, I thought of this. The issue that needs to be handled is that, as I understand it, you don't know where your string is going to be broken when calling slime output. So you have to manage remembering that you are in the middle of a potential marker regular expression and holding on to previous output until a subsequent call resolves that you have or have not seen a marker. This is what the bridge does. It is a non trivial amount of work so I decided to use their code for a first pass, so we could see if the idea even felt ok. But I didn't realize about the non-dedicated stream case. I'm not averse to making this work, though I think I'd wait to make sure some other showstopper doesn't show up so I don't waste my time.
Looking forward to your future thoughts and comments,
-Alan
-
Alan Ruttenberg -
Antonio Menezes Leitao -
Helmut Eller -
Luke Gorrie -
Marco Baringer -
Nikodemus Siivola -
R. Mattes