I have some concerns regarding #'swank:find-definitions-for-emacs, which is documented to "Return a list ((DSPEC LOCATION) ...) of definitions for NAME." MCLIDE uses this functionality for the button on its Apropos dialog that opens the definition for a symbol in the editor.
In slime-2009-12-18, #'swank:find-definitions-for-emacs gives a significantly different result depending on the lisp implementation. This leads to compatibility issues and complicates client side use of the result. Evaluate:
(swank:find-definitions-for-emacs "swank::find-definitions")
Clozure Common Lisp 1.4-r13119 returns:
(("#'SWANK-BACKEND:FIND-DEFINITIONS" (:LOCATION (:FILE "/Volumes/ Mirror/Users/Terje/DEV/MCLIDE_DEV/MCLIDE 1.0b1-2/MCLIDE.app/Contents/ Resources/Slime/swank-backend.lisp") (:POSITION 28699) (:SNIPPET "(definterface find-definitions (name) "))))
LispWorks 5.1.0 returns:
(("(DEFUN SWANK-BACKEND:FIND-DEFINITIONS)" (:LOCATION (:FILE "/ Volumes/Mirror/Users/Terje/DEV/MCLIDE_DEV/MCLIDE 1.0b1-2/MCLIDE.app/ Contents/Resources/Slime/swank-lispworks.lisp") (:FUNCTION-NAME "FIND- DEFINITIONS") NIL)) ("(DEFUN SWANK-BACKEND:FIND- DEFINITIONS)" (:LOCATION (:FILE "/Volumes/Mirror/Users/Terje/DEV/ MCLIDE_DEV/MCLIDE 1.0b1-2/MCLIDE.app/Contents/Resources/Slime/swank- backend.lisp") (:FUNCTION-NAME "FIND-DEFINITIONS") NIL)))
Of particular concern is the DSPEC, which Clozure returns as "#'SWANK- BACKEND:FIND-DEFINITIONS" and LispWorks returns as "(DEFUN SWANK- BACKEND:FIND-DEFINITIONS)". It would be helpful if the DSPEC standardized on the same symbol to denote that the definition is for a function, as in (FUNCTION SWANK-BACKEND:FIND-DEFINITIONS).
To facilitate client side use and flexibility in presentation, I also suggest that the DSPEC is returned as a list instead of as a string, with the type as a symbol and the function name encoded as a string, as in (FUNCTION "SWANK-BACKEND:FIND-DEFINITIONS").
-- Terje Norderhaug terje@in-progress.com
* Terje Norderhaug [2009-12-18 20:32+0100] writes:
[...]
Of particular concern is the DSPEC, which Clozure returns as "#'SWANK- BACKEND:FIND-DEFINITIONS" and LispWorks returns as "(DEFUN SWANK- BACKEND:FIND-DEFINITIONS)". It would be helpful if the DSPEC standardized on the same symbol to denote that the definition is for a function, as in (FUNCTION SWANK-BACKEND:FIND-DEFINITIONS).
To facilitate client side use and flexibility in presentation, I also suggest that the DSPEC is returned as a list instead of as a string, with the type as a symbol and the function name encoded as a string, as in (FUNCTION "SWANK-BACKEND:FIND-DEFINITIONS").
Assigning a standard meaning would be pretty difficult as implementations differ considerably in this area. E.g Allegro has fspecs while Lispworks has dspecs. Fspecs are for functions only but dspecs are indented for all kinds of definitions and new dspecs can be added by users. It's much simpler to treat dspecs as strings and just display them as label.
Helmut
On Dec 19, 2009, at 12:59 AM, Helmut Eller wrote:
- Terje Norderhaug [2009-12-18 20:32+0100] writes:
[...]
Of particular concern is the DSPEC, which Clozure returns as "#'SWANK- BACKEND:FIND-DEFINITIONS" and LispWorks returns as "(DEFUN SWANK- BACKEND:FIND-DEFINITIONS)". It would be helpful if the DSPEC standardized on the same symbol to denote that the definition is for a function, as in (FUNCTION SWANK-BACKEND:FIND-DEFINITIONS).
To facilitate client side use and flexibility in presentation, I also suggest that the DSPEC is returned as a list instead of as a string, with the type as a symbol and the function name encoded as a string, as in (FUNCTION "SWANK-BACKEND:FIND-DEFINITIONS").
Assigning a standard meaning would be pretty difficult as implementations differ considerably in this area. E.g Allegro has fspecs while Lispworks has dspecs. Fspecs are for functions only but dspecs are indented for all kinds of definitions and new dspecs can be added by users. It's much simpler to treat dspecs as strings and just display them as label.
MCLIDE needs to be able to filter definitions based on their type, as well as present the dspec in custom ways.
How can we make swank provide the information required for clients to make sense out of the specs? That is, not just display a pre-formated string, but being able to do filtering and other processing based on the dspecs, including custom presentation.
Perhaps the backend for the different implementations could provide clients with a dspec type hierarchy or mapping from canonical definition types to the type symbols used in their dspecs. That way, clients could determine that when LispWorks provides a dspec of type DEFUN and Clozure provides a dspec of type FUNCTION, both are dpecs for functions.
Take the dpecs in my original post, where each lisp implementation provided a different dspec in response to (swank:find-definitions-for- emacs "swank::find-definitions"):
Clozure => "#'SWANK-BACKEND:FIND-DEFINITIONS" LispWorks =>"(DEFUN SWANK-BACKEND:FIND-DEFINITIONS)"
Ignore the extra complication that Clozure reports (FUNCTION SWANK- BACKEND:FIND-DEFINITIONS) as #'SWANK-BACKEND:FIND-DEFINITIONS.
For a client to know that either of these dpecs are for function definitions, there could be a implementation specific backend function that declares the definition types provided by the implementation, e.g.:
(definterface definition-types () "Type hierarchy or map from canonical definition types to the type symbols used in their dspecs." nil)
Clozure:
(defimplementation definition-types () '((:function function) (:class class) ...))
LispWorks:
(defimplementation definition-types () '((:function defun) ...))
SBCL:
(defimplementation definition-types () '((:function defun) (:class defclass) (:optimizer :defoptimizer) ...))
-- Terje Norderhaug terje@in-progress.com
* Terje Norderhaug [2009-12-22 01:52+0100] writes:
Assigning a standard meaning would be pretty difficult as implementations differ considerably in this area. E.g Allegro has fspecs while Lispworks has dspecs. Fspecs are for functions only but dspecs are indented for all kinds of definitions and new dspecs can be added by users. It's much simpler to treat dspecs as strings and just display them as label.
MCLIDE needs to be able to filter definitions based on their type, as well as present the dspec in custom ways.
Why is that needed?
Helmut
On Dec 22, 2009, at 12:24 AM, Helmut Eller wrote:
- Terje Norderhaug [2009-12-22 01:52+0100] writes:
Assigning a standard meaning would be pretty difficult as implementations differ considerably in this area. E.g Allegro has fspecs while Lispworks has dspecs. Fspecs are for functions only but dspecs are indented for all kinds of definitions and new dspecs can be added by users. It's much simpler to treat dspecs as strings and just display them as label.
MCLIDE needs to be able to filter definitions based on their type, as well as present the dspec in custom ways.
Why is that needed?
It is needed for functionality that improves the usability of MCLIDE.
For example, the Apropos dialog (see screen shot at http://mclide.in- progress.com/tour) lets lisp developers filter the result of an apropos based on type. If the developer chooses to only be interested in the definitions of functions, clicking the Source button goes straight to the function definition even if there are other definitions for the same symbol. Several dialogs also present dspecs from Swank in more uniform custom ways rather than display the implementation dependent raw dspec string.
Adding a definition-types function to swank will aid MCLIDE and other clients make sense of the definitions from swank by providing a type hierarchy or map from canonical definition types to the type symbols used in their lisp implementation specific dspecs. Are there any other solutions we should consider?
-- Terje Norderhaug terje@in-progress.com
* Terje Norderhaug [2009-12-22 17:33+0100] writes:
Why is that needed?
It is needed for functionality that improves the usability of MCLIDE.
For example, the Apropos dialog (see screen shot at http://mclide.in- progress.com/tour) lets lisp developers filter the result of an apropos based on type. If the developer chooses to only be interested in the definitions of functions, clicking the Source button goes straight to the function definition even if there are other definitions for the same symbol. Several dialogs also present dspecs from Swank in more uniform custom ways rather than display the implementation dependent raw dspec string.
Hmm... sounds like quite bit of work for a relatively small improvement. I would estimate that 80% of all symbols have only one definition, i.e. most of the time it wouldn't make a difference.
Adding a definition-types function to swank will aid MCLIDE and other clients make sense of the definitions from swank by providing a type hierarchy or map from canonical definition types to the type symbols used in their lisp implementation specific dspecs. Are there any other solutions we should consider?
I guess you also want to include method signatures. That wouldn't work because in general the symbols for class names don't exist on the client side.
Helmut
On Dec 23, 2009, at 8:18 AM, Helmut Eller wrote:
- Terje Norderhaug [2009-12-22 17:33+0100] writes:
Why is that needed?
It is needed for functionality that improves the usability of MCLIDE.
For example, the Apropos dialog (see screen shot at http://mclide.in- progress.com/tour) lets lisp developers filter the result of an apropos based on type. If the developer chooses to only be interested in the definitions of functions, clicking the Source button goes straight to the function definition even if there are other definitions for the same symbol. Several dialogs also present dspecs from Swank in more uniform custom ways rather than display the implementation dependent raw dspec string.
Hmm... sounds like quite bit of work for a relatively small improvement. I would estimate that 80% of all symbols have only one definition, i.e. most of the time it wouldn't make a difference.
I don't mind going the extra mile to improve usability, so I'd be glad to take a lead. Making the definitions more easy to process on the client can also open for improvements in SLIME, such as custom/ uniform presentation of definitions, and more advanced processing both on the client and server.
The next step is to get an overview of the current dspecs provided to swank by the various lisp implementations. Has somebody documented any of this already? I have Clozure and LispWorks covered, but could benefit from input about other implementations.
-- Terje Norderhaug terje@in-progress.com
* Terje Norderhaug [2009-12-23 23:56+0100] writes:
The next step is to get an overview of the current dspecs provided to swank by the various lisp implementations. Has somebody documented any of this already? I have Clozure and LispWorks covered, but could benefit from input about other implementations.
Allegro's stuff is documented here: http://www.franz.com/support/documentation/8.1/doc/implementation.htm#functi...
SBCL's sb-introspect:find-definition-sources-by-name has a docstring.
For the others the source code is the only documentation.
CLISP and ECL don't track source locations so there isn't much to find anyway.
Helmut
On Dec 24, 2009, at 1:14 AM, Helmut Eller wrote:
- Terje Norderhaug [2009-12-23 23:56+0100] writes:
The next step is to get an overview of the current dspecs provided to swank by the various lisp implementations. Has somebody documented any of this already? I have Clozure and LispWorks covered, but could benefit from input about other implementations.
Allegro's stuff is documented here: http://www.franz.com/support/documentation/8.1/doc/ implementation.htm#function-specs-1
SBCL's sb-introspect:find-definition-sources-by-name has a docstring.
For the others the source code is the only documentation.
CLISP and ECL don't track source locations so there isn't much to find anyway.
Thanks. I took a close look at the dspecs in LispWorks, Clozure CL and SBCL, as well as the xref.lisp module. There is indeed a lot of variation in the dspecs provided by the various lisp implementations. Rather than force them all into the same mold, swank may provide clients with the information required to make sense of the dspecs without changing the dspecs themselves.
-- Terje Norderhaug terje@in-progress.com
On Dec 23, 2009, at 8:18 AM, Helmut Eller wrote:
I guess you also want to include method signatures. That wouldn't work because in general the symbols for class names don't exist on the client side.
Can't we work around that?
-- Terje Norderhaug terje@in-progress.com
* Terje Norderhaug [2009-12-24 03:13+0100] writes:
On Dec 23, 2009, at 8:18 AM, Helmut Eller wrote:
I guess you also want to include method signatures. That wouldn't work because in general the symbols for class names don't exist on the client side.
Can't we work around that?
This is software so anything is possible, but I doubt that dealing with eql specializers etc. will be elegant.
Helmut
On Dec 19, 2009, at 12:59 AM, Helmut Eller wrote:
- Terje Norderhaug [2009-12-18 20:32+0100] writes:
[...]
Of particular concern is the DSPEC, which Clozure returns as "#'SWANK- BACKEND:FIND-DEFINITIONS" and LispWorks returns as "(DEFUN SWANK- BACKEND:FIND-DEFINITIONS)". It would be helpful if the DSPEC standardized on the same symbol to denote that the definition is for a function, as in (FUNCTION SWANK-BACKEND:FIND-DEFINITIONS).
To facilitate client side use and flexibility in presentation, I also suggest that the DSPEC is returned as a list instead of as a string, with the type as a symbol and the function name encoded as a string, as in (FUNCTION "SWANK-BACKEND:FIND-DEFINITIONS").
Assigning a standard meaning would be pretty difficult as implementations differ considerably in this area. E.g Allegro has fspecs while Lispworks has dspecs. Fspecs are for functions only but dspecs are indented for all kinds of definitions and new dspecs can be added by users. It's much simpler to treat dspecs as strings and just display them as label.
It is unfortunate that the current definition specs are made to be human readable strings rather than easily digested by lisp code. This simplicity comes with a trade-off in that it limits client-side processing.
Swank should better aid clients like MCLIDE and SLIME in making sense of the definitions from the various lisp implementations. Two potential solutions have been brought up:
1) Swank uses a unified representation of dspecs shared between all lisp implementations. 2) Swank provides clients with lisp implementation specific information to make sense out of different dspecs.
You mention that implementations differ considerably and that it would be hard to standardize the dspecs. What would be concrete cases of dspecs that are prohibitively hard to unify?
-- Terje Norderhaug terje@in-progress.com
* Terje Norderhaug [2009-12-24 00:00+0100] writes:
Swank should better aid clients like MCLIDE and SLIME in making sense of the definitions from the various lisp implementations. Two potential solutions have been brought up:
- Swank uses a unified representation of dspecs shared between all
lisp implementations. 2) Swank provides clients with lisp implementation specific information to make sense out of different dspecs.
3) let Swank choose the representation but provide a backend function if someone thinks there is some value in dspecs.
You mention that implementations differ considerably and that it would be hard to standardize the dspecs. What would be concrete cases of dspecs that are prohibitively hard to unify?
setf and user defined defining forms.
Helmut
On Dec 24, 2009, at 1:20 AM, Helmut Eller wrote:
- Terje Norderhaug [2009-12-24 00:00+0100] writes:
Swank should better aid clients like MCLIDE and SLIME in making sense of the definitions from the various lisp implementations. Two potential solutions have been brought up:
- Swank uses a unified representation of dspecs shared between all
lisp implementations. 2) Swank provides clients with lisp implementation specific information to make sense out of different dspecs.
- let Swank choose the representation but provide a backend function
if someone thinks there is some value in dspecs.
The latter is a reasonable direction. We may as well stick to the current dspec representation but add a backend function that can help clients make sense out of the dspecs. That way the functionality can be added without introducing compatibility issues and disruption for SLIME users.
-- Terje Norderhaug terje@in-progress.com
-
Helmut Eller -
Terje Norderhaug