Dear all,
Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects.
However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one.
Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html.
There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome!
P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know.
My apologies to those receiving this message multiple times.
Best regards, Alexander Fedorov.
-------- Date: Sat, 6 Jul 2024 18:56:15 +0200 From: Alexander Fedorov varedif.a.s@gmail.com
Dear all,
Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects.
However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one.
Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html.
There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome!
P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know.
My apologies to those receiving this message multiple times.
Best regards, Alexander Fedorov.
Alexander,
I have not read this specification draft, but when asked for plns by a customer I found it easy to implement years ago with little controversy from my point of view, even on top of our Hierarchichal Packages implementation. The only discrepancy I had found (which might now be corrected, since it was in 2019) was that sbcl's documentation described plns as not being abot to co-exist with global names. I made the assumption that the documentation was either incorrect or out-of-date, based on my experiments with plns in sbcl as I was implementing them in Allegro CL.
The documentation for Allegro CL has been available since the implementation was provided in a patch for 10.1, and the current description is here:
https://franz.com/support/documentation/current/packages.html#local-nickname...
Hopefully there are not too many conflicts with your proposed spec.
Duane Rettig
Hello,
Off the top of my head, the main differences are in the standardization issues, more specifically issues 2 -- PRINT-READ consistency, 8 -- Interaction with FORMAT (this is about the tilde slash ~// directive) and 9 -- Empty string as a local nickname. The last one actually is not very related to PLNs, but rather can be viewed as a separate issue, see the recent WSCL issue: https://github.com/s-expressionists/wscl/issues/63 .
Thanks for the link to the documentation, I'll take a closer look :)
Best regards, Alexander Fedorov
On Sat, 6 Jul 2024 at 22:04, Duane Rettig duane@franz.com wrote:
Date: Sat, 6 Jul 2024 18:56:15 +0200 From: Alexander Fedorov <varedif.a.s@gmail.com> Dear all, Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects. However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one. Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html. There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome! P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know. My apologies to those receiving this message multiple times. Best regards, Alexander Fedorov.Alexander,
I have not read this specification draft, but when asked for plns by a customer I found it easy to implement years ago with little controversy from my point of view, even on top of our Hierarchichal Packages implementation. The only discrepancy I had found (which might now be corrected, since it was in 2019) was that sbcl's documentation described plns as not being abot to co-exist with global names. I made the assumption that the documentation was either incorrect or out-of-date, based on my experiments with plns in sbcl as I was implementing them in Allegro CL.
The documentation for Allegro CL has been available since the implementation was provided in a patch for 10.1, and the current description is here:
https://franz.com/support/documentation/current/packages.html#local-nickname...
Hopefully there are not too many conflicts with your proposed spec.
Duane Rettig
There is a very subtle semantic problem in this proposal. Focus on this seemingly-harmless requirement on the :local-nicknames arg to {def, make-}package:, so subtle that probably no one else has noticed it yet
Name conflict errors are handled by the underlying calls to add-package-local-nickname.
where apln is specified to signal a package error under certain circumstances. It is in the design of ANSI CL -- though rarely exploited in practice -- that normal errors can be handled and recovered predictably and harmlessly. There are many circumstances in CL where an error might be signaled by a standard function, but all the ones I can think of (e.g. vector-push or open of a file) would be expected to leave the world in an unaltered state (in any way that can be detected within the ANS) when an error is signaled. (I'm discounting bumps to *gensym-counter* when a compilation fails.) But this "handled by" specification strongly implies implementation and the order of function side effects; apln cannot be called until after the package exists, but when t, so subtle that probably no one else has noticed it yethe error is signaled and the defpackage "fails" is the incompletely-created named package left around?
Quite possibly this problem already exists somewhere within the ANS package system -- haven't thought about it very hard -- but it isn't the sort of thing conformance test suites would test for.
On Sat, Jul 6, 2024 at 1:04 PM Duane Rettig duane@franz.com wrote:
Date: Sat, 6 Jul 2024 18:56:15 +0200 From: Alexander Fedorov <varedif.a.s@gmail.com> Dear all, Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects. However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one. Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html. There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome! P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know. My apologies to those receiving this message multiple times. Best regards, Alexander Fedorov.Alexander,
I have not read this specification draft, but when asked for plns by a customer I found it easy to implement years ago with little controversy from my point of view, even on top of our Hierarchichal Packages implementation. The only discrepancy I had found (which might now be corrected, since it was in 2019) was that sbcl's documentation described plns as not being abot to co-exist with global names. I made the assumption that the documentation was either incorrect or out-of-date, based on my experiments with plns in sbcl as I was implementing them in Allegro CL.
The documentation for Allegro CL has been available since the implementation was provided in a patch for 10.1, and the current description is here:
https://franz.com/support/documentation/current/packages.html#local-nickname...
Hopefully there are not too many conflicts with your proposed spec.
Duane Rettig
Thanks for your reply!
About MAKE-PACKAGE - good catch, will fix that! This is definitely not what I have intended, nor what is currently implemented by any other implementation. I have blindly copied it from DEFPACKAGE, not noticing that it delegates errors to MAKE-PACKAGE as well.
About DEFPACKAGE - this is analogous to the :USE clause. Note that the "incompletely-created named package" problem indeed exists in the ANSI standard, since when the :SHADOW clause is present, it must call MAKE-PACKAGE with an empty :USE list. After that it calls SHADOW, and finally calls USE-PACKAGE. If a name conflict happens at the last stage, a correctable error is signaled; and if you don't resolve those conflicts but instead exit the debugger via the ABORT restart, you will end up with a defined named package with an empty USE-LIST.
Best regards, Alexander Fedorov.
On Sun, 7 Jul 2024 at 05:34, Steve Haflich shaflich@gmail.com wrote:
There is a very subtle semantic problem in this proposal. Focus on this seemingly-harmless requirement on the :local-nicknames arg to {def, make-}package:, so subtle that probably no one else has noticed it yet
Name conflict errors are handled by the underlying calls to add-package-local-nickname.
where apln is specified to signal a package error under certain circumstances. It is in the design of ANSI CL -- though rarely exploited in practice -- that normal errors can be handled and recovered predictably and harmlessly. There are many circumstances in CL where an error might be signaled by a standard function, but all the ones I can think of (e.g. vector-push or open of a file) would be expected to leave the world in an unaltered state (in any way that can be detected within the ANS) when an error is signaled. (I'm discounting bumps to *gensym-counter* when a compilation fails.) But this "handled by" specification strongly implies implementation and the order of function side effects; apln cannot be called until after the package exists, but when t, so subtle that probably no one else has noticed it yethe error is signaled and the defpackage "fails" is the incompletely-created named package left around?
Quite possibly this problem already exists somewhere within the ANS package system -- haven't thought about it very hard -- but it isn't the sort of thing conformance test suites would test for.
On Sat, Jul 6, 2024 at 1:04 PM Duane Rettig duane@franz.com wrote:
Date: Sat, 6 Jul 2024 18:56:15 +0200 From: Alexander Fedorov <varedif.a.s@gmail.com> Dear all, Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects. However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one. Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html. There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome! P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know. My apologies to those receiving this message multiple times. Best regards, Alexander Fedorov.Alexander,
I have not read this specification draft, but when asked for plns by a customer I found it easy to implement years ago with little controversy from my point of view, even on top of our Hierarchichal Packages implementation. The only discrepancy I had found (which might now be corrected, since it was in 2019) was that sbcl's documentation described plns as not being abot to co-exist with global names. I made the assumption that the documentation was either incorrect or out-of-date, based on my experiments with plns in sbcl as I was implementing them in Allegro CL.
The documentation for Allegro CL has been available since the implementation was provided in a patch for 10.1, and the current description is here:
https://franz.com/support/documentation/current/packages.html#local-nickname...
Hopefully there are not too many conflicts with your proposed spec.
Duane Rettig
On Sat, 6 Jul 2024 18:56:15 +0200, Alexander Fedorov said:
Dear all,
Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects.
However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one.
Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html.
There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome!
P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know.
My apologies to those receiving this message multiple times.
Best regards, Alexander Fedorov.
Firstly, thanks for writing this up. I like the idea of having a specification for PLNs. Possibly it is inevitable at this stage that it is heavily based on the current implementations with minor cleanups, but it was never clear to me if the quirks in the SBCL implementation were deliberate or accidental.
Most of the issues you have identified only affect new APIs that use PLNs, but Issue 3 (Local nicknames effect on DEFPACKAGE, MAKE-PACKAGE and others) is a bigger problem IMHO because it affects other code by the accidental value of *PACKAGE*. I think the most pervasive part of this issue is actually IN-PACKAGE, not DEFPACKAGE or MAKE-PACKAGE, which is why LispWorks doesn't look up PLNs for IN-PACKAGE.
How about another proposal for issue 3: FIND-PACKAGE-AND-READ-PRINT-ONLY, meaning that only FIND-PACKAGE and the reader/printer are affected? To allow callers of the other package functions to detect PLNs, add an optional second argument "package" to FIND-PACKAGE, which would be used in the lookup of the first argument and would default to nil, meaning no PLN lookup.
That would allow code such as:
(intern name (find-package "FOO"))
to work without any ambiguity about what FOO means.
My opinions on the issues:
2.1: I very weakly prefer 2.1.7, NICKNAMED-PACKAGE-IF-SUCCESSFUL. If we're going to return anything other than T, we might as well return the value that carries the most information. But I doubt anyone will ever care, though I agree that it should be standardized.
2.2: The strongest point I want to make here is that a correctable error should be signalled whenever a package becomes fully shadowed by another package (all of its global names are shadowed), with a restart that requests a new local nickname for the shadowed package in the shadowing package(s). We want to alert the user to the situation, and strongly encourage them to add a new nickname (global or local) in the relevant source file so as to make the fully shadowed package accessible again. Meanwhile, we supply the restart to make it easy to remove the shadowing in the current session.
If we do that, then I think it won't matter too much how the printer deals with a fully-shadowed package, as it won't have to do so very often. I would go with 2.2.7 (proposal PRINT-UNREADABLY). I can't see introducing an obscure and cryptic syntax, that users probably won't recognize, for such a rare case; better to make it completely clear what is going on.
2.3: I strongly support Martin Simmons's new proposal. In my view, the purpose of local nicknames is to shorten frequently typed package prefixes. Using them for anything else seems potentially unsafe. The catch, of course, is that implementing this behavior is likely to break existing code, though OTOH the fix is not difficult.
2.4: I strongly support 2.4.4, NO-EFFECT.
2.5: I would simply disallow it, with an uncorrectable error. What are we trying to do here, win the Obfuscated Lisp Contest?
2.6: I agree with 2.6.3, EXTRA-KEYWORD-ARGUMENT-LIST.
2.7: package-locally-nicknamed-by-list doesn't seem felicitously defined to me in the first place. Shouldn't it return an alist of package and local nickname? Other than that, I have no opinion.
2.8: I agree with 2.8.4, NOT-AFFECTED-BY-LOCAL-NICKNAMES.
2.9: I don't think it should be allowed to use the empty string as the name or nickname of any package other than the keyword package. (See above crack about the Obfuscated Lisp Contest.)
-- Scott
On Sat, Jul 6, 2024 at 9:58 AM Alexander Fedorov varedif.a.s@gmail.com wrote:
Dear all,
Over 10 years ago, SBCL implemented the Package-Local Nicknames (PLNs) extension. Since then, PLNs have also been adopted by multiple implementations (ABCL, CCL, ECL, Clasp, Allegro CL and LispWorks) and are now widely used in many projects.
However, PLNs are still considered experimental, as stated in the SBCL manual, since there is no formal specification for them, and each implementation interprets various corner cases differently. While the need for a specification has been previously discussed, I was unable to find any publicly accessible draft for one.
Therefore, I have drafted a specification intended to become a CDR document. I initially wrote this draft about a year ago and have recently revised it. You can find the latest version here: https://gleefre.github.io/cdr-package-local-nicknames/index.html.
There are currently nine unresolved standardization issues, each with at least one proposed resolution. Input on those issues would be particularly helpful, but any feedback is welcome!
P.S. I am sharing this link across multiple platforms to reach as many lispers as possible. This includes various mailing lists (Lisp Pro, CDR-discuss and -devel for various CL implementations); IRC channels (#commonlisp and implementation-specific ones); the Lisp Discord server; several Telegram groups; and a post on Reddit. If you have any suggestions for additional places to share the link, please let me know.
My apologies to those receiving this message multiple times.
Best regards, Alexander Fedorov.
On Mon, Jul 8, 2024 at 9:05 PM Scott L. Burson Scott@sympoiesis.com wrote:
2.2: The strongest point I want to make here is that a correctable error should be signalled whenever a package becomes fully shadowed by another package (all of its global names are shadowed), with a restart that requests a new local nickname for the shadowed package in the shadowing package(s). We want to alert the user to the situation, and strongly encourage them to add a new nickname (global or local) in the relevant source file so as to make the fully shadowed package accessible again. Meanwhile, we supply the restart to make it easy to remove the shadowing in the current session.
I think signalling an error at the point where a package becomes completely shadowed is a bad idea . It destroys the locality of the local nicknames and brings us back to the problem where you can't load two systems at the same time because they happened to both want to use the same convenient name. Arguably one of them has a bad name, and should have other names, but lots of CL code is old and un- or under-maintained, and we shouldn't be annoying users about them just for a very rare edge case.
In my view, the purpose of local nicknames is to shorten frequently typed package prefixes. Using them for anything else seems potentially unsafe.
Another intended use case was people who want to substitute one package for another without having to change the code (either picking from multiple implementations depending on requirements, or replacing an old lib with a compatibility layer, etc). I've seen people describe doing that with :USE, and package local nicknames should support it as well, and more safely in cases where any of those packages might change over time.
Hi
Thanks for the writeup. Apologies for the CDR-DISCUSS mishaps.
I am one of the major polluters of the packages' namespace as I happily abuse :nicknames.
I perused the specification, I like it and I agree with the comments, but I think there is an extra useful bit that is missing. I would like to be able to write
*(in-package "A-VERY-LONG-AND-HAIRY-PACKAGE-NICKNAME" ; This may even be a 'global nickname'!*
* :as "HAIRY-PKG" ; The 'local nickname'. )*
It seems quite natural to me. Did I miss something?
All the best
Marco
On Tue, Jul 9, 2024 at 8:19 AM Bart Botta 00003b@gmail.com wrote:
On Mon, Jul 8, 2024 at 9:05 PM Scott L. Burson Scott@sympoiesis.com wrote:
2.2: The strongest point I want to make here is that a correctable error
should be signalled whenever a package becomes fully shadowed by another package (all of its global names are shadowed), with a restart that requests a new local nickname for the shadowed package in the shadowing package(s). We want to alert the user to the situation, and strongly encourage them to add a new nickname (global or local) in the relevant source file so as to make the fully shadowed package accessible again. Meanwhile, we supply the restart to make it easy to remove the shadowing in the current session.
I think signalling an error at the point where a package becomes completely shadowed is a bad idea . It destroys the locality of the local nicknames and brings us back to the problem where you can't load two systems at the same time because they happened to both want to use the same convenient name. Arguably one of them has a bad name, and should have other names, but lots of CL code is old and un- or under-maintained, and we shouldn't be annoying users about them just for a very rare edge case.
In my view, the purpose of local nicknames is to shorten frequently
typed package prefixes. Using them for anything else seems potentially unsafe.
Another intended use case was people who want to substitute one package for another without having to change the code (either picking from multiple implementations depending on requirements, or replacing an old lib with a compatibility layer, etc). I've seen people describe doing that with :USE, and package local nicknames should support it as well, and more safely in cases where any of those packages might change over time.
On 13 Jul 2024, at 11:49, Marco Antoniotti marco.antoniotti@unimib.it wrote:
Hi
Thanks for the writeup. Apologies for the CDR-DISCUSS mishaps.
I am one of the major polluters of the packages' namespace as I happily abuse :nicknames.
I perused the specification, I like it and I agree with the comments, but I think there is an extra useful bit that is missing. I would like to be able to write
(in-package "A-VERY-LONG-AND-HAIRY-PACKAGE-NICKNAME" ; This may even be a 'global nickname'! :as "HAIRY-PKG" ; The 'local nickname'. )
It seems quite natural to me. Did I miss something?
How is this useful, since once you’re in a package, you access its symbols directly without qualification? How would use use HAIRY-PKG and why?
Sorry. I was so caught up colorizing that I messed it up. I meant USE-PACKAGE
Marco Antoniotti https://dcb.disco.unimib.it
On Sat, 13 Jul 2024 at 16:47, Pascal Bourguignon pjb@informatimago.com wrote:
On 13 Jul 2024, at 11:49, Marco Antoniotti marco.antoniotti@unimib.it wrote:
Hi
Thanks for the writeup. Apologies for the CDR-DISCUSS mishaps.
I am one of the major polluters of the packages' namespace as I happily abuse :nicknames.
I perused the specification, I like it and I agree with the comments, but I think there is an extra useful bit that is missing. I would like to be able to write
*(in-package "A-VERY-LONG-AND-HAIRY-PACKAGE-NICKNAME" ; This may even be a 'global nickname'!*
:as "HAIRY-PKG" ; The 'localnickname'. )*
It seems quite natural to me. Did I miss something?
How is this useful, since once you’re in a package, you access its symbols directly without qualification? How would use use HAIRY-PKG and why?
-- __Pascal Bourguignon__
On 13 Jul 2024, at 16:53, Marco Antoniotti marco.antoniotti@unimib.it wrote:
Sorry. I was so caught up colorizing that I messed it up. I meant USE-PACKAGE
use-package is a function. I think it would be saner to define a separate function to add nicknames, rather than modify use-package.
(use-package :foo.with.long.name) (add-local-nickname :foo.with.long.name :foo)
I see your point, Alas
*import FooBarBaz as FBB*
is what people have gotten used to.
Cheers
Marco
On Sat, Jul 13, 2024 at 5:59 PM Pascal Bourguignon pjb@informatimago.com wrote:
On 13 Jul 2024, at 16:53, Marco Antoniotti marco.antoniotti@unimib.it
wrote:
Sorry. I was so caught up colorizing that I messed it up. I meant
USE-PACKAGE
use-package is a function. I think it would be saner to define a separate function to add nicknames, rather than modify use-package.
(use-package :foo.with.long.name) (add-local-nickname :foo.with.long.name :foo)
-- __Pascal Bourguignon__
-
Alexander Fedorov -
Bart Botta -
Duane Rettig -
Marco Antoniotti -
Martin Simmons -
Pascal Bourguignon -
Scott L. Burson -
Steve Haflich