[asdf-devel] standard io syntax woes
Dear Common Lisp hackers, Inspecting with Anton Vodonosov the latest batch of cl-test-grid issues when running with asdf 2.29.x, we found an interesting case that mirrors the previous failure of iolib 0.7.3 with 2.29. In the hope of making the semantics of asd files more deterministic, with an eye on eventually making .asd files a strict subset of Lisp, I had put in 2.27 a with-standard-io-syntax around the loading of a .asd file. However, this is specified to bind *readtable* and *print-pprint-dispatch* to standard tables that are notionally read-only, though this immutability is NOT enforced on most implementations, instead there being unspecified bad consequences if you do mutate. So, I could conceivably (copy-readtable nil) and (copy-pprint-dispatch nil) every time, but that could be expensive on some implementations. Or I could say "it's the programmer's responsibility to ensure a proper table has been setup before he modifies it", but that would be harsh and a notable backward incompatibility (and there's no equivalent of named-readtables for pprint-dispatch). Or I could preserve the current semantics of a global table that everyone modifies causing "interesting" issues, by rebinding *print-pprint-dispatch* as well as *readtable* within the w-s-i-s, only ensuring that the other syntax variables are standard. Or I could remove the with-standard-io-syntax altogether, and say "yes, if you're doing any global modification, you suck and you're going to break something for someone, but that's none of my business". Anton leans for the latter. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org One can be so anxious to put his "best foot forward" that he doesn't even notice that it isn't his own foot. — Harry Browne (HIFFIAUW)
In 2.29.9, I pushed a change whereby ASDF keeps passing around whichever value the two syntax tables are globally bound to, leaving on the user the onus to not mutate them in too destructive a way, yet without precluding a change. Allocation is for (copy-readtable nil) and (copy-pprint-dispatch nil) is 1648 and 2192 bytes respectively on CCL 1.9 x86-64, and more like 5300 and 10300 bytes on SBCL 1.1.3 x86-64. rme suggests that 15KiB per .asd file is no big deal and we should just bite the bullet. Others suggest that we should let users fail and keep out of the business of binding syntax variables altogether. I'd agree if I didn't want to eventually move towards "pure" .asd files in a restricted standardized syntax as per https://bugs.launchpad.net/asdf/+bug/541562 If only Common Lisp allowed to portably specify read-only tables, I would just use that, and let users fail when they try to mutate them. Unhappily, it doesn't, and therefore when some user mutates a global table, it will end up causing pain for another user, not himself. Or I could rely on SBCL being used a whole lot and indeed having immutable such default syntax tables with understandable messages to blame whoever tries to mutate those tables without rebinding them first, and push for all implementations to similarly make them read-only. The jury is still out. Please voice your opinion. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Statism is the secular version of salvation through faith: it doesn't matter what bureaucrats do, only that they do it with good intentions. On Wed, Feb 20, 2013 at 3:44 PM, Faré <fahree@gmail.com> wrote:
Dear Common Lisp hackers,
Inspecting with Anton Vodonosov the latest batch of cl-test-grid issues when running with asdf 2.29.x, we found an interesting case that mirrors the previous failure of iolib 0.7.3 with 2.29.
In the hope of making the semantics of asd files more deterministic, with an eye on eventually making .asd files a strict subset of Lisp, I had put in 2.27 a with-standard-io-syntax around the loading of a .asd file. However, this is specified to bind *readtable* and *print-pprint-dispatch* to standard tables that are notionally read-only, though this immutability is NOT enforced on most implementations, instead there being unspecified bad consequences if you do mutate.
So, I could conceivably (copy-readtable nil) and (copy-pprint-dispatch nil) every time, but that could be expensive on some implementations. Or I could say "it's the programmer's responsibility to ensure a proper table has been setup before he modifies it", but that would be harsh and a notable backward incompatibility (and there's no equivalent of named-readtables for pprint-dispatch). Or I could preserve the current semantics of a global table that everyone modifies causing "interesting" issues, by rebinding *print-pprint-dispatch* as well as *readtable* within the w-s-i-s, only ensuring that the other syntax variables are standard. Or I could remove the with-standard-io-syntax altogether, and say "yes, if you're doing any global modification, you suck and you're going to break something for someone, but that's none of my business".
Anton leans for the latter.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org One can be so anxious to put his "best foot forward" that he doesn't even notice that it isn't his own foot. — Harry Browne (HIFFIAUW)
You could specify an extension to Common Lisp that allows users to request read-only readtables, submit this as a CDR, and hope that CL implementations are willing to implement such an extension of the language, and then rely on that. Pascal On 20 Feb 2013, at 22:44, Faré <fahree@gmail.com> wrote:
In 2.29.9, I pushed a change whereby ASDF keeps passing around whichever value the two syntax tables are globally bound to, leaving on the user the onus to not mutate them in too destructive a way, yet without precluding a change.
Allocation is for (copy-readtable nil) and (copy-pprint-dispatch nil) is 1648 and 2192 bytes respectively on CCL 1.9 x86-64, and more like 5300 and 10300 bytes on SBCL 1.1.3 x86-64. rme suggests that 15KiB per .asd file is no big deal and we should just bite the bullet.
Others suggest that we should let users fail and keep out of the business of binding syntax variables altogether. I'd agree if I didn't want to eventually move towards "pure" .asd files in a restricted standardized syntax as per https://bugs.launchpad.net/asdf/+bug/541562
If only Common Lisp allowed to portably specify read-only tables, I would just use that, and let users fail when they try to mutate them. Unhappily, it doesn't, and therefore when some user mutates a global table, it will end up causing pain for another user, not himself. Or I could rely on SBCL being used a whole lot and indeed having immutable such default syntax tables with understandable messages to blame whoever tries to mutate those tables without rebinding them first, and push for all implementations to similarly make them read-only.
The jury is still out. Please voice your opinion.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Statism is the secular version of salvation through faith: it doesn't matter what bureaucrats do, only that they do it with good intentions.
On Wed, Feb 20, 2013 at 3:44 PM, Faré <fahree@gmail.com> wrote:
Dear Common Lisp hackers,
Inspecting with Anton Vodonosov the latest batch of cl-test-grid issues when running with asdf 2.29.x, we found an interesting case that mirrors the previous failure of iolib 0.7.3 with 2.29.
In the hope of making the semantics of asd files more deterministic, with an eye on eventually making .asd files a strict subset of Lisp, I had put in 2.27 a with-standard-io-syntax around the loading of a .asd file. However, this is specified to bind *readtable* and *print-pprint-dispatch* to standard tables that are notionally read-only, though this immutability is NOT enforced on most implementations, instead there being unspecified bad consequences if you do mutate.
So, I could conceivably (copy-readtable nil) and (copy-pprint-dispatch nil) every time, but that could be expensive on some implementations. Or I could say "it's the programmer's responsibility to ensure a proper table has been setup before he modifies it", but that would be harsh and a notable backward incompatibility (and there's no equivalent of named-readtables for pprint-dispatch). Or I could preserve the current semantics of a global table that everyone modifies causing "interesting" issues, by rebinding *print-pprint-dispatch* as well as *readtable* within the w-s-i-s, only ensuring that the other syntax variables are standard. Or I could remove the with-standard-io-syntax altogether, and say "yes, if you're doing any global modification, you suck and you're going to break something for someone, but that's none of my business".
Anton leans for the latter.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org One can be so anxious to put his "best foot forward" that he doesn't even notice that it isn't his own foot. — Harry Browne (HIFFIAUW)
_______________________________________________ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
-- Pascal Costanza
For the record, checking on my machine today: Implementations that make the standard readtable read-only: allegro sbcl Implementations that don't: abcl ccl clisp cmucl ecl lispworks scl Implementations that make the standard pprint-dispatch table read-only: ccl sbcl Implementations that don't: abcl allegro clisp cmucl ecl lispworks scl I don't know if a CDR is needed to change that, but I'm going to bug all the respective vendors. It's just a safety issue wrt the existing standard. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The meeting of two personalities is like the contact of two chemical substances: if there is any reaction, both are transformed. — Carl Jung On Wed, Feb 20, 2013 at 4:50 PM, Pascal Costanza <pc@p-cos.net> wrote:
You could specify an extension to Common Lisp that allows users to request read-only readtables, submit this as a CDR, and hope that CL implementations are willing to implement such an extension of the language, and then rely on that.
Pascal
On 20 Feb 2013, at 22:44, Faré <fahree@gmail.com> wrote:
In 2.29.9, I pushed a change whereby ASDF keeps passing around whichever value the two syntax tables are globally bound to, leaving on the user the onus to not mutate them in too destructive a way, yet without precluding a change.
Allocation is for (copy-readtable nil) and (copy-pprint-dispatch nil) is 1648 and 2192 bytes respectively on CCL 1.9 x86-64, and more like 5300 and 10300 bytes on SBCL 1.1.3 x86-64. rme suggests that 15KiB per .asd file is no big deal and we should just bite the bullet.
Others suggest that we should let users fail and keep out of the business of binding syntax variables altogether. I'd agree if I didn't want to eventually move towards "pure" .asd files in a restricted standardized syntax as per https://bugs.launchpad.net/asdf/+bug/541562
If only Common Lisp allowed to portably specify read-only tables, I would just use that, and let users fail when they try to mutate them. Unhappily, it doesn't, and therefore when some user mutates a global table, it will end up causing pain for another user, not himself. Or I could rely on SBCL being used a whole lot and indeed having immutable such default syntax tables with understandable messages to blame whoever tries to mutate those tables without rebinding them first, and push for all implementations to similarly make them read-only.
The jury is still out. Please voice your opinion.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Statism is the secular version of salvation through faith: it doesn't matter what bureaucrats do, only that they do it with good intentions.
On Wed, Feb 20, 2013 at 3:44 PM, Faré <fahree@gmail.com> wrote:
Dear Common Lisp hackers,
Inspecting with Anton Vodonosov the latest batch of cl-test-grid issues when running with asdf 2.29.x, we found an interesting case that mirrors the previous failure of iolib 0.7.3 with 2.29.
In the hope of making the semantics of asd files more deterministic, with an eye on eventually making .asd files a strict subset of Lisp, I had put in 2.27 a with-standard-io-syntax around the loading of a .asd file. However, this is specified to bind *readtable* and *print-pprint-dispatch* to standard tables that are notionally read-only, though this immutability is NOT enforced on most implementations, instead there being unspecified bad consequences if you do mutate.
So, I could conceivably (copy-readtable nil) and (copy-pprint-dispatch nil) every time, but that could be expensive on some implementations. Or I could say "it's the programmer's responsibility to ensure a proper table has been setup before he modifies it", but that would be harsh and a notable backward incompatibility (and there's no equivalent of named-readtables for pprint-dispatch). Or I could preserve the current semantics of a global table that everyone modifies causing "interesting" issues, by rebinding *print-pprint-dispatch* as well as *readtable* within the w-s-i-s, only ensuring that the other syntax variables are standard. Or I could remove the with-standard-io-syntax altogether, and say "yes, if you're doing any global modification, you suck and you're going to break something for someone, but that's none of my business".
Anton leans for the latter.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org One can be so anxious to put his "best foot forward" that he doesn't even notice that it isn't his own foot. — Harry Browne (HIFFIAUW)
_______________________________________________ asdf-devel mailing list asdf-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
-- Pascal Costanza
"Fare" == Far <Far> writes:
Fare> For the record, checking on my machine today: Fare> Implementations that make the standard readtable read-only: Fare> allegro sbcl Fare> Implementations that don't: Fare> abcl ccl clisp cmucl ecl lispworks scl FWIW, on cmucl, *readtable* is a copy of the standard readtable. The standard readtable isn't writable unless you go out of your way to access the magic standard readtable. So, it's perfectly valid to modify *readtable*. Or are you suggesting that *readtable* should be eq to the magic standard readtable? The same holds for *print-pprint-dispatch*---it's a copy of the standard dispatch table, not eq to it. Ray
On Thu, Feb 21, 2013 at 2:08 PM, Raymond Toy <toy.raymond@gmail.com> wrote:
"Fare" == Far <Far> writes:
Fare> For the record, checking on my machine today: Fare> Implementations that make the standard readtable read-only: Fare> allegro sbcl Fare> Implementations that don't: Fare> abcl ccl clisp cmucl ecl lispworks scl
FWIW, on cmucl, *readtable* is a copy of the standard readtable. The standard readtable isn't writable unless you go out of your way to access the magic standard readtable.
So, it's perfectly valid to modify *readtable*.
Or are you suggesting that *readtable* should be eq to the magic standard readtable?
The same holds for *print-pprint-dispatch*---it's a copy of the standard dispatch table, not eq to it.
I am suggesting indeed that, as per the CLHS for WITH-STANDARD-IO-SYNTAX and the CLHS Glossary for standard reatable, W-S-IO-S should bind *READTABLE* to the standard readtable, and similarly for *PRINT-PPRINT-DISPATCH*. And for the sake of sanity, said standard tables better be readonly. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org To authorize all commercial relations between consenting adults.
"Fare" == Far <Far> writes:
Fare> On Thu, Feb 21, 2013 at 2:08 PM, Raymond Toy <toy.raymond@gmail.com> wrote: >>>>>>> "Fare" == Far <Far> writes: >> Fare> For the record, checking on my machine today: Fare> Implementations that make the standard readtable read-only: Fare> allegro sbcl Fare> Implementations that don't: Fare> abcl ccl clisp cmucl ecl lispworks scl >> >> FWIW, on cmucl, *readtable* is a copy of the standard readtable. The >> standard readtable isn't writable unless you go out of your way to >> access the magic standard readtable. >> >> So, it's perfectly valid to modify *readtable*. >> >> Or are you suggesting that *readtable* should be eq to the >> magic standard readtable? >> >> The same holds for *print-pprint-dispatch*---it's a copy of the >> standard dispatch table, not eq to it. >> Fare> I am suggesting indeed that, Fare> as per the CLHS for WITH-STANDARD-IO-SYNTAX and Fare> the CLHS Glossary for standard reatable, Fare> W-S-IO-S should bind *READTABLE* to the standard readtable, Fare> and similarly for *PRINT-PPRINT-DISPATCH*. Fare> And for the sake of sanity, said standard tables better be readonly. Ah, I misspoke. w-s-io-s does indeed bind *readtable* to the magic standard table. *print-pprint-dispatch* gets a copy of the standard table. I will try to update this and make them not writable in time for the next snapshot. Maybe. Ray
On Wed, Feb 20, 2013 at 11:18 PM, Faré <fahree@gmail.com> wrote:
Implementations that make the standard readtable read-only: allegro sbcl Implementations that don't: abcl ccl clisp cmucl ecl lispworks scl
This is not true
(with-standard-io-syntax (set-dispatch-macro-character #\! #\Y (constantly nil)))
Condition of type: SIMPLE-ERROR Change readtable Available restarts: 1. (RESTART-TOPLEVEL) Go back to Top-Level REPL. Broken at SI:BYTECODES. [Evaluation of: (WITH-STANDARD-IO-SYNTAX (SET-DISPATCH-MACRO-CHARACTER #\! ...))] In: #<process TOP-LEVEL>. I just realized the error message is broken, but ECL does have read-only readtables and a function to lock them (ext:readtable-lock readtable &optional yes-or-no) I am not sure about the pprint dispatch table, though. We just recycled old SBCL/CMUCL code for that. Juanjo -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://juanjose.garciaripoll.googlepages.com
On Thu, Feb 21, 2013 at 4:46 PM, Juan Jose Garcia-Ripoll <juanjose.garciaripoll@gmail.com> wrote:
On Wed, Feb 20, 2013 at 11:18 PM, Faré <fahree@gmail.com> wrote:
Implementations that make the standard readtable read-only: allegro sbcl Implementations that don't: abcl ccl clisp cmucl ecl lispworks scl
This is not true
(with-standard-io-syntax (set-dispatch-macro-character #\! #\Y (constantly nil)))
Condition of type: SIMPLE-ERROR Change readtable
Oops, my tests were not automated enough and I missed that. Sorry.
I just realized the error message is broken, but ECL does have read-only readtables and a function to lock them (ext:readtable-lock readtable &optional yes-or-no)
Thanks.
I am not sure about the pprint dispatch table, though. We just recycled old SBCL/CMUCL code for that.
You would have to do something similar to keep the standard pprint-dispatch table read-only. Thanks a lot! —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Vision is the art of seeing what is invisible to others. — Jonathan Swift
Or I could rely on SBCL being used a whole lot and indeed having immutable such default syntax tables with understandable messages to blame whoever tries to mutate those tables without rebinding them
this approach seems to be the most straightforward to me. especially since we have that great cl-test-grid running on multiple lisps, one of which is sbcl which will catch the wrongdoers. i'd also argue that it's also a potential time waster if asdf rebinds a copy and someone foolhardy tries to modify *readtable*, and will experience zero sideffects contrary to his expectations. he will waste his own time only, so it's already much better, but still not ideal. -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39
On Tue, Feb 26, 2013 at 2:18 AM, Attila Lendvai <attila.lendvai@gmail.com> wrote:
Or I could rely on SBCL being used a whole lot and indeed having immutable such default syntax tables with understandable messages to blame whoever tries to mutate those tables without rebinding them
this approach seems to be the most straightforward to me.
especially since we have that great cl-test-grid running on multiple lisps, one of which is sbcl which will catch the wrongdoers.
i'd also argue that it's also a potential time waster if asdf rebinds a copy and someone foolhardy tries to modify *readtable*, and will experience zero sideffects contrary to his expectations.
he will waste his own time only, so it's already much better, but still not ideal.
I wholly agree with you, and that's why I had started using with-standard-io-syntax indeed, and that's where I want to go eventually. However, until we've fixed all the packages that do bad stuff like that, it's not practical to commit that change now, especially since it's easy to (multiple-value-setq (*readtable* *print-pprint-dispatch*) (with-standard-io-syntax (values *readtable* *print-pprint-dispatch*))) and restore the safe behavior from the compatible behavior, but it's not easy to configure the other way around. I suppose that's also an argument for disabling deferred-warnings by default, though in this case, it's slightly easier to disable than to reenable. (defparameter asdf::*warnings-file-type* nil) vs #+asdf3 (setf asdf:*warnings-file-type* (asdf/lisp-build:warnings-file-type)) But yes, if disabling them by default allows ASDF3 to make it to Quicklisp, I'll do it. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org One can be so anxious to put his "best foot forward" that he doesn't even notice that it isn't his own foot. — Harry Browne (HIFFIAUW)
On 2/20/13 Feb 20 -2:44 PM, Faré wrote:
Dear Common Lisp hackers,
Inspecting with Anton Vodonosov the latest batch of cl-test-grid issues when running with asdf 2.29.x, we found an interesting case that mirrors the previous failure of iolib 0.7.3 with 2.29.
In the hope of making the semantics of asd files more deterministic, with an eye on eventually making .asd files a strict subset of Lisp, I had put in 2.27 a with-standard-io-syntax around the loading of a .asd file. However, this is specified to bind *readtable* and *print-pprint-dispatch* to standard tables that are notionally read-only, though this immutability is NOT enforced on most implementations, instead there being unspecified bad consequences if you do mutate.
So, I could conceivably (copy-readtable nil) and (copy-pprint-dispatch nil) every time, but that could be expensive on some implementations. Or I could say "it's the programmer's responsibility to ensure a proper table has been setup before he modifies it", but that would be harsh and a notable backward incompatibility (and there's no equivalent of named-readtables for pprint-dispatch). Or I could preserve the current semantics of a global table that everyone modifies causing "interesting" issues, by rebinding *print-pprint-dispatch* as well as *readtable* within the w-s-i-s, only ensuring that the other syntax variables are standard. Or I could remove the with-standard-io-syntax altogether, and say "yes, if you're doing any global modification, you suck and you're going to break something for someone, but that's none of my business".
Anton leans for the latter.
I favor the expensive option that keeps safety (copying) and that provides a predictable environment. We don't load ASDF system definitions into COMMON-LISP-USER, or "whatever *PACKAGE* happens to be bound to" and we shouldn't do the equivalent for *readtable* or *print-pprint-dispatch*. Doing something nasty to *READTABLE* while loading your ASDF system *should* break, IMO. Libraries that want to do this should export functions that modify readtables and pprint-dispatch tables. [I'm especially unsympathetic to this because of just this month having gone through hell fixing code that broke because a library felt like it could call SET-DISPATCH-MACRO-CHARACTER without specifying the readtable....] Cheers, r
21.02.2013, 00:45, "Faré" <fahree@gmail.com>:
Or I could remove the with-standard-io-syntax altogether, and say "yes, if you're doing any global modification, you suck and you're going to break something for someone, but that's none of my business".
Anton leans for the latter.
I meant not that this solution is preferred, but that the io-syntax questoin is not entangled with other features ready to be relased, so in case of doubds or difficulties to implement better solutions, the question could be postponed to future.
participants (7)
-
Anton Vodonosov -
Attila Lendvai -
Faré -
Juan Jose Garcia-Ripoll -
Pascal Costanza -
Raymond Toy -
Robert Goldman