Dear lispers,
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
Nothing should have changed in the semantics of ASDF itself since the last run of cl-test-grid, so I don't expect any discrepancy. We did add better immutable-system support thanks to Dave Cooper, but that isn't cover by cl-test-grid, only by our regression tests and his usage.
What we did change was to make UIOP more robust and portable. Robert Goldman and I, with the help of Dave Cooper, ran extensively tests. We have notably debugged a lot of pathname and run-program issues on Windows. These issues recently cropped up since we have added a lot of tests since last release, and since neither of us is a Windows user, they had previously only been run on Unix. We also made an important compatibility fix for SBCL, added CLASP support, enhanced support for new LispWorks 7, fixed chdir for ABCL and XCL.
As far as I'm concerned, this means that portable scripting in Common Lisp is more a reality than ever. If you are careful, you can write portable CL code that runs as is on Unix (including Linux and MacOS) and Windows. As usual, the more portable you want to be, the more careful you have to be; but at least UIOP covers all the bases with a reasonable portable interface (please don't use anymore any of cl-fad, trivial-shell, run-shell-command, trivial-backtrace; use uiop). I recommend CCL for a portable scripting implementation, because SBCL on Windows isn't quite as good (and is not able to call out to CMD.EXE) and CLISP has too many quirks (and is looking for a new maintainer to release it anew after 5 years). For examples of scripting in CL, see https://github.com/fare/fare-scripts
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org In a country where the sole employer is the State, opposition means death by slow starvation. The old principle: who does not work does not eat, has been replaced by a new one: who does not obey shall not eat. — Leon Trotsky
On Jul 10, 2015, at 05:05, Faré fahree@gmail.com wrote:
[…]
Nothing should have changed in the semantics of ASDF itself since the last run of cl-test-grid, so I don't expect any discrepancy. We did add better immutable-system support thanks to Dave Cooper, but that isn't cover by cl-test-grid, only by our regression tests and his usage.
In a minor point on compatibility: I believe that [in the revision to the XDG machinery][1], under Windows the default location of the XDG config directory will change. Unfortunately, I have not had the time to verify that this is the case on my production ABCL instances under Windows.
While I acknowledge that this change is a “bug fix” in ASDF’s interpretation of the XDG spec and the “correct” thing to do, it might help to be a little more verbose in the release notes on the ramification of a Windows-deployed end user’s upgrading from asdf-3.1.4 to asdf-3.1.5. Windows' lack of a symbolic links and NTFS semantics of not allowing a directory to be renamed when opened by a given process may make upgrading ASDF in-place for production instances a little tricky.
[1]: https://gitlab.common-lisp.net/asdf/asdf/commit/7efd83582488c0cd4e3f0bb9aa19...
Otherwise, onwards!
On 7/10/15 Jul 10 -3:24 AM, Mark Evenson wrote:
On Jul 10, 2015, at 05:05, Faré fahree@gmail.com wrote:
[…]
Nothing should have changed in the semantics of ASDF itself since the last run of cl-test-grid, so I don't expect any discrepancy. We did add better immutable-system support thanks to Dave Cooper, but that isn't cover by cl-test-grid, only by our regression tests and his usage.
In a minor point on compatibility: I believe that [in the revision to the XDG machinery][1], under Windows the default location of the XDG config directory will change. Unfortunately, I have not had the time to verify that this is the case on my production ABCL instances under Windows.
While I acknowledge that this change is a “bug fix” in ASDF’s interpretation of the XDG spec and the “correct” thing to do, it might help to be a little more verbose in the release notes on the ramification of a Windows-deployed end user’s upgrading from asdf-3.1.4 to asdf-3.1.5. Windows' lack of a symbolic links and NTFS semantics of not allowing a directory to be renamed when opened by a given process may make upgrading ASDF in-place for production instances a little tricky.
This sounds like a good point. Should we do this in the cover letter, the changelog, manual, or some combination?
My guess is that relatively few people actually upgrade their ASDFs in place -- most just get ASDF from their implementation. But that's not an excuse for making them miserable.
Faré, do you have a sentence about the change in location that I could use?
thanks, r
Otherwise, onwards!
On Sun, Jul 12, 2015 at 1:24 PM, Robert Goldman rpgoldman@sift.net wrote:
This sounds like a good point. Should we do this in the cover letter, the changelog, manual, or some combination?
My guess is that relatively few people actually upgrade their ASDFs in place -- most just get ASDF from their implementation. But that's not an excuse for making them miserable.
Faré, do you have a sentence about the change in location that I could use?
What about this?
"If you are using Windows and taking advantage of the default search path for the configuration and/or source-registry, then mind that this path has just changed in incompatible ways. If you have configuration files, you may have to move or copy them from $LOCALAPPDATA/common-lisp/config/ to $LOCALAPPDATA/config/common-lisp/. Meanwhile your cache will be moved from $LOCALAPPDATA/common-lisp/cache/ to $LOCALAPPDATA/cache/common-lisp/. However, you should not have to move your source code, still in subdirectories of $LOCALAPPDATA/common-lisp/source/"
NB: I have not used Windows, and if any Windows user actually relies on ASDF configuration, double-checking would be great.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org You may call yourself athiest, but I'm athier than you are!
On 7/12/15 Jul 12 -1:17 PM, Faré wrote:
On Sun, Jul 12, 2015 at 1:24 PM, Robert Goldman rpgoldman@sift.net wrote:
This sounds like a good point. Should we do this in the cover letter, the changelog, manual, or some combination?
My guess is that relatively few people actually upgrade their ASDFs in place -- most just get ASDF from their implementation. But that's not an excuse for making them miserable.
Faré, do you have a sentence about the change in location that I could use?
What about this?
"If you are using Windows and taking advantage of the default search path for the configuration and/or source-registry, then mind that this path has just changed in incompatible ways. If you have configuration files, you may have to move or copy them from $LOCALAPPDATA/common-lisp/config/ to $LOCALAPPDATA/config/common-lisp/. Meanwhile your cache will be moved from $LOCALAPPDATA/common-lisp/cache/ to $LOCALAPPDATA/cache/common-lisp/. However, you should not have to move your source code, still in subdirectories of $LOCALAPPDATA/common-lisp/source/"
NB: I have not used Windows, and if any Windows user actually relies on ASDF configuration, double-checking would be great.
I don't use Windows, either, so if there's anyone on the list who would be so kind as to pull the prerelease ASDF, and check it against their running configuration, along the lines laid out above, I would be very grateful.
10.07.2015, 06:06, "Faré" fahree@gmail.com:
Dear lispers,
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
Yes, I've started the tests.
Best regards, - Anton
10.07.2015, 12:20, "Anton Vodonosov" avodonosov@yandex.ru:
10.07.2015, 06:06, "Faré" fahree@gmail.com:
Dear lispers,
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
Yes, I've started the tests.
Some lisps are finished already:
abcl-1.3.2-fasl42-linux-x86 ccl-1.10-r16196-f96-linux-x86 ccl-1.8-r15286m-f95-linux-x86 ccl-1.9-r15756-f96-linux-x86 clisp-2.49-unix-x86 cmu-snapshot-2014-01__20e_unicode_-linux-x86 cmu-snapshot-2014-05-dirty__20e_unicode_-linux-x86 cmu-snapshot-2014-12___20f_unicode_-linux-x86 ecl-13.5.1-unknown-linux-i686-bytecode ecl-13.5.1-unknown-linux-i686-lisp-to-c ecl-15.2.21-ee989b97-linux-i686-bytecode sbcl-1.0.58-linux-x86 sbcl-1.1.11-linux-x86 sbcl-1.1.16-linux-x86 sbcl-1.2.6-linux-x86
So far I see no difference from the git version d70a8f8 we tested recently (off the mailing lists).
Other (elder) lisps are still running. I will report as they finish.
Best regards, - Anton
I'm afraid I've forgotten: would you please send out the results URL(a)?
Sent from my iPhone
On Jul 12, 2015, at 20:34, Anton Vodonosov avodonosov@yandex.ru wrote:
10.07.2015, 12:20, "Anton Vodonosov" avodonosov@yandex.ru:
10.07.2015, 06:06, "Faré" fahree@gmail.com:
Dear lispers,
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
Yes, I've started the tests.
Some lisps are finished already:
abcl-1.3.2-fasl42-linux-x86 ccl-1.10-r16196-f96-linux-x86 ccl-1.8-r15286m-f95-linux-x86 ccl-1.9-r15756-f96-linux-x86 clisp-2.49-unix-x86 cmu-snapshot-2014-01__20e_unicode_-linux-x86 cmu-snapshot-2014-05-dirty__20e_unicode_-linux-x86 cmu-snapshot-2014-12___20f_unicode_-linux-x86 ecl-13.5.1-unknown-linux-i686-bytecode ecl-13.5.1-unknown-linux-i686-lisp-to-c ecl-15.2.21-ee989b97-linux-i686-bytecode sbcl-1.0.58-linux-x86 sbcl-1.1.11-linux-x86 sbcl-1.1.16-linux-x86 sbcl-1.2.6-linux-x86
So far I see no difference from the git version d70a8f8 we tested recently (off the mailing lists).
Other (elder) lisps are still running. I will report as they finish.
Best regards,
- Anton
13.07.2015, 05:01, "Robert P. Goldman" rpgoldman@sift.net:
I'm afraid I've forgotten: would you please send out the results URL(a)?
Ah, sorry, you were not CC'ed.
Faré contacted me for help when he wanted to run cl-test-grid tests himself, and we ended up testing that version (d70a8f8).
The reports for the current version (c3f7c73) are below.
The summary is that I see no errors caused by new ASDF.
The only regression you may be interested in is "COMMON-LISP:TYPE-ERROR : value NIL is not of the expected type STRUCTURE." occurring on CCL 1.8.
It happens because (I think) new ASDF restores special variable values after .asd file is loaded.
CCL 1.8 initializes *PRINT-PPRINT-DISPATCH* to NIL, causing (SET-PPRINT-DISPATCH ...) to fail with this error, unless somebody initialized *PRINT-PPRINT-DISPATCH* e.g. by (COPY-PPRINT-DISPATCH NIL)
This is CCL bug #784 (and duplicated as #960).
The file ~/quicklisp/dists/quicklisp/software/nst-4.0.2/ext/defdoc/defdoc.asd uses this workaround. But with new ASDF, when one of it's fasl files is loaded (nst-4.0.3/ext/defdoc/lisp/core/output.lx32fsl), *PRINT-PPRINT-DISPATCH* is nil. I.e. the workaround has no effect.
The reports:
Full diff https://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-47.html
As you see, there are many "improvements" on old lisps just because many systems use new ASDF features not provided by the ASDF version shipped with those old lisp impls.
For convenience, here is the diff report including only the tests which fail on new ASDF: https://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-48.html
I marked by yellow notes the failures I think not necessary to consider.
Some tests are still running, I will report as they finish.
Best regards, - Anton
On Jul 12, 2015, at 20:34, Anton Vodonosov avodonosov@yandex.ru wrote:
10.07.2015, 12:20, "Anton Vodonosov" avodonosov@yandex.ru:
10.07.2015, 06:06, "Faré" fahree@gmail.com:
Dear lispers,
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
Yes, I've started the tests.
Some lisps are finished already:
abcl-1.3.2-fasl42-linux-x86 ccl-1.10-r16196-f96-linux-x86 ccl-1.8-r15286m-f95-linux-x86 ccl-1.9-r15756-f96-linux-x86 clisp-2.49-unix-x86 cmu-snapshot-2014-01__20e_unicode_-linux-x86 cmu-snapshot-2014-05-dirty__20e_unicode_-linux-x86 cmu-snapshot-2014-12___20f_unicode_-linux-x86 ecl-13.5.1-unknown-linux-i686-bytecode ecl-13.5.1-unknown-linux-i686-lisp-to-c ecl-15.2.21-ee989b97-linux-i686-bytecode sbcl-1.0.58-linux-x86 sbcl-1.1.11-linux-x86 sbcl-1.1.16-linux-x86 sbcl-1.2.6-linux-x86
So far I see no difference from the git version d70a8f8 we tested recently (off the mailing lists).
Other (elder) lisps are still running. I will report as they finish.
Best regards, - Anton
All my lisps finished, reports are updated. The summary remains the same.
The lisp tested:
abcl-1.2.1-fasl42-linux-x86 abcl-1.3.0-fasl42-linux-x86 abcl-1.3.1-fasl42-linux-x86 abcl-1.3.2-fasl42-linux-x86 ccl-1.10-r16196-f96-linux-x86 ccl-1.8-r15286m-f95-linux-x86 ccl-1.9-r15756-f96-linux-x86 clisp-2.49-unix-x86 cmu-snapshot-2014-01__20e_unicode_-linux-x86 cmu-snapshot-2014-05-dirty__20e_unicode_-linux-x86 cmu-snapshot-2014-12___20f_unicode_-linux-x86 ecl-13.5.1-unknown-linux-i686-bytecode ecl-13.5.1-unknown-linux-i686-lisp-to-c ecl-15.2.21-ee989b97-linux-i686-bytecode ecl-15.2.21-ee989b97-linux-i686-lisp-to-c sbcl-1.0.58-linux-x86 sbcl-1.1.11-linux-x86 sbcl-1.1.16-linux-x86 sbcl-1.2.6-linux-x86
Best regards, - Anton
13.07.2015, 17:52, "Anton Vodonosov" avodonosov@yandex.ru:
13.07.2015, 05:01, "Robert P. Goldman" rpgoldman@sift.net:
I'm afraid I've forgotten: would you please send out the results URL(a)?
Ah, sorry, you were not CC'ed.
Faré contacted me for help when he wanted to run cl-test-grid tests himself, and we ended up testing that version (d70a8f8).
The reports for the current version (c3f7c73) are below.
The summary is that I see no errors caused by new ASDF.
The only regression you may be interested in is "COMMON-LISP:TYPE-ERROR : value NIL is not of the expected type STRUCTURE." occurring on CCL 1.8.
It happens because (I think) new ASDF restores special variable values after .asd file is loaded.
CCL 1.8 initializes *PRINT-PPRINT-DISPATCH* to NIL, causing (SET-PPRINT-DISPATCH ...) to fail with this error, unless somebody initialized *PRINT-PPRINT-DISPATCH* e.g. by (COPY-PPRINT-DISPATCH NIL)
This is CCL bug #784 (and duplicated as #960).
The file ~/quicklisp/dists/quicklisp/software/nst-4.0.2/ext/defdoc/defdoc.asd uses this workaround. But with new ASDF, when one of it's fasl files is loaded (nst-4.0.3/ext/defdoc/lisp/core/output.lx32fsl), *PRINT-PPRINT-DISPATCH* is nil. I.e. the workaround has no effect.
The reports:
Full diff https://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-47.html
As you see, there are many "improvements" on old lisps just because many systems use new ASDF features not provided by the ASDF version shipped with those old lisp impls.
For convenience, here is the diff report including only the tests which fail on new ASDF: https://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-48.html
I marked by yellow notes the failures I think not necessary to consider.
Some tests are still running, I will report as they finish.
Best regards,
- Anton
On Jul 12, 2015, at 20:34, Anton Vodonosov avodonosov@yandex.ru wrote:
10.07.2015, 12:20, "Anton Vodonosov" avodonosov@yandex.ru:
10.07.2015, 06:06, "Faré" fahree@gmail.com:
Dear lispers,
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
Yes, I've started the tests.
Some lisps are finished already:
abcl-1.3.2-fasl42-linux-x86 ccl-1.10-r16196-f96-linux-x86 ccl-1.8-r15286m-f95-linux-x86 ccl-1.9-r15756-f96-linux-x86 clisp-2.49-unix-x86 cmu-snapshot-2014-01__20e_unicode_-linux-x86 cmu-snapshot-2014-05-dirty__20e_unicode_-linux-x86 cmu-snapshot-2014-12___20f_unicode_-linux-x86 ecl-13.5.1-unknown-linux-i686-bytecode ecl-13.5.1-unknown-linux-i686-lisp-to-c ecl-15.2.21-ee989b97-linux-i686-bytecode sbcl-1.0.58-linux-x86 sbcl-1.1.11-linux-x86 sbcl-1.1.16-linux-x86 sbcl-1.2.6-linux-x86
So far I see no difference from the git version d70a8f8 we tested recently (off the mailing lists).
Other (elder) lisps are still running. I will report as they finish.
Best regards, - Anton
we're ready to bless ASDF 3.1.4.25 as release 3.1.5, and now is a good time to test it before it's too late. Anton, do you have time and/or resource for a run of cl-test-grid?
FWIW, the hu.dwim universe builds without any issues.
We did add better immutable-system support thanks to Dave Cooper,
is this meant to be the final public API (an exported global variable that holds a hashtable)?
maybe a simple defun SYSTEM-MUTABLE-P and a setf variant would be better?
We did add better immutable-system support thanks to Dave Cooper,
is this meant to be the final public API (an exported global variable that holds a hashtable)?
The final API is (register-immutable-system "foo") and, I suppose, (sysdef-immutable-system-search "foo") though I forgot to export said function... fixed.
maybe a simple defun SYSTEM-MUTABLE-P and a setf variant would be better?
Maybe. I'll let Robert decide if he wants a way to make a system mutable no more. Up until now, the usage scenario was that systems would transition one way only from mutable to immutable, as you prepare an image for delivery.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Uniquely in us, nature opens her eyes and sees that she exists. — Raymond Tallis (1946–)
On 7/10/15 Jul 10 -8:34 AM, Faré wrote:
We did add better immutable-system support thanks to Dave Cooper,
is this meant to be the final public API (an exported global variable that holds a hashtable)?
The final API is (register-immutable-system "foo") and, I suppose, (sysdef-immutable-system-search "foo") though I forgot to export said function... fixed.
maybe a simple defun SYSTEM-MUTABLE-P and a setf variant would be better?
Maybe. I'll let Robert decide if he wants a way to make a system mutable no more. Up until now, the usage scenario was that systems would transition one way only from mutable to immutable, as you prepare an image for delivery.
I'm reluctant to add more code unless there's a demonstrated need for the functionality. I'd remind everyone that ASDF has ballooned in size and for all our efforts that inevitably means it has collapsed in maintainability!
If you'd like an extension to provide either introspection or "mutabilification", please file an enhancement ticket. But unless you provide a use-case with it, it is likely to get short shrift.
Certainly I don't want to hold the 3.1.5 release any longer for such an enhancement.
Best, r
maybe a simple defun SYSTEM-MUTABLE-P and a setf variant would be better?
Maybe. I'll let Robert decide if he wants a way to make a system mutable no more. Up until now, the usage scenario was that systems would transition one way only from mutable to immutable, as you prepare an image for delivery.
I'm reluctant to add more code unless there's a demonstrated need for the functionality. I'd remind everyone that ASDF has ballooned in size and for all our efforts that inevitably means it has collapsed in maintainability!
If you'd like an extension to provide either introspection or "mutabilification", please file an enhancement ticket. But unless you provide a use-case with it, it is likely to get short shrift.
no, i'm fine with the functinality of REGISTER-IMMUTABLE-SYSTEM.
what i wasn't fine with is an exported global holding a hashtable, and at that time i hadn't noticed RIS because i was expecting a different name. REGISTER-IMMUTABLE-SYSTEM doesn't register any systems, it rather marks a system, that is already known by asdf, immutable. it's a different story that the implementation registers it in a hashtable as opposed to e.g. setting a flag on the system object.
so, before i knew about RIS, i proposed an API that i still think would be better, namely SYSTEM-MUTABLE-P.
if it's still feasible i suggest to replace REGISTER-IMMUTABLE-SYSTEM with (SETF SYSTEM-MUTABLE-P) and stop exporting *IMMUTABLE-SYSTEMS*.
On Tue, Jul 14, 2015 at 3:40 PM, Attila Lendvai attila@lendvai.name wrote:
no, i'm fine with the functinality of REGISTER-IMMUTABLE-SYSTEM.
what i wasn't fine with is an exported global holding a hashtable, and at that time i hadn't noticed RIS because i was expecting a different name. REGISTER-IMMUTABLE-SYSTEM doesn't register any systems, it rather marks a system, that is already known by asdf, immutable. it's a different story that the implementation registers it in a hashtable as opposed to e.g. setting a flag on the system object.
Sorry, I don't remember why I did it with a hash-table rather than by adding a boolean slot to the class SYSTEM. I believe the rationale was that like with register-preloaded-system, you might want to the system compilation product as a monolithic .fasl and register without even having to a define with defsystem in the current image.
In any case, the current API is REGISTER-IMMUTABLE-SYSTEM and SYSDEF-IMMUTABLE-SYSTEM-SEARCH.
so, before i knew about RIS, i proposed an API that i still think would be better, namely SYSTEM-MUTABLE-P.
if it's still feasible i suggest to replace REGISTER-IMMUTABLE-SYSTEM with (SETF SYSTEM-MUTABLE-P) and stop exporting *IMMUTABLE-SYSTEMS*.
I think it's too late to make changes for 3.1.5.
Indeed, it's probably a bad idea to export *IMMUTABLE-SYSTEMS*. Maybe to late to fix in 3.1.5, but hopefully will be fixed in 3.2.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Ask not what the government can do for you. Ask what the government is doing to you." — David Friedman, "The Machinery of Freedom", p. 21
if it's still feasible i suggest to replace REGISTER-IMMUTABLE-SYSTEM with (SETF SYSTEM-MUTABLE-P) and stop exporting *IMMUTABLE-SYSTEMS*.
I think it's too late to make changes for 3.1.5.
Indeed, it's probably a bad idea to export *IMMUTABLE-SYSTEMS*. Maybe to late to fix in 3.1.5, but hopefully will be fixed in 3.2.
"Ask not what the government can do for you. Ask what the government is doing to you." — David Friedman, "The Machinery of Freedom", p. 21
I think either way will work for us, but if register-immutable-system goes away, it will mean an application code change for us. We are not accessing *immutable-sytems* directly, so no worries there whether it's exported or not.
We do call register-immutable-systems, however.
For the record, here is how we are using it:
We make pre-built Gendl or Genworks GDL distributions which do not include any ASDF or Quicklisp at all, but they are built with monolithic-compile-bundles generated using asdf in our build environment.
For downstream users of prebuilt distributions who then want to load ASDF and Quicklisp, we provide a function, "load-quicklisp," which we ask users to call, rather than directly loading the quicklisp/setup.lisp themselves.
This function takes a :path keyword argument which defaults to the location of the quicklisp/ directory which we ship with our system. So if the user wants to use a different quicklisp/ directory (unsupported by us if they pick a different quicklisp version than what we built with and shipped), they can do it with that :path argument. The more usual case is that they copy our shipped quicklisp/ directory into a location where they have write access, because they want to put stuff into its local-projects/ or do other things which require write-access to the quicklisp/ directory.
The :path binds the dynamic variable *quicklisp-home*, as used in the following code:
(load (merge-pathnames "setup.lisp" *quicklisp-home*))
(defclass asdf::gdl (asdf::cl-source-file) ((type :initform "gdl"))) (defclass asdf::gendl (asdf::cl-source-file) ((type :initform "gendl"))) (defclass asdf::lisp (asdf::cl-source-file) ())
(let ((preloaded gdl::*already-loaded-systems*)) (dolist (system preloaded) (asdf/find-system:register-immutable-system system)))
So as you can see, we maintain a variable gdl::*already-loaded-systems* (which probably ought to be exported, now that I mention it), which is used to establish the immutable-systems upon loading of quicklisp and asdf into the prebuilt image. And we do a few other extra initialization things after loading the quicklisp/setup.lisp.
On 7/14/15 Jul 14 -3:58 PM, Dave Cooper wrote:
> > if it's still feasible i suggest to replace REGISTER-IMMUTABLE-SYSTEM > with (SETF SYSTEM-MUTABLE-P) and stop exporting *IMMUTABLE-SYSTEMS*. > I think it's too late to make changes for 3.1.5. Indeed, it's probably a bad idea to export *IMMUTABLE-SYSTEMS*. Maybe to late to fix in 3.1.5, but hopefully will be fixed in 3.2. "Ask not what the government can do for you. Ask what the government is doing to you." — David Friedman, "The Machinery of Freedom", p. 21I think either way will work for us, but if register-immutable-system goes away, it will mean an application code change for us. We are not accessing *immutable-sytems* directly, so no worries there whether it's exported or not.
We do call register-immutable-systems, however.
For the record, here is how we are using it:
We make pre-built Gendl or Genworks GDL distributions which do not include any ASDF or Quicklisp at all, but they are built with monolithic-compile-bundles generated using asdf in our build environment.
For downstream users of prebuilt distributions who then want to load ASDF and Quicklisp, we provide a function, "load-quicklisp," which we ask users to call, rather than directly loading the quicklisp/setup.lisp themselves.
This function takes a :path keyword argument which defaults to the location of the quicklisp/ directory which we ship with our system. So if the user wants to use a different quicklisp/ directory (unsupported by us if they pick a different quicklisp version than what we built with and shipped), they can do it with that :path argument. The more usual case is that they copy our shipped quicklisp/ directory into a location where they have write access, because they want to put stuff into its local-projects/ or do other things which require write-access to the quicklisp/ directory.
The :path binds the dynamic variable *quicklisp-home*, as used in the following code:
(load (merge-pathnames "setup.lisp" *quicklisp-home*))
(defclass asdf::gdl (asdf::cl-source-file) ((type :initform "gdl"))) (defclass asdf::gendl (asdf::cl-source-file) ((type :initform "gendl"))) (defclass asdf::lisp (asdf::cl-source-file) ())
(let ((preloaded gdl::*already-loaded-systems*)) (dolist (system preloaded) (asdf/find-system:register-immutable-system system)))
So as you can see, we maintain a variable gdl::*already-loaded-systems* (which probably ought to be exported, now that I mention it), which is used to establish the immutable-systems upon loading of quicklisp and asdf into the prebuilt image. And we do a few other extra initialization things after loading the quicklisp/setup.lisp.
I'm inclined to remove the export of *IMMUTABLE-SYSTEMS*. It hasn't been used in a released version of ASDF AFAIK, so it seems benign to remove it.
Cheers, r
I'm inclined to remove the export of *IMMUTABLE-SYSTEMS*. It hasn't been used in a released version of ASDF AFAIK, so it seems benign to remove it.
isn't that also the case for REGISTER-IMMUTABLE-SYSTEM?
if that export sticks in the release then it'll be a headache down the road (assuming that it is indeed an unfortunate name and not just my lone opinion).
On 7/14/15 Jul 14 -6:04 PM, Attila Lendvai wrote:
I'm inclined to remove the export of *IMMUTABLE-SYSTEMS*. It hasn't been used in a released version of ASDF AFAIK, so it seems benign to remove it.
isn't that also the case for REGISTER-IMMUTABLE-SYSTEM?
if that export sticks in the release then it'll be a headache down the road (assuming that it is indeed an unfortunate name and not just my lone opinion).
How would people feel about keeping REGISTER-IMMUTABLE-SYSTEM now (so as not to break existing Genworks code), with it becoming deprecated later? We would immediately put in IMMUTABLE-SYSTEM-P, as Attila suggests, as a the permanent name?
Thanks, r
How would people feel about keeping REGISTER-IMMUTABLE-SYSTEM now (so as not to break existing Genworks code), with it becoming deprecated later? We would immediately put in IMMUTABLE-SYSTEM-P, as Attila suggests, as a the permanent name?
If we are the only ones using it then you can also just go ahead and get rid of it right now. We don't have the code out in the wild, and can future-proof what goes out from now on.
On 7/14/15 Jul 14 -6:04 PM, Attila Lendvai wrote:
I'm inclined to remove the export of *IMMUTABLE-SYSTEMS*. It hasn't been used in a released version of ASDF AFAIK, so it seems benign to remove it.
isn't that also the case for REGISTER-IMMUTABLE-SYSTEM?
if that export sticks in the release then it'll be a headache down the road (assuming that it is indeed an unfortunate name and not just my lone opinion).
OK, I just had a look at rewriting REGISTER-IMMUTABLE-SYSTEM and it is *not* amenable to a rewrite as a setf-able predicate. There are all kinds of side-effecting in that code that would have to be disentangled to make the "query" form of that function work.
If anyone is interested in carrying this through, I'm attaching my abortive version of find-system.lisp with the beginnings of a rewrite.
But it's way more than I want to do before 3.1.5 is released.
Attila, if you care enough, LMK when you think you could submit a patch. Otherwise we go with what we have (except I remove the export of *immutable-systems*).
Best, r
On Tue, Jul 14, 2015 at 8:34 PM, Robert Goldman rpgoldman@sift.net wrote:
On 7/14/15 Jul 14 -6:04 PM, Attila Lendvai wrote:
I'm inclined to remove the export of *IMMUTABLE-SYSTEMS*. It hasn't been used in a released version of ASDF AFAIK, so it seems benign to remove it.
isn't that also the case for REGISTER-IMMUTABLE-SYSTEM?
Well, note that I exported *IMMUTABLE-SYSTEMS* from ASDF/FIND-SYSTEM but *not* from ASDF/INTERFACE (aka ASDF). I believe that was appropriate; but I don't strongly defend this export, either.
if that export sticks in the release then it'll be a headache down the road (assuming that it is indeed an unfortunate name and not just my lone opinion).
OK, I just had a look at rewriting REGISTER-IMMUTABLE-SYSTEM and it is *not* amenable to a rewrite as a setf-able predicate. There are all kinds of side-effecting in that code that would have to be disentangled to make the "query" form of that function work.
Well, if we change the API to add a boolean slot to SYSTEM, we could remove REGISTER-IMMUTABLE-SYSTEM, and make it the responsibility of the user to create the system, whether through register-preloaded-system or not. Actually, if the slot has an initarg keyword, then you can use register-preloaded-system directly to create an immutable system if not already present, or you'll have to use setf to make an otherwise existing one immutable.
I'd rather you just release what we have for now, but if you want, I can make those changes before 3.1.5 (or after).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org There are two kinds of problems with a tool: either it should do a thing and doesn't, or it shouldn't do a thing yet does. The first kind seems just a limitation, you can address or tolerate it. The other one feels like the tool is your enemy. — Michael Raskin
Well, if we change the API to add a boolean slot to SYSTEM, we could
this may be naive, but what i meant is a very simple change: introduce an exported SYSTEM-MUTABLE-P, together with a SETF version, that messes around with the current hashtable based implementation.
IOW, it's pretty much just a rename of REGISTER-IMMUTABLE-SYSTEM -> (SETF SYSTEM-MUTABLE-P), and a new SYSTEM-MUTABLE-P that queries the internal hashtable. the (setf ... false) version may be a not-yet-implemented error if there's too much state to mess around with before this release.
moving to a slot based implementation is another story that can be delayed, and may not even be preferable (e.g. what if someone sometimes wants a set of systems to be immutable, and some other time not? although, this also applies to every slot of SYSTEM, so...).
remove REGISTER-IMMUTABLE-SYSTEM, and make it the responsibility of the user to create the system, whether through register-preloaded-system or not. Actually, if the slot has an initarg keyword, then you can use register-preloaded-system directly to create an immutable system if not already present, or you'll have to use setf to make an otherwise existing one immutable.
I'd rather you just release what we have for now, but if you want, I can make those changes before 3.1.5 (or after).
another alternative is to just keep all these symbols private and delay their exporting. optionally multiple variants could be added but not exported, so that early adaptors can already use the to-be-published API by using ASDF:: prefix. probably it's only the two of us participating in this thread who are using this new feature.
an untested sketch is attached that merely renames what's already there, and errors at (setf ... false).
but to reiterate: i don't have hard feelings about this. i'm just trying to help avoiding the publishing of a confusing name that will be much harder to change down the road.
On 7/15/15 Jul 15 -2:23 PM, Attila Lendvai wrote:
Well, if we change the API to add a boolean slot to SYSTEM, we could
this may be naive, but what i meant is a very simple change: introduce an exported SYSTEM-MUTABLE-P, together with a SETF version, that messes around with the current hashtable based implementation.
I looked at that, but it looks like the actual code fuses a bunch of stuff about *looking up* a system (the system argument is a system name) together with actually making the system immutable.
I think if we were to do what you want, instead of doing a half-way job by just renaming what's there and adding a SETF method, it would make a lot more sense to decouple the lookup from the setting, and have SYSTEM-MUTABLE-P take a real *SYSTEM* as argument, instead of having an odd API where only a system-NAME is acceptable.
Then there's the question of what to do with the :VERSION argument when we pass a real system, and when SYSTEM-IMMUTABLE-P is used in a query mode.
I started banging on this, and the alarm bells for "this is an ill-thought-out hack that you will regret later" began to go off.
So I'd prefer to do a good job of this, later, instead of digging myself another hole I'll have to fill in later.
I think having a confusing name that we deprecate is better than taking the good name, and implementing something bad on it. This way we can think about a better API and put the better API on the better name.
Best, Robert
IOW, it's pretty much just a rename of REGISTER-IMMUTABLE-SYSTEM -> (SETF SYSTEM-MUTABLE-P), and a new SYSTEM-MUTABLE-P that queries the internal hashtable. the (setf ... false) version may be a not-yet-implemented error if there's too much state to mess around with before this release.
moving to a slot based implementation is another story that can be delayed, and may not even be preferable (e.g. what if someone sometimes wants a set of systems to be immutable, and some other time not? although, this also applies to every slot of SYSTEM, so...).
remove REGISTER-IMMUTABLE-SYSTEM, and make it the responsibility of the user to create the system, whether through register-preloaded-system or not. Actually, if the slot has an initarg keyword, then you can use register-preloaded-system directly to create an immutable system if not already present, or you'll have to use setf to make an otherwise existing one immutable.
I'd rather you just release what we have for now, but if you want, I can make those changes before 3.1.5 (or after).
another alternative is to just keep all these symbols private and delay their exporting. optionally multiple variants could be added but not exported, so that early adaptors can already use the to-be-published API by using ASDF:: prefix. probably it's only the two of us participating in this thread who are using this new feature.
an untested sketch is attached that merely renames what's already there, and errors at (setf ... false).
but to reiterate: i don't have hard feelings about this. i'm just trying to help avoiding the publishing of a confusing name that will be much harder to change down the road.
SYSTEM-MUTABLE-P take a real *SYSTEM* as argument, instead of having an odd API where only a system-NAME is acceptable.
note:
CL-USER> (asdf:coerce-name (asdf:find-system :hu.dwim.def)) "hu.dwim.def"
I think having a confusing name that we deprecate is better than taking the good name, and implementing something bad on it. This way we can think about a better API and put the better API on the better name.
i thought (and stated previously, with a doubt) that RIS is something new, but now that i double checked i realized it's already there since 2014 aug, exported:
https://github.com/fare/asdf/commit/1b38225b8cc5749fafac9f300ac469fd92beba86
it's a lost case then, it's already published, so there's no way other than the deprecation way. in that case it's not an urgent issue, just put it on the TODO.
sorry for the noise!
On Wed, Jul 15, 2015 at 10:32 PM, Attila Lendvai attila@lendvai.name wrote:
SYSTEM-MUTABLE-P take a real *SYSTEM* as argument, instead of having an odd API where only a system-NAME is acceptable.
note:
CL-USER> (asdf:coerce-name (asdf:find-system :hu.dwim.def)) "hu.dwim.def"
I think having a confusing name that we deprecate is better than taking the good name, and implementing something bad on it. This way we can think about a better API and put the better API on the better name.
i thought (and stated previously, with a doubt) that RIS is something new, but now that i double checked i realized it's already there since 2014 aug, exported:
https://github.com/fare/asdf/commit/1b38225b8cc5749fafac9f300ac469fd92beba86
it's a lost case then, it's already published, so there's no way other than the deprecation way. in that case it's not an urgent issue, just put it on the TODO.
It's been written last august, but in a branch that was rebased into master around 3.1.4.15, so not all is lost. The only user so far is Dave Cooper, and he's watching the list, so we can safely change the API. The deeper question is: what exactly is the API we want?
Now that I look at the code, I believe one reason that register-immutable-system works the way it does is so as to still work after all systems are cleared, e.g. by a non-backward-compatible asdf upgrade. See the (register-hook-function '*post-upgrade-cleanup-hook* 'clear-defined-systems nil) form in asdf/find-system.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Never abandon a theory that explains something until you have a theory that explains more. — John McCarthy
(load (merge-pathnames "setup.lisp" *quicklisp-home*))
Minor nit: this code isn't compliant ANSI CL. (make-pathname :name "setup" :type "lisp" :defaults *quicklisp-home*) is the compliant version. Indeed, if *default-pathname-defaults* has non-null HOST and DEVICE that differ from *quicklisp-home*, you lose; and you can't explicitly specify :HOST NIL yourself, it's not portable.
And that's the reason for UIOP:MERGE-PATHNAMES* and UIOP:SUBPATHNAME; but you can't use them when ASDF isn't loaded yet.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The hacker: someone who figured things out and made something cool happen. — Alan Schmitt
Hi!
SBCL on Windows isn't quite as good (and is not able to call out to CMD.EXE)
I encountered the problem recently, reported a bug and I seem to have a workaround. Feel free to experiment with it.
-
73budden . -
Anton Vodonosov -
Attila Lendvai -
Dave Cooper -
Faré -
Mark Evenson -
Robert Goldman -
Robert P. Goldman