[Asdf-devel] Registering preloaded systems for Quicklisp/ASDF
Hi, I know this has been discussed, but what is currently the best practice for registering a preloaded system after loading Quicklisp? That is, I have a pre-built image with some Quicklisp/ASDF systems in it, which were loaded through so-called monolithic-fasls (now known as compile-bundles). So the pre-built image doesn’t itself have ASDF or Quicklisp. A downstream user then loads Quicklisp and ASDF (I take steps to ensure that they are loading the matching version of ASDF and Quicklisp as was used to build the image). Now I need to do something to tell Quicklisp and ASDF that certain systems are already loaded in this image, so it won’t try to re-load them as depended-upon systems. In this specific case, I don't want it to try to fetch and reload Gendl when the user does (ql:quickload ...) on an application which :depends-on :gendl (because gendl is already built into this image). asdf/find-system:register-preloaded-system seems like it should be the answer if we’re talking about plain ASDF, because it will prevent ASDF from raising a “missing-component” error for the system in question, if some other system :depends-on it and yet no source code is found. But in my case I can’t guarantee that “no source code is found.” The source might be there in the quicklisp dist directory. So apparently I need something stronger than register-preloaded-system? Or something else? -- My Best, Dave Cooper
On Fri, Aug 8, 2014 at 10:54 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Hi, I know this has been discussed, but what is currently the best practice for registering a preloaded system after loading Quicklisp? That is, I have a pre-built image with some Quicklisp/ASDF systems in it, which were loaded through so-called monolithic-fasls (now known as compile-bundles). So the pre-built image doesn’t itself have ASDF or Quicklisp.
A downstream user then loads Quicklisp and ASDF (I take steps to ensure that they are loading the matching version of ASDF and Quicklisp as was used to build the image). Now I need to do something to tell Quicklisp and ASDF that certain systems are already loaded in this image, so it won’t try to re-load them as depended-upon systems. In this specific case, I don't want it to try to fetch and reload Gendl when the user does (ql:quickload ...) on an application which :depends-on :gendl (because gendl is already built into this image).
asdf/find-system:register-preloaded-system seems like it should be the answer if we’re talking about plain ASDF, because it will prevent ASDF from raising a “missing-component” error for the system in question, if some other system :depends-on it and yet no source code is found. But in my case I can’t guarantee that “no source code is found.” The source might be there in the quicklisp dist directory. So apparently I need something stronger than register-preloaded-system? Or something else?
Using asdf 3.1, you can use *immutable-systems* for that (see its docstring). There is currently no provision to do it as part of a program-op, so you may have to do it manually between a load-op and a program-op, or as part of an image-dump-hook or image-restore-hook, contingent on a variable that gets reset (the image-dump-hook doesn't work on ECL, and you must use the ECL-specific epilogue instead; the image-dump-hook wastes a little bit of time at restart on other platforms). —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The problem with socialism is that eventually you run out of other people's money. — Margaret Thatcher
Faré wrote:
Using asdf 3.1, you can use *immutable-systems* for that (see its
docstring).
Ok... After initializing Quicklisp and ASDF, I do: (setq asdf:*immutable-systems* (uiop:list-to-hash-set "gendl" ... )) as well as (asdf/find-system:register-preloaded-system "gendl") ... with all the known preloaded systems. But now if I do (ql:quickload :gendl) I still get this error: "Requested system gendl is in the *immutable-systems* set, but not loaded in memory" Apparently, (asdf:system-registered-p "gendl") is still coming back as NIL. Is there another step I should be doing? On Sat, Aug 9, 2014 at 1:09 AM, Faré <fahree@gmail.com> wrote:
Hi, I know this has been discussed, but what is currently the best
for registering a preloaded system after loading Quicklisp? That is, I have a pre-built image with some Quicklisp/ASDF systems in it, which were loaded through so-called monolithic-fasls (now known as compile-bundles). So the pre-built image doesn’t itself have ASDF or Quicklisp.
A downstream user then loads Quicklisp and ASDF (I take steps to ensure
they are loading the matching version of ASDF and Quicklisp as was used to build the image). Now I need to do something to tell Quicklisp and ASDF that certain systems are already loaded in this image, so it won’t try to re-load them as depended-upon systems. In this specific case, I don't want it to try to fetch and reload Gendl when the user does (ql:quickload ...) on an application which :depends-on :gendl (because gendl is already built into this image).
asdf/find-system:register-preloaded-system seems like it should be the answer if we’re talking about plain ASDF, because it will prevent ASDF from raising a “missing-component” error for the system in question, if some other system :depends-on it and yet no source code is found. But in my case I can’t guarantee that “no source code is found.” The source might be
On Fri, Aug 8, 2014 at 10:54 PM, Dave Cooper <david.cooper@genworks.com> wrote: practice that there
in the quicklisp dist directory. So apparently I need something stronger than register-preloaded-system? Or something else?
Using asdf 3.1, you can use *immutable-systems* for that (see its docstring).
There is currently no provision to do it as part of a program-op, so you may have to do it manually between a load-op and a program-op, or as part of an image-dump-hook or image-restore-hook, contingent on a variable that gets reset (the image-dump-hook doesn't work on ECL, and you must use the ECL-specific epilogue instead; the image-dump-hook wastes a little bit of time at restart on other platforms).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The problem with socialism is that eventually you run out of other people's money. — Margaret Thatcher
-- My Best, Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
Here is a hack which seems to work for the moment but clearly is not the Right Way, because it's doing surgery on asdf::*defined-systems*. [assume `known-preloaded-systems' is a list of strings naming the known preloaded systems]: (dolist (system known-preloaded-systems) (setf (gethash system asdf::*defined-systems*) (cons nil (make-instance 'asdf:system :name system)))) But I think this is effectively what I want to achieve here. On Sat, Aug 9, 2014 at 3:27 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Faré wrote:
Using asdf 3.1, you can use *immutable-systems* for that (see its
docstring).
Ok... After initializing Quicklisp and ASDF, I do:
(setq asdf:*immutable-systems* (uiop:list-to-hash-set "gendl" ... ))
as well as
(asdf/find-system:register-preloaded-system "gendl") ...
with all the known preloaded systems.
But now if I do
(ql:quickload :gendl)
I still get this error:
"Requested system gendl is in the *immutable-systems* set, but not loaded in memory"
Apparently,
(asdf:system-registered-p "gendl")
is still coming back as NIL.
Is there another step I should be doing?
On Sat, Aug 9, 2014 at 1:09 AM, Faré <fahree@gmail.com> wrote:
Hi, I know this has been discussed, but what is currently the best
for registering a preloaded system after loading Quicklisp? That is, I have a pre-built image with some Quicklisp/ASDF systems in it, which were loaded through so-called monolithic-fasls (now known as compile-bundles). So
pre-built image doesn’t itself have ASDF or Quicklisp.
A downstream user then loads Quicklisp and ASDF (I take steps to ensure
they are loading the matching version of ASDF and Quicklisp as was used to build the image). Now I need to do something to tell Quicklisp and ASDF that certain systems are already loaded in this image, so it won’t try to re-load them as depended-upon systems. In this specific case, I don't want it to try to fetch and reload Gendl when the user does (ql:quickload ...) on an application which :depends-on :gendl (because gendl is already built into this image).
asdf/find-system:register-preloaded-system seems like it should be the answer if we’re talking about plain ASDF, because it will prevent ASDF from raising a “missing-component” error for the system in question, if some other system :depends-on it and yet no source code is found. But in my case I can’t guarantee that “no source code is found.” The source might be
On Fri, Aug 8, 2014 at 10:54 PM, Dave Cooper <david.cooper@genworks.com> wrote: practice the that there
in the quicklisp dist directory. So apparently I need something stronger than register-preloaded-system? Or something else?
Using asdf 3.1, you can use *immutable-systems* for that (see its docstring).
There is currently no provision to do it as part of a program-op, so you may have to do it manually between a load-op and a program-op, or as part of an image-dump-hook or image-restore-hook, contingent on a variable that gets reset (the image-dump-hook doesn't work on ECL, and you must use the ECL-specific epilogue instead; the image-dump-hook wastes a little bit of time at restart on other platforms).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The problem with socialism is that eventually you run out of other people's money. — Margaret Thatcher
-- My Best,
Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
-- My Best, Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
Sorry, but doesn't it work to set *immutable-systems* AFTER you load everything? I admit there not yet a test for this functionality in the asdf test suite; it may be that the functionality is buggy and/or its API suboptimal, and you're welcome to suggest patches and/or suggestions. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Gun: weapon of individual vs individual. Bomb: weapon of group vs group. That's why collectivists hate guns and love bombs. On Sat, Aug 9, 2014 at 3:49 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Here is a hack which seems to work for the moment but clearly is not the Right Way, because it's doing surgery on asdf::*defined-systems*.
[assume `known-preloaded-systems' is a list of strings naming the known preloaded systems]:
(dolist (system known-preloaded-systems) (setf (gethash system asdf::*defined-systems*) (cons nil (make-instance 'asdf:system :name system))))
But I think this is effectively what I want to achieve here.
On Sat, Aug 9, 2014 at 3:27 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Faré wrote:
Using asdf 3.1, you can use *immutable-systems* for that (see its docstring).
Ok... After initializing Quicklisp and ASDF, I do:
(setq asdf:*immutable-systems* (uiop:list-to-hash-set "gendl" ... ))
as well as
(asdf/find-system:register-preloaded-system "gendl") ...
with all the known preloaded systems.
But now if I do
(ql:quickload :gendl)
I still get this error:
"Requested system gendl is in the *immutable-systems* set, but not loaded in memory"
Apparently,
(asdf:system-registered-p "gendl")
is still coming back as NIL.
Is there another step I should be doing?
On Sat, Aug 9, 2014 at 1:09 AM, Faré <fahree@gmail.com> wrote:
On Fri, Aug 8, 2014 at 10:54 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Hi, I know this has been discussed, but what is currently the best practice for registering a preloaded system after loading Quicklisp? That is, I have a pre-built image with some Quicklisp/ASDF systems in it, which were loaded through so-called monolithic-fasls (now known as compile-bundles). So the pre-built image doesn’t itself have ASDF or Quicklisp.
A downstream user then loads Quicklisp and ASDF (I take steps to ensure that they are loading the matching version of ASDF and Quicklisp as was used to build the image). Now I need to do something to tell Quicklisp and ASDF that certain systems are already loaded in this image, so it won’t try to re-load them as depended-upon systems. In this specific case, I don't want it to try to fetch and reload Gendl when the user does (ql:quickload ...) on an application which :depends-on :gendl (because gendl is already built into this image).
asdf/find-system:register-preloaded-system seems like it should be the answer if we’re talking about plain ASDF, because it will prevent ASDF from raising a “missing-component” error for the system in question, if some other system :depends-on it and yet no source code is found. But in my case I can’t guarantee that “no source code is found.” The source might be there in the quicklisp dist directory. So apparently I need something stronger than register-preloaded-system? Or something else?
Using asdf 3.1, you can use *immutable-systems* for that (see its docstring).
There is currently no provision to do it as part of a program-op, so you may have to do it manually between a load-op and a program-op, or as part of an image-dump-hook or image-restore-hook, contingent on a variable that gets reset (the image-dump-hook doesn't work on ECL, and you must use the ECL-specific epilogue instead; the image-dump-hook wastes a little bit of time at restart on other platforms).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The problem with socialism is that eventually you run out of other people's money. — Margaret Thatcher
-- My Best,
Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
-- My Best,
Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
On Sat, Aug 9, 2014 at 5:16 PM, Faré <fahree@gmail.com> wrote:
Sorry, but doesn't it work to set *immutable-systems* AFTER you load everything?
No, it doesn't, because we never actually load anything via ASDF. Everything was preloaded into the built image, using monolithic-compile-bundles, sans actual ASDF.
I admit there not yet a test for this functionality in the asdf test suite; it may be that the functionality is buggy and/or its API suboptimal, and you're welcome to suggest patches and/or suggestions.
Ok, here is a patch (against 3.1.3.2 of master) which: o Makes it check for sysdef-preloaded-system-search as well as system-registered-p, when doing sysdef-immutable-system-search and when checking for *immutable-systems* in find-system. (previously it was only checking system-registered-p). o Adds a function asdf/find-system:register-immutable-system which adds the requested system to the *immutable-systems* as well as calls register-preloaded-system with it. -- My Best, Dave Cooper, Genworks Support
Dear Dave, your patch looks great except that it introduces tabs. Please no tabs in ASDF. At least SBCL hates them. Also, if you can contrive a test case, maybe as an add-on to the image-op test in test-program.script, that would be great. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org drug, n: A substance that, injected into a rat, produces a scientific paper. On Sun, Aug 10, 2014 at 4:06 PM, Dave Cooper <david.cooper@genworks.com> wrote:
On Sat, Aug 9, 2014 at 5:16 PM, Faré <fahree@gmail.com> wrote:
Sorry, but doesn't it work to set *immutable-systems* AFTER you load everything?
No, it doesn't, because we never actually load anything via ASDF. Everything was preloaded into the built image, using monolithic-compile-bundles, sans actual ASDF.
I admit there not yet a test for this functionality in the asdf test suite; it may be that the functionality is buggy and/or its API suboptimal, and you're welcome to suggest patches and/or suggestions.
Ok, here is a patch (against 3.1.3.2 of master) which:
o Makes it check for sysdef-preloaded-system-search as well as system-registered-p, when doing sysdef-immutable-system-search and when checking for *immutable-systems* in find-system. (previously it was only checking system-registered-p).
o Adds a function asdf/find-system:register-immutable-system which adds the requested system to the *immutable-systems* as well as calls register-preloaded-system with it.
-- My Best,
Dave Cooper, Genworks Support
Here it is untabified. I'll have to study the test-program.script a bit before attempting a test case. On Sun, Aug 10, 2014 at 7:31 PM, Faré <fahree@gmail.com> wrote:
Dear Dave,
your patch looks great except that it introduces tabs. Please no tabs in ASDF. At least SBCL hates them.
Also, if you can contrive a test case, maybe as an add-on to the image-op test in test-program.script, that would be great.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org drug, n: A substance that, injected into a rat, produces a scientific paper.
On Sun, Aug 10, 2014 at 4:06 PM, Dave Cooper <david.cooper@genworks.com> wrote:
On Sat, Aug 9, 2014 at 5:16 PM, Faré <fahree@gmail.com> wrote:
Sorry, but doesn't it work to set *immutable-systems* AFTER you load everything?
No, it doesn't, because we never actually load anything via ASDF.
was preloaded into the built image, using monolithic-compile-bundles, sans actual ASDF.
I admit there not yet a test for this functionality in the asdf test suite; it may be that the functionality is buggy and/or its API suboptimal, and you're welcome to suggest patches and/or suggestions.
Ok, here is a patch (against 3.1.3.2 of master) which:
o Makes it check for sysdef-preloaded-system-search as well as system-registered-p, when doing sysdef-immutable-system-search and when checking for *immutable-systems* in find-system. (previously it was only checking system-registered-p).
o Adds a function asdf/find-system:register-immutable-system which adds
Everything the
requested system to the *immutable-systems* as well as calls register-preloaded-system with it.
-- My Best,
Dave Cooper, Genworks Support
-- My Best, Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
Now that I think of it, I see a problem with your solution: there can be situations where you register your system as immutable and preloaded, but since it's not registered as defined yet, find-system will still look for it on disk. I believe you need to either have you register-immutable-system do the registering a defined system in *registered-systems* rather than trying to use system-registered-p, or further uglify find-system.lisp to short-circuit the search when a system is immutable and preloaded. I believe the former is what you want: make sure the system is always defined. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If Government is the solution to any alleged "failure" of free society, what is the solution to blatant failures of Government? On Sun, Aug 10, 2014 at 8:32 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Here it is untabified.
I'll have to study the test-program.script a bit before attempting a test case.
On Sun, Aug 10, 2014 at 7:31 PM, Faré <fahree@gmail.com> wrote:
Dear Dave,
your patch looks great except that it introduces tabs. Please no tabs in ASDF. At least SBCL hates them.
Also, if you can contrive a test case, maybe as an add-on to the image-op test in test-program.script, that would be great.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org drug, n: A substance that, injected into a rat, produces a scientific paper.
On Sun, Aug 10, 2014 at 4:06 PM, Dave Cooper <david.cooper@genworks.com> wrote:
On Sat, Aug 9, 2014 at 5:16 PM, Faré <fahree@gmail.com> wrote:
Sorry, but doesn't it work to set *immutable-systems* AFTER you load everything?
No, it doesn't, because we never actually load anything via ASDF. Everything was preloaded into the built image, using monolithic-compile-bundles, sans actual ASDF.
I admit there not yet a test for this functionality in the asdf test suite; it may be that the functionality is buggy and/or its API suboptimal, and you're welcome to suggest patches and/or suggestions.
Ok, here is a patch (against 3.1.3.2 of master) which:
o Makes it check for sysdef-preloaded-system-search as well as system-registered-p, when doing sysdef-immutable-system-search and when checking for *immutable-systems* in find-system. (previously it was only checking system-registered-p).
o Adds a function asdf/find-system:register-immutable-system which adds the requested system to the *immutable-systems* as well as calls register-preloaded-system with it.
-- My Best,
Dave Cooper, Genworks Support
-- My Best,
Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
Thanks to both of you for your work on this patch. I'm going to hold off on applying any patches, though, until there's a test case. Dave, I understand that it might be hard to wrap your head around the test script code, so let's start by brainstorming what the test should do, rather than digging in and getting confused with the details of implementation. I believe what is wanted is a test where we make an image with some system preloaded, try to mark it as immutable, and then check to see if it's reloaded. Is that correct? Cheers, r
On Mon, Aug 11, 2014 at 10:19 AM, Robert P. Goldman <rpgoldman@sift.info> wrote:
Thanks to both of you for your work on this patch.
I'm going to hold off on applying any patches, though, until there's a test case.
Dave, I understand that it might be hard to wrap your head around the test script code, so let's start by brainstorming what the test should do, rather than digging in and getting confused with the details of implementation.
I believe what is wanted is a test where we make an image with some system preloaded, try to mark it as immutable, and then check to see if it's reloaded. Is that correct?
I think you'd want to (1) compile a bundle from one slave process, or in the master process then delete suitable packages and clear-defined-systems. (2) in another slave process (or master process, after clean up), load the bundle and call register-immutable-system, then ((re)configure asdf if needed) and try to load the system, and see that the system loading was a success yet that the .asd file wasn't loaded (e.g. the .asd can contain a defpackage or defparameter that you check didn't happen) Considering that it can be done without a slave process if only you clean up, maybe the best place for that isn't the test-program.script, but test-bundle.script. As for what the patch itself should do, I believe the right thing to do would be for register-immutable-system to use (register-system (make-preloaded-system ...)). Note that you might want to preserve the version information for your system, if already registered, and/or if a keyword is given to register-immutable-system (with e.g. the default t meaning check for an already defined system). Finally, you might also want to patch clear-defined-systems so it doesn't undefine the immutable-systems. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org No woman ever falls in love with a man unless she has a better opinion of him than he deserves. — Edgar Watson Howe
On Mon, Aug 11, 2014 at 10:36 AM, Faré <fahree@gmail.com> wrote:
On Mon, Aug 11, 2014 at 10:19 AM, Robert P. Goldman <rpgoldman@sift.info> wrote:
Thanks to both of you for your work on this patch.
I'm going to hold off on applying any patches, though, until there's a test case.
Dave, I understand that it might be hard to wrap your head around the test script code, so let's start by brainstorming what the test should do, rather than digging in and getting confused with the details of implementation.
I believe what is wanted is a test where we make an image with some system preloaded, try to mark it as immutable, and then check to see if it's reloaded. Is that correct?
I think you'd want to (1) compile a bundle from one slave process, or in the master process then delete suitable packages and clear-defined-systems. (2) in another slave process (or master process, after clean up), load the bundle and call register-immutable-system, then ((re)configure asdf if needed) and try to load the system, and see that the system loading was a success yet that the .asd file wasn't loaded (e.g. the .asd can contain a defpackage or defparameter that you check didn't happen)
Considering that it can be done without a slave process if only you clean up, maybe the best place for that isn't the test-program.script, but test-bundle.script.
Now that I think of it, the test should also compare the situation with not using register-immutable-system, so first try to load, cleanup, reload, and check that reload happens; then cleanup, register-immutable-system, and check that reload does not happen. Also, check that the system doesn't disappear when you try to clear it. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org A tautology is a thing which is tautological.
Ok here is an updated patch which does a register-system as well as register-preloaded-system as part of register-immutable-system. It tries to preserve version information according to what Faré recommended. It also patches clear-system and clear-defined-system so as to prevent clearing any systems which occur in the *immutable-systems* table. In the case when a system is immutable, this also now affects the return-value of these clearing functions. On Mon, Aug 11, 2014 at 10:19 AM, Robert P. Goldman <rpgoldman@sift.info> wrote:
Thanks to both of you for your work on this patch.
I'm going to hold off on applying any patches, though, until there's a test case.
Dave, I understand that it might be hard to wrap your head around the test script code, so let's start by brainstorming what the test should do, rather than digging in and getting confused with the details of implementation.
I believe what is wanted is a test where we make an image with some system preloaded, try to mark it as immutable, and then check to see if it's reloaded. Is that correct?
Cheers, r
-- My Best, Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
Getting there! More nits: * Up until now, clear-defined-system had no declared return value; I propose you instead return T if successful, NIL if unsuccessful, and just use an unless rather than if. * It's unclear to me what clear-system does or should do that differs from clear-defined-system. I see that clear-system is documented in asdf.texinfo, but clear-defined-system isn't; furthermore, clear-defined-system doesn't appear anywhere in quicklisp, as I can tell grepping through ~/quicklisp/dists/quicklisp/software after I install all of quicklisp (see my repo git://common-lisp.net/users/frideau/ql-test.git for the helpers I use). I believe it's the fruit of some previous fumbling with a bug, and should be deleted, its contents moved to clear-system. * I would have used a nested if rather than repeat (eql version t), but that's me. * you shouldn't use find-system in register-immutable-system, since it might mess things up; instead, consult the cdr of registered? * don't call system-registered-p system-name twice. * The targeted style is to usually skip one line between function, unless tightly related (0 line) or conceptually separate (two lines). If conceptually unrelated, a new with-upgradability clause. A year ago, I would have fixed it all myself — today I'm just telling you how to do it, and hoping that it helps create new ASDF developers to replace me. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Most economic fallacies derive... from the tendency to assume that there is a fixed pie, that one party can gain only at the expense of another. — Milton Friedman On Mon, Aug 11, 2014 at 12:32 PM, Dave Cooper <david.cooper@genworks.com> wrote:
Ok here is an updated patch which does a register-system as well as register-preloaded-system as part of register-immutable-system. It tries to preserve version information according to what Faré recommended.
It also patches clear-system and clear-defined-system so as to prevent clearing any systems which occur in the *immutable-systems* table. In the case when a system is immutable, this also now affects the return-value of these clearing functions.
On Mon, Aug 11, 2014 at 10:19 AM, Robert P. Goldman <rpgoldman@sift.info> wrote:
Thanks to both of you for your work on this patch.
I'm going to hold off on applying any patches, though, until there's a test case.
Dave, I understand that it might be hard to wrap your head around the test script code, so let's start by brainstorming what the test should do, rather than digging in and getting confused with the details of implementation.
I believe what is wanted is a test where we make an image with some system preloaded, try to mark it as immutable, and then check to see if it's reloaded. Is that correct?
Cheers, r
-- My Best,
Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699
* It's unclear to me what clear-system does or should do that differs from clear-defined-system. I see that clear-system is documented in asdf.texinfo, but clear-defined-system isn't; furthermore, clear-defined-system doesn't appear anywhere in quicklisp, as I can tell grepping through ~/quicklisp/dists/quicklisp/software after I install all of quicklisp (see my repo git://common-lisp.net/users/frideau/ql-test.git for the helpers I use). I believe it's the fruit of some previous fumbling with a bug, and should be deleted, its contents moved to clear-system.
What about clear-defined-systems (plural)? This is exported functionality and it calls clear-defined-system. Should it be changed to clear-systems and made to call clear-system, or kept the same and made to call clear-system? -- My Best, Dave Cooper, Genworks Support
On Mon, Aug 11, 2014 at 1:12 PM, Dave Cooper <david.cooper@genworks.com> wrote:
What about clear-defined-systems (plural)?
This is exported functionality and it calls clear-defined-system.
Should it be changed to clear-systems and made to call clear-system, or kept the same and made to call clear-system?
I believe it should keep its name and just be made to call clear-system. I called it clear-defined-systems rather than clear-system because it clears all the defined systems, as opposed to a specified list of systems. clear-system clears a specified system. I wouldn't rename it. Moreover, since it's exported, renaming should be a maintainer's prerogative. Note though that no one in Quicklisp uses it either. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Speed has always been important otherwise one wouldn't need the computer. —Seymour Cray
On Mon, Aug 11, 2014 at 1:02 PM, Faré <fahree@gmail.com> wrote:
Getting there!
More nits:
Please see attached patch, still against 3.1.3.2 of master.
A year ago, I would have fixed it all myself — today I'm just telling you how to do it, and hoping that it helps create new ASDF developers to replace me.
Thank you for the continued hand-holding! -- My Best, Dave Cooper, Genworks Support
Almost there. You did rename the function in a call to register-hook-function. Oops. Can you run tests on at least one implementation before you submit? Also, for historical reasons, the registered-system thing is a cons cell, not a list. cdr is more appropriate than rest, here. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Science is a cooperative process based on an attitude of logic, imagination and doubt. On Mon, Aug 11, 2014 at 1:38 PM, Dave Cooper <david.cooper@genworks.com> wrote:
On Mon, Aug 11, 2014 at 1:02 PM, Faré <fahree@gmail.com> wrote:
Getting there!
More nits:
Please see attached patch, still against 3.1.3.2 of master.
A year ago, I would have fixed it all myself — today I'm just telling you how to do it, and hoping that it helps create new ASDF developers to replace me.
Thank you for the continued hand-holding!
-- My Best,
Dave Cooper, Genworks Support
Has there been any more progress on this front, since 11 August? That was the last email I saw on the subject. Best, r
On Mon, Aug 25, 2014 at 7:39 PM, Robert P. Goldman <rpgoldman@sift.info> wrote:
Has there been any more progress on this front, since 11 August? That was the last email I saw on the subject.
Dave and I had a brief conversation about it at ILC 2014. My advice was to modify test-bundle.script and not bother with subprocesses, just with detecting side-effects in a .asd file (defpackage or defparameter), and undoing it (if needed) before the next test: compare the situation before and after making the bundle, undo effects; compare the situation before and after loading the bundle, undo effects; mark the systems immutable, touch for modification; compare the situation before and after trying to update. Repeat the test without marking the system immutable. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I hold that America, Champion of the World is a betrayal of its humble beginnings as One Nation Underdog.
Somehow I just now noticed the followups to this thread, sorry for the late response. I would like to keep the thread alive though. I was hoping to have that done and submit the immutable-systems patch before the next major release, but obviously I'm too late for that now -- I happen to be starting the Windows tests for 3.1.4.13 right now, which are apparently the last hurdle for a 3.1.5 release, and I certainly don't want to suggest holding up the 3.1.5 release by trying to cram the immutable-systems patch into it. After 3.1.5 is out the door I will attempt it during the subsequent window of opportunity. I admit I've been shipping my immutable-systems patch privately with the asdf shipped with Gendl and GDL pre-built distributions since last summer. This is obviously dangerous practice on our part and has to stop. On Tue, Aug 26, 2014 at 1:52 AM, Faré <fahree@gmail.com> wrote:
On Mon, Aug 25, 2014 at 7:39 PM, Robert P. Goldman <rpgoldman@sift.info> wrote:
Has there been any more progress on this front, since 11 August? That was the last email I saw on the subject.
Dave and I had a brief conversation about it at ILC 2014. My advice was to modify test-bundle.script and not bother with subprocesses, just with detecting side-effects in a .asd file (defpackage or defparameter), and undoing it (if needed) before the next test: compare the situation before and after making the bundle, undo effects; compare the situation before and after loading the bundle, undo effects; mark the systems immutable, touch for modification; compare the situation before and after trying to update. Repeat the test without marking the system immutable.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I hold that America, Champion of the World is a betrayal of its humble beginnings as One Nation Underdog.
-- My Best, Dave Cooper genworks.com, gendl.org +1 248-330-2979
Dear Dave, I created a branch immutable-systems in which I imported your code. I cleaned it up your code, added a test, found bugs, fixed them. Please try with the code in the branch. It might be a bit late for your patch to make it to 3.1.5 (although it's pretty independent from other code, so should be harmless, as long as the tests still pass), but if not, I'd love to have it in 3.2. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Language independence means you're proud of not calling your barking a language — Faré, about designers of DBMS, OS or other software who claim such
participants (3)
-
Dave Cooper -
Faré -
Robert P. Goldman