Dear CMUCL implementors,
below is the announcement for ASDF 2.016, as posted on the asdf-announce mailing-list, also accessible via its gmane mirror gmane.lisp.asdf.announce. Please have me take your list off my forwarding aliases if unappropriate.
Can you upgrade your implementation to ship with the latest ASDF? Thanks! If you're still using 2.015 (or something even older), there have been many bug fixes, robustness, portability and upgradability enhancements and a few features implemented since.
Also, the CLISP maintainers insist on having (require :asdf) not work and (require "asdf") work, whereas CMUCL does the opposite. Since the CLISP maintainers won't seem to budge, could you have CMUCL accept both?
PS: I noticed you moved the mailing-list to common-lisp.net; the old address @cons.org seems to still exist, but rejects messages as my not being subscribed. Or do both addresses exist? That's confusing.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Consider this: the first hole ever dug on the moon by a man-made machine is now done. It is the most expensive hole in the history of the human race. Now, what does that mean? How do we know whether this is one of man's noblest achievements or if it is a game being played by a small group of lunatics for their own amusement — at our expense?" — Neil Postman, "Teaching as a Subversive Activity"
---------- Forwarded message ---------- From: Faré fahree@gmail.com Date: 7 June 2011 20:30 Subject: ASDF 2.016 released To: asdf-announce@common-lisp.net Cc: ASDF-devel asdf-devel@common-lisp.net
Dear CL users,
I released ASDF 2.015.12 as 2.016.
Please test, and upon success update your implementations and lisp distributions.
Since 2.015, we have the following improvements:
* Portability: Fixes for ECL, RMCL. Tweaks for CLISP. Improvements to Windows configuration, notably for LispWorks.
* Fix to load-source-op (broken in 2.014.13 - oops).
* Avoid infinite loops in defsystem in cases of bad interference between quicklisp and your source-registry.
* Small fixes regarding upgrade, compile-file* :output-file.
* Make defsystem-depends-on actually useful by allowing a system thus depended on to define the class of the system depending on it. The XCVB bridge notably relies on this feature.
* Some documentation updates. Notably, for the sake of CLISP, (require "asdf") is the new canonical way to load ASDF, instead of (require :asdf). Sigh.
* Better tests, notably wrt hot upgrade.
Happy hacking!
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The American diplomat to the Russian: "My country too is free: in the USA too anyone may freely publish the dirty little secrets of president Medvedev!"
On 6/8/11 5:21 AM, Faré wrote:
Can you upgrade your implementation to ship with the latest ASDF? Thanks! If you're still using 2.015 (or something even older), there have been many bug fixes, robustness, portability and upgradability enhancements and a few features implemented since.
Will do. Same message sent to gmane. I'm surprised you could do that. That hasn't worked (I think) in a long time.
Also, the CLISP maintainers insist on having (require :asdf) not work and (require "asdf") work, whereas CMUCL does the opposite. Since the CLISP maintainers won't seem to budge, could you have CMUCL accept both?
I will try. That would be an incompatible change, though, so I'll have to think about that.
PS: I noticed you moved the mailing-list to common-lisp.net; the old address @cons.org seems to still exist, but rejects messages as my not being subscribed. Or do both addresses exist? That's confusing.
Both addresses exist. There was some talk about moving the lists to cl.net so I had the lists created. This occurred when there were problems with the list on cons.org. They have since been fixed and I've been too lazy to do anything about the cl.net lists. All links that I know of say cons.org is the location of the mailing lists.
Ray
On 6/8/11 5:21 AM, Faré wrote:
Can you upgrade your implementation to ship with the latest ASDF? Thanks! If you're still using 2.015 (or something even older), there have been many bug fixes, robustness, portability and upgradability enhancements and a few features implemented since.
Done, in cvs.
Also, the CLISP maintainers insist on having (require :asdf) not work and (require "asdf") work, whereas CMUCL does the opposite. Since the CLISP maintainers won't seem to budge, could you have CMUCL accept both?
I thought this would be a more difficult change, but in fact it's quite simple and compatible. Just needed to add (defmodule "asdf" "modules:asdf/asdf") and now cmucl will accept both :asdf and "asdf".
Will be available in the next snapshot.
Ray
On 8 June 2011 11:54, Raymond Toy toy.raymond@gmail.com wrote:
On 6/8/11 5:21 AM, Faré wrote:
Can you upgrade your implementation to ship with the latest ASDF? Thanks! If you're still using 2.015 (or something even older), there have been many bug fixes, robustness, portability and upgradability enhancements and a few features implemented since.
Done, in cvs.
Also, the CLISP maintainers insist on having (require :asdf) not work and (require "asdf") work, whereas CMUCL does the opposite. Since the CLISP maintainers won't seem to budge, could you have CMUCL accept both?
I thought this would be a more difficult change, but in fact it's quite simple and compatible. Just needed to add (defmodule "asdf" "modules:asdf/asdf") and now cmucl will accept both :asdf and "asdf".
Will be available in the next snapshot.
Thank you so much!
It's not the first time I resent the attitude both uncommon and dogmatic of the CLISP maintainers. All other maintainers are so nice and accommodating!
If I have another wishlist for CMUCL, it would be self-enclosed application delivery into a single executable that includes the image.
SBCL, CLISP, CCL, ECL, LispWorks do it and are supported by XCVB and CL-Launch. Could CMUCL do it? That would tremendously simplify delivery of code.
For extra brownie points, if you have a good story for a "linkkit" allowing to statically compile additional C libraries into the executable being delivered, that's would be wonderful. CLISP and ECL have it, SBCL notably doesn't. If CMUCL is similar to SBCL in that regard, you might have to have some magic linker script and function annotations to ensure that magic functions are located at an address known to Lisp code despite new code being linked into the .text and .data segments. Typically, you'd either ensure the proper objects files are linked first, or you'd create a magic segment(s) different from .text and/or .data, that starts at a magic address, in which to compile the stuff that the Lisp image will expect to be at a known address.
So there.
Thanks a whole lot!
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Arbitrary limits to programs are evil, particularly when they go either enforced or unenforced.
On 6/8/11 2:14 PM, Faré wrote:
If I have another wishlist for CMUCL, it would be self-enclosed application delivery into a single executable that includes the image.
If you had asked a year or so ago, I would have said definitely not. But now that I've changed how executables are created (no more huge gross linker script), it's now much easier to maintain and actually even works on solaris/sparc, which didn't have that before. Now it's a question of how stable I think it is. I personally don't even build custom images and just load up whatever I need. Thus, I don't even use the :executable feature at all either, other than an occasional test.
Don't know if I'd make that the default, but I wouldn't have any problem with creating snapshots that included a single executable. Well, that's not quite right. The executable would include the image, but the unicode info is still stored out in a separate file. Would that be ok? Might be possible to build an executable with that included too, but I'll have to test that.
SBCL, CLISP, CCL, ECL, LispWorks do it and are supported by XCVB and CL-Launch. Could CMUCL do it? That would tremendously simplify delivery of code.
For extra brownie points, if you have a good story for a "linkkit" allowing to statically compile additional C libraries into the executable being delivered, that's would be wonderful. CLISP and ECL have it, SBCL notably doesn't. If CMUCL is similar to SBCL in that
What do you mean to statically compile in libraries since there are usually no static libraries anymore, just shared ones? If you dump an image that includes other shared libraries, cmucl is supposed to reload those libraries automatically when the image is started again. That used to work, but I haven't tested that in ages. Would that be ok? Or do you really mean statically linked?
regard, you might have to have some magic linker script and function annotations to ensure that magic functions are located at an address known to Lisp code despite new code being linked into the .text and .data segments. Typically, you'd either ensure the proper objects files are linked first, or you'd create a magic segment(s) different from .text and/or .data, that starts at a magic address, in which to compile the stuff that the Lisp image will expect to be at a known address.
This is kind of how cmucl creates its executable images. The main thing are that the address of two magic trampoline functions and the addresses of the various spaces are specified to the linker when creating the executable image. Not exactly sure how to extend that right now, but I suppose it's possible.
Ray
Don't know if I'd make that the default, but I wouldn't have any problem with creating snapshots that included a single executable.
That would be great.
Well, that's not quite right. The executable would include the image, but the unicode info is still stored out in a separate file. Would that be ok? Might be possible to build an executable with that included too, but I'll have to test that.
Not so great without it, unless the file is available at standard locations in all Linux distributions as in /usr/share/misc/unicode.gz
What do you mean to statically compile in libraries since there are usually no static libraries anymore, just shared ones? If you dump an image that includes other shared libraries, cmucl is supposed to reload those libraries automatically when the image is started again. That used to work, but I haven't tested that in ages. Would that be ok? Or do you really mean statically linked?
I really mean statically linked. Currently, CFFI and friends produce plenty of .so files for wrappers. I don't want to chase a zillion Lisp-specific .so files; that totally defeats the "delivery into one executable file" strategy. Also, some less-known or fast-changing C libraries might be good candidate for inclusion as static libraries rather than dynamic libraries.
This is kind of how cmucl creates its executable images.
Oh, I didn't realize you already had this feature. Sweet. Did you also fix that age-old bug whereby cmucl parsed -eval arguments passed the end of the lisp image arguments?
The main thing are that the address of two magic trampoline functions and the addresses of the various spaces are specified to the linker when creating the executable image. Not exactly sure how to extend that right now, but I suppose it's possible.
Can you tell me which things they are and point me to the linker magic? I think I could port the trick to SBCL...
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Not only is there no contradiction between egoism and altruism, but no altruism is possible without egoism - for what betterment to wish to an other person devoid of selfish desire, to whom any change is indifferent?
On 6/8/11 3:19 PM, Faré wrote:
Don't know if I'd make that the default, but I wouldn't have any problem with creating snapshots that included a single executable.
That would be great.
Well, that's not quite right. The executable would include the image, but the unicode info is still stored out in a separate file. Would that be ok? Might be possible to build an executable with that included too, but I'll have to test that.
Not so great without it, unless the file is available at standard locations in all Linux distributions as in /usr/share/misc/unicode.gz
Heh. My openSuSE 11.4 machine doesn't have that file. But even if it did, it wouldn't work. The unicode data file is a custom file created from Unicode.txt and a bunch of other unicode files. The only option I can see would be to include the contents of that file into the image. I think that's possible.
What do you mean to statically compile in libraries since there are usually no static libraries anymore, just shared ones? If you dump an image that includes other shared libraries, cmucl is supposed to reload those libraries automatically when the image is started again. That used to work, but I haven't tested that in ages. Would that be ok? Or do you really mean statically linked?
I really mean statically linked. Currently, CFFI and friends produce plenty of .so files for wrappers. I don't want to chase a zillion Lisp-specific .so files; that totally defeats the "delivery into one executable file" strategy. Also, some less-known or fast-changing C libraries might be good candidate for inclusion as static libraries rather than dynamic libraries.
I'll have to think about that. That would take some work to integrate, but might fit in fairly easily with the executable support.
This is kind of how cmucl creates its executable images.
Oh, I didn't realize you already had this feature. Sweet.
Fred came up with the original idea, but at the time he only knew how to do it by creating a custom linker script. I talked to him about it and figured out gcc with a few special arguments would do the same thing. That allowed me to extend the idea to work on solaris/sparc and even Mac OS X.
Did you also fix that age-old bug whereby cmucl parsed -eval arguments passed the end of the lisp image arguments?
This only prints out 42 now:
cmulisp -noinit -eval '(print 42)' -- -eval '(print pi)'
Is that what you mean? Or did you mean something else?
Can you tell me which things they are and point me to the linker magic? I think I could port the trick to SBCL...
The C side of the stuff is in src/lisp/elf.c and src/lisp/mach-o.c and src/lisp/save.c, with a little bit of help from the lisp side in src/code/save.lisp. elf.c and mach-o.c dump the three spaces into valid elf or mach-o object files.
The script that does the linking is src/tools/linker.sh. It takes the arguments created by save.c and does the right thing on the various supported architectures and calls gcc in the right way to tell the executable that an image is attached and to append the three spaces onto the end of the C executable.
Hope this helps. If you have questions understanding what's going on, just drop me a note.
Ray
Heh. My openSuSE 11.4 machine doesn't have that file. But even if it did, it wouldn't work. The unicode data file is a custom file created from Unicode.txt and a bunch of other unicode files. The only option I can see would be to include the contents of that file into the image. I think that's possible.
OK. Note: cl-unicode does something like that, too.
This only prints out 42 now:
cmulisp -noinit -eval '(print 42)' -- -eval '(print pi)'
Is that what you mean? Or did you mean something else?
That's what I meant. Excellent! Thanks a lot.
Can you tell me which things they are and point me to the linker magic? I think I could port the trick to SBCL...
The C side of the stuff is in src/lisp/elf.c […]
Note: I've pushed that information onto https://bugs.launchpad.net/sbcl/+bug/794713
Hope this helps. If you have questions understanding what's going on, just drop me a note.
It's pushed on my (long) TODO list. On top of it, I'm trying to get XCVB to work with ECL, then start trying to encourage people to use XCVB instead of ASDF.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org In its weak form, Utilitarianism sums up as a requirement of observational consistency and behavioral relevance for ethical rules.
On 6/8/11 4:22 PM, Faré wrote:
On top of it, I'm trying to get XCVB to work with ECL, then start trying to encourage people to use XCVB instead of ASDF.
Something that I've meant to try out, but asdf (and mk:defsys) is more than adequate so far for what I've wanted to do.
But if I create a self-contained unicode executable, and you try to get XCVB working with cmucl, I can help you.
Oh. One more thing. The only external format built in to cmucl is iso8859-1. All the other formats are in files that are loaded (and compiled) on demand. Would you want all of the external formats preloaded too? I can imagine just having utf-8 and the composing formats to handle unix, dos, and mac end-of-line would satisfy 99 and 44/100 % of the people out there.
Ray
On top of it, I'm trying to get XCVB to work with ECL, then start trying to encourage people to use XCVB instead of ASDF.
Something that I've meant to try out, but asdf (and mk:defsys) is more than adequate so far for what I've wanted to do.
But if I create a self-contained unicode executable, and you try to get XCVB working with cmucl, I can help you.
Yes, I'm making a push to have XCVB be more usable for more people. Ideally, it would be suitable for CL implementations themselves to be compiled from their cross-compiler and/or kernel. But we're not there yet.
Oh. One more thing. The only external format built in to cmucl is iso8859-1. All the other formats are in files that are loaded (and compiled) on demand. Would you want all of the external formats preloaded too? I can imagine just having utf-8 and the composing formats to handle unix, dos, and mac end-of-line would satisfy 99 and 44/100 % of the people out there.
Even I, these days, have adopted utf-8 as my default and forsaken iso8859-1. Yes, I think those two should cover most everyone's needs.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org All programmers are playwrights and all computers are lousy actors.
On 6/8/11 5:51 PM, Faré wrote:
Oh. One more thing. The only external format built in to cmucl is iso8859-1. All the other formats are in files that are loaded (and compiled) on demand. Would you want all of the external formats preloaded too? I can imagine just having utf-8 and the composing formats to handle unix, dos, and mac end-of-line would satisfy 99 and 44/100 % of the people out there.
Even I, these days, have adopted utf-8 as my default and forsaken iso8859-1. Yes, I think those two should cover most everyone's needs.
I've almost switched to utf-8 too. At least in slime and my terminal windows.
Another issue: The self-contained exec can contain the unicode data base and the external formats. I've tested that. But the executable can't save another executable image because the link shell script can't be found. To be truly self-contained, it would have to be stored in C code and written out at some point. Or the script would need to be converted into C. Neither of these appeal to me, but at least the former could be somewhat automated. Still kind of gross.
Ray
Another issue: The self-contained exec can contain the unicode data base and the external formats. I've tested that. But the executable can't save another executable image because the link shell script can't be found. To be truly self-contained, it would have to be stored in C code and written out at some point. Or the script would need to be converted into C. Neither of these appeal to me, but at least the former could be somewhat automated. Still kind of gross.
I have recently converted a biggish shell script (used to test XCVB) into pure Lisp code using run-program (see xcvb/t/sub-xcvb.lisp), and I'm soooo glad I don't have to write shell scripts anymore.
That said, you could also save the script to a string, and invoke /bin/sh -c '...'
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Lie, n.: A very poor substitute for the truth, but the only one discovered to date.
On 6/8/11 6:13 PM, Faré wrote:
Another issue: The self-contained exec can contain the unicode data base and the external formats. I've tested that. But the executable can't save another executable image because the link shell script can't be found. To be truly self-contained, it would have to be stored in C code and written out at some point. Or the script would need to be converted into C. Neither of these appeal to me, but at least the former could be somewhat automated. Still kind of gross.
I have recently converted a biggish shell script (used to test XCVB) into pure Lisp code using run-program (see xcvb/t/sub-xcvb.lisp), and I'm soooo glad I don't have to write shell scripts anymore.
Lisp code would be nice, but, at least the way it's done now, it's too late to run Lisp code; it has to be in C.
I started to move the shell script into C; that doesn't look too hard, but then I realized that for an executable to be able to create another executable image, you need to also have lisp.a, which is basically the run time. For the executable to be self contained, that would also have to be in the executable.
At this point I'm giving up. I don't really want to store lisp.a inside the executable too.
Not sure what to do at this point. But I will add the necessary function to load up unidata into core in case someone wants that.
Ray
I started to move the shell script into C; that doesn't look too hard, but then I realized that for an executable to be able to create another executable image, you need to also have lisp.a, which is basically the run time. For the executable to be self contained, that would also have to be in the executable.
At this point I'm giving up. I don't really want to store lisp.a inside the executable too.
Not sure what to do at this point. But I will add the necessary function to load up unidata into core in case someone wants that.
Interesting. Well, for delivery, I would be satisfied with not including the lisp.a in the executable -- it is only require when you're using the compiler for development, at which point it's OK to not embed lisp.a (or unidata) into the image. After all, the CLISP linkkit isn't embedded in the clisp executable itself, so you'll be no worse off than other implementations. But yes, that sucks. Of course, you could conceivably "unlink" the current process image (or its executable as found with /proc ?) into a platform-specific .o file to link at a static address, but while a cool hack, it's not necessary. Didn't ELK kind of do that, though?
This all reminds me how cl-launch does it, being a shell script that embeds code that it can either use (after eval'ing it) or dump into shell scripts for delivery. Gah.
-#f
"Fare" == Far <Far> writes:
>> I started to move the shell script into C; that doesn't look too hard, >> but then I realized that for an executable to be able to create another >> executable image, you need to also have lisp.a, which is basically the >> run time. For the executable to be self contained, that would also have >> to be in the executable. >> >> At this point I'm giving up. I don't really want to store lisp.a inside >> the executable too. >> >> Not sure what to do at this point. But I will add the necessary >> function to load up unidata into core in case someone wants that. >> Fare> Interesting. Well, for delivery, I would be satisfied with not Fare> including the lisp.a in the executable -- it is only require when Fare> you're using the compiler for development, at which point it's OK to Fare> not embed lisp.a (or unidata) into the image. After all, the CLISP
The only time you need lisp.a is when you create an executable image. Otherwise, it's just wasting space.
Fare> linkkit isn't embedded in the clisp executable itself, so you'll be no Fare> worse off than other implementations. But yes, that sucks. Of course, Fare> you could conceivably "unlink" the current process image (or its Fare> executable as found with /proc ?) into a platform-specific .o file to Fare> link at a static address, but while a cool hack, it's not necessary. Fare> Didn't ELK kind of do that, though?
I think so. Isn't that also how emacs used to (still does) work too? I'm pretty sure gcl still does something like that.
Ray
BTW, I tried to checkout cmucl from cvs by following the instructions in http://www.cons.org/cmucl/cmucl-cvs.html but they do not work. I get permission denied and that's all, and the ftp server seems down, too. Can you update the instructions and/or make them work?
Oh, I googled the fact that CVS now seems to be on common-lisp.net, and having an account there could find how to do it: CVSROOT=common-lisp.net:/project/cmucl/cvsroot cvs checkout src ; mv src cmucl Can you document that online?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Re: Graphics: A picture is worth 10K words — but only those to describe the picture. Hardly any sets of 10K words can be adequately described with pictures. — Alan Perlis
On 6/8/11 3:48 PM, Faré wrote:
BTW, I tried to checkout cmucl from cvs by following the instructions in http://www.cons.org/cmucl/cmucl-cvs.html
I'm not in control of that page; I even forgot it existed. The canonical place for cmucl is http://ww.cons.org/cmucl, or CMUCL's trac page at trac.common-lisp.net.
The download page http://www.cons.org/cmucl/download.html has the correct instructions, I think. I used them a few days ago to do an anon checkout of cmucl.
Oh, I googled the fact that CVS now seems to be on common-lisp.net, and having an account there could find how to do it: CVSROOT=common-lisp.net:/project/cmucl/cvsroot cvs checkout src ; mv src cmucl
Oh, you really are out-of-date! :-) CMUCL moved to common-lisp.net many, many years ago when cons.org crashed and was unavailable for several months. Just never bothered to move the mailing lists.
Ray
-
Faré -
Raymond Toy