April 2012 - asdf-devel - mailman3.common-lisp.net

[asdf-devel] Faster loading for deployed systems?
by Erik Pearson 22 Apr '12

22 Apr '12

Hi, I am trying to implement a method of using a "batteries included" lisp image (ccl, linux, 64bit) which can be used for running a variety of lisp tasks in a batch mode. I chose to pre-load the image because it was just too slow to load the individual fasl files at run time, either via asdf or through a generated load-list. I'm using ASDF to load a small system at run-time, but finding that even with small system (say, one file), performance is sluggish, over 2 seconds. It looks like the problem crops up when there are many dependencies. Poking around the asdf code this appears to be because operate always traverses the system dependencies, and pokes the asd files to find if they have changed. This makes sense for asdf in development mode, since it needs to discover changed systems. Not so good for a deployed system. ASDF assumes that since the system def file has not changed, the system hasn't been changed either (I know there has been discussion of this assumption.) But lets assume that at least part of the reason for that is because it is very time consuming to check the entire set of files every time the developer wants to recompile and load just a single file. Why not assume in a deployment mode that NO system will have changed if it is already loaded in memory, and unless the application has asked that it be reloaded? This would be for performance as well as for safety -- we really don't want dependency checking to result in any filesystem checks, etc. I feel that this problem is really pushing my grokking of asdf, but I'm delving ahead anyway. I haven't tested it extensively, but I did just waste a perfect sunday morning fiddling with this: extended module with a slot "up-to-date-p", but surely there is a better name, this I just chose while in early play mode: added these methods: (defmethod operation-done-p ((o load-op) (c module)) (module-up-to-date-p c)) (defmethod mark-operation-done ((operation load-op) (c module)) (setf (module-up-to-date-p c) t)) tweaked the operate method so that it exits early if the module has been marked as up-to-date (if (operation-done-p op system) (return-from operate (values op nil))) and do-traverse to only collect "kids" if it is not done (unless (operation-done-p operation c) (while-collecting (internal-collect) Note that this code goes in the special section of operate for handling modules. I think that is all that I did. I found that the time to handle the require for my current project dropped from about 2 1/2 to about 1/2 second. Thoughts?

3 2

[asdf-devel] Testing with RMCL....
by Robert Goldman 22 Apr '12

22 Apr '12

I am willing to run the ASDF tests using RMCL, since my Mac still has Snow Leopard, instead of Lion. *However*, as a non-RMCL user, I have no idea how to run it from the shell, so that we can run the test suite on RMCL. Can someone please offer me a clue? I can't be trying to run this from the GUI every time it needs a test. I had to give up testing LispWorks for the same reason. thanks, r

4 8

[asdf-devel] Please test 2.20.16
by Faré 21 Apr '12

21 Apr '12

OK, so I propose we freeze the current state of ASDF for a release 2.21, hopefully some time next week. Please test 2.20.16. Changes since 2.20: * New features: * Most importantly, encodings. ASDF now lets you specify an :encoding for a system, module or component, that is used when loading or compiling Lisp files. See the documentation. By default, the only useful value is :utf-8, and we recommend you use UTF-8 everywhere. We intend to make it the default in the future (current default is the legacy behavior of using whichever implicit default your underlying implementation is currently configured to use). An extension asdf-encodings is available that supports more encodings, including autodetection of encoding in the near future. * You can now specify :hostname in your asdf-output-translations, so you can easily share a home directory yet split its fasl cache between several subtly different machines. * Bug Fixes: * lp#982285. since 2.014.4, the default source-registry was skipping anything from /usr or XDG_DATA_DIRS, because of a missing test in getenv-absolute-pathnames. Fixed in .15. * In 2.019.9 aka 2.20, ECL support was broken by using a function in the wrong package (defined in cl-user). Fixed in .1. * logical-pathname support was somewhat broken since 2.017.6, due to the way most implementations fail to read physical namestrings when *default-pathname-defaults* is a logical-pathname. Fixed, together with other logical-pathname issues, and a test case was added to the test suite to ensure no further regression. Works great modulo quirks around implementation bugs on CLISP and Allegro. * Minor tweaks: * Use :unspecific in pathname components on more implementations. * export and/or document more utilities. * add a few missing compatfmt for Genera. I do not intend to make any further changes to ASDF before the 2.21 release, except for fixes to any bug found, of course. Experiments with encodings and autodetection are redirected to the asdf-encodings system for the time being. Discussion of a possible merge of asdf-encodings into ASDF will happen only after asdf-encodings itself is reasonably stable. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The mice which helplessly find themselves between the cats teeth acquire no merit from their enforced sacrifice. — Mahatma Gandhi

4 14

[asdf-devel] ASDF and character encodings
by Daniel Herring 21 Apr '12

21 Apr '12

This topic came up at the Boston Lisp meeting last night. Here was my takeaway from the discussion. ASDF does not need to support arbitrary encodings in system definition files. UTF-8 is the simplest and most portable solution for non-ASCII characters in system definitions. Thus ASDF can simply declare that all system definitions are read using UTF-8. On systems where unicode support is not available, system definitions will be read as "extended ASCII", and the non-ASCII characters will simply print incorrectly. Filenames should only be specified using ASCII characters to avoid portability issues. Authors do need a way to support arbitrary encodings in other source files, but this should be restricted to cases where there is an actual need to use a non-UTF-8 character set. Example use cases are locale-specific libraries, a toolchain that converts between character sets, or possibly a character-set-detection library. Again, other character sets can be a major portability issue. There are only a handful of libraries that use something other than ASCII or UTF-8. These should be easy to fix, given approval from their authors. Hopefully that's an accurate record of the general consensus and this writeup will be helpful to others in some way. - Daniel

2 2

[asdf-devel] Tested all on 2.20.12 Mac OS X 10.6
by Robert Goldman 18 Apr '12

18 Apr '12

Seems to work fine. Still not testing on CMUCL, waiting for patch to that. cheers, r

3 4

[asdf-devel] Ran the tests on Mac OS X
by Robert Goldman 16 Apr '12

16 Apr '12

...all have passed. rpg% echo $ASDF_TEST_LISPS ccl clisp sbcl ecl abcl allegro allegromodern Cheers, r

1 0

[asdf-devel] source file encoding
by Anton Vodonosov 16 Apr '12

16 Apr '12

Hello. How to specify source files encoding in ASD2? I.e. what value to pass to the :external-format argument of the compile-file? In the old ASDF I did it by (let ((asdf:*compile-file-external-format* #+:clisp charset:cp1251 #+:sbcl :cp1251)) (asdf:operate 'asdf:load-op :mysystem)) But today the variable asdf:*compile-file-external-format* does not exist. Also, it of course would be better to specify the encoding in the .asd file (the system author knows the files encoding, and should care about this, but not the system user). Could you advise me how to do it? Best regards, - Anton

13 61

[asdf-devel] logical pathname fixes
by Faré 13 Apr '12

13 Apr '12

I added a more functional test for logical pathnames, and made a round of fixes for logical pathnames. Some things are still broken on CLISP and Allegro, but I blame the implementations (see test/test-logical-pathname.script for details). p-cos, janderson, pjb: I believe you are the three known users of logical pathnames. Can you test the latest ASDF in your respective setups? (currently 2.20.10). —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Think you can, or think you can't — either way, you'll be right. — Henry Ford

2 6

[asdf-devel] What are current plans for RMCL?
by Pascal Costanza 12 Apr '12

12 Apr '12

Hi, I'd like to get an idea what the current plans for RMCL are, if any. Since the switch to OS X 10.7, there is no default support for old PowerPC applications on OS X anymore, so RMCL doesn't work anymore, at least not without a major effort. Are there any plans to find a remedy for this situation? Or will RMCL effectively become deprecated? I'm asking for the following reasons: When ASDF was changed from 1.x to 2.x, this caused some problems for RMCL, which I eventually resolved by using Common Lisp's logical pathnames for the systems I maintain (primarily Closer to MOP and ContextL). However, the current maintainers of ASDF have an unjustified very low regard for logical pathnames, which causes a lot of pain - basically, whenever a new version of a Common Lisp implementation comes bundled with a new ASDF version, I have to deal with bugs in ASDF that in one way or the other break my setup with logical pathnames. Since my time is limited and is better served on things other than producing bug reports for ASDF (which is a tool that should be much more stable than it currently is), I decided now that it is better to drop logical pathnames and go for the Unix-like names that the ASDF maintainers seem to strongly prefer. RMCL would be the only Common Lisp implementation that would cause a problem in this regard. So, what's the verdict? Thanks a lot for any hints! Best, Pascal -- Pascal Costanza The views expressed in this email are my own, and not those of my employer.

3 7

[asdf-devel] Latest commit
by Robert Goldman 12 Apr '12

12 Apr '12

I just ran the noupgrade tests (with CMUCL disabled) on Mac OS X, and all passed. I assume that I don't need to bother with Linux testing.... r

1 0