- asdf-devel - mailman3.common-lisp.net

RFC: Slots on operations, and bundle filenames
by Robert Goldman 28 Jul '16

28 Jul '16

Once upon a time in old ASDF, there was an idea that one might put slots on operations. IIRC, this worked oddly, at best, because it wasn't clear how to propagate these slot values around in the plans that ASDF built. OTOH, we could really use the ability to add information to some of the ASDF operations. For example, now that we have an IMAGE-OP, we really don't want the images hidden away in the .cache directory where they are hard to find. Justification: it's fine to have fasl files hidden away there, because the fasls are loaded by ASDF:LOAD-OP based on computed fasl file locations. But if you want to load an image, you need to find that image *before* you have ASDF -- even before you have lisp -- so an invisible file, whose location is computed, is really inappropriate. Note also that by Faré's design principle -- the one who knows should specify -- this information cannot be somehow programmed into the system -- it must be supplied by the user. I'm inclined to think we could allow the user to set an output-file for some of the bundle operations with a keyword argument to OPERATE, and then jam the value of the keyword argument into the bundle-op for it to be written. This works, I think, because it's a trivial case. If we try to put some properties on an operation and need them to be propagated around the plan, I think we just lose. Comments?

2 1

Using ASDF as part of an interactive app framework - storing components in memory?
by Andrew Sengul 27 Jul '16

27 Jul '16

I'm interested in using ASDF to build interactive applications. Here's an example of the kind of thing I'd like to do. Imagine creating a Web-based multiple choice poll. When someone submits an entry to the poll, a data file containing all the responses is updated with the new entry. Two SVG graphics, a bar chart and a pie chart of the answers, are also updated according to the updated data set. If I wanted to implement something like this in Common Lisp, ASDF seems like a good foundation to build on, since it allows me to create a complex dependency graph of code and data files. In the example above, the SVG visualizations are dependent on the answers data set, and they get updated when the answer data is updated. However, I don't know how well this approach would scale because in order to use ASDF this way, a file needs to be written to disk and other files must be loaded from disk and compiled each time an entry is made to the poll. Would it make sense to create a new component type that represents a quoted piece of code stored as a variable in a running program rather than a file? If the poll is implemented with a persistent Web server, it could keep the answer data and the graphics in its memory instead of on disk, and use some means to periodically back up the contents of those variables to disk. I'm still new to ASDF so I'm not sure if there's a simple way I could add in this functionality or if it's a good idea. I know that the Web poll example I gave above would work better if the graphics generation and answer storage was just written into the Web server instead of using ASDF, but the poll is just an example; my goal is to develop a system that can handle much more complex code/data graphs. Any feedback appreciated. Thanks, Andrew

3 4

Request for advice on handling SERIOUS-CONDITIONs in ASDF tests
by Robert P. Goldman 25 Jul '16

25 Jul '16

Previously, in the ASDF test suite, we would run tests, checking for errors, and if there were errors, we would exit with a non-zero status, signaling to our shell script that a test had failed. However, experimenting with ECL, I found that there were cases where ECL would fail with a SERIOUS-CONDITION that was not an error. These were typically various flavors of STORAGE-CONDITION, and need to be caught so that non-interactive tests don't hang when there's some resource exhaustion. Accordingly, I modified the ASDF test suite to catch not just ERRORs, but also SERIOUS-CONDITIONs. Unfortunately, this breaks the tests on CCL. The problem appears in one of our tests where we call (EXIT-LISP 0). This is (ccl:quit 0) on CCL. Unfortunately, it seems that CCL signals a CCL:PROCESS-RESET when quitting and PROCESS-RESET is a SERIOUS-CONDITION. So how should I modify the tests so that I catch the other SERIOUS-CONDITIONs (like STORAGE-CONDITIONs) without mistakenly catching these PROCESS-RESET conditions? I suppose I can either catch all SERIOUS-CONDITIONs *except* PROCESS-RESET, and just decline those? Or could I have the definition of EXIT-LISP handle the PROCESS-RESET condition? Without more knowledge, it seems possible that could mess up graceful exit. Could you clarify PROCESS-RESET a little for me? The spec indicates that SERIOUS-CONDITIONs are "All conditions serious enough to require interactive intervention if not handled should inherit from the type serious-condition." I'm not sure how that aligns with the use of PROCESS-RESET as described in the manual. Thanks, R

1 0

Fixed the release branch
by Robert Goldman 21 Jul '16

21 Jul '16

I seem to have overlooked updating the "release" branch on common-lisp.net. So the 3.1.7 tag was fine, but if someone tried to pull from the release branch, they would get an old version. This is fixed now. Thanks to RME of Clozure Associates for bringing this to my attention. Best, r

1 0

Depending on and generating files in system definitions
by Jan Moringen 29 Jun '16

29 Jun '16

Hi. It seems to be a common pattern to maintain a project's version information as a string in a separate file, e.g. version.sexp, and use something like (:version (:read-file-form "version.sexp")) to get that information into the system definition. This seems like a good thing to do as long as version changes are rare and performed manually. Unless there is a better way for this simple case, no problems or questions so far. There are, however, situations in which this doesn't work. My use-case is a project which is maintained in a GIT repository and uses the following versioning scheme MAJOR.MINOR.NUMBER-OF-COMMITS-SINCE-RELEASE This is useful for things like bug reports or determining compatibility of input files with the current version of the software. The crucial part is maintaining NUMBER-OF-COMMITS-SINCE-RELEASE automatically. Previously, I implemented this behavior using some horrible hacks that I would rather not talk about. I started from scratch and came up with a potential solution (detailed description attached). Since this seems like a common problem, I would like to ask whether the outlined approach is the proper way to do this. If so, I can try to turn the description into something that could be added to the FAQ section of the ASDF manual. Thanks in advance and kind regards, Jan

3 4

Re: Wildcard modules support
by Ricardo 27 Jun '16

27 Jun '16

The intended purpose of wild-module as defined in asdf-contrib is to specify a module composed of the set of files that match a glob pattern. For example: (defsystem "my-system" ... :components ((:file ...) ... (:wild-module "plugins" :pathname "plugins/*.lisp"))) something like that, although the pathname specifier should rather be constructed in a platform-independent way. The usefulness of this feature is that it avoids having to perform manual maintenance on the system definition file every time a "plugin" is added or removed from the system. This is a pretty common thing on many projects and is easily handled by most (if not all) the system building tools I've come across in other platforms/languages. However, I put aside half a day yesterday to try to hack a similar thing into asdf and I found the project code in a less than ideal state with regards to documentation, clarity and overall structure and methodology, which surprised me because asdf3 was introduced as a rewrite in the hopes of achieving better extensibility, consistency and configurability. It's not clear at all where should someone plug these kind of features in or how to merge the behavior of a new specific component (not just a source-file subclass) into the whole control flow. I'd like to contribute but I can't spend any more evenings on this when hardly anyone seems to need it. For my own purposes, the solution proposed in http://stackoverflow.com/questions/24171268/how-to-make-defsystem-use-every… works fairly well (and, actually, something as simple as this might as well be a built-in feature). Sorry. Take care, rcn > I am, and I'm open to including this, but as I said before, not in its > current state of non-documentation. > > If we can get a description of its intended purpose, suitable for the > manual and to check its implementation, I will review it for inclusion. > > But ASDF has really grown, and I'm not in a position to incorporate any > more bits that aren't clearly documented. Rather the contrary: I'm > interested in better understanding and better documenting. > > Best, > r

3 2

Re: Wildcard modules support
by rcn 26 Jun '16

26 Jun '16

But loading all the required files from an "all.lisp" file makes asdf unaware of the existence of those files, doesn't it? I can use that as a workaround but it has several disadvantages: The relationships between those files and the rest of the system will be lost, and any changes with regards to compilation/loading configuration (bundling, loading from pre-compiled files) will have to be done manually in all.lisp, hidden from the project .asd file. Given that a workaround is necessary for this, using a reader macro in the system definition to unfold a list of :file components from a glob pattern seems like a better solution, but it's a shame that such a basic feature isn't supported out of the box. In my opinion, if you really want to push asdf as the standard system definition for Common Lisp, these kind of things (what the users need, after all) should be given more attention. It's unusual for a programming ecosytem to have a powerful system definition tool and still having to rely on Makefiles and roll-your-own hacks for some things. Don't get me wrong, I think asdf is a great tool and I thank you all for the effort you put on it. Kind regards, rcn >wild-modules were an example extension and/or contrib from the asdf 1 >days. There is a test for them in test/ but otherwise I don't believe >anyone uses them. If you have the asdf git installed, you can use them >with package-inferred-system. > >I believe that package-inferred-system is a better way to achieve a >similar effect, that allows for dependencies between sub-systems, >unlike wild-modules. I suppose a wildcard option to load all files in >a directory while using package-inferred-system dependencies would be >a nice feature to add. But so far, I found it more practical to have >an explicit file named "all.lisp" that includes all the files that >matter.

3 2

Wildcard modules support
by rcn 24 Jun '16

24 Jun '16

Hi all, Will wild-module support ever be fully and "officially" supported? Currently there's what appears to be a sample implementation as part of asdf/contrib but, if I recall correctly, asdf/contrib is bound to disappear eventually (actually, the documentation says this is an empty package). However, it's not clear how to use it, it's not documented and it's not clear if it will ever be a part of asdf or not. In my mind, this is an useful feature. Not for small systems (which, sadly, are the majority of Common Lisp open source projects), but for large systems where new files may be added to a directory to incorporate additional functionalities (a plug-in kind of thing). In these scenarios, wild-module definitions would ease development and system maintenance greatly. As far as I know, asdf is perfectly capable of implementing it and it looks to me like an implementation-agnostic feature, so I see no reason not to include as a core component. Thanks, rcn

3 2

Introspection of system version
by Frank 17 Jun '16

17 Jun '16

Hello, In the system definition we can specify a version; for example: (asdf:defsystem :foo :description "foo utility" :license "LGPL" :version (:read-file-form "VERSION") :serial t :components ((:file "foo"))) For example I can query the system license like this: (asdf:system-licence (asdf:find-system "foo")) How can I query the version of system foo at run time? Thanks, Frank

3 3

Fixed performance bug in UIOP:DIRECTORY-FILES
by Faré 14 Jun '16

14 Jun '16

I bumped ASDF to 3.1.7.1 after I fixed a performance bug in directory-files that made it quadratic instead of linear, due to some remove-duplicates used for logical-pathnames. The fix was to only use this quadratic behavior for LPNs. If you use LPNs, you deserve to lose on large directories. LPNs are for legacy Lisp code with relatively few files, only. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The best of all regimes is one where I am the head honcho of a worldwide empire. — Winston Churchill (his most famous quip, as translated to plain English)

3 6