- asdf-devel - mailman3.common-lisp.net

One bug down, one bug up
by Faré 03 Jan '18

03 Jan '18

Dear future maintainers of ASDF, I have great news for you if you were looking for an opportunity to learn about the guts of ASDF: a new bug that will cover a lot of ground, yet is relatively well-understood, and isn't urgent at all. I wouldn't make it anyone's first bug, but once you have some experience, it might be great bug to get really into it. Context: I did this interactive debugging session, but it went quite slow and after two hours we only narrowed down the root issue: we needed the primary-system-* functions to distinguish between "syntactic" and "semantic" primariness, since the two notions aren't identical in the existing codebase, and my just ignoring the issue caused these spurious rebuilds. I once again failed to properly record the 2.5 hour interaction, but one of the participants might have done so, though it's not as valuable as I'd have hoped. Then, It took me the next morning to slowly develop a bug fix. See explanation in the bug and code in the MR: https://bugs.launchpad.net/asdf/+bug/1739514 https://gitlab.common-lisp.net/asdf/asdf/merge_requests/91 Now now now, while thinking very hard about the correct fix to the above bug, and especially about the up-to-date-p condition in find-system, I realized that there was a lingering bug in it, of the kind order as the bug that broke ASDF 2, except in a much rarer situation: if you update the defsystem-depends-on of a loaded system a that could cause a change in how the a.asd file would be evaluated; then this should trigger a reload of a.asd when you next operate on a; but it won't because the timestamp is not propagated, because find-system fails to integrate well enough with the dependency graph usage and construction (which integration is new with 3.3). https://bugs.launchpad.net/asdf/+bug/1741083 This seems to be like it's "just" a variant at the meta-level of the famous bug that broke ASDF 2 (see my 25 page paper on the history of ASDF about that). Except that now the infrastructure to propagate timestamps, what more across evaluation phases, already exists; therefore fixing this bug should mostly "just" use the existing infrastructure. Then again, it could possibly break the universe again, though I bet it won't: I believe there should be "just" an update of find-system and the creation of a new operation class prepare-define-op. In any case, not my problem anymore, because it's clearly not a regression. But a great opportunity for whoever steps up and fixes this bug and whatever else they find. You better step forward while I'm still here to answer questions: considering how hard it was for me to page back the crucial elements of the algorithm into memory after just one year, don't wait. PS: Another exercise for a future ASDF maintainer would be to do the boring work and fix all the 300+ systems on Quicklisp that trigger the bad-system-name warning, one by one. This is also a great exercise for getting to know what kind of clients are around. And for fixing them according to my best_practices document. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Every great truth begins as heresy and ends as superstition. — Thomas Henry Huxley

1 0

ASDF debugging on Hangouts?
by Faré 27 Dec '17

27 Dec '17

Dear CLers, with ASDF 3.3.1 released and the syntax-control branch pending review, there's not much left for me to do with ASDF before I actually retire. Now I'm looking for hackers to whom to pass the torch of ASDF hacking. It so happens that I seem to have introduced (probably with 3.3.0) a regression that breaks static image dumping of workout-timer with cffi-toolchain. If people are interested, I could do the entire debugging session over Google Hangouts, from isolating the bug and filing a bug report to writing regression test and submitting a merge request. That would be a good way to introduce newcomers to ASDF hacking. Having a public ask questions as I make progress would be a good way to keep the video relevant. Is anyone interested? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If there's a book you really want to read but it hasn't been written yet, then you must write it. — Toni Morrison [same with software, —♯f]

8 23

Relativising FASL Cache Paths
by Nicolas Hafner 17 Dec '17

17 Dec '17

Hello everyone. I have a rather particular problem that I have so far not been able to solve on my own without brittle workarounds. Specifically, I'm looking for a way to ensure that the directory paths stored in the FASL cache directory are, if possible, truncated according to some other path (by enough-pathname or a similar facility). The reason I want this is for Portacle <https://portacle.github.io/>. The Portacle directory should be relocatable and you should even be able to take it with you on a flash drive to run on other machines. However, this usually means that the absolute position of the directory on the file system will change, even if the Lisp files and FASLs within it remain at the same relative locations. I would thus like to stop ASDF from recompiling everything when the directory is moved absolutely, and instead rely on paths relative to Portacle's own "root" directory if possible. My current approach involves setting asdf/output-translations::*output-translation-function* to a modified version that attempts to relativise the path as seen here <https://github.com/portacle/config/blob/master/sbcl-init.lisp#L70>. However, this sometimes leads to issues <https://github.com/portacle/portacle/issues/62> as the translated path might be relative and other components down the line seem to expect absolute paths. Any advice on how to proceed on this would be appreciated. Sincerely, Nicolas Hafner

3 6

ASDF + QL maintenance needed.
by Faré 12 Dec '17

12 Dec '17

ASDF needs volunteers to replace its retiring maintainers. One easy way to start, and a good filter for people non ready for the job, is the simple and boring yet essential task of looking at Quicklisp build failures and either fixing other people's build files, or fixing ASDF, depending on who introduced the bug (often a combination of the two). Recent Quicklisp build report: http://report.quicklisp.org/2017-12-11/failure-report.html I've started these two pull requests: https://github.com/marijnh/Postmodern/pull/115 https://github.com/zkat/chanl/pull/12 If there are candidate maintainers, we could do some more of these together. This would build up: * Good practice for dealing with build issues in quicklisp * Knowledge of what is or isn't the contract provided by ASDF 3.3. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If instead of teaching other people what government should be and should do, you'd teach yourself what government actually is and does do, you'd be a libertarian.

1 1

ASDF 3.3.1 is released
by Robert Goldman 17 Nov '17

17 Nov '17

Today we release ASDF 3.3.1, the first bugfix release for the 3.3.0 release series. Of primary interest in this release are 1. a fix for an incompatibility in the UIOP stamp-comparison functions. The handling of infinity as a bound in these functions changed between 3.2 and 3.3, potentially causing problems in backward compatibility. In order to maintain compatibility, the old behavior was restored to the stamp-comparison functions, and new functions -- TIMEstamp comparison functions -- were added to carry the new behaviors. 2. a fix for upgrade from version 3.2.1 3. fixes for bugs that can occur when the default readtable is manipulated inside a dependency loaded by DEFSYSTEM-DEPENDS-ON. We urge implementations that are currently bundling previous versions of ASDF -- and especially those bundling 3.3.0 -- to upgrade to 3.3.1 at their earliest convenience. Here is the changelog entry for 3.3.1: cl-asdf (2:3.3.167-1) unstable; urgency=low New milestone: * UIOP compatibility fix: Introduced new replacement "timestamp" comparison functions, because of inadvertent change in the API. Functions that are compatible with the old semantics are retained as "stamp" comparison functions, but will eventually be deprecated. * Upgrade fix: Upgrade from 3.2.1 needed repair. * Syntax manipulation: Fix for bugs that could be introduced when the default readtable was manipulated during the loading of a defsystem-depends-on system. * Tests: tests for new capabilities and bugs * Documentation: a number of improvements and clarifications.

2 1

RFS: cl-asdf/3.3.1 - Another System Definition Facility
by Kambiz Darabi 15 Nov '17

15 Nov '17

Hello, I have packaged the new upstream release 3.3.1 of the Common Lisp ASDF software and looking for a sponsor: Package name : cl-asdf Version : 2:3.3.1-1 Upstream Author : Robert P. Goldman <rpgoldman(a)sift.info> URL : https://common-lisp.net/project/asdf/ License : Expat Section : lisp The cl-asdf source package builds this binary package: cl-asdf - Another System Definition Facility The package appears to be lintian (2.5.59) clean, apart from: - testsuite-autopkgtest-missing which is in level 'info'. I will try to fix it for the next release after reading the documentation. To access further information about this package, please visit the following URL: http://mentors.debian.net/package/cl-asdf Alternatively, you can download the package with dget using this command: dget -x http://mentors.debian.net/debian/pool/main/c/cl-asdf/cl-asdf_3.3.1-1.dsc Changes since the last upload: New upstream release 3.3.1: * UIOP compatibility fix: Introduced new replacement "timestamp" comparison functions, because of inadvertent change in the API. Functions that are compatible with the old semantics are retained as "stamp" comparison functions, but will eventually be deprecated. * Upgrade fix: Upgrade from 3.2.1 needed repair. * Syntax manipulation: Fix for bugs that could be introduced when the default readtable was manipulated during the loading of a defsystem-depends-on system. * Tests: tests for new capabilities and bugs * Documentation: a number of improvements and clarifications. Thank you Kambiz Darabi -- m-creations gmbh Acker 2 55116 Mainz Germany

2 1

Can not upgrade to ASDF 3.3.0
by Vince 07 Nov '17

07 Nov '17

Hello lispers ! I wanted to install the latest ASDF 3.3.0 but got a bunch of errors. I extracted the tar into quicklisp's local-projects and loaded the installation script: (load #p"~/quicklisp/local-projects/asdf/tools/install-asdf.lisp") While evaluating the form starting at line 22, column 0 of #P"/home/vince/quicklisp/local-projects/asdf/tools/load-asdf.lisp": While evaluating the form starting at line 46, column 0 of #P"~/quicklisp/local-projects/asdf/tools/install-asdf.lisp": stacktrace: When attempting to set the slot's value to NIL (SETF of SLOT-VALUE), the slot ASDF/FORCING:FORCED is missing from the object #<ASDF/PLAN:SEQUENTIAL-PLAN {1006E82343}>. [Condition of type SIMPLE-ERROR] Restarts: 0: [RETRY] Retry ASDF operation. 1: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration. 2: [RETRY] Retry EVAL of current toplevel form. 3: [CONTINUE] Ignore error and continue loading file "/home/vince/quicklisp/local-projects/asdf/tools/load-asdf.lisp". 4: [ABORT] Abort loading file "/home/vince/quicklisp/local-projects/asdf/tools/load-asdf.lisp". 5: [RETRY] Retry EVAL of current toplevel form. --more-- Backtrace: 0: ((:METHOD SLOT-MISSING (T T T T)) #<unused argument> #<ASDF/PLAN:SEQUENTIAL-PLAN {1006E82343}> ASDF/FORCING:FORCED SETF NIL) [fast-method] 1: ((LAMBDA (SB-PCL::NEW-VALUE SB-PCL::OBJECT) :IN SB-PCL::SLOT-MISSING-INFO) NIL #<ASDF/PLAN:SEQUENTIAL-PLAN {1006E82343}>) 2: ((:METHOD INITIALIZE-INSTANCE :AFTER (ASDF/PLAN:PLAN-TRAVERSAL)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1006E82343}> :FORCE NIL :FORCE-NOT NIL :SYSTEM NIL) [fast-method] 3: ((SB-PCL::EMF INITIALIZE-INSTANCE) #<unused argument> #<unused argument> #<ASDF/PLAN:SEQUENTIAL-PLAN {1006E82343}> :FORCING #<ASDF/SESSION:FORCING :PERFORMABLE-P T>) 4: ((:METHOD MAKE-INSTANCE (CLASS)) #<STANDARD-CLASS ASDF/PLAN:SEQUENTIAL-PLAN> :FORCING #<ASDF/SESSION:FORCING :PERFORMABLE-P T>) [fast-method] 5: ((LAMBDA NIL :IN ASDF/PLAN:MAKE-PLAN)) 6: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/FIND-SYSTEM:DEFINE-OP > #<ASDF/SYSTEM:SYSTEM "asdf"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-method] 7: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/FIND-SYSTEM:DEFINE-OP > #<ASDF/SYSTEM:SYSTEM "asdf">) 8: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE)) 9: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/FIND-SYSTEM:DEFINE-OP > #<ASDF/SYSTEM:SYSTEM "asdf">) [fast-method] 10: ((LAMBDA NIL :IN ASDF/SYSTEM:FIND-SYSTEM)) 11: (ASDF/SESSION:CONSULT-ASDF-CACHE (ASDF/SYSTEM:FIND-SYSTEM "asdf") #<CLOSURE (LAMBDA NIL :IN ASDF/SYSTEM:FIND-SYSTEM) {1006E7BB6B}>) 12: ((:METHOD ASDF/COMPONENT:FIND-COMPONENT (STRING T)) "asdf" NIL :REGISTERED NIL) [fast-method] 13: ((:METHOD ASDF/OPERATE:OPERATE (SYMBOL T)) ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL) [fast-method] 14: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL) 15: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE)) 16: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL) [fast-method] 17: (ASDF/OPERATE:LOAD-SYSTEM :ASDF :VERBOSE NIL) 18: (ASDF/UPGRADE:UPGRADE-ASDF) 19: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE)) --more-- To Continue a few times does not help. Any hints on that ? I'm on SBCL 1.3.19 and ASDF 3.2.1. Now the reason I want to do this is that it looks like my last solution to build an executable with ECL (I'd like to build a smaller executable than with SBCL). I was hit by the fact that ASDF did not ship make-build, which is the build procedure documented by ECL. Other options like program-op are not really documented either. I explained and keep track of the issue here: https://stackoverflow.com/questions/46520876/building-an-executable-with-ec… :) Regards, Vincent

4 3

Introducing Seed, an interactive software environment built on ASDF
by Andrew Sengul 05 Nov '17

05 Nov '17

Hi everyone, I've created a new software environment based on Common Lisp and it'd be great to get your feedback. Called Seed, this system can be used to create Lisp programs and interact with them in many ways. It's a bit like a web-based IDE, but unlike most IDEs, it uses a visual programming language as its foundation. Seed displays Lisp code in the form of an interactive tree grid. It's possible to create customized interface elements within the tree grid; these elements are converted into Lisp code for compilation. A simple example is a color picker for use choosing colors inside a HTML template. Drop-downs and toggle switches are also easy to insert, and these elements can be composited to create more sophisticated interfaces. And unlike Emacs and other editors that are designed around text files, Seed is designed around editing ASDF systems. When using Seed, the top-level interactive elements are called "systems" - contrast to "buffers" in Emacs. Each Seed system is an ASDF system with particular extensions. Specifically, Seed systems all have a .seed file in their source directory that defines their behavior within the scope of the Seed environment. Check out this video to see Seed in action and a brief tutorial: https://vimeo.com/237947324 This is a slide deck that goes into further detail: https://www.dropbox.com/s/ssjd05bmrfnzg5e/SeedPresentationTechnical.pdf?dl=0 I don't know if Seed is the first general-purpose visual programming environment, but I'm sure its paradigm is among the most flexible. It is implemented as a Web application, with each atom and form within a Lisp program represented by a module manifested using the React framework for JavaScript. Because Seed's interface is implemented in React, the entire ecosystem of React components can be brought to bear when building custom software interfaces. Ready to try Seed out yourself? It's free software and the repo is here: https://github.com/phantomics/seed The installation instructions start here: https://github.com/phantomics/seed#the-first-step-installing-and-running-se… Read the usage tutorial here: https://github.com/phantomics/seed/wiki/Introductory-Tutorial So far, only SBCL has been tested. This is alpha software and you will encounter bugs aplenty. I envision Seed as something that could one day encompass most of the desktop productivity functions that people now use office apps for. The data-driven nature of the interface will make it easy to mash up and modify these tools to design custom workflows.With Lisp's homoiconicity it may even be possible to develop machine learning algorithms to tailor Seed systems to the jobs they're intended to do. Emacs has always been seen as the flagship of Lisp-based environments, but because of its byzantine interface and its tight coupling to the text buffer as the main interactive unit, I can't see a future where Emacs is usable by regular people for desktop productivity. I think Seed may be able to get there. Let me know what you think of Seed, particularly as an ASDF use case. Best, Andrew Sengul

2 6

uiop non-problems
by Faré 21 Oct '17

21 Oct '17

Dear Xach, I saw your blog post: http://lispblog.xach.com/post/166534341423/uiop-330-problems There are no comments on your blog, so I'll just clarify things on the asdf-devel list: * The situation you describe is not at all a problem specific to uiop 3.3.0, or any other version of uiop. uiop.asd hasn't changed since 3.2.0, and the "issue" already existed before, and exists when there is no uiop.asd available but only the builtin uiop. * The situation you describe, where some libraries are loaded multiple times, is a bug in ASDF, that was fixed in 3.3.0 --- actually, fixing this situation and other related situations was the very reason why I had to teach ASDF about build phases, which lead to the big refactoring done in 3.3.0. * All build systems have to either face the issue of build system extension, or put their head in the sand and do the wrong thing. At least ASDF 3.3.0 does the right thing now, whereas earlier versions put their head in the sand as far as that issue was concerned. (I also know that Bazel has a proper solution, though it involves using a non-extensible DSL for extensions). * At least in that case, the issue is not too bad: just some system recompiled, that will also cause more recompilation the next time, but don't cause an infinite loop, just a waste of CPU time. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org In Italia i fascisti si dividono in due categorie: i fascisti e gli antifascisti. — Ennio Flaiano

3 4

Re: Warning: Computing just-done stamp in plan NIL for action
by Stas Boukarev 18 Oct '17

18 Oct '17

ASDF keeps inventing good reasons all the time... On Wed, Oct 18, 2017 at 7:43 PM Faré <fahree(a)gmail.com> wrote: > Oh, is it a case where you're insisting on using a secondary system > name that doesn't follow the convention B/A when the primary system is > B ? That might explain it. There is also a warning against such a > thing, and for good reason. > > —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• > http://fare.tunes.org > Doing well is the result of doing good. That's what capitalism is all > about. > — Ralph Waldo Emerson (1803–1882) > > > On Wed, Oct 18, 2017 at 12:15 PM, Stas Boukarev <stassats(a)gmail.com> > wrote: > > cat b.asd > > (defsystem A) > > > > (defsystem B > > :depends-on (A)) > > > > (asdf:load-system 'b) is enough to trigger it. > > > > On Wed, Oct 18, 2017 at 6:51 PM Faré <fahree(a)gmail.com> wrote: > >> > >> See section 4 of the ASDF 3.3 paper at ILC 2017 for a quick overview: > >> https://github.com/fare/asdf2017 > >> > >> To support proper phase separation, ASDF has a new operation, > >> define-op, that tracks the dependencies of loading a .asd file (i.e. > >> other systems you operate on, e.g., via load-system or > >> defsystem-depends-on, etc.) > >> > >> I don't know exactly what is your system A and how you use it, but > >> apparently it's unhappy about depending on the primary system for > >> define-op yet being earlier in the .asd file, so the other system > >> hasn't been defined yet. > >> > >> Maybe ASDF should refrain from registering a dependency here between > >> sibling systems. Or not. It was quite subtle to debug about a year > >> ago, and it fell out of my working cache. > >> > >> Is there anything special you do between A and B except a :depends-on ? > >> > >> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• > >> http://fare.tunes.org > >> You think war is economically beneficial? Then why share those benefits > >> with > >> dirty foreigners? Let's have a civil war — A war we're sure to win! > >> > >> > >> On Wed, Oct 18, 2017 at 11:34 AM, Stas Boukarev <stassats(a)gmail.com> > >> wrote: > >> > It’s a regular depends-on, but why did it work before without > problems? > >> > > >> > On Wed, 18 Oct 2017 at 18:32 Faré <fahree(a)gmail.com> wrote: > >> >> > >> >> On Wed, Oct 18, 2017 at 11:25 AM, Stas Boukarev <stassats(a)gmail.com> > >> >> wrote: > >> >> > B is primary, but A has to be loaded before it. It precedes B, and > B > >> >> > depends > >> >> > on it. > >> >> > > >> >> If it's a regular :depends-on, then B should be able to appear before > >> >> A, and that should actually work better. > >> >> > >> >> If it's a :defsystem-depends-on, it used to be that A must be before > >> >> B, but the new situation is that they have to be in separate files > for > >> >> define-op to work at all. > >> >> > >> >> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• > >> >> http://fare.tunes.org > >> >> For ultimately, the most visible trait of a just man is to have no > >> >> desire > >> >> at all to rule others, and only want to rule himself. This decides > >> >> everything. In other words, the worst people will rule. — Alain > >> >> > >> >> > >> >> > >> >> > On Wed, Oct 18, 2017 at 6:23 PM Faré <fahree(a)gmail.com> wrote: > >> >> >> > >> >> >> On Wed, Oct 18, 2017 at 11:13 AM, Stas Boukarev < > stassats(a)gmail.com> > >> >> >> wrote: > >> >> >> > Looks like this happens because I have two systems in B.asd, A > and > >> >> >> > B, > >> >> >> > and B > >> >> >> > depends on A. > >> >> >> Maybe changing the order of the two systems will help. > >> >> >> Which is primary? Which depends on the other? > >> >> >> > >> >> >> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• > >> >> >> http://fare.tunes.org > >> >> >> All honest people are welcome to come and live here. > >> >> >> All dishonest people are welcome to come and die here. > >> >> >> — Libertarian Open-Borders, Open-Skulls Policy >

2 4