Stelian Ionescu <sionescu@cddr.org> writes:

On Thu, 2014-03-13 at 13:40 -0500, Robert P. Goldman wrote:

...

I'm a little concerned about making BUILD-OP be the default operation.

It seems to me that "BUILD" is not a good synonym for "LOAD," which is how BUILD-OP is currently interpreted.

I agree.

...

I think the conventional interpretation of the word "build" would suggest to the user that

(build "foo-system")

would compile and NOT load "foo-system," instead of performing LOAD-OP as now. To me "build" does not connote "load."

Is this just me? What's the sense of the community?

Should we use a different term? I realize that LOAD is taken, and shadowing CL:LOAD would be a big PITA. Is there a synonym we can use?

Not necessarily a PITA. Does any package :use ASDF ?

It's not uncommon to see a given system file define a package like: (defpackage foo-system (:use cl asdf) ...) Zach

Stelian Ionescu wrote:

On Thu, 2014-03-13 at 13:40 -0500, Robert P. Goldman wrote:

...

I'm a little concerned about making BUILD-OP be the default operation.

It seems to me that "BUILD" is not a good synonym for "LOAD," which is how BUILD-OP is currently interpreted.

I agree.

...

I think the conventional interpretation of the word "build" would suggest to the user that

(build "foo-system")

would compile and NOT load "foo-system," instead of performing LOAD-OP as now. To me "build" does not connote "load."

Is this just me? What's the sense of the community?

Should we use a different term? I realize that LOAD is taken, and shadowing CL:LOAD would be a big PITA. Is there a synonym we can use?

Not necessarily a PITA. Does any package :use ASDF ? Otherwise you'd get away with qualifying cl:load in uiop-build/load* and shadowing it in a few packages.

Let's continue this discussion until we get somewhere we're happy with. I'm going to put the BUILD-OP changes into a topic branch for now. If necessary, build-op (by some name) can wait until after 3.2... Best, r

For the record, it's not that I'm objecting to the build-operation idea. I'm sorry if you got that idea, and felt that you had to spend a lot of time convincing me! My concern was a much more limited one: that the word "build" doesn't properly convey what is going to happen. I believe that the operation is "prepare the direct object system for me to use it." I don't think "build" is the right word for this, but I am happy to see some sort of "do the default operation," as long as we can come up with a name that conveys the meaning to the user. I think "build" in its normal sense has a connotation much closer to the things that bundle-op or save-image-op would do. Cheers, r

Of course, I forgot to attach the patch. Here it is. If you think it's satisfactory, I can push. Hopefully, the added test will convince you that it is all working. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org To converse at the distance of the Indes by means of sympathetic contrivances may be as natural to future times as to us is a literary correspondence. — Joseph Glanvill, 1661 On Thu, Mar 13, 2014 at 7:11 PM, Faré <fahree@gmail.com> wrote:

On Thu, Mar 13, 2014 at 4:24 PM, Robert P. Goldman <rpgoldman@sift.info> wrote:

...

For the record, it's not that I'm objecting to the build-operation idea. I'm sorry if you got that idea, and felt that you had to spend a lot of time convincing me!

My concern was a much more limited one: that the word "build" doesn't properly convey what is going to happen.

Well, ASDF itself has long been described as a build system or build tool (including in our ILC 2010 article), just like make, ant, etc. See also Wikipedia pages for each of these. What do these programs do? They build. I don't love the word, but I don't know a better one.

...

I believe that the operation is "prepare the direct object system for me to use it."

Here are some suggestion:

prepare (shorthand: p) update (shorthand: u) build (shorthand: b) build-and-update (shorthand: bu) build-and-maintain (shorthand: bnm, or |bnm,|, which perfectly fits the asdf keyboard pattern) build-operation or build-op (shorthand: bo, or bop) default-operation (shorthand: do, shadowing cl:do, or dop) operate 'build-op (shorthand: op, or ob) operate-default (shorthand: od) ok (shorthand: o) ensure-component-is-built-and-update-current-image-for-component (shorthand: ecibaucifc)

If we pick the latter, we have utterly failed the user. Actually, I already have failed the user by naming the function build-system, which is too long a name, and deceiving because it's not just for systems: you can (build-system '("system" "module" "component")) to build a targetted component. Since obviously no one is using it yet, I propose we remove it from 3.1.

...

I don't think "build" is the right word for this, but I am happy to see some sort of "do the default operation," as long as we can come up with a name that conveys the meaning to the user.

I think "build" in its normal sense has a connotation much closer to the things that bundle-op or save-image-op would do.

I don't think it's right, only that it's the rightest available so far.

PS: here is a second patch on top of the former one, that makes string designators for operations, so that :build-operation "foo::op" works. Added a suitable test for defsystem-depends-on to the test suite. Without this feature, build-op is much less useful.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The Party will yield power when and only when it is in the best interest of its members. Don't fight the Party: buy It with the profits of Liberty.

Faré wrote:

On Thu, Mar 13, 2014 at 4:24 PM, Robert P. Goldman <rpgoldman@sift.info> wrote:

...

...

For the record, it's not that I'm objecting to the build-operation idea. I'm sorry if you got that idea, and felt that you had to spend a lot of time convincing me!

My concern was a much more limited one: that the word "build" doesn't properly convey what is going to happen.

Well, ASDF itself has long been described as a build system or build tool (including in our ILC 2010 article), just like make, ant, etc. See also Wikipedia pages for each of these. What do these programs do? They build. I don't love the word, but I don't know a better one.

I get it, but now ASDF does more than one kind of build, including some things (with the bundle-op) that look a lot more like what 'make' does than what ASDF has done up to now. Up to now, there's been no real need to distinguish between "build" and "load", but now there are things that look like a conventional C program build, rather than like what DEFSYSTEM did before. Let's not rush into this: as Attila has pointed out, whatever we choose, we'll have to live with for a long while. Unfortunately, I think both DO and LOAD are non-starters because of the pain they would impose on anyone who wants to USE-PACKAGE ASDF. What about something bland like "start"? Or we could get all Star Trek and use "make it so" ;-) cheers, r

On Thu, Mar 13, 2014 at 11:47 PM, Faré <fahree@gmail.com> wrote:

Dear Robert,

I pushed my changes to your build-op branch. Easier than mailing patches.

...

...

Well, ASDF itself has long been described as a build system or build tool (including in our ILC 2010 article), just like make, ant, etc. See also Wikipedia pages for each of these. What do these programs do? They build. I don't love the word, but I don't know a better one.

I get it, but now ASDF does more than one kind of build, including some things (with the bundle-op) that look a lot more like what 'make' does than what ASDF has done up to now.

Indeed. On the other hand, you just acknowledged that all these things are a kind of "build".

...

Up to now, there's been no real need to distinguish between "build" and "load", but now there are things that look like a conventional C program build, rather than like what DEFSYSTEM did before.

Hence my recommending making load-system not the main interface anymore, but build. Loading is just the common case for CL systems.

...

Let's not rush into this: as Attila has pointed out, whatever we choose, we'll have to live with for a long while.

There's no rush, but I'd really like it as part of ASDF 3.1.1, and I'm really hoping for a release this month.

...

Unfortunately, I think both DO and LOAD are non-starters because of the pain they would impose on anyone who wants to USE-PACKAGE ASDF.

Of course, I only suggested the former as a joke. Although UIOP provides the :MIX option to DEFINE-PACKAGE that nicely deals with all these conflicts.

...

What about something bland like "start"? Or we could get all Star Trek and use "make it so" ;-)

I think this is even more misleading than "build". It feels like you are starting a program. "make" could do it.

Note that if we rename it make, it would also be nice to rename build-op to make-op before anyone uses it. We should also rename :build-operation to :make-operation or :make-op or :make, and just have some #-asdf3.1 in asdf.asd to cope with upgrade from the one and only current client of it, i.e. current versions of asdf3. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Mathematicians are like lovers. Grant a mathematician the least principle, and he will draw from it a consequence which you must also grant him, and from this consequence another. — Bernard Le Bouyer de Fontenelle

...

Note that if we rename it make, it would also be nice to rename build-op to make-op before anyone uses it. We should also rename :build-operation to :make-operation or :make-op or :make, and just have some #-asdf3.1 in asdf.asd to cope with upgrade from the one and only current client of it, i.e. current versions of asdf3.

MAKE is less cumbersome than my proposed DO-DEFAULT-OPERATION.

On the other hand, I have already code with a MAKE-OP (and a MAKE-SYSTEM class) defined, because I have an ASDF extension that allows ASDF to invoke a Makefile through make as part of a compilation.

Is your make-op in the package ASDF? If not, there is no conflict.

MAKE-OP seems like not an unusual thing for people to want to do in their systems.....

In this connection, it might be time to talk about the package protocol for ASDF extensions. Forcing all such extensions into the ASDF package is not really a scalable approach for a set of code that's not centrally managed. But can people effectively use package-qualified names for classes and operations in ASDF system definitions? It may be that the answer to that is "naturally yes," but I'd have to investigate further. I'm pretty sure that the current way we process DEFSYSTEM-DEPENDS-ON would not handle such names well (the package for the package-qualified name would not yet exist, causing a reader error), forcing people to back off to using procedural (ASDF:LOAD-SYSTEM "my-asdf-extension") in their .asd files...

That's a great question, and I had only thought about it yesterday. Look at the build-op branch: I did improve the protocol so that strings may be used as designators for either operation classes or component classes, that will be read as a symbol while in package ASDF, but a : or :: prefix can override the package. This makes the component class parsing in parse-defsystem oh so slightly backward incompatible, so we'll have to cl-test-grid the change, but I suspect no one uses the undocumented corner cases that are backward incompatible. Now you can: (defsystem foo :defsystem-depends-on (foo-ext) :class "foo-ext:system" :components (("foo-ext:file" "file1") (:some-class-in-package-asdf "file2"))) —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org My dad encouraged us to fail. Growing up, he would ask us what we failed at that week. If we didn't have something, he would be disappointed. It changed my mindset at an early age that failure is not the outcome, failure is not trying. Don't be afraid to fail. — Sara Blakely

...

There's no rush, but I'd really like it as part of ASDF 3.1.1, and I'm really hoping for a release this month.

I'm concerned that these two objectives may conflict.

I need to finish the ASDF manual overhaul, and we need the new default directory in place in order to release.

Given other work on my plate right now, that's already eating evenings and weekends, getting BUILD-OP out there is going to be very challenging.

I'm ready to help. My "harder" desire is that I would like my asdf3 article to accurately describe release 3.1.1, and so the release to ready before the paper goes final (which if it's accepted at ELS 2014, is April 21, 2014). —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Common Lisp makes it easy for you to grow your own language; however, it makes it difficult for that language to be the same as anyone else's. — Faré

On Fri, Mar 14, 2014 at 9:28 AM, Robert P. Goldman <rpgoldman@sift.info> wrote:

Faré wrote:

...

...

I get it, but now ASDF does more than one kind of build, including some

...

things (with the bundle-op) that look a lot more like what 'make' does than what ASDF has done up to now.

Indeed. On the other hand, you just acknowledged that all these things are a kind of "build".

I don't think there IS a single word that captures these two radically different things, so what's the advantage to papering over their differences by using a single word?

I.e., I push a button, and it either loads a system into my running lisp image (possibly without building *anything*), or builds a stand-alone executable. Why would I want a single button, or a single word, to use for both of these operations?

I suppose the counter-argument is that there IS a thing those two operations have in common: each is the default operation for the corresponding system.

That seems reasonable: the system author might want to create and deliver a system that is intended to be a standalone executable, and is never meant to be integrated into a running lisp image.

My theory is that we've been calling ASDF an "(in-image) build system", so what it does is build stuff (in (the current) image). Therefore, it is appropriate to call the main asdf verb BUILD or BUILD-IN-IMAGE. Is BNM an OK shorthand for Build-iN-iMage ? ASDF:BNM is in the natural keyboard line noise starting with B. Otherwise, BII might do. Although, with POIU and maybe some future XCVB-like extension, the in-image would become misleading — into-image might be more correct, then. I don't claim the work is perfect, just that it's the least bad alternative I think of, with a valid justification. ASDF:MAKE can be justified by the analogy with Unix make. In itself, the name is just as good/bad and has the same casual language connotations as build. However and has the disadvantage that we don't call ASDF a make system or make replacement, and indeed, that we might want to avoid the confusion with Unix make.

In that case, maybe instead of trying to take a real English word that already has a meaning, we should take a short phrase that means exactly what we will do:

DO-DEFAULT-OPERATION-ON-SYSTEM to be shortened to DDO

[I don't like DoS or DDoS for obvious reasons! ;-)]

I think DEFAULT-OP is a better name than BUILD-OP because it says what it means.

While semantically correct, that is quite ugly, and ultimately not very helpful, because it fails to convey intent to someone who is not familiar with ASDF already, and someone who is familiar doesn't need to be reminded such thing. Also, the word "default" might be ambiguous in this context: default if who/what/where didn't specify something? The "default" is precisely about the system having (maybe) specified something that is not the default *load-system-operation* — so the name, while precisely correct in some context, only raises ambiguity in other important contexts. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Down to Gehenna or up to the Throne, He travels the fastest who travels alone. — Rudyard Kipling

...

: Faré : Robert

Well, but consider this hypothetical person who doesn't know what's going to happen and who isn't familiar with ASDF already.

S/he types (asdf:make "foo") and *either* gets foo loaded into his/her lisp image or.... an executable file gets dropped onto his/her disk?

Whichever makes sense for the given system, and that's the feature: the user doesn't have to know which newfangled operation the system is using, that will do the Right Thing™. Once again, the person who knows the Right Thing™ is the system author, so they get to specify the operation via :build-operation. The user calls (asdf:build "foo") or (asdf:make "foo"), and the Right Thing™ happens. The user justs want to use the system and gets it to do its thing.

This is going to be confusing no matter *what* word we use.

Or *unconfusing*: the user doesn't need to first dig into the README to figure out which is the correct operation to use.

I can't imagine why we would want to have a command that is "either build or run or do something else with this system."

It's just like "make" running with the first target when not given a target. It just does The Right Thing™, and the user doesn't have to look at the README or at comments in the Makefile to figure out what that is.

To be honest, I am not too hung up over this proposed command, simply because I can't imagine myself ever wanting to use it instead of LOAD-SYSTEM, because I don't see the utility of issuing a command that will do something that I don't know and can't predict. Or more accurately, that MIGHT do what I expect and might do something radically different.

It's not something you can't predict, it's "the Right Thing™ for this system". And the user probably can tell what that thing should be, since he's using the system. If they specifically want to load it and not do that "Right Thing™", they can already use load-system.

It seems like you have two objectives with this build operation:

1. Have a default command. This seems primarily useful for systems that will *not* be loaded, and where the user expects something else to happen. In that case, it seems to me we would be better off NOT making this mostly be an alias to LOAD-SYSTEM. If LOAD-SYSTEM is the right thing, we should just tell ASDF to load the system.

A non-default default is indeed only useful when the default default isn't the thing to do, and the point is precisely that the user doesn't need to know which is which. Maybe the system defines a new variant of load-op, and specifically wants to be loaded with that instead of load-op, or maybe it wants to initialize some server connection, etc.

2. Make a *shorter* alias to LOAD-SYSTEM. This seems to me to be a different objective than #1, and I'm not sure trying to achieve both with a single new addition to ASDF is The Right Thing.

That's indeed a different thing. I'm advocating #1 for the same reason that "make" has a default target, the first one. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Any time you're asking the user to make a choice they don't care about, you have failed the user — Jeff Atwood

On Sun, Mar 16, 2014 at 11:25 AM, Robert P. Goldman <rpgoldman@sift.info> wrote:

...

Whichever makes sense for the given system, and that's the feature: the user doesn't have to know which newfangled operation the system is using, that will do the Right Thing™. Once again, the person who knows the Right Thing™ is the system author, so they get to specify the operation via :build-operation. The user calls (asdf:build "foo") or (asdf:make "foo"), and the Right Thing™ happens. The user justs want to use the system and gets it to do its thing.

I still don't follow this claim. Consider my example again. I'm the user. I don't know whether (MAKE "FOO-SYSTEM") is going to load FOO-SYSTEM into my current lisp image or dump a standalone executable.

So I type (MAKE "FOO-SYSTEM"). In what sense do I know how to get it to do its thing now? I don't know whether I should be exiting lisp and returning to the shell to invoke an executable, or whether I should be calling some API functions.

I apologize if I seem dim, but I don't see how this hypothetical MAKE function actually creates more simplicity than confusion.

The thing is, make does not *replace* load-system, but complements it. If the user knows he wants to load things only, and not do anything else that might interfere with his interaction, he keeps using load-system. If he wants to do the "default thing for the system", he uses make. Often, the system can instruct him to use (asdf:make :foo) instead of (asdf:oos 'asdf:build-op :foo), or worse (asdf:oos "some-weird:operation" "foo") I consider it a bug that some Lisps always quit on dumping an executable, rather than offering the option to fork before dump — but a bug too onerous to fix at present. Problem is, it's not always safe to fork, when there are multiple threads. But if the subset of Lisps that quit after dump is small enough and intersects with a subset of Lisps where we know how to fork safely (or cerror before dump). dump should take a quit key arg, possibly so might image-op and program-op, and a cerror should happen if the program needs to quit but wasn't requested and/or was explicitly asked not to. create-image instead of dump-image, doesn't quit: ecl, mkcl dump-image doesn't quit: clisp, allegro, lispworks, dump-image quits: sbcl, ccl, cmucl error while dumping: gcl neither dump-image nor dump-image: abcl (but has abcl-jar), xcl. Not tested: corman, scl, rmcl, genera. Fork could be made to work with SBCL and CMUCL. Actually, I already have code in POIU for SBCL to fork, and CMUCL is similar, SCL probably the same. The only tricky one, then, is CCL, that cannot safely fork unless you started it with my "single-threaded-ccl" hotpatch. In any case, this looks like a separate issue.

The use case I *do* understand is one where, for example, the system provider knows that to make the system usable I should *both* do LOAD-SYSTEM *and* (OOS 'DOC-OP <SYSTEM>) because the system provider knows that the system has a manual, which requires formatting; something the user cannot be expected to know.

I'm also thinking also in terms of secondary systems: if you (MAKE :FOO), it might load it with LOAD-OP but if you (MAKE :FOO/TEST) it might test it with TEST-OP and if you (MAKE :FOO/DOC) if might produce use MY-DOC:DOC-OP, which you can't even refer to right now until after you (LOAD-SYSTEM :MY-DOC). Then, even if, say, there is no convention as to which documentation generating system is best, there might be a convention that FOO/DOC is the documentation for FOO. As a demonstration, ASDF could replace the asdf/Makefile for all but bootstrap purposes: the Makefile could delegate all requests to ASDF except for the initial building of asdf.lisp itself (in case there's no asdf yet to build it). Not to pollute asdf.asd and everything that loads it (i.e., everything), this infrastructure would be in a separate make-asdf.asd.

I'm not going to kick and scream too much about this since, as I said, I don't think it's actively harmful. But I am not ready to make the end-to-end modifications to the ASDF manual required to substitute MAKE for LOAD-SYSTEM, and I'm unconvinced that this is the Right Thing.

Not substitute. Complement. Maybe new default. But not substitute.

I will therefore leave this on the BUILD-OP topic branch until after the next release.

I think that with or without any "substitution", the code would be better before the release, but you're the master. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org We reject: kings, presidents and voting. We believe in: rough consensus and running code. — David Clark for the IETF

Usability demands an asdf:build function. The name may vary, but shorter is better. asdf:b or asdf:bnm (build 'n' munge?) might do, too.

these sound awful to me, but then i'm against abbreviations in general and always have fuzzy completion on my TAB... a perfect candidate for this semantics would be 'operate', but sadly that's already taken. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- I wish there was more: Grace to accept with serenity the things that cannot be changed, Courage to change the things that should be changed, and Wisdom to distinguish the one from the other. — a secular version of The Serenity Prayer (http://youtu.be/zZvBknPV6hk?t=3h52m00s)