On Fri, Dec 2, 2011 at 12:28 AM, Nikodemus Siivola <nikodemus@random-state.net> wrote:

On 1 December 2011 23:12, Juan Jose Garcia-Ripoll

...

I _now_ think that the compiler macro processor should simply recognize the failure to parse the form and return the original form, unprocessed

I'm not convinced.

Firstly, the (admittedly non-normative) final example in the DEFINE-COMPILER-MACRO dictionary entry shows that this isn't exactly traditional or something writers of portable code can expect.

Ah, I should have looked at that example. Still, your argument, though it is a good one, applies somewhat obliquely to Juanjo's question. If our purpose is to make incorrect compiler macros show up early, seems to me, DEFINE-COMPILER-MACRO itself should signal an error (or at least, warn) if the lambda list uses &KEY but not also &REST. This is going to catch a far larger fraction of incorrect complier macros than waiting for the almost vanishingly rare case in which a nonconstant key is used. (Compiler macros are not invoked on APPLY forms; see 3.2.2.1.1.) To put it another way: by your argument, I think the answer to Juanjo's question doesn't matter very much. While it's true that a compiler macro that could get a keyword parsing error must be wrong, waiting for the parsing error to actually occur is not a very effective way of detecting this class of mistake. There is another way that a compiler macro's keyword parsing could fail, that seems to me to be much more likely than the use of a nonconstant key: where a function and a compiler macro for it are written, and then some time later, a new keyword is added to the function but the need to do likewise with the compiler macro is overlooked. Although one could argue that getting an error when compiling a form with the new keyword is desirable, as it reminds one to fix the compiler macro, I think the nature of compiler macros is that it's nice when they work but one isn't necessarily counting on them to work, and so ignoring the error is often preferable. And then, of course, there's the point that a keyword parsing error is not the only kind of error a compiler macro could signal; by the argument I just made, one might well want to ignore all of them. I guess what I'm muddling towards here is that there should be a compiler parameter *IGNORE-COMPILER-MACRO-ERRORS* that users can set to their preference. -- Scott

On 2 December 2011 21:42, Juan Jose Garcia-Ripoll

A different issue is the utility of &key. I do not buy the argument that &key should always be used with &allow... in compiler macros. This forfeits any use of &key at all because the values are _never_ going to be trusted at all: one of the arguments might be a variable name and the parser will confuse the user about expectations.

Well, they /can/ be used to provide defaults. (Not that that's so useful since you /still/ need to walk &REST, but...)

If on the other hand this definition

(define-compiler-macro foo (&rest my-args &key some-key-arg) ...)

implies that the compiler macro will refuse parsing (without an error) when some argument is not an allowed keyword, then this actually has some utility.

Point taken. Also, I just found a hilarious --apparently universal-- bug in this area: (defun foo (&key ((a ax) t)) (format nil "a=~S" ax)) (define-compiler-macro foo (&key ((a ax) t)) (format nil "a=~S" ax)) (compile nil `(lambda () (foo 'a 42))) =| ERROR, unknown keyword 'A (funcall (compile nil `(lambda () (foo a 42)))) => "a=42" SBCL, CCL, Clisp, and Lispworks all share this beauty. Now, fixing this (at least in SBCL) is simple enough, but raises the question of what to do about non-constant form where a keyword argument is expected. Complaining about "unknown keyword argument FOO" when FOO is actually a variable that may evaluate to a perfectly valid keyword at runtime seems a bit suspect. So, on reflection, I've made a 180 degree turnabout and now think that declining to expand compiler-macros with unknown or variable keyword arguments is The Right Thing to do unless &ALLOW-OTHER-KEYS is explicitly specified. Cheers, -- nikodemus

Interestingly enough, that also means that there is no ways to use &KEY to specify non-keyword keyword arguments in a compiler-macro lambda-list that doesn't run roughshod across the evaluation model.

I think you mean "non-constant" rather than "keyword". There is nothing special about keyword (which is a constant) and any other constant that eventually evaluates to a symbol. The real open issuse behind compiler-macros are deeper. Back in 1989 JonL White (then of Lucid) and I (then and still of Franz) threatened to hold our breath until we turned blue at an X3J13 meeting unless we were permitted to enter a proposal for compiler macros. We worked overnight (I think it was at JonL's house in Palo Alto) to come up with a proposal. It was accepted, subject to a few later emendations, and that is what is in the ANS. Unfortunately, there was not time to give the proposal the traditional trial-by-implementation-and-use, so compiler-macros are not defined completely enough for effective portable use. (But I'm nonetheless glad they are there.) The ANS is silent on what happens if a compiler-macro signals an error -- not just an error from an unmatchable "keyword" argument. I suggest that the compiler _should_ have been specified to wrap an error handler around compiler-macro expansion, and if any error occurs, the expansion should be treated as ther equivalent of returning &whole. But even this is not sufficient. The ANS allows a compiler to ignore compiler-macros altogether, or whenever it likes. This provision was necessary to get compiler macros into the standard (to avoid imposing new features on implementations unwilling or unable to conform) but this makes the facility portably unusable. Now, many uses of compiler macros presume nonportable dependencies, such as translating certain references into nonportable lower-level accessors, but IMO it should be possible somehow for portable code to assume that a compiler macro _will_ be expanded in compiled portable code. There ought be some sort of mechanism so the programmer can determine that a compiler macro has signalled error, or that a compiler macro has returned &whole. (Those if us who use compiler-macros currentyl often use disassemble.) I have no proposal how to implement such facilities -- they would live too close to the real estate owned by the programming environment.