Today, Joerg-Cyril Hoehle Joerg-Cyril.Hoehle@t-systems.com wrote:
Hi,
you're invited to look up comp.lang.lisp on this subject where I wrote:
"I believe in its over 10 years of existence, it's never been able to correctly handle this form: (macrolet ((over(x) `(collect ,x))) (iterate (for n in '(1 2 3)) (flet ((over(x)(declare (ignore x)) (collect 2))) (over n)))) ; would yield (2 2 2) if correct One reason is that it does not contain a code walker aware of lexical bindings. ..." [BTW, I added that to iterate-test.lisp :)]
Great! (-:
I believe it's possible to add restricted support for macrolet. The restriction is that the macro expanders neither contain nor expand into iterate clauses (the body may contain clauses). This would mean that Iterate would play nice with uses of macrolet which are independent on it, e.g. hashtable iterators.
Right. Restricted macrolet support for stuff like e.g. with-hashtable-iterator (which doesn't expand into iterate clauses) would be a sensible valuable thing to have.
I did the non-sensible thing instead and hacked together a more or less macrolet/flet/labels-aware code walker that uses custom lexical environments. It gives the right answer with your MACROLET/FLET example, and the right answer if you swap the FLET/MACROLET forms. with-hashtable-iterator seems to work with sbcl, cmucl, clisp and ACL (personal edition 6.x), as do a few other examples I tried.
The whole thing is a bit ugly, but for the macrolets I tried it with, it seems to work. Here's a diff (against 1.0.9):
(The code really isn't beautiful. Most of it could stand being improved in some places. But as a proof of concept, it seems to work pretty well. I thought it would be much harder to get this to work) (-:
Together with support for locally (and fixing some more subtle bugs I did not yet talk about), I believe Iterate would be more valuable if
a) it accepted macrolet instead of giving an error b) there would be some restrictions b2) Iterate would issue (warn ...) upon macrolet c) the above shows that it is hard to have Iterate generate correct code in all cases, but users can hopefully live with that.
I'm not entirely sure that mine is the way to go. But yours (as the more conservative one) sounds good, from a maintainer's point of view.
My fear is that the Iterate user may not realize that some situations are deadly. E.g. I've often thought about an implementation style as follows: (defmacro loop-finish () (error "Use of LOOP-FINISH outside LOOP") ;; or `(error ...)) (defmacro loop (&body) `(macrolet (loop-finish () `(go loop-end)) ...)) The idea is to provide both global and local macro definitions. The global one is solely to ease debugging.
Iterate would generate incorrect code when this loop definition would be used somewhere inside the iterate body. A real problem is that the user may not even notice that this situation arises.
What do you think?
With working custom environments, this doesn't seem like a big problem. The problem lies in the OAO20TM nature of the whole custom environment thing. I guess we should give your
Notice that correct macrolet handling requires correct lexical parsing. In my posting to comp.lang.lisp I'm asking how/whether this can be portably done.
A hacky solution isn't hard. One that is guaranteed to work with all macrolet/flet combinations (recognizing only macro lambda lists, for example) will be quite a bit harder. But that's the problem with code walkers /-: