- iterate-devel - mailman3.common-lisp.net

Re: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 29 Nov '04

29 Nov '04

Stonewall, Thanks for the traces and your efforts. The following trace is really what makes me wonder. Inside Iterate, my patch should not expand m-v-b anymore and call walk-m-v-b instead. ((special-form? (car form)) ; handle known special operators first (walk-special-form form)) ((macro-function (car form) *env*) (walk (macroexpand form *env*))) ((special-operator-p (car form)) Yet both macroexpand and macroexpand-1 show that they are called upon m-v-b. So far, I don't understand where this comes from. 0 MACROEXPAND > ... >> SYSTEM::FORM : (MULTIPLE-VALUE-BIND (#:G831 #:G832) (FOO) (LET* NIL (VALUES (SETQ A #:G831) (SETQ B #:G832)))) >> SYSTEM::ENV : #<Environment venv (#<Venv 269101724 B> #<Venv 269101828 A>) fenv NIL benv ((TEST-ITER . #<Block TEST-ITER>)) tenv NIL> 0 MACROEXPAND < ... << VALUE-0 : (SYSTEM::MULTIPLE-VALUE-BIND-CALL (FUNCTION (LAMBDA (&OPTIONAL #:G831 #:G832 &REST #:EXTRA-VALS833) (DECLARE (IGNORE #:EXTRA-VALS833) (COMPILER::WEAK-LAMBDA-NAME COMPILER::DUMMY-LAMBDA)) (LET* NIL (VALUES # #)))) (FOO)) << VALUE-1 : T There are 3 things worth trying (declaim (notinline walk)) before definition or use (declare (optimize debug)) inside walk replace macroexpand in walk with macroexpand-1 >It's interesting that this expansion turns M-V-BIND into M-V-CALL, >while the first two traces above produced a M-V-BIND-CALL. Indeed. >I'm also a little suspicious of your revised definition: >> (or (assoc symbol *special-form-alist*))) >since it has a superfluous OR. I dropped the OR. I'm sorry that my saving a little time and typing caused even more loss of time and typing to you and me. I should have simplified the form after dropping the second or clause. >You can get the package from ><http://www.lispworks.com/downloads/lw-personal-edition.html>. I >imagine that the same error occurs on all the supported platforms. Installing packages cost me a lot of work. I can make no promise as to when I'll be able to do that. But it would be reasonable to install a collection of Lisps here so I can play with comparisons and code that does not work with either CLISP or Corman, when time becomes available. Regards, Jorg.

1 0

Re: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 26 Nov '04

26 Nov '04

Hi, Stonewall Ballard wrote: >> A) I can't find m-v-BIND-call in the expansion. Is there a typo? >No. I don't know where it is, or what it is. Describe says it has no >bindings of any sort. >> B) Could you please try out the following: >After those changes, compiling: >(defun test-iter (a b) > (iter (repeat 4) > (multiple-value-setq (a b) (foo)))) >Warning: Iterate, in clause (LET* NIL (MULTIPLE-VALUE-BIND (G3378 >G3379) (FOO) (LET* NIL (VALUES (SETQ A G3378) (SETQ B G3379))))): >Unknown special form (MULTIPLE-VALUE-BIND-CALL (FUNCTION (LAMBDA >(&OPTIONAL G3378 G3379 &REST EXTRA-VALS3380) (DECLARE (IGNORE >EXTRA-VALS3380) (WEAK-LAMBDA-NAME DUMMY-LAMBDA)) (LET* NIL (VALUES >(SETQ A G3378) (SETQ B G3379))))) (FOO)) -- won't be walked by Iterate > >BTW, this fails in exactly the same way if I use (setf (values a b) >(foo)) instead of m-v-setq. o It would be nice to know how this m-v-BIND-call appears out of the blue. Does it happen in interpreted and/or compiled mode? Could you trace iter::walk, cl:macroexpand and also cl:macroexpand-1? o It seems to me like Lispworks is not conformant. The CL idea is that user code must be able to walk foreign code, therefore macroexpansions must be available for all but the fixed set of special operators that CL defines. o Could you provide output for (macroexpand-1'(multiple-value-setq (a b) (foo))) (macroexpand-1'(setf (values a b) (foo))) (macroexpand-1'(multiple-value-bind (x y) (foo) (setq a x b y))) o I feel like little is possible about this issue except to (selectively?) shut up the warning message. o what's m-v-bind-call home package? o Portable Iterate should IMHO continue to recognize the case of an unknown (and therefore IMHO violating ANSI) special operator. The open issue is what to do. It could either be treated as a function call and walked accordingly, or left untouched. The latter is conservative, probably safer, but may miss clauses, and it's what the current code does. o What't the value of (let (a b) (list (multiple-value-list (multiple-value-setq (a b) (values 1 2 3))) a b)) It should be ((1) 1 2) but the Iterate message above suggests it could be ((1 2) 1 2)?!? o Finally, are you sure my changes were executed? iter::walk-multiple-value-bind should have been executed, while the output suggests that a LW-specific expansion was used instead, introducing m-v-BIND-call, a lambda and &rest #:extra-vals3380. What I mean is that the transformed clause (LET* NIL (MULTIPLE-VALUE-BIND (G3378 G3379) (FOO) (LET* NIL (VALUES (SETQ A G3378) (SETQ B G3379))))) should be walked solely by Iterate's walk-xyz handlers, without further need for macroexpansion (except for (foo) and values). Be aware that code may keep a reference to the original #'walk even when you replace WALK in the REPL later. What do you think? Jorg Hohle.

1 0

AW: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 25 Nov '04

25 Nov '04

Hi, Stonewall Ballard wrote: >In Lispworks 4.3 on Mac OS X, Iterate 1.0.9, I get a warning: >Unknown special form MULTIPLE-VALUE-BIND-CALL. The lisp environment >claims that MULTIPLE-VALUE-BIND-CALL is a special operator, >but ITERATE doesn't know how to handle it. >when I put a multiple-value-setq inside an iterate like this: >The loop expands to: > (MULTIPLE-VALUE-CALL #'(LAMBDA (&OPTIONAL A) I can't find m-v-BIND-call in the expansion. Is there a typo? B) Could you please try out the following: (defun special-form? (symbol) (or (assoc symbol *special-form-alist*))) and change (defun walk (form) ... to ((special-form? (car form)) ; handle known special operators first (walk-special-form form)) ((macro-function (car form) *env*) (walk (macroexpand form *env*))) ((special-operator-p (car form)) (clause-warning "Unknown special form ~A -- won't be walked by Iterate" form) (list form)) ... starts-clause? What do you get? Regards, Joerg Hoehle

1 0

Re: [iterate-devel] Macroexpansion, bindings and information flow
by Hoehle, Joerg-Cyril 25 Nov '04

25 Nov '04

[augment-environment etc.] FWIW, the thread "Re: constantp values always available at macro expansion" in comp.lang.lisp these days (11 Nov 2004) discusses possible future CLRFI standardization and Franz' future open source environment-access API Duane Rettig writes:" Message-ID: <ZJadnf74lMdIog7cRVn-rA(a)speakeasy.net> It might be premature - there are still mundane issues to decide that the spec and implementations require (e.g. notably, CLOS/PCL and their walker), and I'd not want to drag other vendors into yet) but I would be more than happy to discuss any issues that you might have at the interface level, or take suggestions if you want to get into those details. Start with the docs (http://www.franz.com/support/documentation/7.0/doc/environments.htm Might be a good place to start) and send me some email." Regards, Jorg Hohle

1 0

[iterate-devel] Iterate error with multiple-value-setq
by Stonewall Ballard 19 Nov '04

19 Nov '04

(I originally sent this to Andreas Fuchs. No reply, then I found this address.) In Lispworks 4.3 on Mac OS X, Iterate 1.0.9, I get a warning: Unknown special form MULTIPLE-VALUE-BIND-CALL. The lisp environment claims that MULTIPLE-VALUE-BIND-CALL is a special operator, but ITERATE doesn't know how to handle it. when I put a multiple-value-setq inside an iterate like this: (defun test-iter (a b) (iter (repeat 4) (multiple-value-setq (a b) (foo)))) The loop expands to: (LET* ((#:COUNT181 NIL)) (BLOCK NIL (TAGBODY (SETQ #:COUNT181 4) LOOP-TOP-NIL (IF (<= #:COUNT181 0) (GO LOOP-END-NIL)) (VALUES (LET* () (MULTIPLE-VALUE-CALL #'(LAMBDA (&OPTIONAL #:G1829 #:G1830 &REST #:|m-v-b-&rest1831|) (DECLARE (IGNORE #:|m-v-b-&rest1831|)) (LET* () (VALUES (SETQ A #:G1829) (SETQ B #:G1830)))) (FOO)))) (SETQ #:COUNT181 (1- #:COUNT181)) (GO LOOP-TOP-NIL) LOOP-END-NIL) NIL)) - Stoney -- Stonewall Ballard stoney(a)sb.org http://stoney.sb.org/

1 0

AW: [iterate-devel] Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 17 Nov '04

17 Nov '04

Hi, Andreas Fuchs wrote: >I did the non-sensible thing instead and hacked together a more or >less macrolet/flet/labels-aware code walker that uses custom lexical >environments. I saw that you use a custom implementation of augment-environment. Before you go further into this, you may investigate the following: Many years ago (1995), Barry Margolin wrote in comp.lang.lisp about augment-environment (and the accompanying functions present in CLtL2 but not in ANSI-CL): "It was removed. There were some problems with the design of the environment-related functions" You may wish to investigate what the deficiencies are. I can't judge if despite deficiencies, it may be enough for the purpose of Iterate. But it may still be better than nothing (or error out and refuse to expand). For similar reasons, I'm thinking about how to allow cl:locally (from trivial to complex). Regards, Jorg Hohle.

1 0

[iterate-devel] critique of finding such-that on-failure clause
by Hoehle, Joerg-Cyril 17 Nov '04

17 Nov '04

Hi, (iter(finding 1 such-that t on-failure (error "not found"))) does not do what one might expect. Neither does (iter (finding 1 such-that t) (finally (error "not found"))) on-failure is a IMHO misnomer, because the argument is always evaluated. So it acts more like the third argument to gethash -- a default. It's evaluated in the loop prologue. The manual does not say where on-failure is to be evaluated, but there's no easy way to move it to the epilogue because the manual states that the epilogue will be evaluated -- we'd need an extra flag to tell "in epilogue but not on successful finding". As a result, finding such-that is rather different from (if test (leave expr)) (finally (return on-failure-expr) or (thereis (funcall test expr)) (finally (return-on-failure-expr)) Well, I think that instead of changing the current behaviour (process or not the epilogue), it's better (for compatibility) to leave it as is and to show alternatives in the manual: (thereis (evenp x)) (finally (error "not found"))) (if (evenp x) (leave x)) (finally (error "not found")) Regards, Jörg Höhle

1 0

[iterate-devel] prerelease (was: Macroexpansion, bindings and information flow)
by Hoehle, Joerg-Cyril 16 Nov '04

16 Nov '04

Andreas, >I think I'll integrate your patches (& make a new release, perhaps >1.1pre1?) tomorrow, btw. Thanks for doing it so fast. It helps me since I have been delaying making changes to all files since I sent you the patches. I only added small notes to a separate file. As soon as you'll have it out, I can add further patches. I'd call it an interim release, because IMHO the next release must restore the iterate-1.2 behaviour of looking at the specials it knows to handle first, then macroexpand. As I reported, the present order causes uncontrolled behaviour (for example in CLISP). And I suggest you name it 1.3preX ^ -- am I right that J. Amsterdam's last version was 1.2? BTW, I tested it with Corman 2.01 (2.5 is current). As a result, I made a note to avoid using ~< (pretty printer) in the error messages. Old errors don't make use of them either, so it can be seen as a matter of code consistency. Regards, Jorg Hohle

1 0

Re: [iterate-devel] Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 16 Nov '04

16 Nov '04

Hi, Andreas Fuchs wrote: >I did the non-sensible thing instead and hacked together a more or >less macrolet/flet/labels-aware code walker that uses custom lexical >environments. Now that's cool. Congratulations! >I thought it would be much harder to get this to work) I also wouldn't have expected this to be possible in so little code. >I'm not entirely sure that mine is the way to go. But yours (as the >more conservative one) sounds good, from a maintainer's point of view. I don't know what you mean by "yours". I feel quite uncomfortable at using a package that may generate incorrect code without warnings. >A hacky solution isn't hard. One that is guaranteed to work with all >macrolet/flet combinations [...] will be quite a bit harder. I'll have to review whether your code is correct or what the restrictions are. I still find it hard to believe that a *macroexpand-hook* might suffice. Please explain why you consider your code hacky (or restricted). Similarly, it would have helped if J.Amsterdam had laid out what exactly is hairy or subtle (e.g. what about multiple-values in with clause and other notes). I have a general feeling that it would be better to have the implementation do most of the expansion work, especially w.r.t. error reporting. The current Iterate walkers assume syntactically correct code and I expect implementations to be good at reporting syntax and other errors. >OAO20TM ?!? Google finds nothing about this. Regards, Jorg Hohle

2 1

[iterate-devel] Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 15 Nov '04

15 Nov '04

Hi, you're invited to look up comp.lang.lisp on this subject where I wrote: "I believe in its over 10 years of existence, it's never been able to correctly handle this form: (macrolet ((over(x) `(collect ,x))) (iterate (for n in '(1 2 3)) (flet ((over(x)(declare (ignore x)) (collect 2))) (over n)))) ; would yield (2 2 2) if correct One reason is that it does not contain a code walker aware of lexical bindings. ..." [BTW, I added that to iterate-test.lisp :)] I believe it's possible to add restricted support for macrolet. The restriction is that the macro expanders neither contain nor expand into iterate clauses (the body may contain clauses). This would mean that Iterate would play nice with uses of macrolet which are independent on it, e.g. hashtable iterators. Together with support for locally (and fixing some more subtle bugs I did not yet talk about), I believe Iterate would be more valuable if a) it accepted macrolet instead of giving an error b) there would be some restrictions b2) Iterate would issue (warn ...) upon macrolet c) the above shows that it is hard to have Iterate generate correct code in all cases, but users can hopefully live with that. My fear is that the Iterate user may not realize that some situations are deadly. E.g. I've often thought about an implementation style as follows: (defmacro loop-finish () (error "Use of LOOP-FINISH outside LOOP") ;; or `(error ...)) (defmacro loop (&body) `(macrolet (loop-finish () `(go loop-end)) ...)) The idea is to provide both global and local macro definitions. The global one is solely to ease debugging. Iterate would generate incorrect code when this loop definition would be used somewhere inside the iterate body. A real problem is that the user may not even notice that this situation arises. What do you think? Notice that correct macrolet handling requires correct lexical parsing. In my posting to comp.lang.lisp I'm asking how/whether this can be portably done. Regards, Jörg Höhle.

2 1