November 2004 - iterate-devel - mailman3.common-lisp.net

Re: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 08 Dec '04

08 Dec '04

Stonewall Ballard wrote: >I spent some time tracing the code, and found that multiple-value-bind >is being expanded into multiple-value-bind-call because m-v-b is a >macro. Perhaps it's not in other Lisps. It is a macro in ANSI-CL. >The first test in the cond of WALK (macro-function (car form) *env*) >was true of m-v-b, so it dutifully expanded it and recursed. I believe you missed part of my patch: the macro-function test is the *second* test my patched walk does, because it first checks for the special forms it knows how to handle: ((special-form? (car form)) ; handle known special operators first (walk-special-form form)) ((macro-function (car form) *env*) (walk (macroexpand form *env*))) Therefore, m-v-b should be handled by walk-special-form which then calls walk-m-v-b. macro-function would never be called upon '(m-v-b ...). Could you try to trace both of these and see that they are indeed traversed? There shouldn't be a need to add to *special-form-alist* because m-v-b is there! But it'll only be found if this list is tested first, not macro-function. Regards, Jorg Hohle.

3 2

Re: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 29 Nov '04

29 Nov '04

Stonewall, Thanks for the traces and your efforts. The following trace is really what makes me wonder. Inside Iterate, my patch should not expand m-v-b anymore and call walk-m-v-b instead. ((special-form? (car form)) ; handle known special operators first (walk-special-form form)) ((macro-function (car form) *env*) (walk (macroexpand form *env*))) ((special-operator-p (car form)) Yet both macroexpand and macroexpand-1 show that they are called upon m-v-b. So far, I don't understand where this comes from. 0 MACROEXPAND > ... >> SYSTEM::FORM : (MULTIPLE-VALUE-BIND (#:G831 #:G832) (FOO) (LET* NIL (VALUES (SETQ A #:G831) (SETQ B #:G832)))) >> SYSTEM::ENV : #<Environment venv (#<Venv 269101724 B> #<Venv 269101828 A>) fenv NIL benv ((TEST-ITER . #<Block TEST-ITER>)) tenv NIL> 0 MACROEXPAND < ... << VALUE-0 : (SYSTEM::MULTIPLE-VALUE-BIND-CALL (FUNCTION (LAMBDA (&OPTIONAL #:G831 #:G832 &REST #:EXTRA-VALS833) (DECLARE (IGNORE #:EXTRA-VALS833) (COMPILER::WEAK-LAMBDA-NAME COMPILER::DUMMY-LAMBDA)) (LET* NIL (VALUES # #)))) (FOO)) << VALUE-1 : T There are 3 things worth trying (declaim (notinline walk)) before definition or use (declare (optimize debug)) inside walk replace macroexpand in walk with macroexpand-1 >It's interesting that this expansion turns M-V-BIND into M-V-CALL, >while the first two traces above produced a M-V-BIND-CALL. Indeed. >I'm also a little suspicious of your revised definition: >> (or (assoc symbol *special-form-alist*))) >since it has a superfluous OR. I dropped the OR. I'm sorry that my saving a little time and typing caused even more loss of time and typing to you and me. I should have simplified the form after dropping the second or clause. >You can get the package from ><http://www.lispworks.com/downloads/lw-personal-edition.html>. I >imagine that the same error occurs on all the supported platforms. Installing packages cost me a lot of work. I can make no promise as to when I'll be able to do that. But it would be reasonable to install a collection of Lisps here so I can play with comparisons and code that does not work with either CLISP or Corman, when time becomes available. Regards, Jorg.

1 0

Re: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 26 Nov '04

26 Nov '04

Hi, Stonewall Ballard wrote: >> A) I can't find m-v-BIND-call in the expansion. Is there a typo? >No. I don't know where it is, or what it is. Describe says it has no >bindings of any sort. >> B) Could you please try out the following: >After those changes, compiling: >(defun test-iter (a b) > (iter (repeat 4) > (multiple-value-setq (a b) (foo)))) >Warning: Iterate, in clause (LET* NIL (MULTIPLE-VALUE-BIND (G3378 >G3379) (FOO) (LET* NIL (VALUES (SETQ A G3378) (SETQ B G3379))))): >Unknown special form (MULTIPLE-VALUE-BIND-CALL (FUNCTION (LAMBDA >(&OPTIONAL G3378 G3379 &REST EXTRA-VALS3380) (DECLARE (IGNORE >EXTRA-VALS3380) (WEAK-LAMBDA-NAME DUMMY-LAMBDA)) (LET* NIL (VALUES >(SETQ A G3378) (SETQ B G3379))))) (FOO)) -- won't be walked by Iterate > >BTW, this fails in exactly the same way if I use (setf (values a b) >(foo)) instead of m-v-setq. o It would be nice to know how this m-v-BIND-call appears out of the blue. Does it happen in interpreted and/or compiled mode? Could you trace iter::walk, cl:macroexpand and also cl:macroexpand-1? o It seems to me like Lispworks is not conformant. The CL idea is that user code must be able to walk foreign code, therefore macroexpansions must be available for all but the fixed set of special operators that CL defines. o Could you provide output for (macroexpand-1'(multiple-value-setq (a b) (foo))) (macroexpand-1'(setf (values a b) (foo))) (macroexpand-1'(multiple-value-bind (x y) (foo) (setq a x b y))) o I feel like little is possible about this issue except to (selectively?) shut up the warning message. o what's m-v-bind-call home package? o Portable Iterate should IMHO continue to recognize the case of an unknown (and therefore IMHO violating ANSI) special operator. The open issue is what to do. It could either be treated as a function call and walked accordingly, or left untouched. The latter is conservative, probably safer, but may miss clauses, and it's what the current code does. o What't the value of (let (a b) (list (multiple-value-list (multiple-value-setq (a b) (values 1 2 3))) a b)) It should be ((1) 1 2) but the Iterate message above suggests it could be ((1 2) 1 2)?!? o Finally, are you sure my changes were executed? iter::walk-multiple-value-bind should have been executed, while the output suggests that a LW-specific expansion was used instead, introducing m-v-BIND-call, a lambda and &rest #:extra-vals3380. What I mean is that the transformed clause (LET* NIL (MULTIPLE-VALUE-BIND (G3378 G3379) (FOO) (LET* NIL (VALUES (SETQ A G3378) (SETQ B G3379))))) should be walked solely by Iterate's walk-xyz handlers, without further need for macroexpansion (except for (foo) and values). Be aware that code may keep a reference to the original #'walk even when you replace WALK in the REPL later. What do you think? Jorg Hohle.

1 0

AW: [iterate-devel] Iterate error with multiple-value-setq
by Hoehle, Joerg-Cyril 25 Nov '04

25 Nov '04

Hi, Stonewall Ballard wrote: >In Lispworks 4.3 on Mac OS X, Iterate 1.0.9, I get a warning: >Unknown special form MULTIPLE-VALUE-BIND-CALL. The lisp environment >claims that MULTIPLE-VALUE-BIND-CALL is a special operator, >but ITERATE doesn't know how to handle it. >when I put a multiple-value-setq inside an iterate like this: >The loop expands to: > (MULTIPLE-VALUE-CALL #'(LAMBDA (&OPTIONAL A) I can't find m-v-BIND-call in the expansion. Is there a typo? B) Could you please try out the following: (defun special-form? (symbol) (or (assoc symbol *special-form-alist*))) and change (defun walk (form) ... to ((special-form? (car form)) ; handle known special operators first (walk-special-form form)) ((macro-function (car form) *env*) (walk (macroexpand form *env*))) ((special-operator-p (car form)) (clause-warning "Unknown special form ~A -- won't be walked by Iterate" form) (list form)) ... starts-clause? What do you get? Regards, Joerg Hoehle

1 0

Re: [iterate-devel] Macroexpansion, bindings and information flow
by Hoehle, Joerg-Cyril 25 Nov '04

25 Nov '04

[augment-environment etc.] FWIW, the thread "Re: constantp values always available at macro expansion" in comp.lang.lisp these days (11 Nov 2004) discusses possible future CLRFI standardization and Franz' future open source environment-access API Duane Rettig writes:" Message-ID: <ZJadnf74lMdIog7cRVn-rA(a)speakeasy.net> It might be premature - there are still mundane issues to decide that the spec and implementations require (e.g. notably, CLOS/PCL and their walker), and I'd not want to drag other vendors into yet) but I would be more than happy to discuss any issues that you might have at the interface level, or take suggestions if you want to get into those details. Start with the docs (http://www.franz.com/support/documentation/7.0/doc/environments.htm Might be a good place to start) and send me some email." Regards, Jorg Hohle

1 0

[iterate-devel] Iterate error with multiple-value-setq
by Stonewall Ballard 19 Nov '04

19 Nov '04

(I originally sent this to Andreas Fuchs. No reply, then I found this address.) In Lispworks 4.3 on Mac OS X, Iterate 1.0.9, I get a warning: Unknown special form MULTIPLE-VALUE-BIND-CALL. The lisp environment claims that MULTIPLE-VALUE-BIND-CALL is a special operator, but ITERATE doesn't know how to handle it. when I put a multiple-value-setq inside an iterate like this: (defun test-iter (a b) (iter (repeat 4) (multiple-value-setq (a b) (foo)))) The loop expands to: (LET* ((#:COUNT181 NIL)) (BLOCK NIL (TAGBODY (SETQ #:COUNT181 4) LOOP-TOP-NIL (IF (<= #:COUNT181 0) (GO LOOP-END-NIL)) (VALUES (LET* () (MULTIPLE-VALUE-CALL #'(LAMBDA (&OPTIONAL #:G1829 #:G1830 &REST #:|m-v-b-&rest1831|) (DECLARE (IGNORE #:|m-v-b-&rest1831|)) (LET* () (VALUES (SETQ A #:G1829) (SETQ B #:G1830)))) (FOO)))) (SETQ #:COUNT181 (1- #:COUNT181)) (GO LOOP-TOP-NIL) LOOP-END-NIL) NIL)) - Stoney -- Stonewall Ballard stoney(a)sb.org http://stoney.sb.org/

1 0

AW: [iterate-devel] Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 17 Nov '04

17 Nov '04

Hi, Andreas Fuchs wrote: >I did the non-sensible thing instead and hacked together a more or >less macrolet/flet/labels-aware code walker that uses custom lexical >environments. I saw that you use a custom implementation of augment-environment. Before you go further into this, you may investigate the following: Many years ago (1995), Barry Margolin wrote in comp.lang.lisp about augment-environment (and the accompanying functions present in CLtL2 but not in ANSI-CL): "It was removed. There were some problems with the design of the environment-related functions" You may wish to investigate what the deficiencies are. I can't judge if despite deficiencies, it may be enough for the purpose of Iterate. But it may still be better than nothing (or error out and refuse to expand). For similar reasons, I'm thinking about how to allow cl:locally (from trivial to complex). Regards, Jorg Hohle.

1 0

[iterate-devel] critique of finding such-that on-failure clause
by Hoehle, Joerg-Cyril 17 Nov '04

17 Nov '04

Hi, (iter(finding 1 such-that t on-failure (error "not found"))) does not do what one might expect. Neither does (iter (finding 1 such-that t) (finally (error "not found"))) on-failure is a IMHO misnomer, because the argument is always evaluated. So it acts more like the third argument to gethash -- a default. It's evaluated in the loop prologue. The manual does not say where on-failure is to be evaluated, but there's no easy way to move it to the epilogue because the manual states that the epilogue will be evaluated -- we'd need an extra flag to tell "in epilogue but not on successful finding". As a result, finding such-that is rather different from (if test (leave expr)) (finally (return on-failure-expr) or (thereis (funcall test expr)) (finally (return-on-failure-expr)) Well, I think that instead of changing the current behaviour (process or not the epilogue), it's better (for compatibility) to leave it as is and to show alternatives in the manual: (thereis (evenp x)) (finally (error "not found"))) (if (evenp x) (leave x)) (finally (error "not found")) Regards, Jörg Höhle

1 0

[iterate-devel] prerelease (was: Macroexpansion, bindings and information flow)
by Hoehle, Joerg-Cyril 16 Nov '04

16 Nov '04

Andreas, >I think I'll integrate your patches (& make a new release, perhaps >1.1pre1?) tomorrow, btw. Thanks for doing it so fast. It helps me since I have been delaying making changes to all files since I sent you the patches. I only added small notes to a separate file. As soon as you'll have it out, I can add further patches. I'd call it an interim release, because IMHO the next release must restore the iterate-1.2 behaviour of looking at the specials it knows to handle first, then macroexpand. As I reported, the present order causes uncontrolled behaviour (for example in CLISP). And I suggest you name it 1.3preX ^ -- am I right that J. Amsterdam's last version was 1.2? BTW, I tested it with Corman 2.01 (2.5 is current). As a result, I made a note to avoid using ~< (pretty printer) in the error messages. Old errors don't make use of them either, so it can be seen as a matter of code consistency. Regards, Jorg Hohle

1 0

Re: [iterate-devel] Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 16 Nov '04

16 Nov '04

Hi, Andreas Fuchs wrote: >I did the non-sensible thing instead and hacked together a more or >less macrolet/flet/labels-aware code walker that uses custom lexical >environments. Now that's cool. Congratulations! >I thought it would be much harder to get this to work) I also wouldn't have expected this to be possible in so little code. >I'm not entirely sure that mine is the way to go. But yours (as the >more conservative one) sounds good, from a maintainer's point of view. I don't know what you mean by "yours". I feel quite uncomfortable at using a package that may generate incorrect code without warnings. >A hacky solution isn't hard. One that is guaranteed to work with all >macrolet/flet combinations [...] will be quite a bit harder. I'll have to review whether your code is correct or what the restrictions are. I still find it hard to believe that a *macroexpand-hook* might suffice. Please explain why you consider your code hacky (or restricted). Similarly, it would have helped if J.Amsterdam had laid out what exactly is hairy or subtle (e.g. what about multiple-values in with clause and other notes). I have a general feeling that it would be better to have the implementation do most of the expansion work, especially w.r.t. error reporting. The current Iterate walkers assume syntactically correct code and I expect implementations to be good at reporting syntax and other errors. >OAO20TM ?!? Google finds nothing about this. Regards, Jorg Hohle

2 1

[iterate-devel] Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 15 Nov '04

15 Nov '04

Hi, you're invited to look up comp.lang.lisp on this subject where I wrote: "I believe in its over 10 years of existence, it's never been able to correctly handle this form: (macrolet ((over(x) `(collect ,x))) (iterate (for n in '(1 2 3)) (flet ((over(x)(declare (ignore x)) (collect 2))) (over n)))) ; would yield (2 2 2) if correct One reason is that it does not contain a code walker aware of lexical bindings. ..." [BTW, I added that to iterate-test.lisp :)] I believe it's possible to add restricted support for macrolet. The restriction is that the macro expanders neither contain nor expand into iterate clauses (the body may contain clauses). This would mean that Iterate would play nice with uses of macrolet which are independent on it, e.g. hashtable iterators. Together with support for locally (and fixing some more subtle bugs I did not yet talk about), I believe Iterate would be more valuable if a) it accepted macrolet instead of giving an error b) there would be some restrictions b2) Iterate would issue (warn ...) upon macrolet c) the above shows that it is hard to have Iterate generate correct code in all cases, but users can hopefully live with that. My fear is that the Iterate user may not realize that some situations are deadly. E.g. I've often thought about an implementation style as follows: (defmacro loop-finish () (error "Use of LOOP-FINISH outside LOOP") ;; or `(error ...)) (defmacro loop (&body) `(macrolet (loop-finish () `(go loop-end)) ...)) The idea is to provide both global and local macro definitions. The global one is solely to ease debugging. Iterate would generate incorrect code when this loop definition would be used somewhere inside the iterate body. A real problem is that the user may not even notice that this situation arises. What do you think? Notice that correct macrolet handling requires correct lexical parsing. In my posting to comp.lang.lisp I'm asking how/whether this can be portably done. Regards, Jörg Höhle.

2 1

Re: [iterate-devel] Re: Macroexpansion, bindings and information flow (Iterate package)
by Hoehle, Joerg-Cyril 15 Nov '04

15 Nov '04

Andreas Fuchs wrote: >> I believe in its over 10 years of existence, it's never been able to >> correctly handle this form: >> (macrolet ((over(x) `(collect ,x))) >> (iterate >> (for n in '(1 2 3)) >> (flet ((over(x)(declare (ignore x)) (collect 2))) >> (over n)))) ; would yield (2 2 2) if correct >Actually, the current version of iterate can handle that clause pretty >well Did you try it out? > (since iterate--main--1.0--patch-6, since when we pass the *env* >argument to macroexpand calls). I don't have GnuArch so I don't don't what this version refers to, but my copy of iterate-1.0.9 has *env*, &environment and two-argument calls to macroexpand. I claim that it does not handle it because Iterate will apply the MACROLET expander to the code, because it ignores the shadowing function definition. The problem is independent on macrolet. defmacro would yield the same behaviour. Does your copy of Iterate really return (2 2 2) ?!? >It seems like most implementations offer code walker support similar >to ClTl2's. While not ANSI CL, these might be your best bet. What are your thoughts w.r.t. a release of Iterate which is not portable, but customized to a restricted set of implementations (that we could put our hands on -- I only have CLISP and Corman Lisp here ...)? -- supposing that all interesting implementations export their code walkers in a useable way. Regards, Jörg Höhle

2 1

[iterate-devel] value of the top-level restriction?
by Hoehle, Joerg-Cyril 15 Nov '04

15 Nov '04

Hi, why are some forms restricted to top-level, while others are not? For example, the following looks like a good replacement of FOR IN which is not restricted to top-level iteration: (iter (repeat 5) (let () ; not at top-level (with l = '(1 2 3)) (for el do-next (if (null l) (terminate) (dsetq el (pop l)))) (princ (cons "hi" el)))) (hi . 1)(hi . 2)(hi . 3) This could be wrapped via iterate:defxyz and even be used as a generator. Likewise, the in-whole-vector example in the manual IMHO provides a better iterator than the built-in IN-VECTOR: it is not restricted to top-level -- it uses FOR DO-NEXT. (iterate(repeat 5)(let()(for x in-vector #(1 2 3))(princ x))) Iterate, in (FOR X IN-VECTOR #(1 2 3)): Clause can occur only at top-level [Condition of type SIMPLE-ERROR] Does anybody know the reason of the top-level check? Is it a kind of user safeguard? Should the meaning of "TOP-LEVEL" in Iterate be redefined? Regards, Jörg Höhle.

1 0

[iterate-devel] towards cl:locally support in Iterate
by Hoehle, Joerg-Cyril 15 Nov '04

15 Nov '04

Hi, J.amsterdam wrote: ;;; - Consider allowing (locally...). The semantics would be quite ;;; odd, however. I have some code for locally, but there's one thing open: A) Should the forms therein be considered top-level or not w.r.t. Iterate? ANSI-CL processes locally at top-level, in a possibly non-null lexical environment... That's ANSI, but here we care about Iterate: shall it accept clauses like (FOR ...) which are otherwise limited to Iterate's top-level? My implementation is currently as follows: locally is walked with declarations therein ignored. Sorry, that's already too vague... What do you think? (iterate (repeat 10) (locally (declare whatever) (for i in '(1 2 3)) (declare (fixnum i)))) B1) allow top-level clause or not? B2) if allowed: recognize/create global fixnum declaration or not? not how (declare fixnum) is not at the right place for locally, but at a typical place of iterate clauses. Sorry, maybe this is too confusing. Maybe I should just go ahead and implement something, and you tell me if you're happy. The issue is somewhat similar to macrolet and lexical scoping: there could be cases where the code is incorrect, yet Iterate could still be useful -- "good enough for daily use". Possible incorrectness is not quite the environment that I appreciate, but... BTW, I'd appreciate if somebody could come up with an example of a non-top-level clause endangered by use of locally. I'm looking for something different from the trivial (declare special) corruption. C) Would you mind if B2) differed from (iterate (repeat 10) (locally (declare whatever) (declare (fixnum i)) (for i in '(1 2 3)))) or would you take this to conclude C2) that every declaration information from within locally shall be thrown away, not just the first ones? C3) Top-level inside locally is bad anyway? Regards, Jörg Höhle.

1 0

[iterate-devel] Re: Macroexpansion, bindings and information flow (Iterate package)
by Andreas Fuchs 15 Nov '04

15 Nov '04

(This is the first time I've opened c.l.l in a long time, and I didn't intend to post anything, but then... (-:) I'm cross-posting this to the iterate-devel mailing list. On Mon, 15 Nov 2004 10:15:35 +0100, Joerg Hoehle wrote: > I came across J.Amsterdam's Iterate package and would like to address > its main weakness: correct, lexical aware code expansion. > > I believe in its over 10 years of existence, it's never been able to > correctly handle this form: > (macrolet ((over(x) `(collect ,x))) > (iterate > (for n in '(1 2 3)) > (flet ((over(x)(declare (ignore x)) (collect 2))) > (over n)))) ; would yield (2 2 2) if correct Actually, the current version of iterate can handle that clause pretty well (since iterate--main--1.0--patch-6, since when we pass the *env* argument to macroexpand calls). What it can't handle (and what I think you meant to post, anyway) are nested macrolet clauses, like: (iterate (macrolet ((over(x) `(collect ,x))) (for n in '(1 2 3)) (flet ((over(x)(declare (ignore x)) (collect 2))) (over n)))) ; would yield (2 2 2) if correct because the code walker does not know how to extend the environment with this nested macrolet. > One reason is that it does not contain a code walker aware of lexical > bindings. Right. I think that's partly due to the fact that it's impossible to have a macrolet (and symbol-macrolet)-aware code walker in portable CL without reimplementing half of common lisp. (-: > I actually wonder whether Iterate actually needs a full code walker, > thereby duplicating functionality found in every Lisp implementation, > or whether Iterate could be written without one, possibly by clever > use of MACROLET and/or *MACROEXPAND-HOOK*? An iterate that doesn't code walk could end up being more friendly to its maintainer. Iterate in its current form relies very heavily on the code walker, so you'd end up re-implementing most of it from scratch (probably not such a bad thing, either (-:) *macroexpand-hook*, now there's an idea. From the code walker, it might be possible to put a closure in there which can expand the macrolet by itself. That would be interesting to try out (you'd still have to create a macro function from the macrolet's definition. hmm) > Here's one of the problems: Iterate allows a (COLLECT x INTO var) > clause to appear anywhere deep inside the body of (Iterate ...), > contrary to LOOP. As a result, it's only after COLLECT has been > encountered that Iterate knows which bindings it must create in its > top level expansion. Information flows from deep inside to the > outermost level. How to do that? > > E.g. > (iterate ... (collect x into a) ... (collect ... into b) ...) > -> `(let (a b) ... ...) > (iterate ... no-collect-at-all ...) > -> `(let () ...) or (progn ...) > > Do you have an idea how such a COLLECT INTO macro could be written > without duplicating a code walker? Unfortunately, ANSI CL doesn't specify the order in which macros are expanded, but if it did, you /might/ be able to signal conditions which the outer *macroexpand-hook* functions could know how to treat. All this is highly speculative, but it would be a pretty neat hack. (-: > This seems hairy to me if one wants it to work both in interpreted > and compiled modes, since with an interpreter, the COLLECT form may > only be encountered upon evaluation of (COLLECT #) --really late. Right. That's probably part of the reason why ANSI doesn't specify macroexpand order. > If it's not possible without a code walker, are there portable means > to use the implementation's code walking engine? It seems a pity to me > that something like AUGMENT-ENVIRONMENT (in CLtL2) was not > standardized in ANSI-CL. > Section 8.5, Environments, of CLtL2 see > http://www.supelec.fr/docs/cltl/clm/node102.html It seems like most implementations offer code walker support similar to ClTl2's. While not ANSI CL, these might be your best bet. Cheers, -- Andreas Fuchs, <asf(a)boinkor.net>, asf(a)jabber.at, antifuchs

1 0

[iterate-devel] Jorg's iterate patches against 1.0.9
by Hoehle, Joerg-Cyril 15 Nov '04

15 Nov '04

Hi, here's my current set of patches against iterate-1.0.9 One thing I want to mention here is to bump the version number. I believe J.Amsterdam last was 1.2, so the 200X version should start with 1.3. I did so in the source. The diffs are in -u3 format, and if Andreas perfers small increments, I've included my RCS ,v file. Andreas, are you going to put up a new release based on that? If you do so, I'll base my next patches upon it. What I did not include so far is o changed order of special-form/macro recognition. Are there any more arguments to debate? o walk-locally (currently not quite correct) o some restricted macrolet support o some more manual clarifications Here's a mini-Changelog o package.lisp + use declaim + use (export list): (export +clause-names+ '#:iterate) + export defsynonym defmacro-driver o iterate.lisp + (declaim (declaration declare-variables)) + listp -> consp + (prog1 (car body) (pop body)) -> (pop body) + (code-char 0) is portable, #\nul or #\null is not + counting uses '(subst (var exp) #) instead of expr + bang-vars-1: tail-call optimized o for in-package(s) evaluates package(s) argument o for in-package(s) work as generators + use #:start/beginning to (hopefully) work with Allegro modern style + walk-flet: accepts declarations + remove walk-identity and use (x . nil) consistently in *special-form-alist* o iter-man.tex + drop note about future with-hashtable/package-iterator o iter-pg.lisp + multiple-value-prog1 + avoid zerop length on lists ; ---------------- DONE o minor CL optimizations: consp, pop, export, declaim o for in-package(s) evaluates package(s) argument o for in-package(s) works as generator o for in-hashtable usable as generator o for in-hashtable supports destructuring o for in-packages was broken because of 'pkgs symbol-types are not evaluated in with-package-iterator and must not be empty => consp o (for vars DO-NEXT) now binds vars o use dsetq instead of destructuring-bind for consistency of templates o a few more package exports o added more testcases o (dsetq (values x nil y)) works Regards, Jörg Höhle.

1 0

[iterate-devel] for i in-vector downto 0 by 2 yields what?
by Hoehle, Joerg-Cyril 12 Nov '04

12 Nov '04

Hi, I'm adding a few test cases to iterate-test.lisp. I wonder about the interaction of in-vector and by when moving backwards. ITER> (iter (for i index-of-vector #(0 1 2 3 4) downto 0 by 2) (collect i)) (3 1) I'd have expected (4 2 0) here. What do you think? By analogy: ITER> (iter(for x downfrom 4 by 2)(repeat 3)(collect x)) (4 2 0) ITER> (loop for x downfrom 4 by 2 repeat 3 collect x) (4 2 0) ITER> (iter (for i index-of-vector #(0 1 2 3 4) downto 0 by 1) (collect i)) (4 3 2 1 0) That is fine with me. I'd expect behaviour like (iter (for x rom (1- (length array)) downto 0 [by N]) (collect x)) (4 2 0) ; when by 2 Rationale: the first index/element it the rightmost, then walk by N. The current behaviour looks to me like an unintended side effect of the implementation, moving down from (length array)=5 by 2 to yield 3,1. Regards, Jörg Höhle.

3 2

[iterate-devel] walk-special-forms first (was Iterate broken on A CL 7.0 beta?)
by Hoehle, Joerg-Cyril 12 Nov '04

12 Nov '04

Hi, walking through the archives, I came across this thread. Indeed, I believe Allegro 7.0 to be broken to macroexpand (cond ...) into (cond ...) again. But Iterate-1.0.9 is IMHO broken as well. I recommend doing the following steps: 1. if operator is in *special-form-alist*, call handling function 2. if macro-function, tail-recursively expand it 3. if still special-operator-p, complain (and possibly default to function call) 4. if clause, handle it 5. else function call Rationale: The argument "To make maintenance of ITERATE easier, I prefer expanded macros over special forms that I have to treat specially." is IMHO moot. Iterate MUST recognize and handle the known special forms, because that's what can come out of macroexpansion. So it may as well do it immediately. This has the additional benefit that it can handle these consistently across platforms. For the sake of Allegro, walk-cond form must remain, otherwise it'll loop endlessly. I expect no other platform specific hacks. My claim of "portable behaviour" is backed by this observation: In CLISP, declare behaves as a macro and expands to NIL. I can't judge whether this is bogus, weird or not. As a result, walk-declare is never called. This explains why declarations are not kept in the code as the manual claims when using CLISP with iterate-1.0.9. Only when Iterate treats known special forms first can it guarantee that its walkers will be used. Changing the order to do so makes declarations show up in CLISP. Handling known special operators first is what iterate-1.0.lisp and -1.2.lisp did. Steps 3 and 4 are interchangeable, as a matter of taste. Regard, Jörg Höhle

1 0

[iterate-devel] iterate package source code server
by Hoehle, Joerg-Cyril 12 Nov '04

12 Nov '04

Hi, it's a great idea to put valuable but old packages from the AI repository or whereever to an open source server on the net! I've done several changes that I'll release soon, but there are are a few points I'd like to discuss before: o Participation: GNUArch to me sounds like yet another CVS system. It causes me overhead as I have to install yet anoter SW on the PC. CVS, subversions, GNUArch, where will it end? Is that really necessary? o "to quote or not to quote" -- see separate e-mail. Here's a list of changes which I've done so far: o minor CL optimizations: consp, pop, export, declaim o for in-package(s) evaluates package(s) argument o for in-package(s) works as generator o for in-hashtable usable as generator o for in-hashtable supports destructuring o for in-packages was broken because of 'pkgs symbol-types are not evaluated in with-package-iterator and must not be empty => consp o (for vars DO-NEXT) now binds vars o use dsetq instead of destructuring-bind for consistency of templates + (code-char 0) is portable, #\nul or #\null is not + use #:start/beginning to (hopefully) work with Allegro modern style + tiny changes to manual o added more testcases Planned: support (dsetq (values x nil y) (foo)) to ignore second value Regards, Jörg Höhle.

2 1

[iterate-devel] iterate proposal: "maybe quote or not" is bad design: remove from manual
by Hoehle, Joerg-Cyril 11 Nov '04

11 Nov '04

Hi, I'd like to change the few places where the manual says "may be quoted, but need not be". Rationale: don't make the user believe something is evaluated when it's not! Despite the manual says so, (collect i at 'beginning) barfs. Instead of changing the code to make this work, I propose to change the manual to *not* accept quoted forms. 1. These confuse the user. They fool the user into believing arbitrary forms could be used (if (foo) 'beginning 'end) and that it's a run-time choice when it's not. 2. These make implementations contain broken heuristics like (and consp (eq car 'quote)) I believe there's more value in making such forms accept various forms of symbols (like loop keywords), e.g. :end, end END etc. instead. On a similar issue, I have criticized UFFI for containing similar heuristics instead of clear descriptions of what types are evaluated and what not (cf. uffi-users list of October 2004). Kevin Rosenberg is about to change something there. Cost of change to users: it's a pure matter of style. Users using 'end/'beginning/'start need to use other forms, e.g. (collect foo :at :start) (collect foo at :start) (collect foo :at #:start) (collect foo #:at start) etc. All of these can be supported cheaply (in terms of run-time) and I'm sure all users will find a form that pleases them visually. Prior to making these changes in (my copy of) the code, I wanted to discuss the issue at first. Regards, Jörg Höhle

1 0