- iterate-devel - mailman3.common-lisp.net

Re: [iterate-devel] Between-each
by Hoehle, Joerg-Cyril 02 Dec '05

02 Dec '05

Hi, Mészáros Levente wrote: >I may be wrong about this, but I could not find support for a >quite commonly used loop pattern, namely when you want >something to be evaluated between each loop cycles. Please accept my apology for responding that late. I felt a little uncomfortable (see below) and did not quite know how to reply. >(def-special-clause BETWEEN-EACH (between) > (return-code :body (list (if-1st-time nil (list (walk-expr between)))))) Correct code (like implementation of IF-FIRST-TIME). >Maybe I'm wrong, could somebody tell how to do this with >iterate? If there's no support for this perhaps you could put >something like this into the main branch. You could use a macro-substitution and expand to the exported and documented IF-FIRST-TIME and DEFMACRO-CLAUSE, instead of using package internals like return-code. Your implementation shows that there's the same relationship between IF-FIRST-TIME and your BETWEEN-EACH like there's between WHEN and UNLESS. So why not add it, even it would be redundant to purists? However, given that analogy, I find the given name quite misleading, as it's very different from AFTER-EACH. >And it works like: >(iter (for i from 1 to 3) (between-each (collect (+ i 100))) > (collect i)) >(1 102 2 103 3) Hmm, I feel uncomfortable with the name, given the functionality you show. "Between" is something I'd expect to happen after one iteration, while you use it within the body. I.e., my wording shows I'm thinking in terms of AFTER-EACH. Well, there's AFTER-EACH, which is subject to code movement, but wouldn't it work for you? Another question is: how does this extension behave when used outside the top-level of the loop. Does the given name adequately reflect the functionality? E.g. (iter (for i...) (if (oddp i) (between-each (collect i)))) What would a reader expect as value from such a form? What about BUT-FIRST[-TIME] as a name? Would you like to show more elaborate loop examples? Regards, Jörg Höhle.

1 0

Re: [iterate-devel] fishy iterate use of delete...
by Hoehle, Joerg-Cyril 02 Nov '05

02 Nov '05

Faré wrote: >Indeed, (delete ...) has side-effects and doesn't preserve its second >argument. Although I can hardly imagine any implementation where DELETE on lists would operate in ways that break the code (except for an ANSI-compliance stress tester mode), it's not correct use of DELETE by the CLHS. Thus, a minimally disruptive patch would be to replace >(delete entry index) with (setf index (delete entry index)) twice, Funny. It seems the code has an extra element in the list precisely to avoid the well-known "delete yields empty list not reflected in variable" Lisp FAQ issue, but CLHS precisely disables such optimization when using DELETE (instead of a self-written naive implementation). >Someone who understands the code >better audit what is being done by the considered functions. > and (delete nil temps) with (remove nil temps). Assuming you're talking about make-mv-dsetqs: (let ((mv-setq `(multiple-value-setq ,vars ,value)) (temp-vars (delete nil temps))) Great, that's not just being picky about standard compliance, you discovered a real bug in the code! temps is still used afterwards and DELETE destroys it. REMOVE MUST be used instead. Here's a testcase I came up with: (let (x y z) (dsetq (values nil x (y z)) (values 1 'a '(b c))) (list x y z)) must yield (a b c), not (a nil nil). Regards, Jörg Höhle.

1 0

[iterate-devel] fishy iterate use of delete...
by Faré 02 Nov '05

02 Nov '05

Dear Iterate developers, while I was compiling cl-pdf, sbcl detected some invalid uses of delete in iterate.lisp. I checked that these errors were there in the iterate 1.4 release available from asdf-install. Indeed, (delete ...) has side-effects and doesn't preserve its second argument. Thus, a minimally disruptive patch would be to replace (delete entry index) with (setf index (delete entry index)) twice, and (delete nil temps) with (remove nil temps). Someone who understands the code better audit what is being done by the considered functions. Patch attached. Happily these errors didn't seem to affect the behaviour of cl-pdf. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] You may easily play a joke on a man who likes to argue -- agree with him. -- Ed Howe

2 1

[iterate-devel] Between-each
by Mészáros Levente 25 Oct '05

25 Oct '05

Hi, I may be wrong about this, but I could not find support for a quite commonly used loop pattern, namely when you want something to be evaluated between each loop cycles. It means if the loop runs N times it will be evaluated N - 1. I did a patch like: (def-special-clause BETWEEN-EACH (between) (return-code :body (list (if-1st-time nil (list (walk-expr between)))))) And it works like: (iter (for i from 1 to 3) (between-each (collect (+ i 100))) (collect i)) (1 102 2 103 3) Maybe I'm wrong, could somebody tell how to do this with iterate? If there's no support for this perhaps you could put something like this into the main branch. Please send it to my direct mail address too, because I'm not on the list. Levy

1 0

[iterate-devel] final version of "conservative" Iterate: 1.4
by Hoehle, Joerg-Cyril 23 Jun '05

23 Jun '05

Hi, I've sent a new version of Iterate to Marco, Marc, Kevin, Peter and a few others, so it should appear in the usual places soon. Changes are too numerous to mention (and it depends on which version you're currently using:) and still extremely conservative. I'll have to supply the updated iter-man.pdf a little later, because my current Ubuntu laptop generates much uglier fonts therein than the old Suse one (maybe the default printer which is ljfour is the reason??). o supports nested in-hashtable and in-package o adjoin :at :start did not honour :test o fixed in-file/in-stream destructuring o do-next forgot about finally etc. clauses o works as case-sensitive package in CLISP (don't know about Allegro) o various optimizations, less bugs, more tests o tested with clisp, cmucl, sbcl, gclcvs This is what I consider the last version of 1.4, called "conservative". 1.5 will do some possibly incompatible changes to allow more optimizations (see recent e-mail of mine about my version plan). Regards, Jörg Höhle.

1 0

Re: [iterate-devel] Version plan for Iterate 1.4 and 1.5
by Hoehle, Joerg-Cyril 17 Jun '05

17 Jun '05

Marc Battyani wrote: >>code-movement [possible changes] >Those parts (prologue/epilog/var bindings) are under- (and >badly) specified >in LOOP and implementations differ on this. So let's keep it >in a sane way. I find it underspecified in Iterate as well. I want to change that state :-) E.g. Andreas' with-*-iterator based in-hashtable and in-package has very different initialization behaviour from the old ones (see below). It occured to me that there's another issue: iterate-pg may not work as expected, for similar reasons. (iter (with foo = connection) (with x on-relation foo connection bar)) #) ;won't work Reason: WITH-PG-TRANSACTION is move outside the whole Iterate form, thus foo and bar get evaluated before any other thing in Iterate(!) My current belief is that the following order would be a better match: (let (...) (wrappers: unwind-protect and with-pg-transaction, in order of apperance (block (tagbody #)))) Currently, it's (wrappers: with-package-iterator, with-hashtable-iterator, with-pg-transaction (let # (unwind-protect (block (tagbody #))))) Therefore (iter (with table = (foo)) (for (k v) in-hashtable table)) used to work with the old maphash based Iterate, but fails since Andreas' changes in 2004. I think this is also a result of underspecification of Iterate, together with a quite complex package and lots of feature interactions (initialization time, generators, declarations, top-level only clauses, support for PREVIOUS etc.) As a result, simple examples (e.g. the testsuite) work, but subtle forms fail. I'm going to change that as well (1.5). At the present, one can think of two forms of body wrappers: a) around the whole body (e.g. current with-*-iterator) b) around BLOCK, but inside LET (e.g. unwind-protect) I think I'm going to provide for only variant b, as shown above. Subtle interactions between unwind-protect clauses can still happen. I think that's a sign of limitations of Iterates design. All in all, packages needing a code-walker are not that great (they appear like a can of worms, e.g. Screamer was well, with some forms that work as expected and others that fail surprisingly (cf. primordial in Screamer). There's quite some truth when Bruno Haible expresses that CL somehow fails to be its own extension language. LOOP's weakness is also its strength: not needing a code walker, being limited to top-level forms, having a grammar so its transformation does not require macroexpand and remains syntactic. Regards, Jörg Höhle

1 0

[iterate-devel] Iterate in-hashtable expansion, with-hashtable-iterator vs. mapha sh
by Hoehle, Joerg-Cyril 15 Jun '05

15 Jun '05

Hi, first, some good news: I think I found an easy way to make Iterate support nested in-hashtable, together with with-hashtable-iterator: Just map with-hashtable-iterator to walk-cddr-with-declarations in *special-form-alist* That should basically work good enough for a start, but it does not walk the hashtable parameter. Next: a possible change -- what do you think? Some packages like SCREAMER have explicitly chosen to support Iterate instead of loop because they walk code and suffer from similar limitations about macrolet as Iterate does. In the past, Iterate never expanded to forms that Screamer was unable to analyse. Starting with Andreas Fuchs' change, he made IN-HASHTABLE expand to with-package-iterator, which is based on MACROLET. Similarly for in-package. As a result, Screamer would error out should it be updated to use one of ours iterate.lisp. Therefore, I'm thinking about introducing a choice, either as a *settable-variable*, or as new iteration keywords, for how to Iterate through hashtables (and packages). I first thought about in-hashtable: use with-hashtable-iterator on-hashtable: use maphash to build up list upfront. Then I realized that it should rather be the other way round, or somebody would have to modify the Screamer examples, and all other unknown packages built using Screamer (and possibly Series, I don't know). An alternative may be to get Screamer support macrolet, but that's IMHO even farther away than having Iterate do that. So what do you prefer? a) in-hashtable -> maphash, on-hashtable -> with-*-iterator and same for packages "reintroduce old behaviour" b) on-hashtable -> maphash, in-hashtable -> with-*-iterator and same for packages "affirm changed behaviour, need to change random packages" c) defvar iterate::*use-with-iterators* Screamer could set such a variable, or at least the examples should do so (since only the examples depend on Iterate, not Screamer itself) I'm biased towards b). Most users will benefit from the iterator, and I'd need to convince the Screamer authors anyway to package a more recent version of Iterate with it, so the examples could be updated at that time, or to at least have the debian cl-screamer package not include but depend on cl-iterate (and similarly for cl-pdf etc.). Still, the users would have to change their Screamer code. (and no, I haven't yet tested Screamer with my version of Iterate, this is just code review and thinking loud, yet I expect no problems beyond that). BTW, having a maphash-based iterator allows to modify any hashtable item while iterating it, instead of just the current one (forbidden by ANSI-CL and many other language specifications). That can come handy sometime. While we are at it, there's a difference between the old in-package and the new (since Andreas Fuchs in 2004): the new (with-package-iterator based) one does not evaluate the :external-only argument any more -- How I hate it when documentations does not precisely state what is evaluated and what not! it always makes simple examples work and complex ones fail. With-package-iterator does not provide for evaluation of this, while the old in-hashtable dispatched to either do-symbols or do-external-symbols. So far, I plan to stick to that change, or does anybody wave hands? Regards, Jörg Höhle.

1 0

Re: [iterate-devel] COMPILE FAILURE UNDER ALLEGRO CL 7
by Hoehle, Joerg-Cyril 13 Jun '05

13 Jun '05

Mark L. Williams CIV NSWC-PC wrote: >>;;; Compiling file [path]/iterate.lisp >>; While compiling (:top-level-form "iterate.lisp" 85868): >>Error: DEFCLAUSE: required part of template (FOR var-spec &sequence) >>is of odd length Since I am suspecting a "modern" Allegro package problem, could you please take a look at (defun define-clause (define-form clause-template body generator?) in the source? In my current source (since RCS 1.24, mid-April), it says therein: (if (and (symbolp last) (string= last '&sequence)) In iterate-1.0.9 and still in November 2004, it said: (if (and (symbolp last) (string= last "&SEQUENCE")) a) what does yours say? b) if "&SEQUENCE", does '&sequence help? BTW, I'm about to send Marc, Marco and Peter a new version. It would be nice to hear if b) fixes your problem. Regards, Jörg Höhle

1 0

Re: [iterate-devel] COMPILE FAILURE UNDER ALLEGRO CL 7
by Hoehle, Joerg-Cyril 13 Jun '05

13 Jun '05

Hi, Mark L. Williams CIV NSWC-PC wrote: >This is a strictly newbie question and I'm not on any mailing lists, so >please reply directly. > >I have downloaded the current version of iterate and have tried to >compile it with Allegro CL (Sparc version). Actually, I got it as part >of the the CL-pdf / CL-typesetting packages but have pulled it >independently because... Did you get cl-pdf and/or cl-typesetting to compile the iterate which is part of it? What version of iterate are you using? Please report something like file dates of the archive (not extraction date), since the file itself doesn't mention any precise enough version number. >the asdf build (executed by attempting to evaluate (asdf:operate >'asdf:load-op 'iterate) after establishing the necessary symbolic link >to the iterate.asd file) fails with the message: > >;;; Compiling file [path]/iterate.lisp >; While compiling (:top-level-form "iterate.lisp" 85868): >Error: DEFCLAUSE: required part of template (FOR var-spec &sequence) is >of odd length > >I've taken a quick look for similar-sounding bugs and/or any other hints >that might help, but I haven't found anything and I'm too new to >interpret and debug myself. Any help will be greatly appreciated. Never seen that myself. Are you using some beta version of Allegro-CL/sparc? Do you nkow about "modern" packages (i.e. case-sensitive reader)? Could some "modern" package get in the way? Can you try (apropos "&sequ") [or (apropos "&SEQU")]? How many symbols does it show? My suspicion is that there maybe 2 symbols of similar names, but distinct in case, and this may be causing trouble (i.e. &sequence and &SEQUENCE). Can you load, and/or compile the file by hand? (of course, need to load package.lisp initially) Can you load first, then compile? Can you compile in a fresh Lisp (not loaded first)? And finally, is there a problem with your mailer? I got your message as a text attachment within an attachment within an attachment. Maybe you should subscribe to iterate-devel for as long as your bug-report is unsolved, instead of hoping for some gentle moderator to forward your message to the list. iterate-devel has low traffic. Regards, Jörg Höhle

1 0

[iterate-devel] COMPILE FAILURE UNDER ALLEGRO CL 7
by Williams, Mark L CIV NSWC-PC 09 Jun '05

09 Jun '05

This is a strictly newbie question and I'm not on any mailing lists, so please reply directly. I have downloaded the current version of iterate and have tried to compile it with Allegro CL (Sparc version). Actually, I got it as part of the the CL-pdf / CL-typesetting packages but have pulled it independently because... the asdf build (executed by attempting to evaluate (asdf:operate 'asdf:load-op 'iterate) after establishing the necessary symbolic link to the iterate.asd file) fails with the message: ;;; Compiling file [path]/iterate.lisp ; While compiling (:top-level-form "iterate.lisp" 85868): Error: DEFCLAUSE: required part of template (FOR var-spec &sequence) is of odd length I've taken a quick look for similar-sounding bugs and/or any other hints that might help, but I haven't found anything and I'm too new to interpret and debug myself. Any help will be greatly appreciated. VR/ Mark L. Williams

1 0