- iterate-devel - mailman3.common-lisp.net

Re: [iterate-devel] Integration with CL-ENUMERATIONS and ITERATE
by Hoehle, Joerg-Cyril 11 Mar '05

11 Mar '05

Marco Antoniotti wrote: >I would like to provide an ITERATE extension to leverage the >ENUMERATION instances. I went through the manual but I am a little >unclear about how to user DEFMACRO-DRIVER etc. >Could anybody help me? Sure, send this to the list. I admit that I'm also confused by the proliferation of terms and lack of definitions (or hard to find definitions in flowtext), e.g. defclause defclause-driver defmacro-driver defmacro-clause defclause-sequence What's a clause? what's driver? etc. My finding so far is that a driver comes into play when FOR can be replaced by GENERATE. defmacro-* is best suited for users writing extensions, drawing from the macro concept, i.e. the body expands to other iterate clauses and CL forms. C.f. iterate-test.lisp I'll make a note that the manual should provide a glossary, or at least make the index work -- where does a TeX .ind file come from? Also unclear to me is top-level vs. anywhere forms. Why are some clauses (mostly FOR, but not all of them) restricted to the top-level? As an exercise, I just looked at iterate-pg.lisp. It appears that it cannot currently be written using only names exported from the ITERATE package. The only thing lacking is an interface to add-loop-body-wrapper for the sake of with-pg-cursor. Other than that, it could be written like (defmacro-driver (FOR var-spec IN-RELATION query ON-CONNECTION conn) (add-loop-body-wrapper `(with-pg-cursor ,cursor ,conn ,query)) `(for ,var-spec = (first (or (pg-result #) (finish)))) or `(for/generate ,var-spec next (first (or (pg-result #) (terminate)))) ) Alternatively, with-pg-cursor might be "inlined" into the expansion using iterate:finally-protected and with clauses. That would also prevent double evaluation of CONN. I recommend that you stick to the manual as long as possible and write a macro-clause or -driver. Don't look at the internals, don't use return-driver-code, make-binding etc. Actually, it may be a worthwhile exercise to rewrite some part of Iterate in a system construction manner, building drivers on top of others. It could make code shorter and possibly easier to maintain. E.g. FOR ... IN could be written as follows: (defmacro-driver (FOR var IN list &optional BY (step ''cdr)) (let ((for/gen (if generate 'generate 'FOR))) `(,for/gen (,var) ON ,list BY ,step))) and the expansion is only slightly worse than now (one temporary let) Regards, Jorg Hohle

1 0

[iterate-devel] Integration with CL-ENUMERATIONS and ITERATE
by Marco Antoniotti 10 Mar '05

10 Mar '05

Hi I just submitted a new package to common-lisp.net called CL-ENUMERATION. It is a rendition of the Java Enumeration/Iterator interfaces in CL. It is essentially a way to let Java, C++, Python etc programmer migrate to CL. I would like to provide an ITERATE extension to leverage the ENUMERATION instances. I went through the manual but I am a little unclear about how to user DEFMACRO-DRIVER etc. Could anybody help me? Thanks Marco -- Marco Antoniotti http://bioinformatics.nyu.edu NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488 715 Broadway 10th FL fax. +1 - 212 - 998 3484 New York, NY, 10003, U.S.A.

1 0

[iterate-devel] devel access?
by Hoehle, Joerg-Cyril 23 Feb '05

23 Feb '05

Hi, >Maybe it would be a good idea to also give a write access to >Jörg Höhle? I'm not at all against it. My experience with CLISP is that I prefer to get my stuff in *and then* let people possibly change it, instead of having somebody play the filter first in the line, breaking my stuff and causing me more work. However, I've some technical hurdles. I'm coming from within a corporate firewall, permitting FTP, HTTP and HTTPS and I think that's all. I manage to reach the CVS devel access to sourceforge (for CLISP) because they provide cvs-ssh (i.e. the connection looks to firewalls and proxies like a typical SSL one). I can't use anon-cvs AFAIK. I haven't yet used GNU Arch and already forgot what their webpages say about firewall interaction and limitations. What's your POV in this GNUArch/CVS/FW issue? Regards, Jörg Höhle

3 2

Re: [iterate-devel] iterate 1.3?
by Hoehle, Joerg-Cyril 23 Feb '05

23 Feb '05

Marco, >2) i'd like to make sure no one else has > other fixes lying around. Well, I have more stuff that I haven't yet sent to anyone. >1) antifuchs recently gave me write access to the iterate project on > common-lisp.net. i'd like to use my new power to create a new > release of iterate (there's been a lot of work since > 1.3pre1). anybody have problems with this? For the record, my code is like a fork on iterate-1.0.9. 'Antifuchs' 1.3pre1 is based on his augment-environment code, which I had somewhat promised to review, but never integrated into my code branch (for I was generally suspicious about macroexpansion-hooks to help macrolet). I haven't yet found the time to write about my recent review, although it's ready in my mind. The executive summary is that I demonstrate that it IMHO fixes too little for too many possibilities to break things, and I demonstrate that a *macroexpand-hook* cannot reliably expand macros or prevent the "expansion" of functions. That's general about *macroexpand-hook*. Then I also point at a concrete bug in the code. The only benefit of that code is that it makes nested in-hashtable iterations work. But I can provide examples with macrolet which will lead to incorrect code with 1.3pre1. My code I sent you in January/February was one with many bug fixes and few extensions, the last one I wanted to create before I would myself dig into and face the macrolet issue. My current code (not yet sent to anyone) (and which I consider not yet finished) does the following: + symbol-macrolet is accepted and does not cause expansion errors. The expansion must not generate iterate clauses, but that's a restriction people can live with, I believe. I added testcases involving with-slots and with-accessors. + macrolet is accepted, but even stronger restrictions apply. Iterate now signals a warning "don't know how to walk ..., so iterate clauses inside it will not be seen". That's as far as one can go with macrolet and still expect stability, robustness, and correctness. But it does not work with (iter outer in-hashtable (iter in-hashtable (in outer ...))) since that would require expansion within a macrolet body. I would like to release that (when it'll be ready), because + it is progress against any previous stable versions which simply rejected all occurrences of macrolet, so general code (not involving iterate) can now compile. + it is still stable. + releasing it would allow people to play with it and give feedback about these limitations. They will certainly be happy that Iterate now processes general code, e.g. containing CL:LOOP and loop-finish that result from expansion of arbitrary forms. Since then, I have noticed one more bug and added even more testcases (the lack of testcases is appaling, but I've steadily been increasing coverage). So here's what I suggest: + Please release what I sent you in January/February (not the 13th of December, which is quite "old" already :) + Give me time to write that review about *macroexpand-hook and augment-environment. Now that I said so much about it, I think I owe you all to write it up in detail. + Give me a little time to finish my current limited macrolet support + Then produce a new release with macrolet, yet another bugfix and more testcases that I've added and continue to add. Regards, Jorg Hohle.

1 0

[iterate-devel] iterate 1.3?
by Marco Baringer 22 Feb '05

22 Feb '05

1) antifuchs recently gave me write access to the iterate project on common-lisp.net. i'd like to use my new power to create a new release of iterate (there's been a lot of work since 1.3pre1). anybody have problems with this? 2) i'd like to commit the latest set of iterate patches to the arch repo, what's the "best" version of iterate lying around. i'm guessing this is jorg's but i'd like to make sure no one else has other fixes lying around. -- -Marco Ring the bells that still can ring. Forget the perfect offering. There is a crack in everything. That's how the light gets in. -Leonard Cohen

2 1

Re: [iterate-devel] destructuring-bind bug
by Hoehle, Joerg-Cyril 18 Feb '05

18 Feb '05

Hi, >The following form never terminates. >(macroexpand '(iter (for index in '((1 2))) (collect >(destructuring-bind (a b) index (+ a b))))) My version of Iterate not only macroexpands, but it also executes it and yields (3) in CLISP-2.33.some-CVS on Linux, meeting my expectations. For Iterate, it's very important to say which Lisp implementation you're using, since it heavily depends on macroexpansion. Expansion of typical CL macros differ a lot among implementations and this is known to have caused different behaviour of Iterate across implementations. Also, you did not say which version you're using. Please submit details, e.g. Lisp version, what does destructuring-bind macroexpand-1 to, etc. Or forget about this trouble, and merely update Iterate. I believe it's also included with Marc Battyani's cl-pdf and Marco Baringer's UCW and/or arnesi packages. Regards, Jörg Höhle.

2 1

[iterate-devel] destructuring-bind bug
by Julian Stecklina 17 Feb '05

17 Feb '05

Hello, The following form never terminates. (macroexpand '(iter (for index in '((1 2))) (collect (destructuring-bind (a b) index (+ a b))))) Regards, Julian Stecklina __________________________________________________________ Mit WEB.DE FreePhone mit hoechster Qualitaet ab 0 Ct./Min. weltweit telefonieren! http://freephone.web.de/?mc=021201

1 0

[iterate-devel] thereis and always/never conflict revisited
by Hoehle, Joerg-Cyril 07 Feb '05

07 Feb '05

Hi, I've been thinking about this way of removing the ambiguity of contradicting defaults for always/never and thereis. CLtL2's loop thereis does not mention a default. ANSI-CL does. My reading of CLtL2 is compatible with an implementation of thereis as `when expr return it' -- no default value is supplied. Maybe CLtL2 didn't mention it because the default return of loop is NIL anyway, but who knows? Now, I propose to implement thereis in Iterate exactly like this. Benefit: o Some meaning associated to a form previously rejected as ambiguous (always/never with thereis). empty iterations with always+thereis would yield T. o thereis can be used with any other clause, including collect, always etc. Drawback: o Some meaning associated to a form previously rejected as ambiguous :) and which formally, still is ambiguous. This is precisely why e.g. Bruno Haible rejected a change to CLISP's loop -- which also rejects always+thereis as erroneous -- that I suggested some month ago. Indeed, the form is still ambiguous from a mathematical POV. ALWAYS and THEREIS do not really fit together. Am I sacrificing too much for too little gain? E.g., my proposed thereis could be rewritten as a macro: `(let ((,var ,expr)) (if ,var (leave ,var))) so, when that's needed, the user could do it themselves. Regards, Jorg Regards, Jorg Hohle.

1 0

[iterate-devel] how to generate pdf from tex?
by Hoehle, Joerg-Cyril 01 Feb '05

01 Feb '05

Hi, I've been able to generate a .pdf from a .tex in a non-straight-forward way, so I'm wondering how it should actually be done. 1. using pdf or latex (dvi output), TeX will complain about the missing iter-man.ind file. o Where does it come from? o is there an easy substitute (instead of using null.tex/sty)? I used iter-man.idx instead, but this was not understood by the pdftex driver 2. pdftex pdftex iter-man -> error about \documentstyle pdfvirtex &pdfcslatex iter-man (I found the dir with the pdfcslatex.fmt and other .fmt in my Suse 8.1 distribution in /var/lib/..., but no pdflatex.fmt) works, after adding symlink of pdftex.cfg into the current directory The table of content did not generate clickable entries, nor bookmarks. I thought this was the big advantage of using pdftex? dvipdf iter-man.dvi generated a .pdf (smaller in size), this uses dvips internally, and a %pdf-1.3 header, while pdftex generates %pdf-1.4 Both appear with similar quality on screen, is that normal? I included that .pdf from pdfcslatex in the distribution I can send on request (see previous e-mail of mine, from today). Regards, Jorg Hohle.

1 0

[iterate-devel] more bug-fixes available
by Hoehle, Joerg-Cyril 01 Feb '05

01 Feb '05

Hi, I have fixed some more bugs or problem areas in my copy of iterate, and I can send that to anybody who wishes. You may want to try the following test in your original version: (deftest or.1 (iter (generate x in '(a nil nil 1)) (generate y in-vector '#(2 #\c #\d)) (collect (or (next x) (next y)))) (a 2 #\c 1)) >From my changes file: o (if (next x) ...) works -- the test form is walked too + allow better compiler optimizations by keeping macros like and, or, prog1 etc. in the expanded code. The expansion also becomes more readable (closer to the source). + #+clisp support ((setf foo) new-value args...) function syntax + #+clisp simple support for handler-bind (not walking handlers) + free-vars correctly handles locally, symbol-macrolet and declare + use macroexpand-1 only to ensure all special handlers get used and still more test cases Marco, I don't know how you modified your UCW and CPS converter for use with CLISP, but I recommend you do not use sys::setf-... (clisp internals) -- I'm not even sure your patch works in all conceivable cases. I don't know what the requirements for ucw/cps are, but here's what I use for iterate: #+clisp ; some macros expand into ((setf foo) value other-args...) ;; reported by Marco Baringer on 24 Jan 2005 ((typep form '(cons (cons (eql setf) *) *)) (apply #'walk-cdr form)) You can see that the setf form is treated like any other function call, and there's no need to make use of (possibly changing) internals. Things I changed previously are: o (DSETQ (values x nil y)) works o restore v 1.0/2 special/macro order, e.g. change special-form? etc. o in-vector/in-sequence downwards by 2 did not start with last element (iter (for i index-of-vector #(0 1 2 3 4) downto 0 by 2) (collect i)) yielded (3 1), expect (4 2 0) by similarity with (iter (for i downfrom 4 by 2) (collect i)) and (iter (for i from 4 downto 0 by 2) (collect i)) + locally accepts [0..N] declarations + constant? and self-evaluating? accept all self-evaluating forms, including keywords and T/NIL, but no other symbols. + avoid ~< pretty-printer (not in Cormanlisp) in error messages, leave that to the implementation + if-1st-time & special-clause ELSE: type 'boolean instead of symbol + #+clisp "explicit declarations are always placed in the generated code" as the manual says. + #+lispworks multiple-value-bind is recognized (again) + make-next-code N <> 1 generates correct code Regards, Jorg Hohle

1 0