- iterate-devel - mailman3.common-lisp.net

Re: [iterate-devel] Version plan for Iterate 1.4 and 1.5
by Hoehle, Joerg-Cyril 02 Jun '05

02 Jun '05

Marc Battyani wrote: >> Plans for 1.5: >> + Move more code from the initially / loop prologue section >inside the let >> binding initialization, which might cause some user code to >fail because of code movement >Please don't wreck that! It's the reason I initially started >to use iterate. >Those parts (prologue/epilog/var bindings) are under (and >badly) specified >in LOOP and implementations differ on this. So let's keep it >in a sane way. What I mean is that Iterate is inconsistent. Did you know that WITH clauses get completely placed inside the surrouding let, while FOR ... ON <list> gets set in the :initially section, while other (e.g. reducing ...) also get set inside LET? I wanted to move as much as possible inside the surrouding LET. There's a *small* posibility that user code will break. In fact, I expect none of the current tests to fails as a result of such a change. >> For a farther future: move to another design for iteration >Do you already have some ideas on this ? No. gclcvs (which passed most Iterate tests) reminded me of the other Iterate package featuring gathering. I'll look at that. What really disturbs me with Iterate has a design weakness about referential transparency and inadvertent capture, e.g. (iter (with-some-other-macro (collect foo))) when with-some-other-macro itself expands to an iterate loop. It similar to those cases when an implicit block would change the target of a return statement. There's IMHO no way around that deficiency in Iterate. LOOP OTOH is robust against that, since it knows no clauses nested inside normal Lisp forms. Regards, Jörg.

1 0

[iterate-devel] Version plan for Iterate 1.4 and 1.5
by Hoehle, Joerg-Cyril 02 Jun '05

02 Jun '05

Hi, here's how I see the next steps w.r.t. Iterate. I've already sent a version of Iterate 1.4 to some of you (and it's on common-lisp.net thanks to Marco). I want 1.4 to be the last really compatibly-conservative Iterate around, before I'll make some possibly incompatible changes in 1.5. Plans for minor updates to 1.4: + some more bug fixes (e.g. did you know that in-stream did not work correctly with destructuring)? -- some already done (but not yet sent to you). + most important: additions to the documentation: - a glossary (what's the loop epilogue anyway?) - a reference of which clause is allowed at top-level only, as a generator etc. Plans for 1.5: + Move more code from the initially / loop prologue section inside the let binding initialization, which might cause some user code to fail because of code movement + harmonize "clause identifiers" across defmacro-driver, remove-clause and display-iterate-clause + possible change to thereis (different interaction with other clauses like always etc.) Somewhere in between: + other minor modifications (e.g. avoid duplication of code in some clauses) + deprecate add-loop-body-wrapper (a misnomer) and export add-iterate-wrapper or wrap-iterate-form or wrap-iterate-with wrap-iterate-using -- what do you suggest? + numerous other topics I have on my list For a farther future: move to another design for iteration that does not need a code-walker and as such suffers from the problematic of pre-order code traversal, or consider using a portable code walker library or implementation-dependent code. BTW, I just run Iterate through gclcvs (after delete-package "iterate" already present there) and it mostly worked, except for problems with macro-function, symbol-macrolet etc. Regards, Jorg Hohle.

2 1

Re: [iterate-devel] iterate 1.3?
by Hoehle, Joerg-Cyril 02 Jun '05

02 Jun '05

Hi, back in February I had promised: > + Give me time to write that review about *macroexpand-hook >and augment-environment. Now that I said so much about it, I >think I owe you all to write it up in detail. I realize I've now forgotten most of my analysis about why the cltl2 augment-environment is limited and doesn't scale to the large. The one thing I can remember was one concrete bug in Andreas Fuchs' code, independent on the weakness of the augment-environment design of CLtL2. Andreas binds *macroexpand-hook* to some handler of his own. Obviously, he does so because he needs the functionality. Correct working of his code depends on this binding. But if his code doesn't recognize an interesting form, he calls #'FUNCALL on the expander. That's bogus and won't work in the large. Consider another macro package of similar complexity, which also depends on rebinding *macroexpand-hook*. This package will fail mysteriouly because Andreas' code bypasses it. Or consider another Andreas' code expanding another macro that behave as badly: Andreas' handler will never get called, and Iterate will fail somehow. The only correct way for a handler is, instead of #'funcall, to invoke the previous value of *macroexpand-hook*. It's only such a weak form of "declining" that's available to the handler. BTW, CLHS says it as well: "Users who put their own function into *macroexpand-hook* should consider saving the previous value of the hook, and calling that value from their own." So that was the bug, and I'm again sorry I forgot most of my reflection about the limits of the augment-environment and brothers design. Regards, Jorg Hohle

1 0

Re: [iterate-devel] devel access?
by Hoehle, Joerg-Cyril 20 May '05

20 May '05

Hi, this is a short note that I'm making progress towards better net connectivity despite corporate firewalls. So far, I've accessed: o cvs-ssh.suorceforge.net (because they redirect port 80+443 to the ssh port and I have developer access there, but not yet managed to migrate my key from F-Secure (commercial) SSH to Ubuntu/Debian's Openssh on the laptop. o cvs-pserver.sourceforge.net (redirect 80+443 to 2401) o common-lisp.net:iterate because Arch (baz) works via http I couldn't get to common-lisp.net:slime because the CVS (2401) port is not accessible. I started browsing through the Arch/baz docs and tutorials, but boy, I feel there are too many commands out there. Since Ubuntu/Debian is such a nice setting, I installed cmucl and sbcl (and gcl) packages (and slime-1.2.1 promptly failed for sbcl and clisp :-( Why did nobody tell me before that both cmucl and sbcl have some failing iterate tests? Now I believe I need to generate a gpg key (instead of pgp ones) to get write access to common-lisp.net? Regards, Jörg Höhle.

1 0

Re: [iterate-devel] type declarations from the Iterate package
by Hoehle, Joerg-Cyril 29 Mar '05

29 Mar '05

Hi, Marc Battyani wrote: [Thanks for your feedback] >> I was wondering (given zero e-mail traffic on this subject here) if >>somebody had noticed that Iterate outputs quite dubious type declarations >when being asked for them (i.e. (declare (iterate:declare-variables)))? >I used it a few times. The (declare (iterate:declare-variables)). > IIRC it worked for me. I always tend to express myself too generally :-( For *some* constructs, Iterate outputs wrong type declarations. It depends on a) the construct and b) the implementation (more precisely the results of TYPE-OF). E.g. in CLISP, (type-of 0) -> BIT and (macroexpand-1'(iter:iter(declare(iter:declare-variables))(iter:repeat 0))) reveals (LET* ((#:COUNT3 0)) (DECLARE (TYPE BIT #:COUNT3)) ...) whose type is evidently broken. Obviously the code was written with (type-of 0) -> FIXNUM in mind. Widening that topic, I'm wondering whether generally, even the implementation's LOOP or other iteration constructs always produce correct declarations in border cases. E.g. suppose array-dimension-limit = most-positive-fixnum Loop code traversing a vector could possibly be written like: (declare (fixnum running-var)) ; because array-dimension fits a fixnum (incf running-var step) ; might escape fixnum limit (when (> running-var limit) (loop-finish)) but not like this (let ((var (+ running-var step))) ; no type declaration for var, ; could be a bignum after last iteration (if (> var limit) (loop-finish)) (setq running-var var)) ; in this case var is not too large Similarly, consider (dotimes (i most-positive-fixnum) (declare (fixnum i)) ...) Within the user-visible body, the type declaration holds. Yet dotimes could be implemented using (incf i) (when (> i #:limit) ...), thereby breaking the type declaration. Is that just academic and I'm worrying too much? Regards, Jörg Höhle.

1 0

[iterate-devel] type declarations from the Iterate package
by Hoehle, Joerg-Cyril 18 Mar '05

18 Mar '05

Hi, I was wondering (given zero e-mail traffic on this subject here) if somebody had noticed that Iterate outputs quite dubious type declarations when being asked for them (i.e. (declare (iterate:declare-variables)))? On CLISP, nobody notices, since they are ignored anyway, but on CMUCL... Did anybody understand the *intent* of J. Amsterdams type handling explained in the Iterate manual? 1. It looks to me that Iterate's handling of type declarations is somewhat broken, at least in the light of current compiler standards (things looked differently ten years ago). Other than that, I fixed a few unrelated bugs and will soon export remove-clause -- don't hold your breath. After that, I'll try to look into the topic of type declarations. Does anybody care about these at all anyway?? Regards, Jorg Hohle

2 1

[iterate-devel] Integration with CL-ENUMERATIONS and ITERATE
by Hoehle, Joerg-Cyril 18 Mar '05

18 Mar '05

Hi, I wrote: >It appears that it [iterate for Postgresql result sets] > cannot currently be written using only names exported >from the ITERATE package. The only thing lacking is an >interface to add-loop-body-wrapper Well, I find add-look-body-wrapper quite an odd name to be exported from the Iterate package. OTOH, I'm quite against gratuitous name changes to software packages that have been basically working for the last ten years. So far, I have (hopefully) managed to maintain strict compatibility. Should I a) just export iterate:add-loop-body-wrapper b) keep iterate::add-loop-body-wrapper and export a different name for the same function, e.g. create iterate:add-wrapper or iterate:add-wrapping-form c) maintain status quo What do users think? In case you don't remember, the issue at hand is that unless some such functionality is exported, one cannot pretend that Iterate presents an API (visible from the list of exported names) that can be used for extending Iterate in some useful ways (e.g. pretend FOR ... IN-FILE did not exist and provide it). Regards, Jorg Hohle

1 0

Re: [iterate-devel] Integration with CL-ENUMERATIONS and ITERATE
by Hoehle, Joerg-Cyril 11 Mar '05

11 Mar '05

Marco Antoniotti wrote: >I would like to provide an ITERATE extension to leverage the >ENUMERATION instances. I went through the manual but I am a little >unclear about how to user DEFMACRO-DRIVER etc. >Could anybody help me? Sure, send this to the list. I admit that I'm also confused by the proliferation of terms and lack of definitions (or hard to find definitions in flowtext), e.g. defclause defclause-driver defmacro-driver defmacro-clause defclause-sequence What's a clause? what's driver? etc. My finding so far is that a driver comes into play when FOR can be replaced by GENERATE. defmacro-* is best suited for users writing extensions, drawing from the macro concept, i.e. the body expands to other iterate clauses and CL forms. C.f. iterate-test.lisp I'll make a note that the manual should provide a glossary, or at least make the index work -- where does a TeX .ind file come from? Also unclear to me is top-level vs. anywhere forms. Why are some clauses (mostly FOR, but not all of them) restricted to the top-level? As an exercise, I just looked at iterate-pg.lisp. It appears that it cannot currently be written using only names exported from the ITERATE package. The only thing lacking is an interface to add-loop-body-wrapper for the sake of with-pg-cursor. Other than that, it could be written like (defmacro-driver (FOR var-spec IN-RELATION query ON-CONNECTION conn) (add-loop-body-wrapper `(with-pg-cursor ,cursor ,conn ,query)) `(for ,var-spec = (first (or (pg-result #) (finish)))) or `(for/generate ,var-spec next (first (or (pg-result #) (terminate)))) ) Alternatively, with-pg-cursor might be "inlined" into the expansion using iterate:finally-protected and with clauses. That would also prevent double evaluation of CONN. I recommend that you stick to the manual as long as possible and write a macro-clause or -driver. Don't look at the internals, don't use return-driver-code, make-binding etc. Actually, it may be a worthwhile exercise to rewrite some part of Iterate in a system construction manner, building drivers on top of others. It could make code shorter and possibly easier to maintain. E.g. FOR ... IN could be written as follows: (defmacro-driver (FOR var IN list &optional BY (step ''cdr)) (let ((for/gen (if generate 'generate 'FOR))) `(,for/gen (,var) ON ,list BY ,step))) and the expansion is only slightly worse than now (one temporary let) Regards, Jorg Hohle

1 0

[iterate-devel] Integration with CL-ENUMERATIONS and ITERATE
by Marco Antoniotti 10 Mar '05

10 Mar '05

Hi I just submitted a new package to common-lisp.net called CL-ENUMERATION. It is a rendition of the Java Enumeration/Iterator interfaces in CL. It is essentially a way to let Java, C++, Python etc programmer migrate to CL. I would like to provide an ITERATE extension to leverage the ENUMERATION instances. I went through the manual but I am a little unclear about how to user DEFMACRO-DRIVER etc. Could anybody help me? Thanks Marco -- Marco Antoniotti http://bioinformatics.nyu.edu NYU Courant Bioinformatics Group tel. +1 - 212 - 998 3488 715 Broadway 10th FL fax. +1 - 212 - 998 3484 New York, NY, 10003, U.S.A.

1 0

[iterate-devel] devel access?
by Hoehle, Joerg-Cyril 23 Feb '05

23 Feb '05

Hi, >Maybe it would be a good idea to also give a write access to >Jörg Höhle? I'm not at all against it. My experience with CLISP is that I prefer to get my stuff in *and then* let people possibly change it, instead of having somebody play the filter first in the line, breaking my stuff and causing me more work. However, I've some technical hurdles. I'm coming from within a corporate firewall, permitting FTP, HTTP and HTTPS and I think that's all. I manage to reach the CVS devel access to sourceforge (for CLISP) because they provide cvs-ssh (i.e. the connection looks to firewalls and proxies like a typical SSL one). I can't use anon-cvs AFAIK. I haven't yet used GNU Arch and already forgot what their webpages say about firewall interaction and limitations. What's your POV in this GNUArch/CVS/FW issue? Regards, Jörg Höhle

3 2