Denis,

I got to thinking after I sent the last email and realized that my example was a bit confusing, but you did an excellent job deciphering it. To sum up what it seems you are proposing:

Symbols in the iterate package like FOR, WHILE, COLLECT, APPENDING, will be blessed in such a way that they can be accessed via symbols in the keyword package. Writing extensions to Iterate via defmacro-clause and friends will remain the same, it will take a symbol, not a keyword, and will not try to institute the same keyword short cut as the blessed clauses.

Is that a fair description? Is the root of your desire to do this the fact that you want to use the symbols FOR, WHILE, COLLECT, etc in the same package that you want to use Iterate?

As for your points, I will work backwards:

Most of iterate keywords are in a keyword package already as they are used in common lisp loop macro. What kind of collision do you mean?

People don't usually worry about keyword collisions because keywords act as syntactic markers and do not hold bindings. I am reminded of a recent post by Xach regarding binding certain functions to keywords in your .rc file like...

(defun :go (&optional (thing *)) "View THING with some appropriate viewer." ;; On my Mac laptop, I use "open" instead. (sb-ext:run-program "gnome-open" (list (princ-to-string thing)) :search t) thing)

These are functions that serve to make the Lisp environment a bit more appealing, not to extend a program's capabilities. This is a fine thing to do right up until the point that someone decides to add these definitions to some utility library. Once that happens, loading that library will clobber any definitions that were in place before. This is an example of how the keyword package is prone to collisions. The collision above is in the space of function definitions, but in Iterate we have a similar space of clauses. Currently those clauses are represented as symbols which afford us all of the protections that the package system gives us to avoid those kinds of collisions.

Probably the most confusing thing that my last message did was use the code in the manual dedicated to extending FOR. This made it seem like I was proposing a means that we could extend FOR while I was talking about working within the framework Iterate defines to build a different kind of clause that happens to be named TEST:FOR. Let's say, as a better example, I wanted to define that in my package, that happens to use some data structure that indexes for 1 instead of 0, for loops in iterate are to always default to starting index 1 rather than 0. One way to do this is to define my own clause, FOR that is local to the package and has the exact definition as ITERATE:FOR except that that initial default 0 is turned into a 1. Now I'm not saying that I would ever do this, I'm not saying that this is a good idea, I'm not even saying (now that I see what you are proposing) that I cannot do this with your proposed change in place, but I am saying that the way Iterate is currently set up, INDEX-FROM-ONE:FOR would be on equal syntactic footing as ITERATE:FOR. I do feel that this is an extremely clean mechanism that holds a lot of similarity with standard CL functionality.

And with that, I will walk back my objection from, "it might break stuff" to "it isn't necessary and it doesn't look consistent". So I suppose that it is a matter of taste. However, as you pointed out, but I was unaware of, Iterate has defsynonym. Why isn't your patch nothing more than a single file that looks like:

(iterate:defsynonym :for iterate:for) (iterate:defsynonym :collect iterate:collect) ... (iterate:defsynonym :appending iterate:appending)

Why isn't this just a small separate library called iterate-keyword that only exports ITERATE?

Note that including this in Iterate distribution would enable usage of keywords as an alternate syntax for those clauses, drivers, and collectors that happen to have been blessed by Iterate during its creation, but it would also break any body's code that decided to (defmacro-clause :for ...). I'm not sure that this is worth worrying about, but from my point of view this is wrong in the same way someone defining a library that has (defun :go ...) in it is wrong.

One last thing, I would hope that you wouldn't actually attempt to patch iterate to not export FOR, WHILE, etc as that really would break code.

Zach

On Tue, Oct 16, 2012 at 1:06 AM, Denis Budyak budden73@gmail.com wrote:

Hi Zach, list!

...

If you encode FOR and COLLECT as special tokens that Iterate's parser/compiler just understands you break the extensibility.

Iterate-keywords makes minimal change to iterate. Roughly, it makes (defsynonym :any-clause-head any-clause-head), but only for standard iterate clauses, not for user defined. As synonyms do not work with special clauses like (next-iteration) (it can be considered a bug in synonym mechanism), patch to iterate itself was required.

Consequently, it does not break your example. %(iterate-keywors:iter (test:for i in-whole-vector #(1 3)) (:collect i)) (1 3)

%(iterate-keywors:iter (:for i in-whole-vector #(1 3)) (:collect i)) ...errs...

For is intended to be extensible via second keyword, in-whole-vector in our case. If we define it as in manual, %(in-package :iterate-keywords) %(defmacro-clause (FOR var IN-WHOLE-VECTOR v) ...), It will work with iterate-keywords as intended: %(in-package:cl-user) %(iterate-keywords:iter (:for i in-whole-vector #(1 3)) (:collect i)) (1 3) If we define any new clause, iterate-keywords won't touch it. We can define it in :test package and use the symbol from that package, that is, %(iter (test:our-clause)) Additionally, we can make %(defsynonym :our-clause test:our-clause) and thus enable ourselves to use keyword: %(iter (:our-clause))

...

and even then, it would suffer from name collisions inherent in the keyword package

Most of iterate keywords are in a keyword package already as they are used in common lisp loop macro. What kind of collision do you mean?