Why do some CL library functions have :key arguments?
I am asking because I am working on some statistics functions, and the design choice came up. Specifically, I can write functions like
(defun quantiles (sequence quantiles &key (key #'identity)) ...)
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
but this conses a bit more. Not a major concern at the moment, but if it ever becomes one, I could just define a compiler macro to take care of forms like this and transform to something like
(quantile-with-keys vector quantiles key)
I also thought of defining something like
(defgeneric map1 (function object) (:method (function (list list)) (mapcar function list)) ...)
for the sole purpose of mapping objects to similar objects (eg lists to lists, arrays to arrays, etc) and allow it to be optimized away (like above) by compiler macros.
I would appreciate advice on this. I am especially interested in the reason why some CL functions have :key arguments: is it because of efficiency, backward-compatibility/history, or something else?
Thanks,
Tamas
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp tkpapp@gmail.com wrote:
Why do some CL library functions have :key arguments?
[...]
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
This not only conses "a bit more", it also duplicates traversal efforts - The original list must be traversed, and the consed-up list of key values as well. I think it is prudent that the CL library functions offer ways to reduce consing for cases where "a bit" is too much (and "a bit" can become a lot if a program operates on long lists).
-Hans
On Mon, 04 Jul 2011 11:39:39 +0200, Hans Hübner wrote:
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp tkpapp@gmail.com wrote:
Why do some CL library functions have :key arguments?
[...]
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
This not only conses "a bit more", it also duplicates traversal efforts
- The original list must be traversed, and the consed-up list of key
values as well. I think it is prudent that the CL library functions offer ways to reduce consing for cases where "a bit" is too much (and "a bit" can become a lot if a program operates on long lists).
I understand this. My main question is: why not do this with compiler macros? Is there any reason for this, other than historical?
Note that I am not complaining about the standard, I just want to learn the reason for this design choice so that I can take it into account when writing my own libraries.
Best,
Tamas
Tamas Papp tkpapp@gmail.com writes:
On Mon, 04 Jul 2011 11:39:39 +0200, Hans Hübner wrote:
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp tkpapp@gmail.com wrote:
Why do some CL library functions have :key arguments?
[...]
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
This not only conses "a bit more", it also duplicates traversal efforts
- The original list must be traversed, and the consed-up list of key
values as well. I think it is prudent that the CL library functions offer ways to reduce consing for cases where "a bit" is too much (and "a bit" can become a lot if a program operates on long lists).
I understand this. My main question is: why not do this with compiler macros? Is there any reason for this, other than historical?
Because it's not easy to do with compiler macros.
Note that I am not complaining about the standard, I just want to learn the reason for this design choice so that I can take it into account when writing my own libraries.
I find (foo sequence :key #'key) much nicer than (foo (map 'sequence #'key sequence))
On Mon, 04 Jul 2011 14:20:32 +0400, Stas Boukarev wrote:
Tamas Papp tkpapp@gmail.com writes:
On Mon, 04 Jul 2011 11:39:39 +0200, Hans Hübner wrote:
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp tkpapp@gmail.com wrote:
Why do some CL library functions have :key arguments?
[...]
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
This not only conses "a bit more", it also duplicates traversal efforts - The original list must be traversed, and the consed-up list of key values as well. I think it is prudent that the CL library functions offer ways to reduce consing for cases where "a bit" is too much (and "a bit" can become a lot if a program operates on long lists).
I understand this. My main question is: why not do this with compiler macros? Is there any reason for this, other than historical?
Because it's not easy to do with compiler macros.
Can you (or someone) please elaborate on that? I have just started reading up on compiler macros, and I don't understand why.
Thanks,
Tamas
Tamas Papp tkpapp@gmail.com writes:
On Mon, 04 Jul 2011 14:20:32 +0400, Stas Boukarev wrote:
Tamas Papp tkpapp@gmail.com writes:
On Mon, 04 Jul 2011 11:39:39 +0200, Hans Hübner wrote:
On Mon, Jul 4, 2011 at 11:31 AM, Tamas Papp tkpapp@gmail.com wrote:
Why do some CL library functions have :key arguments?
[...]
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
This not only conses "a bit more", it also duplicates traversal efforts - The original list must be traversed, and the consed-up list of key values as well. I think it is prudent that the CL library functions offer ways to reduce consing for cases where "a bit" is too much (and "a bit" can become a lot if a program operates on long lists).
I understand this. My main question is: why not do this with compiler macros? Is there any reason for this, other than historical?
Because it's not easy to do with compiler macros.
Can you (or someone) please elaborate on that? I have just started reading up on compiler macros, and I don't understand why.
Suppose you have a function SUM,
(defun sum (list) (loop for i in list sum i))
Now if you want to optimize (sum (map 'list KEY list)), you would need to write an additional function.
(defun sum-key (list key) (loop for i in list sum (funcall key i)))
And then you would need to write a compiler macro, which would match (map 'list KEY sequence) and transform it into (sum-key sequence KEY), which is not that hard, if that's only what you have. Now what if you want it to work with (mapcar KEY list) too. It becomes more and more cumbersome as it gets more general.
And now you need two functions, a clever compiler macro (perhaps using some pattern matching library), and which may not do what the user expects.
While you could have all that with just one function:
(defun sum (list &key key) (loop for i in list sum (if key (funcall key i) i))) Other disadvantages of compiler-macros:
* They are not guaranteed to be expanded. Some implementations may ignore them. * They can't be used with APPLY or FUNCALL. * You can't compose them e.g. if you wanted to write (defun two-sum (list1 list2) (+ (sum list1) (sum list2))) You would need to write a second compiler macro. While with KEY keyword you could just pass it along.
On 4 July 2011 14:46, Stas Boukarev stassats@gmail.com wrote:
- They can't be used with APPLY or FUNCALL.
Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
Compiler-macros are, however, of decent tool for optimizing common cases of :KEY and :TEST arguments.
Cheers,
-- nikodemus
Nikodemus Siivola nikodemus@random-state.net writes:
On 4 July 2011 14:46, Stas Boukarev stassats@gmail.com wrote:
- They can't be used with APPLY or FUNCALL.
Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
Compiler-macros are, however, of decent tool for optimizing common cases of :KEY and :TEST arguments.
Remember however, that you cannot write in a conforming program a compiler macro for a function named in the CL package.
clhs define-compiler-macro: "The consequences of writing a compiler macro definition for a function in the COMMON-LISP package are undefined;"
On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon pjb@informatimago.com wrote:
Nikodemus Siivola nikodemus@random-state.net writes:
On 4 July 2011 14:46, Stas Boukarev stassats@gmail.com wrote:
- They can't be used with APPLY or FUNCALL.
Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
Compiler-macros are, however, of decent tool for optimizing common cases of :KEY and :TEST arguments.
Remember however, that you cannot write in a conforming program a compiler macro for a function named in the CL package.
clhs define-compiler-macro: "The consequences of writing a compiler macro definition for a function in the COMMON-LISP package are undefined;"
You, the user, can't, but Nikodemus, the implementer, can! ;)
On 4 July 2011 16:11, Alessio Stalla alessiostalla@gmail.com wrote:
On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon pjb@informatimago.com wrote:
Nikodemus Siivola nikodemus@random-state.net writes:
On 4 July 2011 14:46, Stas Boukarev stassats@gmail.com wrote:
- They can't be used with APPLY or FUNCALL.
Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
clhs define-compiler-macro: "The consequences of writing a compiler macro definition for a function in the COMMON-LISP package are undefined;"
You, the user, can't, but Nikodemus, the implementer, can! ;)
There's a misunderstanding here. I didn't mean -- nor do I think Stas meant -- writing a compiler-macro for APPLY or FUNCALL, but rather having a compiler-macro take effect when the function in question is called using FUNCALL or APPLY:
(apply #'has-a-compiler-macro ...) ; compiler-macro will not fire
(funcall #'has-a-compiler-macro ...) ; compiler-macro should (if supported) fire
Cheers,
-- nikodemus
Nikodemus Siivola nikodemus@random-state.net writes:
On 4 July 2011 16:11, Alessio Stalla alessiostalla@gmail.com wrote:
On Mon, Jul 4, 2011 at 3:09 PM, Pascal J. Bourguignon pjb@informatimago.com wrote:
Nikodemus Siivola nikodemus@random-state.net writes:
On 4 July 2011 14:46, Stas Boukarev stassats@gmail.com wrote:
- They can't be used with APPLY or FUNCALL.
Actually, they can be used with FUNCALL.
(Otherwise, I echo pretty much everything that Stas said.)
clhs define-compiler-macro: "The consequences of writing a compiler macro definition for a function in the COMMON-LISP package are undefined;"
You, the user, can't, but Nikodemus, the implementer, can! ;)
There's a misunderstanding here. I didn't mean -- nor do I think Stas meant -- writing a compiler-macro for APPLY or FUNCALL, but rather having a compiler-macro take effect when the function in question is called using FUNCALL or APPLY:
(apply #'has-a-compiler-macro ...) ; compiler-macro will not fire
(funcall #'has-a-compiler-macro ...) ; compiler-macro should (if supported) fire
I don't think so.
First you're discussing here of a (function f), not (quote f), and COMPILER-MACRO-FUNCTION takes a function name, not a function or function designator. So even if the compiler did the optimization of avoiding the call to CL:FUNCALL, it could still have difficulty and scruples of building a source form from a function object.
But even if we considered:
(funcall (quote has-a-compiler-macro) args...)
the compiler just could not call the compiler macro of that function name, for this reason:
(defun f (args...) (do-something args...))
(define-compiler-macro f (&environment env &whole form args...) (if (special-f-case-p env form) (generate-special-f-form env form args...) form))
(flet ((f (args...) (do-something-else args...))) (funcall (quote f) args...))
The funcall doesn't refer the same function as if we had written:
(f args...)
and therefore we cannot pass that form to the compiler macro, because it doesn't correspond to the function of the compiler macro!
[The case of (function f) is even more telling in this case:
(flet ((f (args...) (do-something-else args...))) (funcall (function f) args...))
since in that case, the function denoted has no compiler macro!]
And even if the implementation supports compiler macros, there's no reason it should implement the optimization of avoiding the call to funcall.
Therefore:
1- compiler macros won't apply when the function is called by FUNCALL as well as when it's called by APPLY.
2- in my opinion, even in presence of a highly optimizing compiler, synthesizing source code for the benefit of macros or compiler macros would be a bad idea and rarely applicable without problems.
On 4 July 2011 19:42, Pascal J. Bourguignon pjb@informatimago.com wrote:
I don't think so.
I disagree strongly, and I'm pretty sure CLHS agrees with me, since it goes to the trouble of specifying what happens with FUNCALL.
CLHS, DEFINE-COMPILER-MACRO: "The &whole argument is bound to the form argument that is passed to the compiler macro function. The remaining lambda-list parameters are specified as if this form contained the function name in the car and the actual arguments in the cdr, but if the car of the actual form is the symbol funcall, then the destructuring of the arguments is actually performed using its cddr instead."
(I'm not really interested in fencing re. compiler-macros here, just trying to keep the record only moderately crooked. This is a sidetrack of epic proportions already: the OP asked about :KEY, not compiler-macros.)
Cheers,
-- nikodemus
On Mon, 04 Jul 2011 20:11:56 +0300, Nikodemus Siivola wrote:
On 4 July 2011 19:42, Pascal J. Bourguignon pjb@informatimago.com wrote:
I don't think so.
I disagree strongly, and I'm pretty sure CLHS agrees with me, since it goes to the trouble of specifying what happens with FUNCALL.
CLHS, DEFINE-COMPILER-MACRO: "The &whole argument is bound to the form argument that is passed to the compiler macro function. The remaining lambda-list parameters are specified as if this form contained the function name in the car and the actual arguments in the cdr, but if the car of the actual form is the symbol funcall, then the destructuring of the arguments is actually performed using its cddr instead."
(I'm not really interested in fencing re. compiler-macros here, just trying to keep the record only moderately crooked. This is a sidetrack of epic proportions already: the OP asked about :KEY, not compiler-macros.)
I am very happy to learn about these things. Currently I am working on the algorithms and my main concern is to ensure correctness; speed is secondary at this point, but even though I am not optimizing, I want to keep my code optimizable later on.
My problem with the key argument is that it complicates the interface. I would like to use the same interface for sample statistics and random variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have
(mean #(1d0 2d0 3d0)) ; => 2, a sample mean (mean (r-normal 2 1)) ; => 2d0, mean of a univariate normal distribution
If I had a :KEY argument, I would have to check that it is EQ to #'identity or not provided in methods for random variables.
APPLY is not a major concern for me at the moment, all of these functions have a fixed number of arguments (usually one or two). So compiler macros still look attractive: I guess I could just write them for the function I define (eg MAP1), with the understanding that if the user wants speed, he should stick to mapping with this function.
I also thought of the following possibility using runtime dispatch:
(defstruct (w/key (:constructor w/key (key object))) key object)
(defgeneric mean (object) (:method ((obj w/key)) (mean-w/key (w/key-object obj) (w/key-key obj))) (:method ((obj sequence)) (/ (reduce #'+ obj) (length obj))))
(defmethod mean-w/key ((obj sequence) key) (/ (reduce #'+ obj :key key) (length obj)))
(mean #(1 2 3)) ; => 2 (mean (w/key #'1+ #(1 2 3))) ; => 3
Tamas
On Jul 5, 2011, at 11:51 , Tamas Papp wrote:
I am very happy to learn about these things. Currently I am working on the algorithms and my main concern is to ensure correctness; speed is secondary at this point, but even though I am not optimizing, I want to keep my code optimizable later on.
My problem with the key argument is that it complicates the interface. I would like to use the same interface for sample statistics and random variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have
(mean #(1d0 2d0 3d0)) ; => 2, a sample mean (mean (r-normal 2 1)) ; => 2d0, mean of a univariate normal distribution
If I had a :KEY argument, I would have to check that it is EQ to #'identity or not provided in methods for random variables.
But this is exactly where compiler macros can help. With the &key argument you keep a consistent (and useful) interface. The check whether to do away with a possible IDENTITY can be done in an appropriate compiler-macro.
APPLY is not a major concern for me at the moment, all of these functions have a fixed number of arguments (usually one or two). So compiler macros still look attractive: I guess I could just write them for the function I define (eg MAP1), with the understanding that if the user wants speed, he should stick to mapping with this function.
I also thought of the following possibility using runtime dispatch:
(defstruct (w/key (:constructor w/key (key object))) key object)
(defgeneric mean (object) (:method ((obj w/key)) (mean-w/key (w/key-object obj) (w/key-key obj))) (:method ((obj sequence)) (/ (reduce #'+ obj) (length obj))))
(defmethod mean-w/key ((obj sequence) key) (/ (reduce #'+ obj :key key) (length obj)))
(mean #(1 2 3)) ; => 2 (mean (w/key #'1+ #(1 2 3))) ; => 3
You can have your cake and eat it too. Why limit yourself?
Cheers -- Marco
Tamas Papp tkpapp@gmail.com writes:
My problem with the key argument is that it complicates the interface. I would like to use the same interface for sample statistics and random variables, eg currently in CL-NUM-UTILS and CL-RANDOM I have
If that complicates the interface then don't use them!
&KEY are designed to simplify the interface. If that doesn't work in your case, don't use it.
(mean #(1d0 2d0 3d0)) ; => 2, a sample mean (mean (r-normal 2 1)) ; => 2d0, mean of a univariate normal distribution
If I had a :KEY argument, I would have to check that it is EQ to #'identity or not provided in methods for random variables.
Or you can just call the key in all cases.
(mean #(1 2 3)) ; => 2 (mean (w/key #'1+ #(1 2 3))) ; => 3
This will never be faster than keys.
Some compilers (such as sbcl), generate different entry points for functions with &key parameters, so that depending on the lexical keys specified at the call point, it can jump directly to the right parameter processing code.
Basically, the compiler behaves like if it generated automatically for:
(defun f (a &key k1 k2) ...)
the following code:
(defun f-0 (a) (let ((k1) (k2)) ...)) (defun f-k1 (a k1) (let ((k2)) ...)) (defun f-k2 (a k2) (let ((k1)) ...)) (defun f-k1-k2 (a k1 k2) ...) (defun f (a &rest r) (let ((k1 (get-key :k1 r)) (k2 (get-key k2 r))) ...))
;; only smarter, it avoids duplicating the ... code.
and when you call:
(f 42 :k1 11)
it generates:
(f1-k1 42 11)
So what you would do by hand could be done by the compiler.
Only if you called (apply 'f 42 keys) or (f 42 some-key 33) would it call the complex parameter processing entry point.
Nikodemus Siivola nikodemus@random-state.net writes:
On 4 July 2011 14:46, Stas Boukarev stassats@gmail.com wrote:
- They can't be used with APPLY or FUNCALL.
Actually, they can be used with FUNCALL.
I meant that it can be use with FUNCALL when it's called not on a known function.
On Mon, Jul 4, 2011 at 5:57 AM, Tamas Papp tkpapp@gmail.com wrote:
I understand this. My main question is: why not do this with compiler macros? Is there any reason for this, other than historical?
3.2.2.1.3 might offer some insight: "However, no language processor (compiler, evaluator, or other code walker) is ever required to actually invoke compiler macro functions, or to make use of the resulting expansion if it does invoke a compiler macro function."
-Nathan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 04.07.2011 11:31, schrieb Tamas Papp:
Why do some CL library functions have :key arguments?
I am asking because I am working on some statistics functions, and the design choice came up. Specifically, I can write functions like
(defun quantiles (sequence quantiles &key (key #'identity)) ...)
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
but this conses a bit more.
Doesn't quantiles just pass the key to sort? Is there some smarter algorithm I am not aware of right now?
Also, passing the key lets you return the complete objects (or whatever), not just the keys as in the map call. For example, compare
(quantiles #((a 1) (b 4) (c 2) (d 2) (e 5)) 2 :key #'second)
with
(quantiles (map 'vector #'second #((a 1) (b 4) (c 2) (d 2) (e 5))) 2)
In the first case, you could return (d 2), but not in the second.
Maybe I'm just missing the point. :)
Best wishes,
Svante
- -- Svante Carl v. Erichsen Wentorfer Str. 96 21029 Hamburg
+49-(0)40-34923721 +49-(0)160-6941474 Svante.v.Erichsen@web.de
On Mon, 04 Jul 2011 12:12:33 +0200, Svante Carl v. Erichsen wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 04.07.2011 11:31, schrieb Tamas Papp:
Why do some CL library functions have :key arguments?
I am asking because I am working on some statistics functions, and the design choice came up. Specifically, I can write functions like
(defun quantiles (sequence quantiles &key (key #'identity)) ...)
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
but this conses a bit more.
Doesn't quantiles just pass the key to sort? Is there some smarter algorithm I am not aware of right now?
Eg
@inproceedings{greenwald2001space, title={Space-efficient online computation of quantile summaries}, author={Greenwald, M. and Khanna, S.}, booktitle={ACM SIGMOD Record}, volume={30}, number={2}, pages={58--66}, year={2001}, organization={ACM} }
I am making quantiles a generic function: it should work on objects returned by the method above, and also on vectors. My problem was that
(quantiles quantile-summary #(0.25 0.5 0.75) :key something)
makes little sense, since the summary is already accumulated. So I will drop key for the moment, and will probably use compiler macros.
I also thought of a lazy solution that would just save the function and the object, and traverse once, eg
(quantiles (with-key function object) ...)
but I need to think about that.
Also, passing the key lets you return the complete objects (or whatever), not just the keys as in the map call. For example, compare
That's a good point, I didn't think of that.
Best,
Tamas
On Mon, 4 Jul 2011 09:31:31 +0000 (UTC), Tamas Papp said:
Why do some CL library functions have :key arguments?
I am asking because I am working on some statistics functions, and the design choice came up. Specifically, I can write functions like
(defun quantiles (sequence quantiles &key (key #'identity)) ...)
but it is a bit cumbersome. I can make my code simpler by relying on calls like
(quantiles (map 'vector key vector) quantiles)
That approach doesn't work with many of the CL functions (e.g. find, delete, substitute, remove-duplicates) because they operate on (or return) the elements of the sequence, not the keys. The :key argument just affects the arguments to the :test function.
Tamas Papp tkpapp@gmail.com writes:
I would appreciate advice on this. I am especially interested in the reason why some CL functions have :key arguments: is it because of efficiency, backward-compatibility/history, or something else?
The rationals for Common Lisp are generally explained in the first chapter of CLHS, and in the "Issues".
In short, the reason CL has &KEY is because most of the languages it purposed to generalize had &KEY. The reason why some functions have a :KEY argument is because they had it in (at least some of) the original languages.
http://www.lispworks.com/documentation/HyperSpec/Body/01_ab.htm http://www.lispworks.com/documentation/HyperSpec/Issues/iss017_w.htm http://www.lispworks.com/documentation/HyperSpec/Issues/iss109_w.htm
Now about :KEY, each function processing sequence has the opportunity of having to work from a "key" value of the element instead of directly on the element. The typical example is sort:
(sort seq (lambda (a b) (< (element-key a) (element-key b))))
But remember that there are a lot of such functions:
(merge 'list seq1 seq2 (lambda (a b) (< (element-key a) (element-key b))))
Now this lambda is not too much of a problem, we could even write HOFs to make them easily, like: (compose-2 '< 'element-key). But the problem here is that element-key will be called much more than it is necessary. Of course, the alternative is to wrap the sequence, into a keyed sequence that caches it:
(defun wrap (seq key) (map 'vector (lambda (item) (cons (funcall key item) item)) seq)) (defun wrapped-key (wrapped-item) (car wrapped-item)) (defun unwrap (original-seq wrapped-seq) (map-into original-seq (function cdr) wrapped-seq))
(unwrap seq (sort (wrap seq (function element-key)) (lambda (a b) (< (wrapped-key a) (wrapped-key b)))))
(unwrap list (sort (wrap list (function car)) (lambda (a b) (string< (wrapped-key a) (wrapped-key b)))))
(unwrap (make-list (+ (length seq1) (length seq2))) (merge 'vector (wrap seq1 (function element-key)) (wrap seq2 (function element-key)) (lambda (a b) (< (wrapped-key a) (wrapped-key b)))))
; and so on.
You may kind of notice some boilerplate coding there.
So what's the same, and what changes from one call to the other? What's the same is the boilerplate unwrap/wrap/lambda. What changes is the sequence, the lessp function and the key function.
Therefore we shall write:
(sort seq lessp key) (merge result-type seq1 seq2 lessp key)
and leave the boilerplate inside those functions. Some of them may even spare some further calls to the (possibly costly) key function, if they don't need to wrap the whole sequence to do their work.
Now, of course, 90% of the time, you will call:
(sort seq lessp (function identity)) (merge result-type seq1 seq2 lessp (function identity))
so it would be nice if that was optional. No problem, we have &OPTIONAL. But then we have all those functions such as POSITION, FIND, MEMBER, etc, who not only have an optional :KEY parameter, but also, for the same reason, an optional :TEST or :START :END or any other parameter, and it becomes difficult to use if you want the third optional but not the previous. Hence the invention of &key parameters.
Remember that &KEY covers &REST parameters. &KEY parameters are only &REST parameters structured in a specific way, structured like a p-list.
And since for functions such as POSITION, FIND, MEMBER, etc we have to use &KEY parameters for :KEY, we generalize and use &KEY parameters even for functions such as SORT and MERGE that only would have a :KEY parameter.
Obviously it was done so that it would be a no brainer, so that YOU (the language user) wouldn't have to think about it, but it failed lamentably. :-)