(This is a new topic.)
Many times, people have presented a major problem with object-oriented programming as it is often/idiomatically used. The good thing is that OOP, when done right (in Lisp, that means, among other things, always using defgeneric explicitly), provides a great abstarct interface to the caller. It's good for modularity, because if you want to changes the implementation of the class, you know what you can change easily, and what you cannot because callers depend on it.
If all of the callers are under your control, you can easily make incompatible changes, but if you're providing a general-purpose library, that people all over the world are using, then it's a lot harder.
I call the set of defgenerics (plus the factory functions) the "protocol". The word "type" is sort of right but carries a lot of connotations and freight that I'd rather avoid.
Often all of the CLOS classes that implement a given protocol inherit from a common abstract base class. There are two reasons to do this. First, a common base class can provide implementations of some of the generic functions all by itself. My favorite simple example is an "output stream" protocol, that has a write-character operation and a write-string operation. The common base class provides an implementation of write-string that works by iterating over the characters of the string and calling write-character. Any output stream that can write strings in a more efficient way can override that method.
The second reason is to define a Lisp type that all of the subclasses will be of.
In my mind, as far as the first reason goes, there should be NO requirement that every type that implements the protocol *must* inherit from a common abstract base class. The abstract base class is merely an implementation convenience. In the case of the fhash library, I do not have any common base class. As a matter of fact, it's not even implemented using CLOS at all. (The caller doesn't know, and should not.) From the source file:
;;; An fhash is represented as a one-dimensional array. It can have two ;;; representations: a linear search table, or a hash table (from the ;;; underlying Common Lisp implementation). An fhash is represented as ;;; a vector, starting with a header. The 0th element says whether this ;;; fhash instance is :linear or :hash. If it's :hash, the 1th element ;;; contains the hash table, and the rest of the elements are ignored.
As far as the second reason goes, fortunately Lisp has a powerful way to let you define types. In fhash:
(defun fhash-table-p (fhash)
"Return true if the argument is an FHASH. Otherwise return NIL. This is not perfect, since some array might just happen to have these values, but it seems unlikely."
(and (vectorp fhash) (member (aref fhash +fhash-kind+) '(:linear :hash))))
(deftype fhash-table () '(satisfies fhash-table-p))
This is not perfect. A programmer might just happen to create a vector and put one of those keywords into the +fhash-kind+ slot. (+fhash-kind+ is zero but that's a mere detail.) But it's sort of good enough.
OK, back to OOP. Abstraction of the object from the caller is just fine. But, the commonly-heard objection to OOP as it is really used is that there is *not* a clean modular definition isolating *subclasses*.
Smalltalk didn't even try. CLOS, I believe, does not try and there is not an idiomatic way to do it. The only language I know that makes a good stab in this direction is C++, which has "public", "protected", and "private". Whatever else you say about C++, Bjarne understood this issue and tried to fix it. The C++ solution isn't perfect but it's sure a lot better than anything else.
How can we provide the same thing in Common Lisp? And without changing CLOS? In fact, without adding any new language feature at all?
With packages, fhash has a defpackage that provides the protocol seen by callers. If you do an abstract type with CLOS, you ought to do the same thing. (You might have one package for many types.) So the callers can "use" fhash and then call putfhash and so on.
(Digression: A slightly different approach is to define a package not intended to be "use"'ed. I did this with the API for the logging facility in QRes. For example: log:add-rule. If you did a "use", it might be confusing to remember that add-rule is from logging, and indeed you might "use" another package that has an "add-rule" function. This is a name conflict. The *whole idea* of packages is to avoid name conflicts! So I think "use" is opposed to the whole idea of packages and should be used sparingly.
What people usually do in Common Lisp, in my experience, is to name their functions so that the ones in one module (or, more generally, related ones) all have names starting with a common prefix. Well, that's what packages were invented: so that you don't have to do that! In QRes, even though there is a "qconfig" package for the configuration stuff, the functions are still all named "config-, and everybody uses "use". I think this is not as good as using packages as they were designed. Rather than, e.g., config-descriptor-property, use config:descriptor-property. Anyway.)
Now, there's a great thing about CL packages: you can have more than one defpackage for the *same* symbols. So we could have one package for the code that calls into our library, which would have the usual protocol, which we can call the "external protocol". Then we could have a *second package*, which presents a second protocol that subclases would use! This gives us modular separation, just as "protected" does.
Of course, there's nothing stopping any code at all from including a symbol from this second package in its code. But we already let people go into internal symbols using "::". Common Lisp's general attitude is that we don't make these things impossible; you're just not supposed to do them. It's very often useful to do this kind of violation when doing interactive debugging, or when writing white-box unit tests. (It would be great to have a static analysis tool that flag you when you "cheat", that you could run before releasing a new version of your stuff.)
I have not actually written anything this way, but it seems to me as if it would work. And if experience shows that it does provide what I claim it will provide, I would love for that to become idiomatic/standard usage in Common Lisp.
I admit that there's a significant problem, namely the same problem that we always have with packages. Because resolution of packages happens at runtime, it is difficult-to-impossible, in some cases, to change the package declarations during interactive debugging and have the effect that the environment behaves as if you had changed the package declaration and recompiled from scratch. All I can say is that we really ought to do something about the problems with packages, even if you don't buy anything I'm saying here.
-- Dan