So your only chance is to define an operation-class metaclass, and make sure that operation is an instance of operation-class (or similar).
This is quite a crippling limitation, then. Adding a metaclass would add another quite substantial amount of compatibility code to ASDF to permit us to use the MOP (even in a relatively limited way) on all of the supported implementations.
Can you think of any other way we might detect the creation of subclasses of OPERATION?
It would be nice if ASDF could do without pulling in closer-mop or substantial parts thereof.
Adding a metaclass might also in and of itself be an incompatibility, which might require all clients to be updated to use the metaclass, in addition to requiring a punt during hot upgrade (i.e. dropping existing state on the ground). I'm not sure it is worth the cost at this point.
As suggested previously, a :before or :after method on make-instance for operation objects or such should be enough for this particular case, although this defers the warning until the operation is used, rather than when it is defined.
The big problem, however, in this case, is still how do you distinguish between a "good" class that shouldn't trigger a warning, and a "bad" class that should.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org May you live all the days of your life. — Jonathan Swift