On 4 Aug 2020, at 12:02, Ilya Perminov wrote:

I think protobuf and CFFI structure their operations in a very similar way
- process-op is analogous to proto-to-lisp, it takes a "specification" file
and generates a lisp file and some other files. protobuf generates
lisp(fasl) files only, so it does not need to do anything special to
support bundle operations. CFFI's process-op generates some .o and .so
files that a bundle operation may need. The current implementation adds .o
and .so files to outputs of compile-op and it causes the problem I
described.
I do not know what methods need to be defined on process-op to make bundle
operations to pick up its output files. From my very limited understanding
of ASDF I do not think there is a way to do it. Method
"component-depends-on ((o gather-operation) (s system))" determines
input-files of a bundle-op. The method returns dependencies of one
operation only (e.g. compile-op), but in case of CFFI's wrapper-file we
need output files of two operations: process-op and compile-op.

I don't claim to understand this process, but wouldn't it be possible for you to make your own input-files :around method for gather-operation that would collect the outputs from the process-op's and add them to what you want?

Here's the existing definition of what I think is the relevant input-files method:

  (defmethod input-files ((o gather-operation) (c system))
    (unless (eq (bundle-type o) :no-output-file)
      (direct-dependency-files
       o c :key 'output-files
           :test (pathname-type-equal-function (bundle-pathname-type (gather-type o))))))

This invokes map-direct-dependencies which invokes the component-depends-on method for gather-operation on the system which ... I don't really understand, but I believe it's the compile-op's.

I think it would be easiest to write your own method that collects up all of the process-op outputs, drops any .lisp files (which will be superseded by the .fasl files), and adds them to the return value of call-next-method.

If you do that, and drop the .o and .so files from the output-files of compile-op, I think that would get what you want: you would collect the .o and .so files from the process-op, and you wouldn't get ASDF trying to regenerate the files when it's not necessary.

That said, I can think of a simpler, and easier method, and that would be to override the operation-done-p method for the compile-op so that it knows that the .o and .so files are generated by the compile-op, and only pays attention to the relationship between bindings-file.lisp and bindings-file.fasl.

One thing you didn't say was what it means that the compile-op is done over and over -- is it only compiling the bindings-file, or is it doing something to the object files? I don't believe it should change the .o and .so files, because those are already compiled by process-op, right?