Functions COMPILED-FILE-P and ABI-VERSION

Author

Sam Steingold

Related

ANSI Common Lisp standard function compile-file.

Abstract

A facility to determine whether a file is a valid compiled file for the specific implementation and to identify the current compiled file format.

Rationale

Build tools, like defsystem or asdf, have to determine whether a file needs to be recompiled.

Obviously, when the compiled file is older than the source file, recompilation is in order.

Alas, there are other situations when this might be necessary, e.g., when the implementation changes the compiled file format or when two implementations use the same name for their compiled files (.fasl is used by both SBCL and ACL).

Traditionally, system definition facilities have taken the route of creating a separate directory for each combination of implementation type, version, operating system, and architecture. This is wasteful because the the compiled file format does not necessarily change between versions and does not even have to depend on OS and architecture.

The proposed functions will simplify the build directory tree structure and reduce the number of binary distribution bundles.

Current Practice

Implementation-dependent.

Cost of adoption

For COMPILED-FILE-P, probably tiny: an implementation must be able to check for compiled file validity, so all it takes is to export the necessary functionality, e.g.:

#+clisp
(defun compiled-file-p (file-name)
  (with-open-file (in file-name :direction :input :if-does-not-exist nil)
    (and in (char= #\( (peek-char nil in))
         (let ((form (ignore-errors (read in nil nil))))
           (and (consp form)
                (eq (car form) 'SYSTEM::VERSION)
                (null (nth-value 1 (ignore-errors (eval form)))))))))

For ABI-VERSION, it probably depends on the implementation; for some it might be trivial:

#+clisp
(defun abi-version ()
  (car (system::version)))

and for others it might not.

Cost of non-adoption

Users will suffer random errors when trying to load invalid binary files.

Specification

Function COMPILED-FILE-P

Function

(compiled-file-p file-name) ==> valid-p

Returns

true

if the file appears to be a valid compiled file (i.e., exists, is readable, and the contents appears to be valid for this implementation),

false

otherwise.

Implementations are required to inspect the contents (e.g., checking just the pathname type is not sufficient). Although the completeness of the inspection is not required, this function should be able to detect, e.g., file format changes between versions.

Exceptional situations

• Signals an error of type type-error when the argument is not a pathname designator.

Examples

(compiled-file-p "foo.lisp") ==> NIL
(compiled-file-p (compile-file "foo.lisp")) ==> T

Function ABI-VERSION

Function

(abi-version &optional object) ==> object

When called without arguments, returns an implementation-defined object which uniquely identifies the compiled file format produced by the implementation.

The return value must satisfy two conditions:

1. (prin1-to-string value) must be a valid logical pathname component
2. (equalp value (read-from-string (prin1-to-string value))) must be true

which guarantee that it can be used to name directories where the compiled files are stored.

When called with an argument, returns a generalized boolean:

true

if the argument identifies an ABI version which can be consumed by the implementation,

false

otherwise.

Exceptional situations

• May signal an error of type type-error when the argument is not a valid ABI version name for this implementation.

Examples

(abi-version (abi-version)) ==> T

Reference Implementation

See above.

History

The compiled-file-p spec was accepted as CLRFI-2 (in 2004).

Notes

The trivial implementation:

(defun compiled-file-p (file-name)
  (not (nth-value 1 (ignore-errors (load file-name)))))

is wrong because,

1. load may fail even though the file is valid: even when foo.lisp contains calls to error,

   (compiled-file-p (compile-file "foo.lisp"))
   

   should still return T.
2. this is not side-effect-free, i.e., this may define new functions and macros (or, worse yet, redefine some existing functions and macros or execute some malicious code).

If we could require (abi-version file) to return either the abi-version of the implementation which produced the compiled file, or nil if the file is not a compiled file for this implementation, then we could define

(defun compiled-file-p (file) (equalp (abi-version) (abi-version file)))

however, it is not obvious that all implementation can actually do this without unwelcome invasive changes.