[Sorry; I accidentally sent the last email before finishing it (C-c C-c instead of C-c C-e).]
The problem:
Using the excellent cffi-wrappers on platforms that are not Allegro does not work well with delivery, because the path to the generated shared object (presumably near the source code) is hardcoded into the image, and the image will crash very early, before entering user code if this shared object is not available.
The solution:
Close all shared libraries before delivery. After delivery reopen them, searching *foreign-library-directories*.
Brief summary of the patch:
(1) The wrapper generation code is changed to use define-foreign-library and not to explicitly wedge the path into the library name, but to put the directory where it is generated into *foreign-library-directories*.
(2) cffi:close-all-foreign-libraries closes all open defined foreign libraries. Call this before delivery.
(3) (cffi:do-reload-all-compile-time-open-libraries) is a macro that will load all the required libraries again. Put this in your main function.
What I want:
Feedback and comments; maybe this patch touches too much? Apparently Stelian started doing something a little similar but half finished? If it looks viable for the mainline, we can add test cases and documentation.
Examples:
Here are a couple of example build procedures that add the application's binary directory to *foreign-library-directories*, so that the .so files can be put there. (Maybe it would be better to fully set *foreign-library-directories*.)
I have tested it on AMD64 Linux Lispworks and SBCL.
For example, Lispworks:
(compile (defun main () (setf *debugger-hook* 'mtsnmp::mtsnmp-debugger-hook) (pushnew (make-pathname :name nil :type nil :version nil :defaults (truename (first sys:*line-arguments-list*))) cffi:*foreign-library-directories* :test 'equalp) (handler-case (progn (cffi:do-reload-all-compile-time-open-libraries) (prog1 (apply 'my-application:main sys:*line-arguments-list*) (finish-output))) (condition (c) (ignore-errors (format *error-output* "Error: ~A~&" c) (finish-output *error-output*)) (lispworks:quit :status 1))) (lispworks:quit :status 0)))
(cffi:close-all-foreign-libraries)
(deliver 'main (multiple-value-bind (second minute hour date month year day daylight-p zone) (get-decoded-time) (format nil "binaries/~4,'0D~2,'0D~2,'0D-~2,'0D~2,'0D-~A-~A-~A" year month date hour minute (lisp-implementation-type) (lisp-implementation-version) (expt 2 (ceiling (log (log most-positive-fixnum 2) 2))))) 1 :multiprocessing t :keep-eval t :keep-conditions :all :keep-top-level nil :keep-debug-mode nil)
For example, SBCL
(in-package #:cl-user)
(defun main () (unwind-protect (handler-case (progn (pushnew (make-pathname :name nil :type nil :version nil :defaults (truename (first *posix-argv*))) cffi:*foreign-library-directories* :test 'equalp) (cffi:do-reload-all-compile-time-open-libraries) (apply 'my-application:main *posix-argv*) (sb-ext:quit :unix-status 0)) (condition (c) (ignore-errors (format *error-output* "Error: ~A~&" c)) (sb-ext:quit :unix-status 1))) (sb-ext:quit :unix-status 100)))
(defun make-image () (sb-ext:disable-debugger)
(cffi:close-all-foreign-libraries) (sb-ext:save-lisp-and-die (multiple-value-bind (second minute hour date month year) (get-decoded-time) (declare (ignore second)) (format nil "binaries/~4,'0D~2,'0D~2,'0D-~2,'0D~2,'0D-~A-~A-~A" year month date hour minute (lisp-implementation-type) (lisp-implementation-version) (expt 2 (ceiling (log (log most-positive-fixnum 2) 2))))) :executable t :toplevel 'main :purify t))
(make-image)