On Sun, Jun 14, 2009 at 9:33 AM, Elliott Slaughter < elliottslaughter@gmail.com> wrote:

On Sun, Jun 14, 2009 at 10:29 AM, Luís Oliveira <luismbo@gmail.com> wrote:

...

On Fri, Jun 12, 2009 at 9:24 PM, Elliott Slaughter<elliottslaughter@gmail.com> wrote:

...

Of course, with :dont-save t, the user then has to call use-foreign-library again when starting up from the saved image, but I find this preferable to the crash I receive otherwise.

IIRC, you can close the foreign libraries before saving an image and reload them manually afterwards. Would that be an acceptable work-around?

Let me see if it works; I'll get back to you after I've tried it.

I'm believe I'm seeing this issue again, but none of my previous workarounds seem to be working. I am using SBCL 1.0.29 on Windows XP, and when I save and run an executable, I keep getting undefined alien errors. I have tried both the :dont-save t hack that I initially suggested, and have tried calling (close-foreign-library ...) on all of the open libraries before saving the executable, and nothing helps. Can you suggest a course of action? Thanks. Here is the backtrace from when I try to run the executable: This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" debugger invoked on a UNDEFINED-ALIEN-ERROR: Undefined alien: "SDL_Linked_Version" Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. (no restarts: If you didn't do this on purpose, please report it as a bug.) (SB-SYS:ENSURE-DYNAMIC-FOREIGN-SYMBOL-ADDRESS "SDL_Linked_Version" #<unused argument>) 0] WARNING: Starting a select without a timeout while interrupts are disabled. ba 0: (SB-SYS:ENSURE-DYNAMIC-FOREIGN-SYMBOL-ADDRESS "SDL_Linked_Version" #<unused argument>) 1: ((FLET SB-THREAD::WITH-RECURSIVE-SYSTEM-SPINLOCK-THUNK)) 2: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-RECURSIVE-SYSTEM-SPINLOCK]196)) 3: (SB-THREAD::CALL-WITH-RECURSIVE-SYSTEM-SPINLOCK #<CLOSURE (FLET SB-THREAD::WITH-RECURSIVE-SYSTEM-SPINLOCK-THUNK) {22FCDD}> #<unavailable argument>) 4: (SB-SYS:UPDATE-LINKAGE-TABLE) 5: (SB-IMPL::FOREIGN-REINIT) 6: (SB-IMPL::REINIT) 7: ((LABELS SB-IMPL::RESTART-LISP)) 8: ("foreign function: #x4120C4") 9: ("foreign function: #x40AF88") -- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay

On Mon, Dec 21, 2009 at 1:26 PM, Luís Oliveira <luismbo@gmail.com> wrote:

...

I'm believe I'm seeing this issue again, but none of my previous workarounds seem to be working. I am using SBCL 1.0.29 on Windows XP, and when I save and run an executable, I keep getting undefined alien errors. I have

On Mon, Dec 21, 2009 at 8:55 PM, Elliott Slaughter <elliottslaughter@gmail.com> wrote: tried

...

both the :dont-save t hack that I initially suggested, and have tried calling (close-foreign-library ...) on all of the open libraries before saving the executable, and nothing helps.

Can you come up with a minimal test case? Something using e.g. libtest.dll would be nice.

Sure. For this test, I used SDL.dll from http://www.libsdl.org/release/SDL-1.2.14-win32.zip . $ cat test.lisp (asdf:oos 'asdf:load-op :cffi) (cffi:define-foreign-library sdl (:windows "SDL.dll")) (cffi:use-foreign-library sdl) (cffi:defcfun ("SDL_Linked_Version" SDL-Linked-Version) :pointer) (defun main () (format t "does it work?~%") (quit)) (cffi:close-foreign-library 'sdl) (save-lisp-and-die #+windows "main.exe" #-windows "main" :toplevel #'main :executable t) $ sbcl --load test.lisp This is SBCL 1.0.29, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" ; loading system definition from C:\Bin\asdf\cffi\cffi.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM CFFI {23FFE3B9}> as CFFI ; loading system definition from C:\Bin\asdf\babel\babel.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM BABEL {242FC621}> as BABEL ; loading system definition from C:\Bin\asdf\alexandria\alexandria.asd into ; #<PACKAGE "ASDF0"> ; registering #<SYSTEM :ALEXANDRIA {2449AB61}> as ALEXANDRIA ; loading system definition from ; C:\Bin\asdf\trivial-features\trivial-features.asd into #<PACKAGE "ASDF0"> ; registering #<SYSTEM TRIVIAL-FEATURES {246A9BF9}> as TRIVIAL-FEATURES WARNING: Saving cores with alien definitions referring to non-static foreign symbols is unsupported on this platform: references to such foreign symbols from the restarted core will not work. You may be able to work around this limitation by reloading all foreign definitions and code using them in the restarted core, but no guarantees. Dynamic foreign symbols in this core: SDL_Linked_Version [undoing binding stack and other enclosing state... done] [saving current Lisp image into main.exe: writing 2888 bytes from the read-only space at 0x22000000 writing 1736 bytes from the static space at 0x22100000 writing 27688960 bytes from the dynamic space at 0x22300000 done] $ ./main.exe This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" debugger invoked on a UNDEFINED-ALIEN-ERROR: Undefined alien: "SDL_Linked_Version" Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. (no restarts: If you didn't do this on purpose, please report it as a bug.) (SB-SYS:ENSURE-DYNAMIC-FOREIGN-SYMBOL-ADDRESS "SDL_Linked_Version" #<unused argument>) 0] WARNING: Starting a select without a timeout while interrupts are disabled. ba 0: (SB-SYS:ENSURE-DYNAMIC-FOREIGN-SYMBOL-ADDRESS "SDL_Linked_Version" #<unused argument>) 1: ((FLET SB-THREAD::WITH-RECURSIVE-SYSTEM-SPINLOCK-THUNK)) 2: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-RECURSIVE-SYSTEM-SPINLOCK]196)) 3: (SB-THREAD::CALL-WITH-RECURSIVE-SYSTEM-SPINLOCK #<CLOSURE (FLET SB-THREAD::WITH-RECURSIVE-SYSTEM-SPINLOCK-THUNK) {22FCDD}> #<unavailable argument>) 4: (SB-SYS:UPDATE-LINKAGE-TABLE) 5: (SB-IMPL::FOREIGN-REINIT) 6: (SB-IMPL::REINIT) 7: ((LABELS SB-IMPL::RESTART-LISP)) 8: ("foreign function: #x4120C4") 9: ("foreign function: #x40AF88") 0] WARNING: Starting a select without a timeout while interrupts are disabled. (quit) -- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay

On Tue, Dec 22, 2009 at 6:56 AM, Elliott Slaughter <elliottslaughter@gmail.com> wrote:

After what feels like beating my head against a wall for several hours, and getting contradictory results from various attempts to boils the failure down to a manageable size, I finally tried my original build recipe again, and it just worked. I'm not really sure what to say, other than that I'm really confused. I don't think the workarounds suggested by myself and others are necessary, so I think we can all forget about this, at least until this comes to haunt me again.

This is something that definitely needs to be improved in CFFI, at least documentation-wise. Care to share your build recipe for dealing with foreign libraries? -- Luís Oliveira http://r42.eu/~luis/

On Mon, Dec 21, 2009 at 1:26 PM, Luís Oliveira <luismbo@gmail.com> wrote:

...

I'm believe I'm seeing this issue again, but none of my previous workarounds seem to be working. I am using SBCL 1.0.29 on Windows XP, and when I save and run an executable, I keep getting undefined alien errors. I have

On Mon, Dec 21, 2009 at 8:55 PM, Elliott Slaughter <elliottslaughter@gmail.com> wrote: tried

...

both the :dont-save t hack that I initially suggested, and have tried calling (close-foreign-library ...) on all of the open libraries before saving the executable, and nothing helps.

Can you come up with a minimal test case? Something using e.g. libtest.dll would be nice.

I think I managed to reproduce this error, and this time, I have a minimal test case! When I put a dll on *foreign-library-directories* and build a Windows executable with SBCL, the generated executable cannot run unless the dll is present at exactly the same absolute path as it was during compile time. If I instead move the dll into the current working directory, then I don't have any problem with absolute paths being saved in my binary. First, the contents of break.lisp: (asdf:oos 'asdf:load-op :cffi) (use-package :cffi) (pushnew (truename #p"lib/") *foreign-library-directories* :test #'equal) (define-foreign-library sdl (:windows "SDL.dll")) (defun main () (use-foreign-library sdl) (format t "It works~%")) (main) (save-lisp-and-die "main.exe" :toplevel #'main :executable t) (quit) Second, I create two binaries. I demonstrate that the first executable is broken by moving the original lib directory after compiling, while the second executable works fine. $ ls * break.lisp bin: lib: SDL.dll $ sbcl --load break.lisp This is SBCL 1.0.29, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. [ ... ] It works [undoing binding stack and other enclosing state... done] [saving current Lisp image into main.exe: writing 2888 bytes from the read-only space at 0x22000000 writing 1736 bytes from the static space at 0x22100000 writing 27582464 bytes from the dynamic space at 0x22300000 done] $ mv main.exe bin/ $ cp lib/SDL.dll bin/ $ cd bin/ $ ./main.exe This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" It works $ mv ../lib/ ../lib2 $ ./main.exe This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" debugger invoked on a SIMPLE-ERROR: Error opening shared object "c:\\Users\\Elliott\\Programming\\CommonLisp\\brea k-cffi\\lib\\SDL.dll": 126. Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [CONTINUE ] Skip this shared object and continue. 1: [RETRY ] Retry loading this shared object. 2: [CHANGE-PATHNAME] Specify a different pathname to load the shared object fr om. (SB-SYS:DLOPEN-OR-LOSE #S(SB-ALIEN::SHARED-OBJECT :PATHNAME #P"c:\\Users\\Elliott\\Programming\\CommonLisp\\break-cffi\\lib\\S DL.dll" :NAMESTRING "c:\\Users\\Elliott\\Programming\\CommonLisp\\break-cffi\\lib\\S DL.dll" :HANDLE NIL :DONT-SAVE NIL)) 0] (quit) $ cd .. $ mv lib2/ lib $ cp lib/SDL.dll . $ rm bin/main.exe $ ls * SDL.dll break.lisp bin: SDL.dll lib: SDL.dll $ sbcl --load break.lisp [ ... ] It works [undoing binding stack and other enclosing state... done] [saving current Lisp image into main.exe: writing 2888 bytes from the read-only space at 0x22000000 writing 1736 bytes from the static space at 0x22100000 writing 27578368 bytes from the dynamic space at 0x22300000 done] $ mv main.exe bin/ $ cd bin/ $ ./main.exe This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" It works $ mv ../lib/ ../lib2 $ ./main.exe This is experimental prerelease support for the Windows platform: use at your own risk. "Your Kitten of Death awaits!" It works -- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay