Greetings! I'm not really sure if this is a CFFI or a CLISP issue, so I'm posting to both ml's.
I'm trying to make Windows executables images, using Clisp 2.41 (built with readline, iconv and sigsegv). My images have cl projects loaded, like clsql or celltk. These projects relly on cffi to load functions from certain dll's (like odbc32.dll). E.g., I load clsql and save an image.
There are two problems here: 1) Unlike AllegroCL or Lispworks, Clisp (and SBCL too, btw) requires me to precede all defcfuns related to a library with:
#+clisp (cffi::use-foreign-library Tcl) (defcfun ("Tcl_FindExecutable" tcl-find-executable) :void (argv0 :string))
If I don't do this, I can't apply this function at run-time (I'm not even talking about saved images, yet).
2) When I load the saved image (using the code from above), I get an error when applying defcfun'ed functions, telling me the library wasn't found: ** - Continuable Error FFI::FOREIGN-CALL-OUT: no dynamic object named "SQLAllocHandle" in library :DEFAULT But then, if I tell clisp where the libraries are, with (cffi:define-foreign-library odbc (t (:default "odbc32"))) (cffi:use-foreign-library odbc)
, I get this: *** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> comes from a previous Lisp session and is invalid
The only workaround I can think of is to make an init-function on my image that loads all these defcfuns (at run-time), but this is not very sane, if you are to use many external CL libraries!
Thanks for the help,
Olá,
On 12/02/07, Edgar Gonçalves edgar.goncalves@gmail.com wrote:
- Unlike AllegroCL or Lispworks, Clisp (and SBCL too, btw) requires me to
precede all defcfuns related to a library with:
#+clisp (cffi::use-foreign-library Tcl) (defcfun ("Tcl_FindExecutable" tcl-find-executable) :void (argv0 :string))
That doesn't sound right. You should only need to call (use-foreign-library tcl) only once, some time before you load your bindings.
- When I load the saved image (using the code from above), I get an error
when applying defcfun'ed functions, telling me the library wasn't found:
Just tested with GNU CLISP 2.41 (2006-10-13) on OSX. Loaded cffi-test, saved an image (tried with ":executable t" too), restarted with the image, and ran (rt:do-tests) and it works. (Except some funcall.* tests fail at first, for some reason. I blame it on some RT weirdness.)
AFAICT, CLISP correctly reloads the libraries at startup. Can you try this with cffi-test or otherwise provide some sort of test case?
Viva,
On 2/12/07, Luís Oliveira luismbo@gmail.com wrote:
On 12/02/07, Edgar Gonçalves edgar.goncalves@gmail.com wrote:
- Unlike AllegroCL or Lispworks, Clisp (and SBCL too, btw) requires me
to
precede all defcfuns related to a library with:
#+clisp (cffi::use-foreign-library Tcl) (defcfun ("Tcl_FindExecutable" tcl-find-executable) :void (argv0 :string))
That doesn't sound right. You should only need to call (use-foreign-library tcl) only once, some time before you load your bindings.
Let me rephrase it a bit, to avoid misunderstandings. Yes, I call (use-foreign-library tcl) only once. The point is that I *need* to call it, whereas with ACL, for instance, I don't - the defcfuns just work for themselves.
- When I load the saved image (using the code from above), I get an error
when applying defcfun'ed functions, telling me the library wasn't found:
Just tested with GNU CLISP 2.41 (2006-10-13) on OSX. Loaded cffi-test, saved an image (tried with ":executable t" too), restarted with the image, and ran (rt:do-tests) and it works. (Except some funcall.* tests fail at first, for some reason. I blame it on some RT weirdness.)
AFAICT, CLISP correctly reloads the libraries at startup. Can you try this with cffi-test or otherwise provide some sort of test case?
Actually, I haven't tried tests, but I can't seem to get past the library bindings.lisp. It seems to be a problem with the dll (I'm running Windows 2003), given that my cl.exe (version 14.00.50727.42 for 80x86 ) won't recognize the <stdint.h>, and that my cygwin (latest, just checked for updates) makes a dll that hangs bindings.fas while loading.
As for a short test case, here's the smallest I can make that fires up my error. After loading clisp, and loading asdf, I eval:
,----- | (asdf:operate 'asdf:load-op :cffi) | | (cffi:define-foreign-library odbc | (:windows (:or "odbc32.dll")) | (t (:default "odbc32.dll"))) | | (cffi:defcfun ("SQLAllocHandle" sql-handle) :short) | | (sql-handle) `-----
The last form brings me to the following error: WARNING: FFI::FOREIGN-LIBRARY-FUNCTION: no dynamic object named "SQLAllocHandle" in library :DEFAULT *** - FUNCALL: undefined function NIL
with the relevant backtrace being:
<11> #<SYSTEM-FUNCTION ERROR> <12> #<COMPILED-FUNCTION SYSTEM::CHECK-VALUE> <13> #<SYSTEM-FUNCTION FUNCALL> 1 EVAL frame for form (VALUES (FUNCALL (LOAD-TIME-VALUE (MULTIPLE-VALUE-BIND (CFFI-SYS::FF ERROR) (IGNORE-ERRORS (FFI::FOREIGN-LIBRARY-FUNCTION "SQLAllocHandle" (FFI::FOREIGN-LIBRARY :DEFAULT) NIL NIL (FFI:PARSE-C-TYPE '(FFI:C-FUNCTION (:ARGUMENTS) (:RETURN-TYPE FFI:SHORT) (:LANGUAGE :STDC))))) (OR CFFI-SYS::FF (WARN (FORMAT NIL "~?" (SIMPLE-CONDITION-FORMAT-CONTROL ERROR) (SIMPLE-CONDITION-FORMAT-ARGUMENTS ERROR)))))))) APPLY frame for call (SQL-HANDLE) <14> #<FUNCTION SQL-HANDLE NIL (DECLARE (SYSTEM::IN-DEFUN SQL-HANDLE)) (BLOCK SQL-HANDLE (VALUES (CFFI-SYS:%FOREIGN-FUNCALL "SQLAllocHandle" :SHORT)))> 0 EVAL frame for form (SQL-HANDLE)
Hope this helps tracing out something, tell me if you need anything more.
Thanks, Edgar Gonçalves
Viva,
On 2/12/07, Luís Oliveira luismbo@gmail.com wrote:
On 12/02/07, Edgar Gonçalves edgar.goncalves@gmail.com wrote:
- Unlike AllegroCL or Lispworks, Clisp (and SBCL too, btw) requires me to
precede all defcfuns related to a library with:
#+clisp (cffi::use-foreign-library Tcl) (defcfun ("Tcl_FindExecutable" tcl-find-executable) :void (argv0 :string))
That doesn't sound right. You should only need to call (use-foreign-library tcl) only once, some time before you load your bindings.
Let me rephrase it a bit, to avoid misunderstandings. Yes, I call (use-foreign-library tcl) only once. The point is that I *need* to call it, whereas with ACL, for instance, I don't - the defcfuns just work for themselves.
- When I load the saved image (using the code from above), I get an error
when applying defcfun'ed functions, telling me the library wasn't found:
Just tested with GNU CLISP 2.41 (2006-10-13) on OSX. Loaded cffi-test, saved an image (tried with ":executable t" too), restarted with the image, and ran (rt:do-tests) and it works. (Except some funcall.* tests fail at first, for some reason. I blame it on some RT weirdness.)
AFAICT, CLISP correctly reloads the libraries at startup. Can you try this with cffi-test or otherwise provide some sort of test case?
Actually, I haven't tried tests, but I can't seem to get past the library bindings.lisp. It seems to be a problem with the dll (I'm running Windows 2003), given that my cl.exe (version 14.00.50727.42 for 80x86 ) won't recognize the <stdint.h>, and that my cygwin (latest, just checked for updates) makes a dll that hangs bindings.fas while loading.
As for a short test case, here's the smallest I can make that fires up my error. After loading clisp, and loading asdf, I eval:
,----- | (asdf:operate 'asdf:load-op :cffi) | | (cffi:define-foreign-library odbc | (:windows (:or "odbc32.dll")) | (t (:default "odbc32.dll"))) | | (cffi:defcfun ("SQLAllocHandle" sql-handle) :short) | | (sql-handle) `-----
The last form brings me to the following error: WARNING: FFI::FOREIGN-LIBRARY-FUNCTION: no dynamic object named "SQLAllocHandle" in library :DEFAULT *** - FUNCALL: undefined function NIL
with the relevant backtrace being:
<11> #<SYSTEM-FUNCTION ERROR> <12> #<COMPILED-FUNCTION SYSTEM::CHECK-VALUE> <13> #<SYSTEM-FUNCTION FUNCALL> 1 EVAL frame for form (VALUES (FUNCALL (LOAD-TIME-VALUE (MULTIPLE-VALUE-BIND (CFFI-SYS::FF ERROR) (IGNORE-ERRORS (FFI::FOREIGN-LIBRARY-FUNCTION "SQLAllocHandle" (FFI::FOREIGN-LIBRARY :DEFAULT) NIL NIL (FFI:PARSE-C-TYPE '(FFI:C-FUNCTION (:ARGUMENTS) (:RETURN-TYPE FFI:SHORT) (:LANGUAGE :STDC))))) (OR CFFI-SYS::FF (WARN (FORMAT NIL "~?" (SIMPLE-CONDITION-FORMAT-CONTROL ERROR) (SIMPLE-CONDITION-FORMAT-ARGUMENTS ERROR)))))))) APPLY frame for call (SQL-HANDLE) <14> #<FUNCTION SQL-HANDLE NIL (DECLARE (SYSTEM::IN-DEFUN SQL-HANDLE)) (BLOCK SQL-HANDLE (VALUES (CFFI-SYS:%FOREIGN-FUNCALL "SQLAllocHandle" :SHORT)))> 0 EVAL frame for form (SQL-HANDLE)
Hope this helps tracing out something, tell me if you need anything more.
Thanks, Edgar Gonçalves
On 13/02/07, Edgar Gonçalves edgar.goncalves@gmail.com wrote:
,----- | (asdf:operate 'asdf:load-op :cffi) | | (cffi:define-foreign-library odbc | (:windows (:or "odbc32.dll")) | (t (:default "odbc32.dll"))) | | (cffi:defcfun ("SQLAllocHandle" sql-handle) :short) | | (sql-handle) `-----
OK, yes, you need to load the library before the defcfun. Some lisps won't mind but others do. They'll all complain by the time the call to sql-handle is evaluated though.
Why aren't you loading the library? What are you trying to accomplish? What am I missing?
On 2/13/07, Edgar Gonçalves edgar.goncalves@gmail.com wrote:
I'll try again making it work with my clsql project, and I'll post back my success.
Eh, guess not, after all. The problem seems to reside somewhere betwee clsql-odbc and CFFI's uffi-compat.lisp.
The following is related to the initialization function of the saved image. It assumes the cl-sql library was already loaded AND used properly before the image was saved.
What I do before using the foreign functions is to call (clsql-sys:database-type-load-foreign :odbc), and this calls uffi:load-foreign-library, defined in cffi's uffi-compat.lisp, with the odbc32.dll filename. This, by its turn, calls cffi:load-foreign-library, which is the same as to call cffi::use-foreign-library. So, even though the result should apparently be the same, I still get this:
** - Continuable Error FFI::FOREIGN-CALL-OUT: no dynamic object named "SQLAllocHandle" in library :DEFAULT
But then again, if I manually call (cffi::use-foreign-library "odbc32.dll"), what I get is something different:
*** - FFI::FOREIGN-CALL-OUT: #<INVALID FOREIGN-POINTER #x00000000> comes from a previous Lisp session and is invalid
What am I missing here?
-- Edgar Gonçalves
Ok, I've just realized that I've been mislocating the problem at hands. It seems that defcfun works just fine, and the issue I'm getting resides on the foreign pointers. I've made tests to cffi, cffi-uffi-compat, ffi (answering to Ken's "CLisp exe vs FFI" post), and all have worked for a simple function definition/call. So I delved again to clsql initial situation, and I've traced the error to a call to %new-environment-handle, that uses a null pointer as an argument to the ff call to SQLAllocHandle. I also noticed that the null pointer was indeed showing up at the error message, so I tried this simple test:
,----- | (asdf:operate 'asdf:load-op 'cffi) | (asdf:operate 'asdf:load-op 'cffi-uffi-compat) | (defvar my-null-ptr (ffi:unsigned-foreign-address 0)) | (format t "- Before loading image, null ptr is: ~A!~%" my-null-ptr) | (ext:saveinitmem "test.exe" | :init-function #'(lambda () | (format t "- After loading image, my-null-ptr yelds: ~A.~%" my-null-ptr) | (ext:quit)) | :NORC t | :script t | :executable t | :quiet t) `-----
The output I get when I run this code is (after the initial load messages), ,----- | (...) | - Before loading image, null ptr is: #<FOREIGN-ADDRESS #x00000000>! | C:\dev\test>test | test | - After loading image, my-null-ptr yelds: #<INVALID FOREIGN-ADDRESS #x00000000>. `-----
What makes a foreign address valid or invalid? Is there some way I can validate a simple null pointer? I could understand if a given non-null pointer in C would be hard to validate after restarting an image, but a null pointer is, and always will be, 0x0, right?
-
Edgar Gonçalves -
Luís Oliveira