Hello.
I have a crash when use CFFI + CCL + callbacks on Windows.
I am not sure the crash is caused by CFFI, there is another suspect. But the CFFI tests on Windows have errors. Could you take a look to the test results (attached)? This is CFFI 0.10.6
About the crash. It happens with CL+SSL.
OpenSSL requires multi-threaded applications to provide a callback, which implements locking, as described here: http://www.openssl.org/docs/crypto/threads.html.
The callback function prototype is
void locking_function(int mode, int n, const char *file, int line)
The callback is installed using
void CRYPTO_set_locking_callback(void (*func)(int mode,int type, const char *file,int line))
CL+SSL has the following definitions:
(cffi:defcallback locking-callback :void ((mode :int) (n :int) (file :string) (line :int)) ;(declare (ignore file line))
;; the callback body here
)
(cffi:defcfun ("CRYPTO_set_locking_callback" crypto-set-locking-callback) :void (fun :pointer))
If during initialization of CL+SSL we do
(crypto-set-locking-callback (cffi:callback locking-callback))
then we have a crash at the first SSL connection to hunchentoot. The crash is 100% reproducible.
It doesn't matter what we have in the callback body, it may be even empty. The symptoms are always the same.
If we comment out the line
;; (crypto-set-locking-callback (cffi:callback locking-callback))
then the crash doesn't happen.
As I said, I am not sure CFFI is the cause of the crash. I tried to print the values passed to the callback, they are not corrupted, valid numbers and strings. Also, the callback is called thousands of times before the crash happen.
But on the other hand, if we don't set the callback, the crash doesn't happen; and setting even empty callback guarantees the crash...
Anyway, If you provide any patches to CFFI, at least to fix the tests, I can run the tests on Windows.
Best regards, - Anton
One more question about my crash.
I have impression, that the callback is called from a non-Lisp thread, i.e. a thread that was not started by list, but was started by foreign code.
Is it known/supposed to work/fail? In CCL (and in general).
Best regards, - Anton
09.04.2011, 19:44, "Anton Vodonosov" avodonosov@yandex.ru:
Hello.
I have a crash when use CFFI + CCL + callbacks on Windows.
I am not sure the crash is caused by CFFI, there is another suspect. But the CFFI tests on Windows have errors. Could you take a look to the test results (attached)? This is CFFI 0.10.6
About the crash. It happens with CL+SSL.
OpenSSL requires multi-threaded applications to provide a callback, which implements locking, as described here: http://www.openssl.org/docs/crypto/threads.html.
The callback function prototype is
void locking_function(int mode, int n, const char *file, int line)
The callback is installed using
void CRYPTO_set_locking_callback(void (*func)(int mode,int type, const char *file,int line))
CL+SSL has the following definitions:
(cffi:defcallback locking-callback :void ((mode :int) (n :int) (file :string) (line :int)) ;(declare (ignore file line))
;; the callback body here
)
(cffi:defcfun ("CRYPTO_set_locking_callback" crypto-set-locking-callback) :void (fun :pointer))
If during initialization of CL+SSL we do
(crypto-set-locking-callback (cffi:callback locking-callback))
then we have a crash at the first SSL connection to hunchentoot. The crash is 100% reproducible.
It doesn't matter what we have in the callback body, it may be even empty. The symptoms are always the same.
If we comment out the line
;; (crypto-set-locking-callback (cffi:callback locking-callback))
then the crash doesn't happen.
As I said, I am not sure CFFI is the cause of the crash. I tried to print the values passed to the callback, they are not corrupted, valid numbers and strings. Also, the callback is called thousands of times before the crash happen.
But on the other hand, if we don't set the callback, the crash doesn't happen; and setting even empty callback guarantees the crash...
Anyway, If you provide any patches to CFFI, at least to fix the tests, I can run the tests on Windows.
Best regards,
- Anton
cffi-devel mailing list cffi-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/cffi-devel
2011/4/10 Anton Vodonosov avodonosov@yandex.ru:
I have impression, that the callback is called from a non-Lisp thread, i.e. a thread that was not started by list, but was started by foreign code.
Is it known/supposed to work/fail? In CCL (and in general).
Not all Lisps support that mode of operation but it works for me on CCL + OSX. See http://article.gmane.org/gmane.lisp.cffi.devel/1667 for more information and a small test to check whether your Lisp supports this feature. You might need to port it to Windows or use the pthread-win32 library. Let me know if that helps.
Cheers,
10.04.2011, 13:19, "Luís Oliveira" luismbo@gmail.com:
2011/4/10 Anton Vodonosov avodonosov@yandex.ru;:
I have impression, that the callback is called from a non-Lisp thread, i.e. a thread that was not started by list, but was started by foreign code.
Is it known/supposed to work/fail? In CCL (and in general).
Not all Lisps support that mode of operation but it works for me on CCL + OSX. See http://article.gmane.org/gmane.lisp.cffi.devel/1667 for more information and a small test to check whether your Lisp supports this feature. You might need to port it to Windows or use the pthread-win32 library. Let me know if that helps.
I just got a confirmation from the openssl-users list that OpenSSL doesn't invoke the callback from any threads not created by user, i.e. the non-Lisp thread theory is not valid.
If you have any hints on how to determine the crash reason - it is very appreciated.
BTW, I tried with CLISP - it doesn't crash. I've run the CFFI tests with CLISP, the result is similar to CCL, but the number of failures is a bit smaller.
If you are interested, the test log is attached.
Best regards, - Anton
On Tue, Apr 12, 2011 at 1:38 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
If you have any hints on how to determine the crash reason - it is very appreciated.
I suggest rewriting the callback bits using CCL's FFI (macroexpanding your current code should give you a head start) and if the issue remains report it as a bug to the Clozure folks.
HTH,
-
Anton Vodonosov -
Luís Oliveira