Hi, dear CFFI Developers
Since LispWorks 5.1 has been released yesterday, I can start to talk about it.
* :long-long and :unsigned-long-long support
LispWorks 5.0 64-bit can already support :long-long and :unsigned-long- long, but the FLI:FOREIGN-TYPED-AREF function cannot support them, and,
LispWorks 5.1 64-bit has fixed this, now FLI:FOREIGN-TYPED-AREF can support 64-bit integers.
* LispWorks *features*
Though LispWorks cannot use a >=version like compile feature like those in Allegro, but since LW is very stable and its version grows very slowly (0.1/year ^_^), I think we can use #+(and lispworks-64bit (not lispworks5.0)) to mean "LispWorks 64-bit version >= 5.1"
I attach a simple patch, which opens the 64-bit integer support on LW 5.0 & 5.1 (64-bit). Hope it can be reviewed and merged into CFFI trunk. I'll always need these changes to compile & load IOLIB project on 64-bit LispWorks.
Regards,
Chun Tian (binghe)
http://www.lispworks.com/documentation/lw51/RNIG/html/readme-339.htm#pgfId-9...
Hi, dear CFFI Developers
Since LispWorks 5.1 has been released yesterday, I can start to talk about it.
- :long-long and :unsigned-long-long support
LispWorks 5.0 64-bit can already support :long-long and :unsigned- long-long, but the FLI:FOREIGN-TYPED-AREF function cannot support them, and,
LispWorks 5.1 64-bit has fixed this, now FLI:FOREIGN-TYPED-AREF can support 64-bit integers.
- LispWorks *features*
Though LispWorks cannot use a >=version like compile feature like those in Allegro, but since LW is very stable and its version grows very slowly (0.1/year ^_^), I think we can use #+(and lispworks-64bit (not lispworks5.0)) to mean "LispWorks 64-bit version >= 5.1"
I attach a simple patch, which opens the 64-bit integer support on LW 5.0 & 5.1 (64-bit). Hope it can be reviewed and merged into CFFI trunk. I'll always need these changes to compile & load IOLIB project on 64-bit LispWorks.
Regards,
Chun Tian (binghe)
<lw64.diff>
On Fri, Mar 28, 2008 at 5:37 AM, Chun Tian (binghe) binghe.lisp@gmail.com wrote:
Though LispWorks cannot use a >=version like compile feature like those in Allegro, but since LW is very stable and its version grows very slowly (0.1/year ^_^), I think we can use #+(and lispworks-64bit (not lispworks5.0)) to mean "LispWorks 64-bit version >= 5.1"
183c193 < #+#.(cl:if (cl:find-symbol "FOREIGN-TYPED-AREF" "FLI") '(and) '(or)) ---
#+#.(cl:if (cl:find-symbol "FOREIGN-TYPED-AREF" "FLI") '(and lispworks-64bit (not lispworks5.0)) '(or))
IIUC, this would disable this optimization for all non-64bit platforms. It seems easier to simply state that CFFI long-longs require LW version 5.1 or greater on 64-bit platforms. Got any objections against that?
Hi,
On Fri, Mar 28, 2008 at 5:37 AM, Chun Tian (binghe) binghe.lisp@gmail.com wrote:
Though LispWorks cannot use a >=version like compile feature like those in Allegro, but since LW is very stable and its version grows very slowly (0.1/year ^_^), I think we can use #+(and lispworks-64bit (not lispworks5.0)) to mean "LispWorks 64-bit version >= 5.1"
183c193
< #+#.(cl:if (cl:find-symbol "FOREIGN-TYPED-AREF" "FLI") '(and) '(or))
#+#.(cl:if (cl:find-symbol "FOREIGN-TYPED-AREF" "FLI") '(and lispworks-64bit (not lispworks5.0)) '(or))
IIUC, this would disable this optimization for all non-64bit platforms. It seems easier to simply state that CFFI long-longs require LW version 5.1 or greater on 64-bit platforms. Got any objections against that?
I ... think CFFI should support LispWorks 5.0 for long-long, because 5.0 (64-bit) do support it and not every 5.0 user can upgrade to 5.1 (new license and money is required). And (I think) LispWorks will continue support 5.0 by release new patches in the future. For me, I'm using both 5.0 and 5.1 right now. Without long-long support, it is impossible to compile IOLIB on LispWorks 64-bit 5.0.
Since it's not hard to consider both versions, I (personal) hope you can support them both, and that will be perfect.
Regards,
Chun Tian (binghe)
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/
2008/4/5 Chun Tian (binghe) binghe.lisp@gmail.com:
I ... think CFFI should support LispWorks 5.0 for long-long
[...]
Since it's not hard to consider both versions, I (personal) hope you can support them both, and that will be perfect.
Can you test the attached patch (diffed against cffi+lotsastuff) and see if it works for both LW 5.1 and 5.0? I only have the 32-bit Personal Edition so I can't test long-long support myself.
By the way, foreign strings seem completely broken. Do the string tests pass for you?
Hi,
2008/4/5 Chun Tian (binghe) binghe.lisp@gmail.com:
I ... think CFFI should support LispWorks 5.0 for long-long
[...]
Since it's not hard to consider both versions, I (personal) hope you can support them both, and that will be perfect.
Can you test the attached patch (diffed against cffi+lotsastuff) and see if it works for both LW 5.1 and 5.0? I only have the 32-bit Personal Edition so I can't test long-long support myself.
Your patch is OK for 5.1, but in 5.0, cannot compile src/cffi- lispworks.lisp:
**++++ Error in CFFI-SYS:%MEM-SET: Unknown block $MEM-REF
I think, in the definition of (DEFINE-COMPILER-MACRO CFFI-SYS:%MEM- SET), this part has something wrong:
#+(and lispworks-64bit lispworks5.0) (when (or (eql type :unsigned-long-long) (eql type :long-long)) (return-from %mem-ref form))
I don't know how to fix it, I'm just a newbie in DEFINE-COMPILER-MACRO.
By the way, foreign strings seem completely broken. Do the string tests pass for you?
How to do this test? On LispWorks 5.1 (amd64-linux), I load the ASDF- TESTS package, and load the tests/run-test.lisp, all seems OK, no error found.
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ <cffi-llispworks-long-long.diff>
2008/4/6 Chun Tian (binghe) binghe.lisp@gmail.com:
Your patch is OK for 5.1, but in 5.0, cannot compile src/cffi-lispworks.lisp:
**++++ Error in CFFI-SYS:%MEM-SET: Unknown block $MEM-REF
The new patch I've attached should work.
By the way, foreign strings seem completely broken. Do the string tests pass for you?
How to do this test? On LispWorks 5.1 (amd64-linux), I load the ASDF-TESTS package, and load the tests/run-test.lisp, all seems OK, no error found.
You can run the tests with (asdf:oos 'asdf:test-op :cffi)
Or, simply run the following snippet. Does this work for you?
(cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s))
Hi, Oliveira
With new patch, I can compile CFFI on LWL64 5.0 now.
2008/4/6 Chun Tian (binghe) binghe.lisp@gmail.com:
Your patch is OK for 5.1, but in 5.0, cannot compile src/cffi-lispworks.lisp:
**++++ Error in CFFI-SYS:%MEM-SET: Unknown block $MEM-REF
The new patch I've attached should work.
By the way, foreign strings seem completely broken. Do the string tests pass for you?
How to do this test? On LispWorks 5.1 (amd64-linux), I load the ASDF-TESTS package, and load the tests/run-test.lisp, all seems OK, no error found.
You can run the tests with (asdf:oos 'asdf:test-op :cffi)
Thank, now I can run CFFI tests. I met following issues:
LWL64 5.0:
Cannot compile tests/foreign-globals.lisp, seems in somewhere the FLI:FORIEGN-TYPED-AREF been called for 64-bit integers which LW 5.0 cannot support where I have no idea. For details, see test-reports-5.0 in attach.
LWL64 5.1:
18 unexpected failures: FUNCALL.DOUBLE, FUNCALL.DOUBLE26, FUNCALL.FLOAT26, DEFCFUN.DOUBLE, DEFCFUN.BFF.1, DEFCFUN.BFF.2, DEFCFUN.DOUBLE26, DEFCFUN.FLOAT26, CALLBACKS.LONG, CALLBACKS.LONG-LONG, CALLBACKS.FLOAT, CALLBACKS.FUNCALL.2, CALLBACKS.DOUBLE26.FUNCALL, CALLBACKS.FLOAT26, CALLBACKS.FLOAT26.FUNCALL, FOREIGN-GLOBALS.READ-ONLY.1, STRING.ENCODING.UTF-16.BASIC, STRING.ENCODINGS.ALL.BASIC.
For details, see test-reports-5.1 in attach. I'll try to look into them to see why they're failed on LispWorks, I don't know whether they're LW 5.1-only bugs until I can run tests on 5.0
Or, simply run the following snippet. Does this work for you?
(cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s))
Both 5.0 and 5.1 give me this result, I think it's OK:
CL-USER 6 > (cffi:with-foreign-string (s "abc") (cffi:foreign-string- to-lisp s)) "abc" 3
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ <cffi-lispworks-long-long.diff>
2008/4/6 Chun Tian (binghe) binghe.lisp@gmail.com:
LWL64 5.0:
Cannot compile tests/foreign-globals.lisp, seems in somewhere the FLI:FORIEGN-TYPED-AREF been called for 64-bit integers which LW 5.0 cannot support where I have no idea. For details, see test-reports-5.0 in attach.
That's odd. What do these two forms output?
(hcl:compiler-macroexpand '(cffi:mem-ref ptr :char))
(hcl:compiler-macroexpand '(cffi:mem-ref ptr :long-long))
Both 5.0 and 5.1 give me this result, I think it's OK:
CL-USER 6 > (cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s)) "abc" 3
No such luck here:
CL-USER 5 : 1 > (cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s))
Error: "��" is not of type SIMPLE-TEXT-STRING.
(lispworks-personal-5-0-1-x86-linux running on an AMD64)
Hi, Oliveira
2008/4/6 Chun Tian (binghe) binghe.lisp@gmail.com:
LWL64 5.0:
Cannot compile tests/foreign-globals.lisp, seems in somewhere the FLI:FORIEGN-TYPED-AREF been called for 64-bit integers which LW 5.0 cannot support where I have no idea. For details, see test-reports-5.0 in attach.
That's odd. What do these two forms output?
(hcl:compiler-macroexpand '(cffi:mem-ref ptr :char))
(hcl:compiler-macroexpand '(cffi:mem-ref ptr :long-long))
CL-USER 2 > (lisp-implementation-version) "5.0.2"
CL-USER 3 > (lisp-implementation-type) "LispWorks"
CL-USER 4 > (hcl:compiler-macroexpand '(cffi:mem-ref ptr :char)) (LOCALLY (DECLARE (OPTIMIZE (SPEED 3) (SAFETY 0))) (FLI:FOREIGN-TYPED-AREF '(SIGNED-BYTE 8) PTR (THE FIXNUM 0))) T
CL-USER 5 > (hcl:compiler-macroexpand '(cffi:mem-ref ptr :long-long)) (CFFI-SYS:%MEM-REF PTR :LONG-LONG 0) T
Both 5.0 and 5.1 give me this result, I think it's OK:
CL-USER 6 > (cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s)) "abc" 3
No such luck here:
CL-USER 5 : 1 > (cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s))
Error: "��" is not of type SIMPLE-TEXT-STRING.
(lispworks-personal-5-0-1-x86-linux running on an AMD64)
Oh, I know this time. I set this in my LispWorks startup script:
(set-default-character-element-type 'simple-char)
Without this line, I'll also get a error:
CL-USER 2 > (cffi:with-foreign-string (s "abc") (cffi:foreign-string-to-lisp s))
Error: "" is not of type SIMPLE-TEXT-STRING. 1 (abort) Return to level 0. 2 Restart top-level loop.
Type :b for backtrace, :c <option number> to proceed, or :? for other options
CL-USER 3 : 1 > :bb #<The COMMON-LISP-USER package, 15/32 internal, 0/4 external>
Condition: "" is not of type SIMPLE-TEXT-STRING. Call to ERROR (offset 80) SYSTEM::ESTRING : TYPE-ERROR SYSTEM::EARGS : (:EXPECTED-TYPE SIMPLE-TEXT-STRING :DATUM "")
Call to CONDITIONS:TYPE-ERROR-FUNCTION (offset 42)
Call to (SUBFUNCTION 3 (DEFVAR CFFI::*FOREIGN-STRING-MAPPINGS*)) (offset 451) BABEL-ENCODINGS::SRC : #<Pointer to type :BYTE = #x0050B450> BABEL-ENCODINGS::START : 0 BABEL-ENCODINGS::END : 3 BABEL-ENCODINGS::DEST : "" BABEL-ENCODINGS::D-START : 0 BABEL-ENCODINGS::U2 : 0 BABEL-ENCODINGS::U3 : 0 BABEL-ENCODINGS::U4 : 0 BABEL-ENCODINGS::U5 : 0 BABEL-ENCODINGS::U6 : 0 BABEL-ENCODINGS::DI : 0 BABEL-ENCODINGS::I : 0 BABEL-ENCODINGS::U1 : 97 DBG::G : 0 DBG::G : #\a
Call to CFFI:FOREIGN-STRING-TO-LISP (offset 652) CFFI::POINTER : #<Pointer to type :BYTE = #x0050B450> DBG::|rest-| : NIL CFFI::OFFSET : 0 COUNT : NIL CFFI::MAX-CHARS : NIL CFFI::ENCODING : :UTF-8 COUNT : 3 CFFI::MAPPING : #<BABEL-ENCODINGS::CONCRETE-MAPPING 40502DA4B3> CFFI::SIZE : 3 CFFI::NEW-END : 3 DBG::EXTRA-VALS : :DONT-KNOW STRING : ""
Call to SYSTEM::%EVAL (offset 275) EXP : (CFFI:FOREIGN-STRING-TO-LISP S)
Call to SYSTEM::%EVAL (offset 275) EXP : (PROGN (CFFI:FOREIGN-STRING-TO-LISP S))
Catch frame: "<* Catch All Object *>"
Call to UNWIND-PROTECT (offset 116) SYSTEM::STUFF : ((PROGN (CFFI:FOREIGN-STRING-TO-LISP S)) (CFFI:FOREIGN-STRING-FREE S))
Call to SYSTEM::%EVAL (offset 275) EXP : (UNWIND-PROTECT (PROGN (CFFI:FOREIGN-STRING-TO-LISP S)) (CFFI:FOREIGN-STRING-FREE S))
Catch frame: #:DYNAMIC-BINDING-CONTOUR61376
Variable environment contour: () Call to MULTIPLE-VALUE-BIND (offset 1116) SYSTEM::X : ((S) (CFFI:FOREIGN-STRING-ALLOC "abc") (UNWIND-PROTECT (PROGN (CFFI:FOREIGN-STRING-TO-LISP S)) (CFFI:FOREIGN-STRING-FREE S)))
Call to SYSTEM::%EVAL (offset 275) EXP : (MULTIPLE-VALUE-BIND (S) (CFFI:FOREIGN-STRING-ALLOC "abc") (UNWIND-PROTECT (PROGN (CFFI:FOREIGN-STRING-TO-LISP S)) (CFFI:FOREIGN- STRING-FREE S)))
Call to SYSTEM::%EVAL (offset 275) EXP : (CFFI:WITH-FOREIGN-STRING (S "abc") (CFFI:FOREIGN-STRING-TO- LISP S))
SYSTEM::%EVAL <- EVAL <- SYSTEM::DO-EVALUATION <- SYSTEM::%TOP-LEVEL-INTERNAL <- SYSTEM::%TOP-LEVEL <- SYSTEM::IN-START-FUNCTION-ONCE <- SYSTEM::IN-START-FUNCTION <- SYSTEM::START-FUNCTION T
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/
On Sun, Apr 6, 2008 at 4:41 PM, Chun Tian (binghe) binghe.lisp@gmail.com wrote:
CL-USER 5 > (hcl:compiler-macroexpand '(cffi:mem-ref ptr :long-long)) (CFFI-SYS:%MEM-REF PTR :LONG-LONG 0)
Looks good. Now, does this work?
(cffi:with-foreign-object (x :long-long) (setf (cffi:mem-ref x :long-long) 42) (cffi:mem-ref x :long-long))
(set-default-character-element-type 'simple-char)
Without this line, I'll also get a error:
Ah, cool. Thanks for the tip! That should greatly help me come up with a fix.
Hi,
On Sun, Apr 6, 2008 at 4:41 PM, Chun Tian (binghe) binghe.lisp@gmail.com wrote:
CL-USER 5 > (hcl:compiler-macroexpand '(cffi:mem-ref ptr :long-long)) (CFFI-SYS:%MEM-REF PTR :LONG-LONG 0)
Looks good. Now, does this work?
(cffi:with-foreign-object (x :long-long) (setf (cffi:mem-ref x :long-long) 42) (cffi:mem-ref x :long-long))
Seems good:
CL-USER 3 > (cffi:with-foreign-object (x :long-long) (setf (cffi:mem-ref x :long-long) 42) (cffi:mem-ref x :long-long)) 42
CL-USER 4 > (lisp-implementation-version) "5.0.2"
(set-default-character-element-type 'simple-char)
Without this line, I'll also get a error:
Ah, cool. Thanks for the tip! That should greatly help me come up with a fix.
Nice.
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/
Regards,
Chun Tian (binghe)
2008/4/7 Chun Tian (binghe) binghe.lisp@gmail.com:
Seems good:
CL-USER 3 > (cffi:with-foreign-object (x :long-long)
(setf (cffi:mem-ref x :long-long) 42) (cffi:mem-ref x :long-long)) 42
Ah, silly me. :long-long is the same size as :long on x86-64. We have to treat :long differently in the compiler macros as well. The attached patch should work better this time.
Hi, Oliveira
2008/4/7 Chun Tian (binghe) binghe.lisp@gmail.com:
Seems good:
CL-USER 3 > (cffi:with-foreign-object (x :long-long)
(setf (cffi:mem-ref x :long-long) 42) (cffi:mem-ref x :long-long)) 42
Ah, silly me. :long-long is the same size as :long on x86-64. We have to treat :long differently in the compiler macros as well. The attached patch should work better this time.
Thanks for fix that. With your new patch, I can run CFFI's test-op on LWL64 5.0 now. Test on LWL64 5.0 has 28 unexpected failures, more than 5.1 (which has 18 unexpected failures) For details, see in attach.
So, what else can I do for you? Does these failure tests means bugs or feature lost in CFFI's LW support?
Regards,
Chun Tian (binghe)
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/ <cffi-lispworks-long-long-v3.diff>
2008/4/7 Chun Tian (binghe) binghe.lisp@gmail.com:
Thanks for fix that. With your new patch, I can run CFFI's test-op on LWL64 5.0 now.
I've pushed it to the repository. BTW, I just found your patch from a few months ago, which I somehow missed: http://article.gmane.org/gmane.lisp.cffi.devel/1222 it looks exactly the same as the one I came up with. Sorry about that.
Test on LWL64 5.0 has 28 unexpected failures, more than 5.1 (which has 18 unexpected failures) For details, see in attach.
So, what else can I do for you? Does these failure tests means bugs or feature lost in CFFI's LW support?
Most likely, 5.1 fails less tests due to bugfixes.
FUNCALL.DOUBLE, DEFCFUN.DOUBLE
These fail because libm is not loaded. Should be fixed now.
FUNCALL.DOUBLE26, FUNCALL.FLOAT26, DEFCFUN.BFF.1, DEFCFUN.BFF.2, DEFCFUN.DOUBLE26, DEFCFUN.FLOAT26, CALLBACKS.LONG, CALLBACKS.LONG-LONG, CALLBACKS.FLOAT, CALLBACKS.FUNCALL.2, CALLBACKS.DOUBLE26.FUNCALL, CALLBACKS.FLOAT26, CALLBACKS.FLOAT26.FUNCALL
These are most likely bugs in Lispworks's 64-bit FFI. Some of those tests are pretty twisted but I'd give priority to CALLBACKS.{LONG,LONG-LONG,FLOAT,FUNCALL.2} which are pretty simple tests.
If you these matter to you, I suggest you report them to Lispworks. You'll probably make their life easier (and have better results) if you translate the tests to LW's FFI. (And you'd be making sure it's not a CFFI bug.)
FOREIGN-GLOBALS.READ-ONLY.1
I get this failure as well, I'll look into it when I get a chance.
STRING.ENCODING.UTF-16.BASIC, STRING.ENCODINGS.ALL.BASIC.
These are bugs in CFFI, babel or cffi-tests that I have to look into. They fail on all lisps/platforms.
2008/4/7 Chun Tian (binghe) binghe.lisp@gmail.com:
Thanks for fix that. With your new patch, I can run CFFI's test-op on LWL64 5.0 now.
I've pushed it to the repository. BTW, I just found your patch from a few months ago, which I somehow missed: http://article.gmane.org/gmane.lisp.cffi.devel/1222 it looks exactly the same as the one I came up with. Sorry about that.
Yes, and that's why I send to mail list again when I have another chance: LispWorks 5.1 released. For all, glad to see this patch be merged.
Test on LWL64 5.0 has 28 unexpected failures, more than 5.1 (which has 18 unexpected failures) For details, see in attach.
So, what else can I do for you? Does these failure tests means bugs or feature lost in CFFI's LW support?
Most likely, 5.1 fails less tests due to bugfixes.
Ah... good
FUNCALL.DOUBLE, DEFCFUN.DOUBLE
These fail because libm is not loaded. Should be fixed now.
FUNCALL.DOUBLE26, FUNCALL.FLOAT26, DEFCFUN.BFF.1, DEFCFUN.BFF.2, DEFCFUN.DOUBLE26, DEFCFUN.FLOAT26, CALLBACKS.LONG, CALLBACKS.LONG-LONG, CALLBACKS.FLOAT, CALLBACKS.FUNCALL.2, CALLBACKS.DOUBLE26.FUNCALL, CALLBACKS.FLOAT26, CALLBACKS.FLOAT26.FUNCALL
These are most likely bugs in Lispworks's 64-bit FFI. Some of those tests are pretty twisted but I'd give priority to CALLBACKS.{LONG,LONG-LONG,FLOAT,FUNCALL.2} which are pretty simple tests.
If you these matter to you, I suggest you report them to Lispworks. You'll probably make their life easier (and have better results) if you translate the tests to LW's FFI. (And you'd be making sure it's not a CFFI bug.)
Actually, I can't feel them until I can run these tests ... I'll try to look into them, and report to LW. If they're planning to release another patch bundle for 5.0, that will be nice for more FFI bug fixed. Thanks for give me this note.
FOREIGN-GLOBALS.READ-ONLY.1
I get this failure as well, I'll look into it when I get a chance.
STRING.ENCODING.UTF-16.BASIC, STRING.ENCODINGS.ALL.BASIC.
These are bugs in CFFI, babel or cffi-tests that I have to look into. They fail on all lisps/platforms.
Great, I just thought why my SBCL on darwin still didn't pass all these tests ...
-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/
-
Chun Tian (binghe) -
Luís Oliveira