Hello, I'm having a problem with RDNZL using ECL. I'm using a send a (lisp) string to the simple .net gui through a callback. It most works, but the string gets mangled when it goes into .net.
The .net consumer looks like this: EditBox_Results.Text = cpc(TextBox_Filename.Text); where EditBox is a rich textbox, and cpc is a "public delegate System.String callback(string input);" which is set in the constructor. The list invocation looks like this: [Application.Run (new "CPC_interface" (new "callback" #'(lambda (filename) (pretty-print-cpc filename t)) ))] where CPC_interface is my .net gui, and pretty-print-cpc is defined as: (defun pretty-print-CPC (filename &optional (echo nil)) (handler-case (let ( (result (calculate-page-coverage filename)) ) (format echo "Cyan: ~$ %~%Magenta: ~$ %~%Yellow: ~$ %~%Black: ~$ %~%Total Byte Count: ~a" (* 100 (car result)) (* 100 (cadr result)) (* 100 (caddr result)) (* 100 (cadddr result)) (car (cddddr result)) )) (file-error () "File Not Found") ))
for the test files I run it on, the result of pretty-print-cpc is: Cyan: 37.50 % Magenta: 37.50 % Yellow: 37.50 % Black: 12.50 % Total Byte Count: 32
however the .net gui is displaying that as: ???????
As I mentioned before, I'm using ECL is my lisp. I'm compiling the .net part using Visual Studio 2005 C# targeting version 2 of the .net framework. Any help that can be provide is appreciated.
On Fri, 01 Feb 2008 18:19:29 -0500, Michael Mills mikaelmills@gmail.com wrote:
I'm having a problem with RDNZL using ECL. I'm using a send a (lisp) string to the simple .net gui through a callback. It most works, but the string gets mangled when it goes into .net.
The .net consumer looks like this: EditBox_Results.Text = cpc(TextBox_Filename.Text); where EditBox is a rich textbox, and cpc is a "public delegate System.String callback(string input);" which is set in the constructor. The list invocation looks like this: [Application.Run (new "CPC_interface" (new "callback" #'(lambda (filename) (pretty-print-cpc filename t)) ))] where CPC_interface is my .net gui, and pretty-print-cpc is defined as: (defun pretty-print-CPC (filename &optional (echo nil)) (handler-case (let ( (result (calculate-page-coverage filename)) ) (format echo "Cyan: ~$ %~%Magenta: ~$ %~%Yellow: ~$ %~%Black: ~$ %~%Total Byte Count: ~a" (* 100 (car result)) (* 100 (cadr result)) (* 100 (caddr result)) (* 100 (cadddr result)) (car (cddddr result)) )) (file-error () "File Not Found") ))
for the test files I run it on, the result of pretty-print-cpc is: Cyan: 37.50 % Magenta: 37.50 % Yellow: 37.50 % Black: 12.50 % Total Byte Count: 32
however the .net gui is displaying that as: ???????
As I mentioned before, I'm using ECL is my lisp. I'm compiling the .net part using Visual Studio 2005 C# targeting version 2 of the .net framework.
I don't have ECL myself, but here are some things you could check to isolate the problem:
1. Can you run the Apropos example that comes with RDNZL without problems? It uses a delegate with the same signature.
2. Does your application work if you use LispWorks instead of ECL?
And which version of ECL are you using, BTW?
Cheers, Edi.
With this invocation, I think pretty-print-CPC would actually return NIL:
[Application.Run (new "CPC_interface" (new "callback" #'(lambda (filename) (pretty-print-cpc filename t)) ))]
so I'm guessing it was actually called with echo set to NIL. I think I can reproduce something like this myself, I don't understand why I haven't seen it before, but it seems something happens with values returned from lisp to .Net. (This is on the 3.5 framework, however, but tested on both lispworks and ACL).
- Can you run the Apropos example that comes with RDNZL without problems? It uses a delegate with the same signature.
It defines a delegate "callback" with the same signature, but it isn't actually used; the delegate actually used is a KeyPressEventHandler, and the closure it wraps doesn't actually return a value, and if what I'm seeing is the same, only return-values passed from lisp to .Net are affected.
Shortest example is, given a delegate type
public delegate Int32 intcallback (String input);
(doing this with a return-type of String crashes my lisp)
(setf callback (new "TestLib.intcallback" (lambda (x) (length x)))) (invoke callback "Invoke" "hey")
returns 1762921488 . I've done a bit of debugging, and there is no doubt that arguments are passed correctly to the lisp function, which is called correctly, and which produces the correct value. Also, the above result is seen not only when passing the result back to lisp again, but also when the callback is called from .Net and for example displayed on screen with a MessageBox. When the callback is called for effect, there is no problem.
I've tried to return boxed values from the lisp-callback as well with the same result. Obviously some pointer is pointing in the wrong direction here, but I haven't gotten deep enough yet to find the actual bug.
For the original problem, if this is the problem and not the character encoding, it should work as a band-aid to call a .net function instead of returning the value as a string, that is, in pretty-print-CPC do something like
(invoke myCPCInterface "RecievePrettyPrintedCPC" (format echo ...
Regards, Iver
On Mon, 4 Feb 2008 11:22:10 +0100, "Iver Odin Kvello" iverodin@gmail.com wrote:
I don't understand why I haven't seen it before, but it seems something happens with values returned from lisp to .Net.
Iver, thanks for the analysis. With that as a starting point, I've managed to find the actual problem. (And this seems to show that Michael was the first one for a long time who actually used callbacks into Lisp which are supposed to return something... :)
The code in DelegateAdapterBuilder.cpp creates the wrong IL code. I don't have the time right now to prepare a new release (maybe in the next days), but if you want to fix this yourself, here's what to do:
1. The error is in "generateInvokeMethod" /after/ "invoke" has been called, i.e. when the return value is handled.
2. In case of a void return value, the code is correct.
3. In case of a value type, the correct loading form (depending on the type) must be emitted after the "Unbox" operation - for System.Int32 this would be "Ldind_I4" for example - to bring the actual value on the stack.
Which loading form has to be emitted must be decided from a small table lookup or so - "Ldind_I1" for booleans, "Ldind_R8" for double floats, and so on.
4. And in the third case the code which is there (which emits "Ldind_I4") is bogus. In this case nothing has to be done and the return value is already correct.
I've confirmed that with these changes I can use Lisp callbacks which return System.Int32 as well as ones which return System.String.
Thanks to you and Michael for catching this one, Edi.
Which loading form has to be emitted must be decided from a small table lookup or so - "Ldind_I1" for booleans, "Ldind_R8" for double floats, and so on.
I tried to muck around with this myself, and discovered the Unbox_Any opcode, present from version 2.0 of the framework; and this also seems to work (see attached patch). Of course, I don't really know what I'm doing, and this isn't present in version 1.0 and 1.1, but could that work also? It makes the fix a one-liner.
Iver
On Wed, 13 Feb 2008 21:49:55 +0100, "Iver Odin Kvello" iverodin@gmail.com wrote:
I tried to muck around with this myself, and discovered the Unbox_Any opcode, present from version 2.0 of the framework; and this also seems to work (see attached patch).
Cool, thanks. Just checking - the IsValueType condition is the correct one?
Of course, I don't really know what I'm doing, and this isn't present in version 1.0 and 1.1, but could that work also?
The next release of RDNZL that I'm currently working on will switch to Visual Studio 2005 anyway as I don't have 2003 on the harddisk of my new laptop anymore. AFAIU this will entail switching to the 2.0 framework which I think is OK. I don't see any particular reason to stick with 1.x.
Thanks, Edi.
On Thu, Feb 14, 2008 at 9:15 AM, Edi Weitz edi@agharta.de wrote:
On Wed, 13 Feb 2008 21:49:55 +0100, "Iver Odin Kvello" iverodin@gmail.com wrote:
I tried to muck around with this myself, and discovered the Unbox_Any opcode, present from version 2.0 of the framework; and this also seems to work (see attached patch).
Cool, thanks. Just checking - the IsValueType condition is the correct one?
I think so, but I don't understand this material thoroughly.
http://msdn2.microsoft.com/en-us/library/system.reflection.emit.opcodes.unbo... says that:
When applied to the boxed form of a value type, the unbox.any instruction extracts the value contained within obj (of type O), and is therefore equivalent to unbox followed by ldobj.
When applied to a reference type, the unbox.any instruction has the same effect as castclass typeTok.
But we know it will be a value-type. For ldobj (which is actually present from version 1.0 I see now) the docs say:
The ldobj instruction copies the value pointed to by addrOfValObj (of type &, *, or native int) to the top of the stack. The number of bytes copied depends on the size of the class (as specified by the class parameter). The class parameter is a metadata token representing the value type.
So given that the ld-operation to use only depends on the returnType, it seems to me that it should work. Here are my tests (with the "generic"-thing that I didn't submit a patch for) showing returns of both IsValueType-types and a String:
RDNZL-USER(60): (setf f1 (new (generic "Func" "Int32" "Int32") #'1+)) #<RDNZL::CONTAINER System.Func`2[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] #xdd69738> RDNZL-USER(61): (setf f2 (new (generic "Func" "Int32" "Single") (lambda (x) (coerce x 'single-float)))) #<RDNZL::CONTAINER System.Func`2[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Single, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] #xdd69e68> RDNZL-USER(62): (setf f3 (new (generic "Func" "Int32" "Double") (lambda (x) (* 1.0 x)))) #<RDNZL::CONTAINER System.Func`2[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Double, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] #xdd6a378> RDNZL-USER(63): (setf f4 (new (generic "Func" "Int32" "String") (lambda (x) (format nil "~@r" x)))) #<RDNZL::CONTAINER System.Func`2[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] #xdd6a888> RDNZL-USER(64): (property (invoke "System.Type" "GetType" "System.String") "IsValueType") NIL RDNZL-USER(65): (property (invoke "System.Type" "GetType" "System.Single") "IsValueType") T RDNZL-USER(66): (invoke f1 "Invoke" 5) 6 RDNZL-USER(67): (invoke f2 "Invoke" 5) 5.0 RDNZL-USER(68): (invoke f3 "Invoke" 5) 5.0 RDNZL-USER(69): (invoke f4 "Invoke" 5) "V"
Regards, Iver
On Thu, 14 Feb 2008 09:51:07 +0100, "Iver Odin Kvello" iverodin@gmail.com wrote:
RDNZL-USER(60): (setf f1 (new (generic "Func" "Int32" "Int32") #'1+))
Nice. Will this (minus the "generic") also work with 0.12.0 now that your patch is in? Maybe you'd like to write a paragraph or two about generic types that we can add to the documentation? (ASCII text is fine, doesn't have to be HTML.)
Oh, BTW, will you also be in Amsterdam?
Nice. Will this (minus the "generic") also work with 0.12.0 now that your patch is in?
Yes - this is using 0.12.0 with the latest dll:
(import-assembly "System.Core") (import-assembly "mscorlib") (import-assembly "System") (use-namespace "System")
(new '("Func" "Int32" "Int32") #'1+)
(invoke * "Invoke" 4) => 5
Maybe you'd like to write a paragraph or two about generic types that we can add to the documentation? (ASCII text is fine, doesn't have to be HTML.)
I will. If the bare list syntax proves hateful, 'GENERIC' would take the same syntax, so that'll be reusable too.
Oh, BTW, will you also be in Amsterdam?
I still haven't decided.
Iver
Maybe you'd like to write a paragraph or two about generic types that we can add to the documentation? (ASCII text is fine, doesn't have to be HTML.)
So I've tried writing some documentation for this now finally, here goes:
=== CUT HERE ====
Generic types in RDNZL
In summary, refer to a generic type with type-arguments filled with a list of type-names like
("System.Collections.Generic.List" "System.Int32")
Motivation
The name of a generic type, when 'closed' with type arguments so it is instantiable, is of the form
"Basetype´arity[ParameterType1,...,ParameterTypeN]"
and type names of this form can in general be used in all contexts like the argument to 'NEW' and so forth. However, for this type to be found, all the parameter types must either lie in the same assembly as the base type or their names must be assembly-qualified. Furthermore, the full 'path' to each type would have to be specified even if their namespaced had been imported with USE-NAMESPACE making this a bit unpractical.
To solve this, all arguments that accept a string as a typename will also accept a list of typenames (including sublists for when type-arguments are themselves generic types) where these lists represent generic types with their parameters. Also, since the length of the list is enough to determine the arity of the type, the "´arity"-part of the type-name can be dropped. Each type-name element of the list will have its name resolved in the imported namespaces.
The upshot is that for one can instantiate the type with full name
System.Func`2[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], System.Core, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
using
(import-assembly "mscorlib") ;; Int32 lives here (import-assembly "System.Core") ;; Func (of diverse arities) lives here (use-namespace "System") (new '("Func" "Int32" "Int32") #'1+)
On Fri, 7 Mar 2008 17:26:54 +0100, "Iver Odin Kvello" iverodin@gmail.com wrote:
So I've tried writing some documentation for this now finally, here goes:
Thanks again. And I've finally added this to the "official" RDNZL documentation. I also added links to Matthews's DataGridView examples. No code changes, though.
On Wed, 13 Feb 2008 21:49:55 +0100, "Iver Odin Kvello" iverodin@gmail.com wrote:
I tried to muck around with this myself, and discovered the Unbox_Any opcode, present from version 2.0 of the framework; and this also seems to work (see attached patch). Of course, I don't really know what I'm doing, and this isn't present in version 1.0 and 1.1, but could that work also? It makes the fix a one-liner.
I've now released a new version which hopefully fixes this. Thanks for your help and thanks to Michael for discovering the bug. A simple test for callbacks is now included in the examples folder.
As I already announced, the new DLL and the updated C++ code are based on Visual Studio 2005 now.
Please test!
-
Edi Weitz -
Iver Odin Kvello -
Michael Mills