- rdnzl-devel@common-lisp.net - mailman3.common-lisp.net

[rdnzl-devel] CL-CLR, an alternative way of working with RDNZL
by Dan Muller 29 Mar '06

29 Mar '06

I recently put together a system that I layered on top of RDNZL, in order to experiment with some alternative ways of interacting with CLR/.NET code. I'm hoping that some of the ideas could be incorporated into RDNZL. Edi's too busy to look it over right now, and he suggested that I offer for perusal here on the mailing list. Aside from making type lookup a little easier, this system tries to make your .NET-using Lisp code a bit more Lisp-like. Performance is shabby because of the layering over RDNZL; much, if not all of this, could eventually be fixed. Nonetheless, I've been using this system successfully for the past few weeks as a vehicle for exploring Direct3D programming on a WinXP system, using the LispWorks trial edition of Lisp. In order to get reasonable performance, I had to use CL-CLR:DEFINE-CLR-CALL (a thin veneer over DEFINE-RDNZL-CALL) for most calls in the rendering loop. This system *should* work with any Lisp that is supported by RDNZL. The package includes several of Microsoft's Direct3D Tutorial programs, translated to Lisp. If you have ASDF, RDNZL, and Direct3D installed, you ought to be able to run the examples with a minimum of fuss. You can download it from here: http:\\..................\cl-clr.zip Comments are welcome. You are also encouraged to laugh at the excessively long TODO.txt file. -- Dan Muller ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 3

[rdnzl-devel] Type lookup problems, and solutions.
by Dan Muller 10 Mar '06

10 Mar '06

(Gruess Gott, Edi, hoffentlich war der Umzug nicht allzu anstrengend!) Remember that problem I was having importing some types? I'm not sure why, but on my system, System.Type.GetTypes is more picky than RDNZL seems to expect: ;;; ;;; To demonstrate these failing tests, start with a clean Lisp image, ;;; then load RDNZL, then load this file. Assemblies can't be ;;; unloaded except by unloading their containing application domain, ;;; and you can't unload the default application domain. ;;; ;;; It's possible that the tests are sensitive to the assemblies ;;; available on my machine, but I can't find an indication in ;;; Microsoft's documentation of why that might be the case. ;;; (in-package :cl-user) (use-package :rdnzl) (import-assembly "System.Windows.Forms") ;; This fails, because the system doesn't find the type. ;; Comment it out and try reloading to move on to the next test. (new "System.Windows.Forms.Form") ;; This fails, too. Same problem. ;; Comment it out and try reloading to move on to the next test. (import-type "System.Windows.Forms.Form") ;; I also tried direct calls to System.Type.GetType with various ;; abbreviations of the fully qualified type name, leaving off ;; trailing pieces of the full assembly name. None of these succeeded. ;; This succeeds. (import-type "System.Windows.Forms.Form" (load-assembly "System.Windows.Forms")) ;; This succeeds, because RDNZL now remembers what assembly it found ;; Form in. (let ((form ;; This fails, because the system doesn't find the type. (new "System.Windows.Forms.Form"))) (invoke form "Dispose")) <end of test code> I spent a lot of time poking around in RDNZL and the .NET framework, and reading documentation. The documentation is for 2.0, FWIW, but I was careful to look only at classes and methods that were available in 1.1. What follows is a rather lengthy discussion of some possible improvements/fixes I've come up with. This goes rather beyond fixing the immediate problem, but please give it some consideration. - Have IMPORT-TYPE always retrieve and cache the assembly-qualified name of the type. Downside: This will include the exact version. This means that in a delivered application involving a saved image, there is no way to use an updated version of a used assembly. (Haven't tested for this, though.) Might be an acceptable short-term fix, though. - Have IMPORT-ASSEMBLY supply the assembly to IMPORT-TYPE. Good idea, since it's currently inconsistent about this vis-a-vis IMPORT-TYPES. But it doesn't address the problem described above. - In MAKE-TYPE-FROM-NAME, do the following. If the name is assembly-qualified (contains an unquoted paren), then use System.Type.GetType() as before. Otherwise, call System.Reflection.Assembly.GetType() on each assembly returned by System.AppDomain.CurrentDomain.GetAssemblies(). The search should be exhaustive in order to detect ambiguities, which should be signaled as errors. In IMPORT-TYPE, don't accept assembly-qualified names, and cache the actual type object instead of the fully-qualified name. During shutdown, discard these type object references. During initialization, do the lookups again. (Changes to assemblies that introduce ambiguities will cause an error during initialization.) Downsides: Names given to RDNZL for resolution must not be assembly-qualified. (Or, perhaps better, detect these and don't cache them.) It is not possible for the system to deal predictably with like-named types in multiple assemblies. All loaded assemblies would be searched, even if they were not loaded via RDNZL. The latter problem might be a liability for advanced uses, e.g. involving dynamic assemblies that are being built by the application. I tried creating a C# console application that used two assemblies with conflicting type names. You get an error only if you actually reference the conflicting type name. The error documentation explains how C# allows this problem to be worked around, using a compiler switch that assigns a prefix to all namespaces in a given assembly: http://msdn2.microsoft.com/en-us/library/64wh5743.aspx AFAIK, implementing a similar solution requires work on the compiler's part. I have not noticed any direct support for this aliasing in the .NET framework. So a similar solution in RDNZL would require tracking assemblies. However, this might be desirable in order to address the other problem mentioned above. If RDNZL would track loaded assemblies, only those assemblies would be searched for type name matches. If we associate an optional alias with an assembly (e.g. via an optional argument to LOAD-ASSEMBLY), then that assembly would only be searched if the type name includes the prefix, and only after removing the prefix. As a further enhancement, I'd like to actually get rid of the need for importing entirely. It doesn't seem unreasonable to search for a type regardless of whether it has been imported or not, provided that it is then always cached in *TYPE-HASH*. This way *TYPE-HASH* builds up information about all types referenced via RDNZL. Users can still make calls to primitives like System.Type.GetType() if they ever want to bypass such activity. This is also true for LOAD-ASSEMBLY, if it starts tracking assemblies. Of course, the aliasing wouldn't work if you bypass RDNZL. I have prototyped some of the low-level pieces needed for these changes, entirely in Lisp using RDNZL. I didn't want to put more work into it unless I'm sure there's interest in including it in RDNZL. (Or at least, I'll do the work differently if there isn't.) I'm particularly uncertain about issues related to saving and loading Lisp images, something I've never tried out, although I think I understand the basic concerns there. -- Dan Muller ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 3

[rdnzl-devel] Re: New version 0.9.3
by Edi Weitz 19 Feb '06

19 Feb '06

[Re-routing to the mailing list.] On Fri, 17 Feb 2006 13:21:14 -0500, "Dan Muller" <s8ctxw402(a)sneakemail.com> wrote: > I'll try it out tonight and let you know how it works. Good. > I'm thinking that a cleaner solution might be an optional > target-type argument to BOX, although that requires RDNZL to pick > the conversion needed. Yeah, maybe. The advantage of my workaround is that you don't have to call BOX explicitely because RDNZL calls it as needed. So, you should be able to just INVOKE a method with arguments like 3.4d2 and RDNZL will convert it to a System.Single automatically. > Or a perhaps even more generally, a way to call a method of an > object or type with some arguments, but get the return value without > conversion, boxed if necessary. With that, I could have called > System.Convert.ToSingle and gotten a boxed Single back. Perhaps > calling it INVOKE-RETURNING-CONTAINER. But you'd have to do this for PROPERTY and FIELD as well. And you'd lose the advantages of the special reader syntax. A bit messy. > Thanks a lot for writing RDNZL. This is ever so much easier than > fooling around with FFIs to call C or C++ APIs. > > As I'm going along, I'm writing a few small utilities to help with > various common tasks. When I've had more experience with this, I'll > offer them for inclusion in RDNZL. That would be nice. Cheers, Edi.

2 16

[rdnzl-devel] (no subject)
by Dan Muller 17 Feb '06

17 Feb '06

My first post to this list... I started playing around with RDNZL, Direct3D 9.0, and the evaluation version of LispWorks for Windows about a week ago. Very nice! But I've run into an interesting problem. Direct3D uses single floats throughout its interface, but in LispWorks for Windows, all floating point types are equivalent to double-float. So I've got this rather difficult situation, where RDNZL can never find a method if it takes a single float argument! Even if it could find it, a down-conversion from double to single float would be needed, which probably shouldn't occur implicitly. I tried explicitly calling System.Convert.ToSingle, but by the time I can get my hands on the return value, it has been turned back into a double! I haven't been able to figure out a way around this without modifying RDNZL, or writing some sort of .NET function that returned a boxed single. But I'm not sure that the latter wouldn't be unboxed, or that it would give the correct type for the method lookup in RDNZL. Any ideas? -- Dan Muller ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

[rdnzl-devel] Problem with invoking static member
by Jim Sokoloff 13 Feb '06

13 Feb '06

(Apologies if this posts twice; I outsmarted myself with mailman, and I tried to cancel the first one that mailman thought was posted by a non list member.) Hi, I'm having trouble invoking a static member on a class. I've reduced the case to the following simple test case, and I suspect I may be missing somewhat simple. (I'm far from a Lisp wizard. Far...) Basically, I'm trying to load a C# assembly, and be able to call a static method on it. I can call instance methods without issue, but make-type-from-name is not working the way I expect it should, and therefore, calling static methods fails. Any help would be appreciated! --Jim Source code and sample interaction below: Lisp interaction: (win32, using slime and clisp-2.38, but I doubt it makes any difference) Some output trimmed for legibility. CL-USER> (load "/c/vp/devenv/lib/clisp/rdnzl-0.9.1/load.lisp") CL-USER> (in-package :rdnzl) RDNZL> (enable-rdnzl-syntax) RDNZL> (setq lisp-sample-assembly (import-assembly [System.Reflection.Assembly.LoadFrom "C:\\temp\\bin\\Debug\\LispSample1.dll"])) RDNZL> (setq p (new (import-type "LispSample1.Parrot" lisp-sample-assembly) "Bob")) #<CONTAINER LispSample1.Parrot #x2929E6A8> RDNZL> (ffi-call-with-args %invoke-instance-member p "SayHello" '(42)) 47 RDNZL> (resolve-type-name "LispSample1.Parrot") "LispSample1.Parrot, LispSample1, Version=1.0.2230.24825, Culture=neutral, PublicKeyToken=null" ;; To me, it appears that that worked. RDNZL> (make-type-from-name (resolve-type-name "LispSample1.Parrot")) #<CONTAINER NULL #x2929E540> RDNZL> ;; ^^^^ This appears to be as close to the root of the problem as ;; I am able to get RDNZL> (invoke p "SayHello" 17) 22 ;; this worked (There is stdout output, but my slime discards it.) RDNZL> (invoke p "StaticSayHello" "Bill" 17) ; Evaluation aborted ;; failed - .NET error (System.Exception): Instance method not found: LispSample1.Parrot::StaticSayHello(System.String,System.Int32) ;; Of course, it's not an instance method... RDNZL> (invoke "LispSample1.Parrot" "StaticSayHello" "Bill" 17) Trying to call function %INVOKE-STATIC-MEMBER with NULL object #<CONTAINER NULL #x292961E0>. C# class is here, compiled into an assembly: C:\temp\bin\Debug\LispSample1.dll using System; namespace LispSample1 { public class Parrot { private string m_name; public Parrot(string name) { m_name = name; } public int SayHello(int value) { Console.WriteLine("Hello {0}.", m_name); return value + 5; } static public int StaticSayHello(string name, int value) { Console.WriteLine("Hello {0}.", name); return value * 2; } } }

2 3

[rdnzl-devel] New release 0.9.0 / Experimental support for SBCL
by Edi Weitz 01 Feb '06

01 Feb '06

ChangeLog: Version 0.9.0 2006-02-01 Experimental support for SBCL/Win32 Download: <http://weitz.de/files/RDNZL.tar.gz> Cheers, Edi.

1 1

[rdnzl-devel] attaching handler / Tablet PC ink/ stroke/ not working
by Richard Fateman 25 Jan '06

25 Jan '06

Is anyone successfully using microsoft "ink" from Lisp via rdnzl? I sent this note to Edi RJF: Could you look at a (1 page) piece of code and see if there is something obvious that (my student and I) are doing wrong trying to use rdnzl? It is at http://www.cs.berkeley.edu/~fateman/temp/mjnet.lisp Edi: I just did and it looks OK to me. ... Nevertheless it doesn't work on either Allegro (RJF) or Lispworks. (Edi) It opens a window and enables "Ink", but if a handler is set up, it stops inking after the first stroke, and never really calls the handler. Oh, you don't need to have a Tablet PC, but it probably helps to have the microsoft Tablet PC SDK downloaded to your Windows machine. (Our project involves reading and speaking mathematics into a computer for processing by computer algebra programs in Lisp.) Thanks for any hints. RJF

2 7

[rdnzl-devel] new rdnzl 0.8.0 doesn't compile for me, Allegro
by Richard Fateman 24 Jan '06

24 Jan '06

This on Allegro 7.0... ;;; Compiling file h:\lisp\RDNZL-0.8.0\port-acl.lisp ; Fast loading from bundle code\efmacs.fasl. ;;; Writing fasl file h:\lisp\RDNZL-0.8.0\port-acl.fasl Error: Object free-dot-net-container cannot be written to a compiled file unless an applicable method is defined on make-load-form. [condition type: program-error] Any help would be appreciated.

2 1

[rdnzl-devel] Re: New versions 0.8.0/0.5.0
by Edi Weitz 15 Jan '06

15 Jan '06

On Mon, 16 Jan 2006 00:58:59 +0000, "lisp" <lisp(a)spikeisland.com> wrote: > I am happy to report that this works! Great! > If you run the function gc-test below in a LWW DLL image, you can > see the delegate adapter for the Click event being reclaimed and > releasing the closure from the callback hash table. > > If you open a process browser window *before* running this, you will > see the process created by LWW to handle the callback from .NET's gc > thread. Inspecting this process's arguments reveals > (rdnzl:%foreign-callable/releasedelegateadapter). This process is > used for all subsequent callbacks from the gc thread. If you kill > it, a new one will be created for the next gc callback. > > If you run this code in a conventional .exe LWW image you do not get > the callback from .NET's gc thread. Thanks for the detailed info. I'll try this tomorrow. > I have run into a further problem - with LWW this time I think. As > far as I can tell, LWW will occasionally run a foreign callback on > the wrong thread resulting in a stack overflow exception condition > being raised. With a form running in a new process and generating > event callbacks, the listener process occasionally reports a stack > overflow and shows the callback happening in *its* backtrace - > presumably confusing the stack check. I've not yet been able to > verify this or reproduce it reliably. > > Any thoughts or suggestions would be welcome... No idea. But it would be /very/ helpful if you could reproduce this and submit a bug report to LispWorks support. As you'll know they're currently working on the 5.0 release which will have some changes in the threading code as well. It'd be nice if they could fix that before they release the new version. Thanks again, Edi.

1 0

[rdnzl-devel] RDNZL - bug in delegate adapter release mechanism
by lisp 13 Jan '06

13 Jan '06

Hi Edi I've tracked down what looks to be a bug in the mechanism for releasing delegate adapters. The foreign callable ReleaseDelegateAdapter is declared with a return type of ffi-void-pointer but tries to return the result of remhash - a boolean. On LWW (at least) this results in: Condition: T cannot be converted to foreign type :POINTER. Call to SYSTEM::FOREIGN-CALLABLE-ENTRY-POINT-NEW-PROCESS-AUX (offset 21) SYSTEM::FUNC : RDNZL::%FOREIGN-CALLABLE/RELEASEDELEGATEADAPTER I would suggest changing it (and the corresponding initialisation and DLL code) to have a void return type: (ffi-define-callable (ReleaseDelegateAdapter ffi-void) ((index ffi-integer)) ;; remove entry from hash table if CLR is done with it (remhash index *callback-hash*)) Also, in DelegateAdapter.cpp you have a comment on the destructor that it doesn't appear to be working: // the destructor notifies Lisp that this instance is no longer used - // at least in theory, this doesn't seem to work right now DelegateAdapter::~DelegateAdapter() { release(indexIntoLisp); } This may have beeen the caused by the callback/threading issues discussed recently. At least it has certainly worked for me here (hence this bug report) when using a DLL based LWW image (many thanks for your recent example of this). Perhaps the .NET GC is running in a different thread, so the callback was failing in a normal .EXE image? The full traceback from the bug above seems to support this - it shows LWW creating a new process to handle the callback: CL-USER 4 : 1 > :bb #<PACKAGE COMMON-LISP-USER> Condition: T cannot be converted to foreign type :POINTER. Call to SYSTEM::FOREIGN-CALLABLE-ENTRY-POINT-NEW-PROCESS-AUX (offset 21) SYSTEM::FUNC : RDNZL::%FOREIGN-CALLABLE/RELEASEDELEGATEADAPTER Catch frame: MP::PROCESS-TAG Catch frame: (NIL) Binding frame: MP:*CURRENT-PROCESS* : NIL Binding frame: WIN32::*DDE-THREAD-INSTANCE* : #<unbound> Binding frame: MQ::*ACTIONS-AFTER-SLEEPING* : NIL Binding frame: MR::*ACTIONS-BEFORE-SLEEPING* : NIL Binding frame: SYSTEM::*READER-STATE* : #<unbound> Binding frame: *PACKAGE* : #<PACKAGE COMMON-LISP-USER> Binding frame: EDITOR::*CURRENT-BUFFER* : NIL Binding frame: EDITOR::*CURRENT-WINDOW* : NIL Binding frame: EDITOR::*EDITOR-STATE* : NIL Binding frame: EDITOR::*EDITOR-INPUT-STYLE* : #S(EDITOR:EDITOR-INPUT- STYLE :KEY-BINDINGS #<EDITOR::KEY-TABLE #<EQUALP Hash Table{332} 20BCA90C> 1 331> :DELETE-SELECTION-MODE NIL :LOGICAL-CHARACTERS #<EQUALP Hash Table{21} 20C3D364> :EXECUTE-MODE #<EDITOR::MODE-OBJECT "Execute" 20CE37FC> :ECHO-MODE #<EDITOR::MODE-OBJECT "Echo Area" 20CE34A4> :PLIST NIL :INTERRUPT-KEYS (#S(SYSTEM::GESTURE-SPEC :DATA 103 :MODIFIERS 2) #S(SYSTEM::GESTURE-SPEC :DATA 71 :MODIFIERS 2)) :STYLE :EMACS :POINT-ALWAYS-VISIBLE T :USE-FACE-TO-FULL-WIDTH-P T) Binding frame: CAPI::*USE-ACCELERATORS* : T Binding frame: CAPI-WIN32-LIB::*ALT-IS-META-KEY* : T Catch frame: #:CATCHER100675 Call to (SUBFUNCTION MP::PROCESS-SG-FUNCTION MP::INITIALIZE-PROCESS- STACK) (offset 347) MS::IGNORED : :DONT-KNOW Catch frame: SYSTEM::EXIT-FOREIGN-CALLABLE Call to SYSTEM::FOREIGN-CALLABLE-ENTRY-POINT-NEW-PROCESS (offset 133) SYSTEM::FUNC : RDNZL::%FOREIGN-CALLABLE/RELEASEDELEGATEADAPTER Call to SYSTEM::%FUNCALL1-ON-LISP-STACK (offset 34) SYSTEM::%FUNCALL1-ON-LISP-STACK T - Dominic Robinson

2 1