(Gruess Gott, Edi, hoffentlich war der Umzug nicht allzu anstrengend!)
Remember that problem I was having importing some types? I'm not sure why, but on my system, System.Type.GetTypes is more picky than RDNZL seems to expect:
;;; ;;; To demonstrate these failing tests, start with a clean Lisp image, ;;; then load RDNZL, then load this file. Assemblies can't be ;;; unloaded except by unloading their containing application domain, ;;; and you can't unload the default application domain. ;;; ;;; It's possible that the tests are sensitive to the assemblies ;;; available on my machine, but I can't find an indication in ;;; Microsoft's documentation of why that might be the case. ;;; (in-package :cl-user) (use-package :rdnzl) (import-assembly "System.Windows.Forms")
;; This fails, because the system doesn't find the type. ;; Comment it out and try reloading to move on to the next test. (new "System.Windows.Forms.Form")
;; This fails, too. Same problem. ;; Comment it out and try reloading to move on to the next test. (import-type "System.Windows.Forms.Form")
;; I also tried direct calls to System.Type.GetType with various ;; abbreviations of the fully qualified type name, leaving off ;; trailing pieces of the full assembly name. None of these succeeded.
;; This succeeds. (import-type "System.Windows.Forms.Form" (load-assembly "System.Windows.Forms"))
;; This succeeds, because RDNZL now remembers what assembly it found ;; Form in. (let ((form ;; This fails, because the system doesn't find the type. (new "System.Windows.Forms.Form"))) (invoke form "Dispose"))
<end of test code>
I spent a lot of time poking around in RDNZL and the .NET framework, and reading documentation. The documentation is for 2.0, FWIW, but I was careful to look only at classes and methods that were available in 1.1. What follows is a rather lengthy discussion of some possible improvements/fixes I've come up with. This goes rather beyond fixing the immediate problem, but please give it some consideration.
- Have IMPORT-TYPE always retrieve and cache the assembly-qualified name of the type.
Downside: This will include the exact version. This means that in a delivered application involving a saved image, there is no way to use an updated version of a used assembly. (Haven't tested for this, though.) Might be an acceptable short-term fix, though.
- Have IMPORT-ASSEMBLY supply the assembly to IMPORT-TYPE. Good idea, since it's currently inconsistent about this vis-a-vis IMPORT-TYPES. But it doesn't address the problem described above.
- In MAKE-TYPE-FROM-NAME, do the following. If the name is assembly-qualified (contains an unquoted paren), then use System.Type.GetType() as before. Otherwise, call System.Reflection.Assembly.GetType() on each assembly returned by System.AppDomain.CurrentDomain.GetAssemblies(). The search should be exhaustive in order to detect ambiguities, which should be signaled as errors. In IMPORT-TYPE, don't accept assembly-qualified names, and cache the actual type object instead of the fully-qualified name. During shutdown, discard these type object references. During initialization, do the lookups again. (Changes to assemblies that introduce ambiguities will cause an error during initialization.)
Downsides: Names given to RDNZL for resolution must not be assembly-qualified. (Or, perhaps better, detect these and don't cache them.) It is not possible for the system to deal predictably with like-named types in multiple assemblies. All loaded assemblies would be searched, even if they were not loaded via RDNZL. The latter problem might be a liability for advanced uses, e.g. involving dynamic assemblies that are being built by the application.
I tried creating a C# console application that used two assemblies with conflicting type names. You get an error only if you actually reference the conflicting type name. The error documentation explains how C# allows this problem to be worked around, using a compiler switch that assigns a prefix to all namespaces in a given assembly:
http://msdn2.microsoft.com/en-us/library/64wh5743.aspx
AFAIK, implementing a similar solution requires work on the compiler's part. I have not noticed any direct support for this aliasing in the .NET framework. So a similar solution in RDNZL would require tracking assemblies. However, this might be desirable in order to address the other problem mentioned above.
If RDNZL would track loaded assemblies, only those assemblies would be searched for type name matches. If we associate an optional alias with an assembly (e.g. via an optional argument to LOAD-ASSEMBLY), then that assembly would only be searched if the type name includes the prefix, and only after removing the prefix.
As a further enhancement, I'd like to actually get rid of the need for importing entirely. It doesn't seem unreasonable to search for a type regardless of whether it has been imported or not, provided that it is then always cached in *TYPE-HASH*. This way *TYPE-HASH* builds up information about all types referenced via RDNZL. Users can still make calls to primitives like System.Type.GetType() if they ever want to bypass such activity. This is also true for LOAD-ASSEMBLY, if it starts tracking assemblies. Of course, the aliasing wouldn't work if you bypass RDNZL.
I have prototyped some of the low-level pieces needed for these changes, entirely in Lisp using RDNZL. I didn't want to put more work into it unless I'm sure there's interest in including it in RDNZL. (Or at least, I'll do the work differently if there isn't.) I'm particularly uncertain about issues related to saving and loading Lisp images, something I've never tried out, although I think I understand the basic concerns there.
Dan Muller s8ctxw402-at-sneakemail.com |RDNZL-devel/via Sneakemail| wrote:
Remember that problem I was having importing some types? I'm not sure why, but on my system, System.Type.GetTypes is more picky than RDNZL seems to expect: [lots more stuff elided]
On further reflection, I've decided to work on this as a separate package layered on top of RDNZL, so that I can try implementing some other features that I'd like, related to the management of package use-lists and the timing of type resolution, which would be very intrusive to incorporate into RDNZL.
- Have IMPORT-TYPE always retrieve and cache the assembly-qualified
name of the type.
Downside: This will include the exact version. This means that in a delivered application involving a saved image, there is no way to use an updated version of a used assembly. (Haven't tested for this, though.) Might be an acceptable short-term fix, though.
- Have IMPORT-ASSEMBLY supply the assembly to IMPORT-TYPE. Good
idea, since it's currently inconsistent about this vis-a-vis IMPORT-TYPES. But it doesn't address the problem described above.
I suggest using one of these techniques for now in RDNZL. In any case, I'll be able to work around my type resolution problem in the package I'll build, which will have its own type object cache.
I have a local copy of RDNZL with some of the changes that I described, which I'm going to set aside for now.
On Fri, 24 Feb 2006 19:34:03 -0500, "Dan Muller" s8ctxw402@sneakemail.com wrote:
On further reflection, I've decided to work on this as a separate package layered on top of RDNZL, so that I can try implementing some other features that I'd like, related to the management of package use-lists and the timing of type resolution, which would be very intrusive to incorporate into RDNZL.
That sounds interesting. Maybe you can make this available somewhere once it's finished. If you want, we can also add it as some kind of "contrib" folder to RDNZL.
Hi Dan!
As I already said in private email: sorry for the long delay.
On Thu, 23 Feb 2006 21:32:25 -0500, "Dan Muller" s8ctxw402@sneakemail.com wrote:
Remember that problem I was having importing some types? I'm not sure why, but on my system, System.Type.GetTypes is more picky than RDNZL seems to expect:
;;; ;;; To demonstrate these failing tests, start with a clean Lisp image, ;;; then load RDNZL, then load this file. Assemblies can't be ;;; unloaded except by unloading their containing application domain, ;;; and you can't unload the default application domain. ;;; ;;; It's possible that the tests are sensitive to the assemblies ;;; available on my machine, but I can't find an indication in ;;; Microsoft's documentation of why that might be the case. ;;; (in-package :cl-user) (use-package :rdnzl)
There's a RDNZL-USER package for this kind of experimenting, BTW.
(import-assembly "System.Windows.Forms")
;; This fails, because the system doesn't find the type. ;; Comment it out and try reloading to move on to the next test. (new "System.Windows.Forms.Form")
;; This fails, too. Same problem. ;; Comment it out and try reloading to move on to the next test. (import-type "System.Windows.Forms.Form")
;; I also tried direct calls to System.Type.GetType with various ;; abbreviations of the fully qualified type name, leaving off ;; trailing pieces of the full assembly name. None of these succeeded.
;; This succeeds. (import-type "System.Windows.Forms.Form" (load-assembly "System.Windows.Forms"))
;; This succeeds, because RDNZL now remembers what assembly it found ;; Form in. (let ((form ;; This fails, because the system doesn't find the type. (new "System.Windows.Forms.Form"))) (invoke form "Dispose"))
<end of test code>
Well, the recommended way to do it is this one:
(import-types "System.Windows.Forms" "Form")
That should succeed.
I spent a lot of time poking around in RDNZL and the .NET framework, and reading documentation. The documentation is for 2.0, FWIW, but I was careful to look only at classes and methods that were available in 1.1. What follows is a rather lengthy discussion of some possible improvements/fixes I've come up with. This goes rather beyond fixing the immediate problem, but please give it some consideration.
- Have IMPORT-TYPE always retrieve and cache the assembly-qualified
name of the type.
Downside: This will include the exact version. This means that in a delivered application involving a saved image, there is no way to use an updated version of a used assembly. (Haven't tested for this, though.) Might be an acceptable short-term fix, though.
I don't like that. Being able to "cleanly" deliver applications was one of my main goals when developing RDNZL.
- Have IMPORT-ASSEMBLY supply the assembly to IMPORT-TYPE. Good
idea, since it's currently inconsistent about this vis-a-vis IMPORT-TYPES. But it doesn't address the problem described above.
- In MAKE-TYPE-FROM-NAME, do the following. If the name is
assembly-qualified (contains an unquoted paren), then use System.Type.GetType() as before. Otherwise, call System.Reflection.Assembly.GetType() on each assembly returned by System.AppDomain.CurrentDomain.GetAssemblies(). The search should be exhaustive in order to detect ambiguities, which should be signaled as errors. In IMPORT-TYPE, don't accept assembly-qualified names, and cache the actual type object instead of the fully-qualified name. During shutdown, discard these type object references. During initialization, do the lookups again. (Changes to assemblies that introduce ambiguities will cause an error during initialization.)
Downsides: Names given to RDNZL for resolution must not be assembly-qualified. (Or, perhaps better, detect these and don't cache them.) It is not possible for the system to deal predictably with like-named types in multiple assemblies. All loaded assemblies would be searched, even if they were not loaded via RDNZL. The latter problem might be a liability for advanced uses, e.g. involving dynamic assemblies that are being built by the application.
I tried creating a C# console application that used two assemblies with conflicting type names. You get an error only if you actually reference the conflicting type name. The error documentation explains how C# allows this problem to be worked around, using a compiler switch that assigns a prefix to all namespaces in a given assembly:
http://msdn2.microsoft.com/en-us/library/64wh5743.aspx
AFAIK, implementing a similar solution requires work on the compiler's part. I have not noticed any direct support for this aliasing in the .NET framework. So a similar solution in RDNZL would require tracking assemblies. However, this might be desirable in order to address the other problem mentioned above.
If RDNZL would track loaded assemblies, only those assemblies would be searched for type name matches. If we associate an optional alias with an assembly (e.g. via an optional argument to LOAD-ASSEMBLY), then that assembly would only be searched if the type name includes the prefix, and only after removing the prefix.
As a further enhancement, I'd like to actually get rid of the need for importing entirely. It doesn't seem unreasonable to search for a type regardless of whether it has been imported or not, provided that it is then always cached in *TYPE-HASH*. This way *TYPE-HASH* builds up information about all types referenced via RDNZL. Users can still make calls to primitives like System.Type.GetType() if they ever want to bypass such activity. This is also true for LOAD-ASSEMBLY, if it starts tracking assemblies. Of course, the aliasing wouldn't work if you bypass RDNZL.
I have prototyped some of the low-level pieces needed for these changes, entirely in Lisp using RDNZL. I didn't want to put more work into it unless I'm sure there's interest in including it in RDNZL. (Or at least, I'll do the work differently if there isn't.) I'm particularly uncertain about issues related to saving and loading Lisp images, something I've never tried out, although I think I understand the basic concerns there.
I'm not sure I fully understand where you're heading at the moment. I think it's obvious that RDNZL can't quite work like, say, a C# compiler due to Lisp's dynamic nature and due to the loose coupling between Lisp and the .NET runtime.
If you can come up with something that's more versatile without complicating matters for RDNZL developers, that's fine for me. Backwards compatibility is not really an issue as I /think/ we don't have more than half a dozen users anyway... :)
Being able to save and re-load images and to deliver applications is an important issue, though. As well as performance. I'm not advocating pre-mature optimization but I think one should shy away from solutions that are slow by design.
I'm sorry if my answer isn't as detailed as you expected - I'm still too busy with other things to think about this ATM. Maybe it'd be helpful if you could demonstrate how the existing RDNZL examples would look like after applying the changes you want to have.
Thanks, Edi.
-
Dan Muller -
Edi Weitz