Hi, Luigi,
I'm posting this message to the mailing list for future reference and in the hope that others might find it useful (this ought all to be properly documented anyhow!). I hope you're okay with this.
To provide some context for those just now tuning in, this is a reply to an inquiry about Objective-CL, an Objective-C bridge I have quietly been working on for the last two months, sadly completely unaware of the existence of CL-ObjC (I most probably wouldn't have started my own project had I known of CL-ObjC). Objective-CL is less featureful than CL-ObjC appears to be (for example, it doesn't let you define classes or methods yet). On the other hand, it's portable not only across CL implementations, but also across operating systems, supporting both Mac OS X and GNUstep targets and potentially lots of processor architectures. The license has not yet been decided upon (meanwhile it's GPL'd, because the GPL is the most restrictive license I'd consider using).
Darcs repository: http://matthias.benkard.de/code/objective-cl
Reference manual: http://matthias.benkard.de/objective-cl/documentation
Now, on to the reply itself.
> Actually, but i have to study better your code first
I've not studied your code thoroughly yet either, so I can't be sure
what I do differently from CL-ObjC, but I'll just try explaining what
goes on in my code.
> i d like to know (and merge) how you manage:
Okay, here goes...
The most obvoius difference between ObjCL and CL-ObjC is probably that
the latter is pure Lisp, while the former is partially implemented in
Objective-C for portability. This is very important, because it
immediately eliminates a number of problems and makes their solution
trivial (more about that later). Furthermore, I can reuse code from
PyObjC in this way, so if Apple decides to switch to LLVM or
something, Objective-CL can be made compatible with it easily simply
by stealing the approprate code from PyObjC. :)
The only Objective-C source files that I have written specifically for
Objective-CL are
libobjcl.m and libobjcl.h. All the others are from
PyObjC, adapted to Objective-CL.
> * Method invocation (my code is slow, it calls compile for every
> invocation, caching the compiled functions for future invocations)
Method invocation is quite simple in principle. First, we convert all
arguments into C values and reserve space for the return value, after
which we call the method, discard the C values and cons up the result
from the return value. This is precisely what LOW-LEVEL-INVOKE does.
The actual method call is done by the Objective-C-layer function
objcl_invoke_with_types. It's not a complicated function. You can
find it in Objective-C/libobjcl.m (like, incidentally, all other
functions in the Objective-C layer that are directly called from Lisp
and implemented by yours truly).
I'm not sure I understand what exactly you're caching here. I do a
bit of caching myself, namely of method signatures and selectors (the
latter by using compiler macros). Interestingly, caching selector
lookups is a huge performance boost, even though you might think that
calling FIND-SELECTOR, which is a wrapper around a fast C function,
should probably not take much time. The reason for this is that
caching the selector objects avoids MAKE-INSTANCEing them anew for
each method call, which takes much longer than the method call itself
(around 4 times as long, IIRC).
Actually, there are two different kinds of caching going on here.
Method signatures are cached by INVOKE-BY-NAME using a weak hash
table, while selectors are loaded into the image at load time by
transforming
(INVOKE obj :STRING-WITH-C-STRING "bla" :ENCODING 0)
into roughly
(INVOKE-BY-NAME obj (LOAD-TIME-VALUE (FIND-SELECTOR
'(:STRING-WITH-C-STRING :ENCODING))) "bla" 0).
Now, this isn't quite correct, of course, because FIND-SELECTOR may
fail. Because of this, the expansion is actually a bit more verbose:
(INVOKE-BY-NAME obj
(LOAD-TIME-VALUE
(HANDLER-CASE
(FIND-SELECTOR '(:STRING-WITH-C-STRING :ENCODING))
(SERIOUS-CONDITION NIL
(WARN
(MAKE-CONDITION 'SIMPLE-STYLE-WARNING :FORMAT-CONTROL
"~S designates an unknown ~
method selector."
:FORMAT-ARGUMENTS
(LIST '(:STRING-WITH-C-STRING :ENCODING))))
'(:STRING-WITH-C-STRING :ENCODING))))
"bla" 0)
This is done by the compiler macro for INVOKE. INVOKE-BY-NAME itself
has a similar compiler macro. Note that a warning is issued by the
generated code at load time when a selector can't be found. This is
very useful, because it essentially means that typos in method names
are caught at load time (which is normally close to compile time).
Note, by the way, that there are two slightly different functions for
method calling. There's PRIMITIVE-INVOKE on the one hand and
INVOKE-BY-NAME on the other (INVOKE is just a wrapper around the
latter). They essentially work the same, but PRIMITIVE-INVOKE
converts arguments in an ad-hoc way, while INVOKE-BY-NAME actually
examines the method's signature in order to correctly convert
arguments to C values. PRIMITIVE-INVOKE is not strictly needed, but
it's very convenient to have, because it enables Objective-CL to
inquire methods about their signatures, for example, without resorting
to functions implemented in Objective-C.
> * Struct by value argument passing and returning (i use an hack
> working on MacOSX x86 i should check also on GNU/Linux)
We do this by using libffi. It's quite simple: What libffi wants is
an array of pointers that point to the arguments that it should pass
to the foreign function, as well as a type specification which tells
it the structure of these arguments. Storing a pointer to a struct
from Lisp is trivial. Generating a type specification is non-trivial,
but, well, the PyObjC code does it for us. :)
The other way around (structs as return values) is probably similar,
but I don't think we actually support it yet. It may just magically
work, or it may not. I haven't tested it at all (frankly, I haven't
tested passing structs either, but I'm confident that if it doesn't
work right now, it's easy to fix).
> * Exception management (i dont support it)
This is next to trivial with Objective-CL's architecture. As
objcl_invoke_with_types is implemented in Objective-C, it can catch
exceptions easily. It then forwards them to the Lisp layer as a
return value. (Note that the return value of the method invocation is
stored by libffi into a previously allocated cell and _not_ returned
as a return value, so we can use the return value in whichever way we
like.)
> * Condition protocol
There's not much of a protocol... ^^'
By the way, we don't yet support raising Objective-C exceptions from
Lisp. I have no idea whether this is hard or not.
> * Memory management (i ignored this subject)
Ah, memory management... This is an interesting topic. I use the
trivial-garbage library in order to send `release' messages to objects
that are collected by the garbage collector on the Lisp side.
memory-management.lisp is very short. It's probably easier for you to
read it than for me to explain the way it works. :)
A note about the hash tables used there, though: Their sole purpose is
caching (again avoiding having to MAKE-INSTANCE every single return
value we get from Objective-C via method calls, which would be very
slow). If performance is not an issue, they can be removed without
changing the behaviour of the system.
> I also interested in your performance hacks like the compiler macros.
I hope that I explained them sufficiently above. Do remember, though,
that the compiler macros are not merely performance boosters. They
also provide a form of compile-time (load-time, actually) checking by
issuing warnings when encountering unknown method names.
Note the style warning:
OBJECTIVE-CL> (compile nil
'(lambda (x)
(invoke x :do 10 :something 20 :weird 42)))
; in: LAMBDA NIL
; (SB-KERNEL:FLOAT-WAIT)
;
; note: deleting unreachable code
; in: LAMBDA (X)
; (MULK.OBJECTIVE-CL:INVOKE MULK.OBJECTIVE-CL::X :DO 10 :SOMETHING
20 :WEIRD 42)
; --> MULK.OBJECTIVE-CL:INVOKE-BY-NAME
; ==>
; (LOAD-TIME-VALUE
; (HANDLER-CASE (MULK.OBJECTIVE-CL:FIND-SELECTOR '(:DO :SOMETHING :WEIRD))
; (SERIOUS-CONDITION NIL
; (WARN
; (MAKE-CONDITION '
MULK.OBJECTIVE-CL::SIMPLE-STYLE-WARNING
; :FORMAT-CONTROL "~S designates an unknown ~
; method selector."
; :FORMAT-ARGUMENTS #))
; '(:DO :SOMETHING :WEIRD))))
;
; caught STYLE-WARNING:
; (:DO :SOMETHING :WEIRD) designates an unknown method selector.
;
; compilation unit finished
; caught 1 STYLE-WARNING condition
; printed 1 note
#<FUNCTION {B862405}>
T
NIL
> Actually in my todo i have other "external" tasks to complete like
> complete the CLOS interface and to really integrate
This is something I'm interested in, by the way. What does the
mapping between CLOS and Objective-C classes look like? Is there
support for Objective-C metaclasses? Do Objective-C methods and CLOS
generic functions relate to each other in some way?
> CL-ObjC/Emacs/Slime into XCode, providing better documentation and
> examples.
I don't know XCode, but integration with Gorm is a must, yeah. Oh, by
the way, have you looked at the JOURNAL file in my repository? I only
recently started it. Its purpose is to document some of my decisions
regarding the development of Objective-CL.
> Despite that i think that a merge with your work looks
> promising.
What I find most remarkable is that we seem to have done quite a lot
of things in a very, _very_ similar way. Even the
function/macro/class names are similar or even identical (OBJC-ID and
DEFINE-OBJC-STRUCT, for example -- although I never actually
implemented the latter).
But don't get me started on CANONICALIZE-<FOO>. Reading that makes my
eyes hurt! The correct spelling is CANONIZE (or CANONISE, as I would
have written it). :)
(I know, I know. English is an evolving language, and there's some
dispute about this particular issue. Still... I like ranting about this kind of stuff.)
Bye,
Matthias