Bugger. I'd hate for this to come to a dead-end, as it was looking like an elegant solution. The Spark tuning guide mentions using Kyro to provide serialization in Spark. It's not entirely free however; you need to 'register' your classes with Kyro for it to work. Would that be sufficient to provide serialisation to the needed Lisp objects?
On Thursday, July 30, 2020, 3:57:53 PM GMT+8, Alessio Stalla alessiostalla@gmail.com wrote:
You may have luck by providing a list. However, I see a deeper problem. Serializable is a marker interface: it has no methods, it only declares the type serializable. However, you cannot just declare that an object is serializable to make it so; all its components must be serializable as well. This includes the invocation handler that ABCL creates under the cover, as well as all the Lisp objects that you use for the implementation, particularly functions and closures. And, bad news – those aren't serializable. So, if Serializable is a requirement because those instances will effectively be serialized – e.g., to persist them to a file or to send them over the network – you're out of luck. Ages ago I had started a branch to make most Lisp objects serializable, but I don't remember how far I got. I don't think it was ever mature enough to be merged, but many years have passed.
On Thu, 30 Jul 2020 at 05:22, Steven Nunez steve_nunez@yahoo.com wrote:
Apologies, when I said "but I can't see adding multiple implementations", I meant multiple interfaces.
On Thursday, July 30, 2020, 11:19:07 AM GMT+8, Steven Nunez steve_nunez@yahoo.com wrote:
Is there a way to implement multiple interfaces on a single Java proxy? This code almost works:
(java:jinterface-implementation "org.apache.spark.api.java.function.Function"
"call" (lambda (s) (length s))) except that the proxy also needs to implement Serializable. The jproxy code in java.lisp seems to suggest that multiple implementations are allowed: (defgeneric jmake-proxy (interface implementation &optional lisp-this) (:documentation "Returns a proxy Java object implementing the provided interface(s)... but I can't see adding multiple implementations in the code. I see there's a few jmake-proxy methods in there though: are there any documentation or examples for their usage? Lsw2 doesn't use this at all and I can't find any other good examples of using ABCL.
Multiple interfaces from the jinterface-implementation function would be ideal, as the above code could then be wrapped with a macro to produce a 'spark-lambda' and be used nearly like the regular ABCL lambda.