November 2022 - armedbear-devel - mailman3.common-lisp.net

Setting different class loader
by Steven Nunez 06 Feb '23

06 Feb '23

Greetings all, I have what I think is a problem with the ABCL class loader. I am working with a 'big data' library, Spark, but run into an issue on line 2 of the programming guide example. I am able to load the JARs from Maven with the ASDF system definition: (asdf:defsystem #:spark :description "Wrapper for Spark 3.0" :serial t :defsystem-depends-on (abcl-asdf) :depends-on (#:jss #:javaparser) :components ((:mvn "org.apache.spark/spark-core_2.12" :version "3.0.0") (:file "package") (:file "spark"))) and can create a SparkConf object: (defvar *spark-conf* #1"new SparkConf() .setAppName("abcl-app") .setMaster("local")" ) But when I try to create a 'context' (defvar *sc* (new 'JavaSparkContext *spark-conf*)) I get an error in the initialisation: Java exception 'java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.package$'. There isn't much on this except from the Jenkins guys, who have attempted to put Spark and Spark applications into a CI system. They seem to think that it's related to a call to get a properties file in the package class and on a StackOverflow discussion suggested that "you should make sure that you set the classloader that Spark was loaded through using the Thread.currentThread().setContextClassLoader(myCustomLoader) call". I've verified with (java:dump-classpath) that the JAR is on the ABCL classpath, and the JAR file does contain the spark-version-info.properties file. I've also tried getting the file myself with: (defvar rs #1"Thread.currentThread() .getContextClassLoader() .getResourceAsStream("spark-version-info.properties")" ) which returns nil, so their theory may be correct. Messing around with class loaders is a bit beyond my 20 year old Java knowledge so I thought I'd ask here if anyone has any ideas on how I can load Spark in way to use the default Java class loader. Alternatively it occurs to me to ask why the ABCL class loader isn't able to find the properties file if the JAR is on the classpath and then to correct whatever that problem is. Cheers, Steve

3 6

Stack trace from condition
by Garrett Dangerfield 17 Nov '22

17 Nov '22

(handler-case (error "hi") (error (e) ;; print stack trace here )) How do I get the stack trace for the error? Thanks, Garrett Dangerfield.

2 1

File I/O Metrics
by Garrett Dangerfield 05 Nov '22

05 Nov '22

I don't want to cause a firestore here but I was doing some simple benchmarks on file i/o between Java, ABCL, and SBCL and I'm a bit shocked, honestly. Reading a 2.5M file in 16M chunks in (using iso-8859-1): - abcl takes a tad over 1 second - sbcl takes 0.04 seconds Reading a 5.8G file in 16M chunks in (using iso-8859-1 for Lisp, for Java it's just bytes): - abcl takes...too long, I gave up - sbcl takes between 20 and 21 seconds - Java takes 1.5 seconds These are all run on the same computer using the same files, etc. What's up with this? Thoughts? I'd heard that SBCL should be as fast as C under at least some circumstances. I'd wager that C is at least as fast as Java (probably faster). Thanks, Garrett Dangerfield. (he/him/his) P.S. Don't get me wrong, I *LOVE* Lisp, I'm trying to get away from Java as fast as I can (the syntax is killing me slowly). I've used ABCL in projects before (it was wonderful, Java doesn't handle XML well). Lisp code: (with-open-file (stream "/media/danger/OS/temp/jars.txt" :external-format :iso-8859-1) ; great_expectations.iso (let ((size (file-length stream)) (buffer-size (* 16 1024 1024)) ; 16M ) (time (loop with buffer = (make-array buffer-size :element-type 'character) for n-characters = (read-sequence buffer stream) while (< 0 n-characters))) ))) Java code: private static final int BUFFER_SIZE = 16 * 1024 * 1024; try (InputStream in = new FileInputStream("/media/danger/OS/temp/great_expectations.iso"); ) { byte[] buff = new byte[BUFFER_SIZE]; int chunkLen = -1; long start = System.currentTimeMillis(); while ((chunkLen = in.read(buff)) != -1) { System.out.println("chunkLen = " + chunkLen); } double duration = System.currentTimeMillis() - start; duration /= 1000; System.out.println(String.format("it took %,2f secs", duration)); } catch (Exception e) { e.printStackTrace(System.out); } finally { System.out.println("Done."); }

7 10