- armedbear-devel - mailman3.common-lisp.net

Re: [armedbear-devel] [j-devel] regression
by Erik Huelsmann 15 Oct '09

15 Oct '09

On Thu, Oct 15, 2009 at 2:21 PM, Alan Ruttenberg <alanruttenberg(a)gmail.com> wrote: > On Thu, Oct 15, 2009 at 5:50 AM, Erik Huelsmann <ehuels(a)gmail.com> wrote: >> Hi Alan, >> >> On Thu, Oct 15, 2009 at 5:53 AM, Alan Ruttenberg >> <alanruttenberg(a)gmail.com> wrote: >>> I'm sorry about this bug report. Basically, a system I've been working >>> with for a long time has ceased to function in odd ways. I suspect it >>> has to do with exception handling changes recently introduced because >>> the symptom seems to be that a java function that I call that has a >>> timeout function continues to run even after the timeout is supposed >>> to have happened. >> >> Are you talking about ABCL? Or about something else? Which version of >> ABCL? Trunk? 0.16? > > Both it appears. The java code being called is from the pellet > reasoner, which has timers which it checks while it does it's work, > and when they are exceeded it throws the exception. Ok. The most helpful would be a version number (or repository revision number) which starts failing. >> Could you provide a bit more information on what the code is doing? One of the things I mean to ask here: are you calling the lisp code from the EVAL function? Or straight SYMBOL.execute() calls? >>> Unfortunately working on isolating the bug isn't something I have time >>> for so I've had to revert back to an earlier version. >>> >>> Wish I could say more at this point, but perhaps just notice that this >>> is happening might ring a bell for someone. >> >> If you're not talking about J, but specifically about ABCL, then, no, >> no bell yet. However, I'd like to look into it, ofcourse. > > abcl. I'll try to narrow it when I have some time. Thanks! Bye, Erik.

1 0

[armedbear-devel] CLOS optimizations
by Erik Huelsmann 15 Oct '09

15 Oct '09

This weekend, I committed an optimization which shows 4 - 12% performance gain on the CL-BENCH common lisp benchmarking toolset tests for CLOS. However, the test collects performance results from a high number of invocations. Low numbers of invocations should show even better improvements. Profiling one of the performance tests for CL-BENCH which shows huge difference with CLISP, it turned out that our CHECK-INITARGS function in the MOP package is taking 40% of the object instantiation time consumed by MAKE-INSTANCE. After the first invocation, it's possible to replace this function by a lookup in a cache. The same applies to args checking for some of the other functions defined to object instantiation and class modification. However, the time required to build this into our CLOS in a structured manner, exceeds "just" a few evenings, meaning that I probably will be delaying this task until I have a more time to undertake this effort. Ofcourse, I remain on the look out for possibilities to increase the performance of our clos implementation. An important input are the issues you experience with our performance, especially if you have example code to demonstrate it. Bye, Erik.

3 4

[armedbear-devel] Implementing non local exits in a runtime with structured exception handling
by Matthew D. Swank 15 Oct '09

15 Oct '09

I am playing with a lisp dialect (at least in my head) that I am thinking about implementing on the CLR. Like Java the CLR requires jumps across method call boundaries to be done using the exception handling mechanism. If I have code like the following in abcl: (block foo (flet ((callback () (return-from foo t))) (something-that-calls-into-java #'callback) ... stuff that won't happen if the callback is invoked)) And the plain old Java I'm calling into is pathological and catches the exception raised by calling the callback without re-throwing it, the non-local return never happens. Is that correct? If so, is there no way around it? Matt -- "You do not really understand something unless you can explain it to your grandmother." -- Albert Einstein.

2 1

[armedbear-devel] Java callbacks
by Alessio Stalla 14 Oct '09

14 Oct '09

Hello, I was thinking about the integration of lisp and java with abcl and it struck me that we don't have an easy way to pass a callback from Java to Lisp (e.g. a function acting as a factory of objects): one can extend Function, Primitive or even LispObject and implement the appropriate execute() method overload; however, one must perform conversion to/from Lisp objects manually, and while this is a minor inconvenience, it adds boilerplate code that could be easily avoided. So I was thinking about adding a Callback class extending Function, with overloads for execute() taking and returning regular java.lang.Object instances, and corresponding execute() based on LispObjects that perform the necessary conversions and call the Java-friendly versions. Example: public LispObject execute(LispObject arg0) { return JavaObject.getInstance(execute(arg0.javaInstance())); } /** To be overridden by the implementing subclass */ protected Object execute(Object arg0) { return error(new WrongNumberOfArgumentsException(this)); } Such a class could have convenience factory methods taking Runnables, Callables and other popular kinds of Java callback objects. Thoughts? Ale

2 2

Re: [armedbear-devel] [armedbear-cvs] r12183 - trunk/abcl/src/org/armedbear/lisp
by Erik Huelsmann 10 Oct '09

10 Oct '09

Mark, The commit below should enable the use of Gray streams with ABCL. I think this means SLIME can now switch to its normal pattern of using Gray streams to wrap its input and output processors. At least, I'd like to remove the SlimeInputStream and SlimeOutputStream Java classes and move the problem to SLIME, if that's possible. Maybe I want to do the same thing for socket streams. Bye, Erik. On Fri, Oct 9, 2009 at 11:31 PM, Erik Huelsmann <ehuelsmann(a)common-lisp.net> wrote: > Author: ehuelsmann > Date: Fri Oct 9 17:31:50 2009 > New Revision: 12183 > > Log: > Fix last Gray stream incompatibilities: generic functions > overlapping with CL functions are no longer have the STREAM- prefix. > > Note: this commit also removes gray stream testing code which does > not belong in the "production" image of our software. > > Modified: > trunk/abcl/src/org/armedbear/lisp/gray-streams.lisp > > Modified: trunk/abcl/src/org/armedbear/lisp/gray-streams.lisp > ============================================================================== > --- trunk/abcl/src/org/armedbear/lisp/gray-streams.lisp (original) > +++ trunk/abcl/src/org/armedbear/lisp/gray-streams.lisp Fri Oct 9 17:31:50 2009 > @@ -94,25 +94,7 @@ > ;;;; > ;;;; Notes > ;;;; ===== > -;;;; CLOSE is not a generic function in this implementation. Instead, > -;;;; the generic is called STREAM-CLOSE and the function CLOSE calls > -;;;; STREAM-CLOSE. The same goes for STREAMP, INPUT-STREAM-P, > -;;;; OUTPUT-STREAM-P and STREAM-ELEMENT-TYPE. The generic functions for > -;;;; these are STREAM-STREAMP, STREAM-INPUT-STREAM-P, > -;;;; STREAM-OUTPUT-STREAM-P and STREAM-STREAM-ELEMENT-TYPE. > ;;;; > -;;;; The standard Corman Lisp streams are not derived from > -;;;; FUNDAMENTAL-STREAM. All the stream functions check to see if the > -;;;; stream is an original Corman Lisp stream and forward on to the > -;;;; original function implementations. > -;;;; > -;;;; The string streams are implemented in this file as Gray streams > -;;;; but do not replace the Corman Lisp string streams. They are only > -;;;; implemented here to test the Gray stream functionality. These methods > -;;;; are called: > -;;;; GRAY-MAKE-STRING-OUTPUT-STREAM > -;;;; GRAY-GET-OUTPUT-STREAM-STRING > -;;;; GRAY-MAKE-STRING-INPUT-STREAM > ;;;; > ;;;; Much of the implementation of the Gray streams below is from the > ;;;; document referenced earlier. > @@ -123,7 +105,6 @@ > (:nicknames "GS") ;; # fb 1.01 > (:export > "FUNDAMENTAL-STREAM" > - "STREAM-CLOSE" > "STREAM-OPEN-STREAM-P" > "STREAM-STREAMP" > "STREAM-INPUT-STREAM-P" > @@ -200,21 +181,14 @@ > > (defclass fundamental-stream ()) > > -(defgeneric stream-close (stream &key abort)) > -(defgeneric stream-open-stream-p (stream)) > -(defgeneric stream-streamp (stream)) > -(defgeneric stream-input-stream-p (stream)) > -(defgeneric stream-input-character-stream-p (stream)) ;; # fb 1.01 > -(defgeneric stream-output-stream-p (stream)) > -(defgeneric stream-stream-element-type (stream)) > +(defgeneric gray-close (stream &key abort)) > +(defgeneric gray-open-stream-p (stream)) > +(defgeneric gray-streamp (stream)) > +(defgeneric gray-input-stream-p (stream)) > +(defgeneric gray-input-character-stream-p (stream)) ;; # fb 1.01 > +(defgeneric gray-output-stream-p (stream)) > +(defgeneric gray-stream-element-type (stream)) > > -(defmethod stream-close (stream &key abort) > - (declare (ignore stream abort)) > - nil) > - > -(defmethod stream-streamp (s) > - (declare (ignore s)) > - nil) > > (defmethod stream-streamp ((s fundamental-stream)) > s) > @@ -225,20 +199,12 @@ > (and (stream-input-stream-p s) > (eq (stream-stream-element-type s) 'character))) > > -(defmethod stream-input-stream-p (s) > - (declare (ignore s)) > - nil) > - > (defmethod stream-input-stream-p ((s fundamental-input-stream)) > (declare (ignore s)) > t) > > (defclass fundamental-output-stream (fundamental-stream)) > > -(defmethod stream-output-stream-p (s) > - (declare (ignore s)) > - nil) > - > (defmethod stream-output-stream-p ((s fundamental-output-stream)) > (declare (ignore s)) > t) > @@ -539,72 +505,6 @@ > (funcall *old-write-byte* integer binary-output-stream) > (stream-write-byte binary-output-stream integer))) > > -(defclass string-input-stream (fundamental-character-input-stream) > - ((string :initarg :string :type string) > - (index :initarg :start :type fixnum) > - (end :initarg :end :type fixnum))) > - > -(defun gray-make-string-input-stream (string &optional (start 0) end) > - (make-instance 'string-input-stream :string string > - :start start :end (or end (length string)))) > - > -(defmethod stream-read-char ((stream string-input-stream)) > - (with-slots (index end string) stream > - (if (>= index end) > - :eof > - (prog1 > - (char string index) > - (incf index))))) > - > -(defmethod stream-unread-char ((stream string-input-stream) character) > - (with-slots (index end string) stream > - (decf index) > - (assert (eql (char string index) character)) > - nil)) > - > -(defmethod stream-read-line ((stream string-input-stream)) > - (with-slots (index end string) stream > - (let* ((endline (position #\newline string :start index :end end)) > - (line (subseq string index endline))) > - (if endline > - (progn > - (setq index (1+ endline)) > - (values line nil)) > - (progn > - (setq index end) > - (values line t)))))) > - > -(defclass string-output-stream (fundamental-character-output-stream) > - ((string :initform nil :initarg :string))) > - > -(defun gray-make-string-output-stream () > - (make-instance 'string-output-stream)) > - > -(defun gray-get-output-stream-string (stream) > - (with-slots (string) stream > - (if (null string) > - "" > - (prog1 > - (coerce string 'string) > - (setq string nil))))) > - > -(defmethod stream-write-char ((stream string-output-stream) character) > - (with-slots (string) stream > - (when (null string) > - (setq string (make-array 64 :slement-type 'character > - :fill-pointer 0 :adjustable t))) > - (vector-push-extend character string) > - character)) > - > -(defmethod stream-line-column ((stream string-output-stream)) > - (with-slots (string) stream > - (if (null string) > - 0 > - (let ((nx (position #\newline string :from-end t))) > - (if (null nx) > - (length string) > - (- (length string) nx 1)))))) > - > (defmethod stream-line-column ((stream stream)) > nil) > > @@ -614,40 +514,26 @@ > nil ;(funcall *old-stream-column* stream) > (stream-line-column stream)))) > > -(defun gray-stream-element-type (stream) > - (if (old-streamp stream) > - (funcall *old-stream-element-type* stream) > - (stream-stream-element-type stream))) > +(defmethod gray-stream-element-type (stream) > + (funcall *old-stream-element-type* stream)) > > -(defun gray-close (stream &key abort) > - (if (old-streamp stream) > - (funcall *old-close* stream :abort abort) > - (stream-close stream :abort nil))) > +(defmethod gray-close (stream &key abort) > + (funcall *old-close* stream :abort abort)) > > -(defun gray-input-stream-p (stream) > - (if (old-streamp stream) > - (funcall *old-input-stream-p* stream) > - (stream-input-stream-p stream))) > +(defmethod gray-input-stream-p (stream) > + (funcall *old-input-stream-p* stream)) > > -(defun gray-input-character-stream-p (stream) > - (if (old-streamp stream) > - (funcall *old-input-character-stream-p* stream) > - (stream-input-character-stream-p stream))) > +(defmethod gray-input-character-stream-p (stream) > + (funcall *old-input-character-stream-p* stream)) > > -(defun gray-output-stream-p (stream) > - (if (old-streamp stream) > - (funcall *old-output-stream-p* stream) > - (stream-output-stream-p stream))) > +(defmethod gray-output-stream-p (stream) > + (funcall *old-output-stream-p* stream)) > > -(defun gray-open-stream-p (stream) > - (if (old-streamp stream) > - (funcall *old-open-stream-p* stream) > - (stream-open-stream-p stream))) > +(defmethod gray-open-stream-p (stream) > + (funcall *old-open-stream-p* stream)) > > -(defun gray-streamp (stream) > - (if (old-streamp stream) > - (funcall *old-streamp* stream) > - (stream-streamp stream))) > +(defmethod gray-streamp (stream) > + (funcall *old-streamp* stream)) > > (defun gray-write-sequence (sequence stream &key (start 0) end) > (if (old-streamp stream) > @@ -659,24 +545,6 @@ > (funcall *old-read-sequence* sequence stream :start start :end end) > (stream-read-sequence stream sequence start end))) > > -(defstruct two-way-stream-g > - input-stream output-stream) > - > -(defun gray-make-two-way-stream (in out) > - (if (and (old-streamp in) (old-streamp out)) > - (funcall *old-make-two-way-stream* in out) > - (make-two-way-stream-g :input-stream in :output-stream out))) > - > -(defun gray-two-way-stream-input-stream (stream) > - (if (old-streamp stream) > - (funcall *old-two-way-stream-input-stream* stream) > - (two-way-stream-g-input-stream stream))) > - > -(defun gray-two-way-stream-output-stream (stream) > - (if (old-streamp stream) > - (funcall *old-two-way-stream-output-stream* stream) > - (two-way-stream-g-output-stream stream))) > - > (setf (symbol-function 'common-lisp::read-char) #'gray-read-char) > (setf (symbol-function 'common-lisp::peek-char) #'gray-peek-char) > (setf (symbol-function 'common-lisp::unread-char) #'gray-unread-char) > @@ -703,8 +571,5 @@ > (setf (symbol-function 'common-lisp::streamp) #'gray-streamp) > (setf (symbol-function 'common-lisp::read-sequence) #'gray-read-sequence) > (setf (symbol-function 'common-lisp::write-sequence) #'gray-write-sequence) > -(setf (symbol-function 'common-lisp::make-two-way-stream) #'gray-make-two-way-stream) > -(setf (symbol-function 'common-lisp::two-way-stream-input-stream) #'gray-two-way-stream-input-stream) > -(setf (symbol-function 'common-lisp::two-way-stream-output-stream) #'gray-two-way-stream-output-stream) > > (provide 'gray-streams) > > _______________________________________________ > armedbear-cvs mailing list > armedbear-cvs(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-cvs >

3 3

[armedbear-devel] bug with initialize-instance and keyword arguments
by Steve Rolls 09 Oct '09

09 Oct '09

This is present in 0.16 and latest from svn, but not in 0.15. Specifying a keyword arg in an initialize-instance method does not make the keyword a valid initarg for make-instance. Armed Bear Common Lisp 0.16.0 Java 1.6.0_12 Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.425 seconds. Startup completed in 2.086 seconds. Type ":help" for a list of available commands. CL-USER(1): (defclass c1() (x)) #<STANDARD-CLASS C1 {5A347448}> CL-USER(2): (defmethod initialize-instance :after ((c c1) &key x2) (setf (slot-value c 'x) x2)) #<STANDARD-METHOD INITIALIZE-INSTANCE :AFTER (C1) {79A1EA46}> CL-USER(3): (make-instance 'c1 :x2 1) Debugger invoked on condition of type PROGRAM-ERROR: Invalid initarg :X2. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(4): Works as expected in 0.15 Armed Bear Common Lisp 0.15.0 Java 1.6.0_12 Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.45 seconds. Startup completed in 2.021 seconds. Type ":help" for a list of available commands. CL-USER(1): (defclass c1() (x)) #<STANDARD-CLASS C1 {615E7597}> CL-USER(2): (defmethod initialize-instance :after ((c c1) &key x2) (setf (slot-value c 'x) x2)) #<STANDARD-METHOD INITIALIZE-INSTANCE :AFTER (C1) {7346B111}> CL-USER(3): (make-instance 'c1 :x2 1) #<C1 {772EDF9A}> CL-USER(4):

3 8

[armedbear-devel] No :DARWIN, "No tag" & no ASDF
by John Pallister 06 Oct '09

06 Oct '09

Hello list, I'm inching my way towards working with ABCL (stupid day job) and I thought I'd mention a few things that have cropped up. I'm running on OS X 10.5.8 using the OpenJDK 6 (for JSR-223, and the server VM). I updated the source from the SVN repo this morning. When I build ABCL, I get the warning "Use of Java version 1.6.0-internal not recommended." Perhaps '<matches string="${java.version}" pattern="1\.6\.0-internal"/>' could be added to the appropriate spot in build.xml to suppress this warning. When I run ABCL, I get :JAVA-1.6 in *FEATURES* but not :DARWIN (or any other OS identifier). (*FEATURES* is (:DARWIN :JAVA-1.6 :ARMEDBEAR :ABCL :COMMON-LISP :ANSI-CL :CDR6). What's :CDR6?) When my '(load ".../start-swank.lisp")' fails, I get an error when I enter ':cont 0': shadow:abcl john$ ./abcl Armed Bear Common Lisp 0.17.0-dev Java 1.6.0-internal Sun Microsystems Inc. OpenJDK Server VM Low-level initialization completed in 1.699 seconds. Startup completed in 8.5 seconds. [on my Dell Mini 9, not real Apple h/w] Type ":help" for a list of available commands. CL-USER(1): (load "/Users/john/src/lisp/abcl/start-swank.lisp") Error loading /Users/john/src/lisp/asdf/asdf.lisp at line 193 (offset 7111) Debugger invoked on condition of type SIMPLE-ERROR: The long form of DEFINE-METHOD-COMBINATION is not implemented. Restarts: 0: TOP-LEVEL Return to top level. [1] ASDF(2): :cont 0 Debugger invoked on condition of type CONTROL-ERROR: No tag named #:G140089 is currently visible Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(3): :res CL-USER(4): Which brings me to the last problem: asdf.lisp won't load, since it uses the long form of DEFINE-METHOD-COMBINATION. Is there a workaround for this? If anyone can shed some light on these issues, that would be great. As an aside, does anyone want to share their .emacs settings for Java development? I've installed JDE (which was a PITA FWIW), CC-Mode etc. and I'm curious as to what other (Emacs-minded) people are using. Cheers, John :^P -- John Pallister john(a)johnp.net john(a)synchromesh.com

5 13

Re: [armedbear-devel] [armedbear-cvs] r12153 - trunk/abcl/src/org/armedbear/lisp
by Erik Huelsmann 05 Oct '09

05 Oct '09

Hi Mark, On Fri, Sep 18, 2009 at 3:43 PM, Mark Evenson <mevenson(a)common-lisp.net> wrote: > Author: mevenson > Date: Fri Sep 18 09:43:42 2009 > New Revision: 12153 > > Log: > Tenative fix for #63: transform caught Go throwable to LispError. > > Normalized printStackTrace() calls through Debug.trace() interface. > > > > Modified: > trunk/abcl/src/org/armedbear/lisp/Lisp.java > > Modified: trunk/abcl/src/org/armedbear/lisp/Lisp.java > ============================================================================== > --- trunk/abcl/src/org/armedbear/lisp/Lisp.java (original) > +++ trunk/abcl/src/org/armedbear/lisp/Lisp.java Fri Sep 18 09:43:42 2009 > @@ -123,7 +123,7 @@ > } > catch (Throwable t) > { > - t.printStackTrace(); > + Debug.trace(t); > } > } > > @@ -277,7 +277,7 @@ > } > catch (Go go) > { > - throw go; > + return error(go.getCondition()); > } > catch (Throw t) > { This hunk causes "havoc" with the command line interface where people select the "go to toplevel" restart. I think this is the major change in this commit? If I want to revert this bit, do I have to revert the rest too, or are those just improvements on what was already there? BTW: With my recent changes, I don't think there's anything wrong with re-throwing the Go anymore: we check for a catching environment before throwing now. > @@ -1042,6 +1042,7 @@ > } > catch (Throwable t) > { > + Debug.trace(t); > return null; > } > } > @@ -2586,7 +2587,7 @@ > } > catch (ClassNotFoundException e) > { > - e.printStackTrace(); > + Debug.trace(e); > } > } > > > _______________________________________________ > armedbear-cvs mailing list > armedbear-cvs(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-cvs >

2 1

[armedbear-devel] Fw: SparseChars patch
by logicmoo＠gmail.com 04 Oct '09

04 Oct '09

ABCL, Here is the SparseChars patch. It shouldn't pass more or less ANSI-TESTs. but will improve performance due to the fact it uses less java heap and copies less. The one thing I might do later is stop using the character iterator (CharHashMap would get a copyToOther(CharHashMap)).. but for now its sane.

2 1

[armedbear-devel] Making tags (in tagbody) and block names unique
by Erik Huelsmann 29 Sep '09

29 Sep '09

On IRC, we had a discussion some days ago where it was found that ABCL doesn't use unique tag-identifications in tagbody, leading to somewhat dynamically scoped tagbodies, instead of perfectly lexically scoped. BLOCK also uses non-unique names (think NIL), leading to the same issue. Both the compiler and the interpreter exhibit the problem. Consider the following code: (labels ((p (b) (tagbody (if (not b) (p #'(lambda () (go :foo))) (funcall b)) (return-from p :bar) :foo (return-from p :foo)))) (p nil)) It should return :foo, but returns :bar on ABCL's interpreter instead. (The fact that the compiler works, means there's probably a bug in it somewhere, because there's no explanation why it would.) I'm now thinking of ways to make the tags/blocknames unique. The one way I could come up with for the interpreter is this: Adding a second identifier (created using "new Object();") to the blocks and tags registered in the interpreter environment. Then, when a Go or Return is thrown, it should include the identifier. The catching tagbodies and blocks can verify that the Go or Return is meant for that block by checking the identifier. The approach above requires an additional "identifier" field in the Binding object. For the compiler, this could work similarly, except that the identifier can't be stored in the environment (because there is none), meaning it should probably be stored in a closure variable which any throwing function can then use when creating the Go or Return object. Any comments? Bye, Erik.

1 2

[armedbear-devel] Performance of ABCL?
by Žiga Lenarčič 27 Sep '09

27 Sep '09

Hi! I've tested Maxima with ABCL. While it runs, which is rather cool, it's fairly slow! On my computer (with -server) Maxima on ABCL is about 10x - 20x slower than SBCL for various operations in Maxima (and using 3-4x more RAM). I don't know much about implementation details - but as far as I know, every lisp object is encapsulated in a Java object, so there is a lot of allocation going on. Compiler is also very slow (compared to SBCL). My question is, is performance ever increase dramatically (perhaps with java 7?) or is ABCL inherently slower because of the JVM? Clojure performs better than ABCL usually and is also running on JVM. I don't know about KAWA. Is JVM bytecode not flexible enough to perhaps reuse some compilation techniques of native compilers (CCL, SBCL) ? I was thinking about writing a GUI for maxima in java, using ABCL to run Maxima. This way the gui could interact directly with the lisp functions of Maxima. But the main advantage of Maxima is IMO that it can be easily extended via Lisp, which can compile to near C speed on native compilers such as SBCL and CCL. Using ABCL (if it is as slow as it is now), this would not be true and the only fast extensions to Maxima could be written in Java perhaps, which is fine, but is not portable to other CLs and Java code is verbose. Regards, Ziga

6 6

[armedbear-devel] Ugh! java.lang.OutOfMemoryError: PermGen space
by Alessio Stalla 26 Sep '09

26 Sep '09

(dotimes (i 100000) (compile 'f (lambda () 42))) abcl dies with java.lang.OutOfMemoryError: PermGen space if I use nil instead of f it doesn't die. Also with nil is noticeably faster. Adding (when (= 999 (mod i 1000)) (room)) in the loop shows that gc is performed and memory is reclaimed, but apparently there's a leak somewhere. I'll try to find out more. Later, Ale

2 2

[armedbear-devel] In-memory compilation!
by Alessio Stalla 26 Sep '09

26 Sep '09

Hello list, I managed to get temp-file-less compilation with COMPILE working. And the hilarious things is: the classloader experiment was not necessary for this, as the compiler already behaved differently according the the value of *file-compilation*, and the use of loadCompiledFunction that troubled me was only done for the file compiler! So it's sufficient to move file creation from inner functions to outer ones and to pass an additional stream parameter to inner functions. Things can be made clearer - now the structure used to represent compilation results, class-file, is required to always have a pathname set, even when no file is created, for example. But I believe we're on the right track. I cannot provide a patch now because I have still have to clean up the classloader changes. Btw I believe that the use of classloaders is a good thing even if it's not used for memory compilation (for one thing, it would play nicer with serialization of lisp functions, if we ever want that). Cheers, Ale

1 1

[armedbear-devel] Some thoughts on classloaders, inner classes and in-memory compilation
by Alessio Stalla 24 Sep '09

24 Sep '09

Hello folks, I'd like to share some thoughts on the "usual" problem of in-memory compilation in abcl. I haven't written any code to support my thoughts - I just hope to make things clearer for me and other people on the list, and I'm asking you to share your ideas and comment mine. Warning - this is going to be a long post. Let's start by describing how abcl works now. Every piece of code in the JVM needs to be contained in a method of some class, so when abcl compiles a function it produces - surprise - a class, or better, a stream of bytes that the JVM knows how to interpret to create a class. Classes produced by the compiler extend a common abcl class providing the methods that will be used to actually invoke the function; the compiler, among other things, will override some of those methods. When using the runtime compiler, the class is immediately loaded; when using the file compiler, it is stored on the filesystem for later use. So far, nothing requires temporary files to work. However, a Lisp function can contain nested functions introduced with FLET or LABELS, and those have to be compiled to classes as well. Also, they need to be loaded contextually with the main function. How does abcl solve this? By adding instructions in the main class to load the local functions when the class itself is loaded, relatively to where it is loaded from. And here lies the problem. That location is always assumed to be a file in some filesystem subtree. So even when using the runtime compiler all the classes must be, at least temporarily, stored in files for the load machinery to work. Changing this is hard; classes are resolved from strings, how do know when a given string represents a file and when it represents some object in memory instead? There are workarounds, but I think it's the approach itself that's brittle. So let's step back a bit and take a look at how the JVM and Java the language work with respect to loading classes. The JVM uses dedicated objects called classloaders. They are responsible of translating from a class' symbolic name (a string) to a class metaobject, much like the CLOS find-class function does. Classloaders are organized hierarchically: every classloader has a parent which is first consulted to see if it already has the class (there is of course a built-in bootstrap classloader to break the circularity); if it has, it returns it, if it hasn't, it is loaded in a manner dependent to the particular classloader (e.g. from a file, from http, from memory, ...). The process of loading a class from a byte array is native in the JVM, so classloaders only get to decide where the byte array comes from and what it contains. Now to the more interesting things: 1. A class never exists in isolation; to do its work it will need to refer to other classes (at a bare minimum, its superclass and any interface it implements). The JVM - automatically! - uses the same classloader to load a class and, at linking time, all of its dependencies. 2. If I had to manually redo in Java what the abcl compiler does with functions, I'd use static inner classes to represent local functions. Inner classes are classes which are textually defined inside another class and share some data with it. Inner classes do not exist at the bytecode level, only at the Java language level: the compiler (javac) translates them to regular classes, with their name mangled. For example, a class Inner defined inside a class Outer will be referred to Outer.Inner in Java, but compiled to Outer$Inner.class by javac. 3. Inner classes then are treated exactly like the others: referred using strings inside code, resolved by a classloader (generally - always? - by the classloader of the containing class). Return to abcl. As you may have guessed, I propose that we no longer make classes autoload their dependencies, but properly use classloaders instead, in a fashion similar to how inner classes work. We will have an InMemoryClassLoader which will load classes from a Map<String, byte[]>, and a slightly extended URLClassLoader to load classes from the filesystem. Both, in addition to load classes, will be used by the compiler to write classes as well, so it will continue to use the class-file abstraction, changing only the code that actually writes the bytes. Every time the compiler would have written a call to loadCompiledFunction(classname) it will now use something like functionFoo.class.getClassLoader().loadClass(classname) where functionFoo is the compiler-generated name of the class representing the compiled Lisp function. Everything else should stay the same. Does this sound convincing? I admit I have left many things to elaborate on, and I haven't rehashed the code in the compiler, going mainly from memory instead. But I believe this approach has not been proposed before and looks doable. The next few days I'll try writing some sketch code to back up my ideas, if no-one finds any serious problem with them. Peace, Alessio

7 23

[armedbear-devel] [BUG] SET-SYNTAX-FROM-CHAR should clear dispatch tables
by Tobias C. Rittweiler 22 Sep '09

22 Sep '09

(let ((rt (copy-readtable nil))) (set-syntax-from-char #\# #\A rt (copy-readtable nil)) (get-dispatch-macro-character #\# #\x rt)) should signal an error as #\# is no longer a dispatching macro character. -T.

1 0

[armedbear-devel] packaging external libs with abcl
by Matthew D. Swank 21 Sep '09

21 Sep '09

I've been messing with running abcl on the google appengine, and I was wondering what was the best way to package libraries, especially the results of loading an asdf system and its dependencies? Thanks, Matt -- "You do not really understand something unless you can explain it to your grandmother." -- Albert Einstein.

3 8

[armedbear-devel] Bug: Attempt to GO to a tag whose extent has ended quits ABCL
by Brian Mastenbrook 21 Sep '09

21 Sep '09

When attempting to GO to a tag whose dynamic extent has ended, ABCL quits and prints the name of the exception used for internally handling GO (org.armedbear.lisp.Go). I see the same behavior in compiled or interpreted code. CL-USER(1): (lisp-implementation-version) "0.16.0" CL-USER(2): (let ((f nil)) (tagbody (setf f (lambda () (go foo))) foo) (funcall f)) org.armedbear.lisp.Go abcl-src-0.16.0 chandler$ -- Brian Mastenbrook brian(a)mastenbrook.net http://brian.mastenbrook.net/

6 10

[armedbear-devel] [BUG] make-thread and :NAME nil
by Tobias C. Rittweiler 21 Sep '09

21 Sep '09

Not providing a :NAME parameter to MAKE-THREAD will currently fail: CL-USER(1): (threads:make-thread (constantly t)) Debugger invoked on condition of type TYPE-ERROR: The value NIL is not of type STRING. Funnily enough, explicitly providing NIL kind of works: CL-USER(2): (threads:make-thread (constantly t) :name nil) #<THREAD "NIL" {1EEB372}> But, as you can see, the passed NIL is taken as a string designator, but probably shouldn't. -T.

2 1

[armedbear-devel] Your NAMED-READTABLES test
by Erik Huelsmann 20 Sep '09

20 Sep '09

Hi Tobias, The named-readtables test you sent me, does show an obvious bug in ABCL, but not the one you're trying to point me at: when loading your files on Windows, I'm getting an Index out of bounds error: the dispatch tables think our characters range from 0 - 255 in value whereas in practice that number is much higher. I'm working on a fix right now (setting CHAR_MAX to a higher value) but I don't feel too comfortable with it: it'll make us consume much more memory on a number of things. However, what's a K of memory worth these days... ? Bye, Erik.

2 2

[armedbear-devel] JSR-223: Returning immutable lists in ScriptEngineFactory
by Clemens Blamauer 20 Sep '09

20 Sep '09

Hi everyone, I was looking through the JSR-223 code in org.armedbear.lisp.scripting as I noticed that AbclScriptEngineFactory is not fully compliant to JSR-223 [1] (p. 101-102 in the Final Draft Specification, v1.0). For the methods: - java.util.List<java.lang.String> getExtensions() - java.util.List<java.lang.String> getMimeTypes() - java.util.List<java.lang.String> getNames() _immutable_ Lists should be returned. Please find a remedying patch below. Cheerz, Clemens [1] http://jcp.org/en/jsr/detail?id=223 ---- Index: AbclScriptEngineFactory.java =================================================================== --- AbclScriptEngineFactory.java (revision 12157) +++ AbclScriptEngineFactory.java (working copy) @@ -21,6 +21,7 @@ package org.armedbear.lisp.scripting; import java.util.ArrayList; +import java.util.Collections; import java.util.List; import javax.script.ScriptEngine; @@ -44,7 +45,7 @@ public List<String> getExtensions() { List<String> extensions = new ArrayList<String>(1); extensions.add("lisp"); - return extensions; + return Collections.unmodifiableList(extensions); } @Override @@ -89,7 +90,7 @@ @Override public List<String> getMimeTypes() { - return new ArrayList<String>(); + return Collections.unmodifiableList(new ArrayList<String>()); } @Override @@ -99,7 +100,7 @@ names.add("cl"); names.add("Lisp"); names.add("Common Lisp"); - return names; + return Collections.unmodifiableList(names); } @Override -- # Trust in code: # lynx http://lxr.linux.no/linux+v2.6.23.14/arch/i386/kernel/cpu/common.c#L677

2 1

[armedbear-devel] [PATCH] incorporating java stack frames into backtrace
by Tobias C. Rittweiler 20 Sep '09

20 Sep '09

When evaluating (ext:make-thread #'(lambda ())) you get an error "The value NIL is not of type STRING." -- this is a bug in ABCL itself, by the way, but that's not the matter of this mail. The concern of this mail is the backtrace that'd you get: Backtrace: 0: (INVOKE-DEBUGGER #<TYPE-ERROR {B1A201}>) 1: (MAKE-THREAD #<FUNCTION (LAMBDA ()) {15343C2}>) 2: (SYSTEM::%EVAL (MAKE-THREAD #'(LAMBDA NIL))) 3: (EVAL (MAKE-THREAD #'(LAMBDA NIL))) ... The only thing the backtrace reveales is that the error was signalled somewhere in MAKE-THREAD. With the patch of this mail, the backtrace will also contain the relevant Java stack frames for errors that were signalled from within ABCL's Java runtime: Backtrace: 0: (INVOKE-DEBUGGER #<TYPE-ERROR {B1A201}>) 1: org.armedbear.lisp.Lisp.error(Lisp.java:375) 2: org.armedbear.lisp.Lisp.type_error(Lisp.java:390) 3: org.armedbear.lisp.LispObject.getStringValue(LispObject.java:1061) 4: org.armedbear.lisp.LispThread.<init>(LispThread.java:111) 5: org.armedbear.lisp.LispThread.<init>(LispThread.java:39) 6: org.armedbear.lisp.LispThread$3.execute(LispThread.java:909) 7: org.armedbear.lisp.Primitive.execute(Primitive.java:108) 8: (MAKE-THREAD #<FUNCTION (LAMBDA ()) {15343C2}>) 9: (SYSTEM::%EVAL (MAKE-THREAD #'(LAMBDA NIL))) 10: (EVAL (MAKE-THREAD #'(LAMBDA NIL))) Another example: (LENGTH 12) Backtrace: 0: (INVOKE-DEBUGGER #<TYPE-ERROR {814AF5}>) 1: org.armedbear.lisp.Lisp.error(Lisp.java:375) 2: org.armedbear.lisp.Lisp.type_error(Lisp.java:390) 3: org.armedbear.lisp.LispObject.length(LispObject.java:403) 4: org.armedbear.lisp.LispObject.LENGTH(LispObject.java:410) 5: org.armedbear.lisp.Primitives$24.execute(Primitives.java:427) 6: (LENGTH 12) 7: (SYSTEM::%EVAL (LENGTH 12)) 8: (EVAL (LENGTH 12)) -T. -- Diese Nachricht wurde auf Viren und andere gefaerliche Inhalte untersucht und ist - aktuelle Virenscanner vorausgesetzt - sauber. Freebits E-Mail Virus Scanner

4 18

[armedbear-devel] JSR-223: Return immutable lists in ScriptEngineFactory
by Clemens Blamauer 20 Sep '09

20 Sep '09

Hi everyone, I was looking through the JSR-223 code in org.armedbear.lisp.scripting as I noticed that AbclScriptEngineFactory is not fully compliant to JSR-223 [1]. For the methods: - java.util.List<java.lang.String> getExtensions() - java.util.List<java.lang.String> getMimeTypes() - java.util.List<java.lang.String> getNames() _immutable_ Lists should be returned. Please find a remedying patch below. Cheerz, Clemens [1] http://jcp.org/en/jsr/detail?id=223 ---- Index: AbclScriptEngineFactory.java =================================================================== --- AbclScriptEngineFactory.java (revision 12157) +++ AbclScriptEngineFactory.java (working copy) @@ -21,6 +21,7 @@ package org.armedbear.lisp.scripting; import java.util.ArrayList; +import java.util.Collections; import java.util.List; import javax.script.ScriptEngine; @@ -44,7 +45,7 @@ public List<String> getExtensions() { List<String> extensions = new ArrayList<String>(1); extensions.add("lisp"); - return extensions; + return Collections.unmodifiableList(extensions); } @Override @@ -89,7 +90,7 @@ @Override public List<String> getMimeTypes() { - return new ArrayList<String>(); + return Collections.unmodifiableList(new ArrayList<String>()); } @Override @@ -99,7 +100,7 @@ names.add("cl"); names.add("Lisp"); names.add("Common Lisp"); - return names; + return Collections.unmodifiableList(names); } @Override

1 0

[armedbear-devel] Build problem
by Paul 20 Sep '09

20 Sep '09

Can anybody help with this build problem? I'm on MACOSX 10.5.8 trying to build from SVN, which I updated a few minutes ago. Buildfile: build.xml abcl.copy.lisp: [copy] Copying 171 files to /Users/pdo/Lisp/ABCL/build/classes abcl.init: [echo] java.version: 1.5.0_19 [echo] Implementation-Source: ${version.src} abcl.java.warning: abcl.jsr-223.notice: [echo] [echo] Notice: JSR-223 support won't be built since it is not [echo] supported, neither natively by your JVM nor by [echo] libraries in the CLASSPATH. [echo] abcl.compile.java: [javac] Compiling 268 source files to /Users/pdo/Lisp/ABCL/build/ classes abcl.fasls.uptodate: abcl.compile.lisp: [java] Armed Bear Common Lisp 0.15.0 [java] Java 1.5.0_19 Apple Inc. [java] Java HotSpot(TM) Client VM [java] Low-level initialization completed in 1.627 seconds. [java] Startup completed in 7.773 seconds. [java] [java] Caught SIMPLE-ERROR while processing --eval option "(compile-system :zip nil :quit t)": [java] *LISP-HOME* is NIL or invalid. [java] Please set *LISP-HOME* to the full pathname of the directory containing the Lisp system files. BUILD FAILED /Users/pdo/Lisp/ABCL/build.xml:211: Java returned: 2 Thanks, Paul

2 2

[armedbear-devel] [PATCH] Add SYS::*INVOKE-DEBUGGER-HOOK*
by Tobias C. Rittweiler 20 Sep '09

20 Sep '09

The first patch adds SYS::*INVOKE-DEBUGGER-HOOK* which is like *DEBUGGER-HOOK* but is observed by INVOKE-DEBUGGER even when BREAK is called. This obliterates the need to redefine SYS::BREAK in swank-abcl.lisp. A patch for swank-abcl is also attached. -T.

2 1

[armedbear-devel] r12147
by Ville Voutilainen 17 Sep '09

17 Sep '09

Thanks to Douglas Miles, I committed a fix to PRINT-LENGTH.INIT.1 ansi test. I'll commit the rest of Douglas' fixes shortly. Douglas, thanks for the contributions!

1 0