- armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] Summarizing "In-memory compilation status"
by Erik Huelsmann 09 Sep '09

09 Sep '09

As pointed out, Alessio added some primitives required to add in-memory compilation. However, the rest of the thread seems to mix a number of issues and their progress. As a result, I thought a summary might be in order. There were several things "in-memory" versus "using files": 1. Compilation of functions in our JSR-223 support (Alessio changed ABCL's behaviour here) 2. Compilation of lisp forms into compiled functions 3. Compilation of lisp files into fasls With respect to (1): This is where Alessio changed ABCL's behaviour to use COMPILE instead of COMPILE-FILE, eliminating the use of a single temporary file. With respect to (2): Currently, ABCL uses a temporary file for each "compiland" (FLET/LABELS defined function, LAMBDA, DEFUN) in the forms to be compiled. This file is created with a ".cls" extention. After it's written, it gets loaded into the running jvm by loading the bytes in a byte array. That byte array is then resolved to a class. ABCL uses its own class loader to do so. Because the the bytes get written and read without this being required by the Java class loader, there should not be a need to write the bytes out to a file. After the file has been read into memory, the temporary file gets deleted. With respect to (3): Each "compiland" in a file to be compiled leads to a ".cls" file, which is created as a temp file, then read into memory and after that written into the ".abcl" file - which as noted is in fact a .zip file. The intermediate temp file is then deleted. This process too takes much more I/O (what if you're compiling to a network drive with this kind of I/O?!) then required. It's with respect to items (2) and (3) that I'd like to see things changed. In case of (2) there's no reason to send output to any type of disc, while in case (3) it would be beneficial to write our output straight into the ZIP file, instead of using intermediary files. I'm looking for a good solution to both cases. One of the things I came up with is some kind of "storage factory" which would be a function to be called which either generates a stream sending output to the ZIP, or an in-memory stream or a file-backed stream if that's still required. So, what are your comments to that? Bye, Erik.

3 2

[armedbear-devel] [OT] best way to structure and deploy a project
by Alessio Stalla 09 Sep '09

09 Sep '09

Hello everyone, I have a question which is not exactly pertinent with the development of ABCL, but which imho can start a useful discussion. Please forgive me if this is too OT. I have a project with sources in two languages, Java and Lisp. It is an Eclipse project, but the IDE does not make difference. The project is a library, not an end-user application. Currently it is structured as follows: src/java - Java source code src/lisp - Lisp source code lib/ - third-party libraries, both Java (jars) and Lisp (directories). The Java code is compiled by the IDE and put in an output directory. The Lisp source files are treated by the IDE as normal, non-source files, and copied verbatim into the same ouput folder as the Java classes. When I run the project from the IDE, I launch a class which compiles and loads the Lisp systems using asdf. This works for me locally, but thinking about distributing the project to users, this approach has some drawbacks: - Lisp files must be true files, they can't be in a Jar (asdf doesn't understand jars). - Lisp files are recompiled every time, since the IDE always overwrites the Lisp source files in the output directory. - If I make a jar, it will contain Lisp source files, not binaries, since binaries are unknown to the IDE. Ideally I'd like to do what ABCL does, i.e. create one big jar containing .class files and compiled Lisp files. Lisp sources could be compiled by the IDE or by a script, I don't care. Still, I would like to maintain the use of asdf (since some of my dependencies use it and loading them manually is too much of a hassle). I don't know how I can do this. I have also thought about leaving everything as-is, and when I'll distribute the library as a jar, have it extract at runtime the Lisp source files from the jar in a temporary directory and instruct asdf to search for systems in that directory. I will also need the lib/ folder to be in a known place, since asdf has to load stuff from there too (but this is less of a problem since I can make the library user configure that place, and use a sensible default otherwise). I am wondering if a cleaner approach exists, and if someone has experience in this regard. Cheers, Alessio

7 19

[armedbear-devel] In-memory compilation status
by John Pallister 08 Sep '09

08 Sep '09

Hello list, I'm interested in trying to get ABCL running (and preferably REPL-ing & compiling) on a certain cloud computing platform that has a JVM with no access to a file system. The Wiki page for JSR-223 and the sample code both mention that "compilation using the runtime compiler has been removed due to inconsistencies with evaluation and file-based compilation", since May '09. Browsing the mailing list archives suggests that people have been working on removing the need for temporary files since then, but I can't find any mention of "inconsistencies". The thread "Save-image w/Serialization - progress report" from April ends with Sig. Stalla in the process of implementing a byte-array-output-stream, then silence... Can someone comment on the state of runtime compilation, preferably entirely in-memory without reference to a file system? Basically I'd like to be able to compile stuff to a byte array, store the array as a blob in a memcache and/or other datastore, then pull it out again as required and load it via a custom classloader. Is this doable, or would it be easier to port ParenScript to Clojure (which is currently my Plan B)? Cheers, and it's great to see the ABCL project still going strong! John :^P P.S. If anyone is going to be in Hamburg at the weekend and wants to chat about this stuff, that'd be great! -- John Pallister john(a)johnp.net john(a)synchromesh.com

3 9

[armedbear-devel] Passing FORMAT.C.4A and FORMATTER.C.4A
by logicmoo＠gmail.com 06 Sep '09

06 Sep '09

Passing FORMAT.C.4A and FORMATTER.C.4A Index: src/org/armedbear/lisp/LispCharacter.java =================================================================== --- src/org/armedbear/lisp/LispCharacter.java (revision 12140) +++ src/org/armedbear/lisp/LispCharacter.java (working copy) @@ -32,6 +32,8 @@ */ package org.armedbear.lisp; +import java.util.HashMap; +import java.util.Map; public final class LispCharacter extends LispObject { @@ -44,7 +46,7 @@ } public final char value; - + private String name; public static LispCharacter getInstance(char c) { try @@ -256,8 +258,11 @@ case 127: sb.append("Rubout"); break; - default: - sb.append(value); + default: + if (name!=null) + sb.append(name); + else + sb.append(value); break; } } @@ -541,6 +546,8 @@ public static final int nameToChar(String s) { String lower = s.toLowerCase(); + LispCharacter lc = namedToChar.get(lower); + if (lc!=null) return lc.value; if (lower.equals("null")) return 0; if (lower.equals("bell")) @@ -605,7 +612,8 @@ case 127: return "Rubout"; } - return null; + if (c<0 || c>255) return null; + return constants[c].name; } // ### char-name @@ -627,6 +635,39 @@ return Character.toUpperCase(c); } + static int maxNamedChar = 0; + static Map<String, LispCharacter> namedToChar = new HashMap<String, LispCharacter>(); + + static void setCharNames(int i, String[] string) { + int settingChar = i; + int index = 0; + int stringLen = string.length; + while(stringLen-->0) { + setCharName(settingChar,string[index]); + index++; + settingChar++; + } + if (maxNamedChar<settingChar) maxNamedChar = settingChar; + } + + static void setCharName(int settingChar, String string) { + if (settingChar>=CHAR_MAX) return; + LispCharacter c = constants[settingChar]; + c.name = string; + namedToChar.put(string.toLowerCase(), c); + } + + static { + new CharNameMaker0(); + } + + static class CharNameMaker0{ + { + setCharNames(0,new String[]{"Null", "Soh", "Stx", "Etx", "Eot", "Enq", "Ack", "Bell", "Backspace", "Tab", "Newline", "Vt", "Page", "Return", "So", "Si", "Dle", "Dc1", "Dc2", "Dc3", "Dc4", "Nak", "Syn", "Etb", "Can", "Em", "Sub", "Escape", "Fs", "Gs", "Rs", "Us"}); + setCharNames(128,new String[]{"U0080", "U0081", "U0082", "U0083", "U0084", "U0085", "U0086", "U0087", "U0088", "U0089", "U008a", "U008b", "U008c", "U008d", "U008e", "U008f", "U0090", "U0091", "U0092", "U0093", "U0094", "U0095", "U0096", "U0097", "U0098", "U0099", "U009a", "U009b", "U009c", "U009d", "U009e", "U009f"}); + } + } + private static final char[] UPPER_CASE_CHARS = new char[128]; static

1 0

[armedbear-devel] [ANNOUNCE] ABCL 0.16.0
by Erik Huelsmann 06 Sep '09

06 Sep '09

On behalf of the developers of ABCL (Armed Bear Common Lisp) - a lisp implementation running on the JVM - I'm glad to be able to announce the 0.16.0 release. This release features - among lots of other things - performance improvements, better type checking for the THE form and ANSI tests fixes. You can find the release notes at: http://common-lisp.net/project/armedbear/release-notes-0.16.shtml If you have questions regarding its license or use, or you find issues, please report back to the development list: armedbear-devel at common-lisp dot net The sources can be downloaded in ZIP or .tar.gz form from SourceForge: http://common-lisp.net/project/armedbear/releases/abcl-src-0.16.0.tar.gz http://common-lisp.net/project/armedbear/releases/abcl-src-0.16.0.zip Signatures are available under: http://common-lisp.net/project/armedbear/releases/abcl-src-0.16.0.tar.gz.asc http://common-lisp.net/project/armedbear/releases/abcl-src-0.16.0.zip.asc Bye, Erik.

1 0

[armedbear-devel] Branching 0.16.x
by Erik Huelsmann 04 Sep '09

04 Sep '09

With the recent changes to our Ant build to create ZIP files with CRLF line endings and LF line endings in the TAR archive, I think we have enough basis to create a branch for release management of our 0.16.x release line. Should further fixes be required, they can be backported. Creating the release branch now frees up the trunk for further development. Are there any last minute fixes that need to be applied? From the silence on the list this last week I concluded there probably are not. I hope to be able to create a release before the end of the weekend, however, no guarantees! Bye, Erik.

2 1

[armedbear-devel] eol-style problem with abcl-src-0.15.tar.gz
by Mark Evenson 01 Sep '09

01 Sep '09

Somehow we seemed to have shipped [the abcl-0.15.0 source][1] with a CR+LF file-ending for 'abcl.in', which means when built under a 'NIX one gets the slightly mysterious message "#!/bin/sh -- interpreter not found. It is slightly surprising that we haven't gotten more complaints on this (or maybe everyone builds from SVN?). The svn:eol-style is now set to 'LF' in SVN for 'abcl.in', which does the "right thing" when building the source tar balls, so abcl-0.16 should be ok, as long as whoever builds the release doesn't somehow override this is local SVN settings Looking through the SVN log for metadata changes to 'abcl.in' doesn't seem to report anything. Maybe there is somehow to report metadata changes, but I don't see how to do this quickly. We have the currently unused ability to fix line endings in the relevant Ant targets. Maybe I should revisit them? If so, what would be the correct policy? [1]: http://downloads.sourceforge.net/armedbear-j/abc/abcl-src-0.15.0.tgz -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."

2 4

[armedbear-devel] Release planning: two month intervals, but which months?
by Erik Huelsmann 31 Aug '09

31 Aug '09

Last year we set a bi-monthly release planning. Last spring, we released once or twice within 6 weeks. This summer it took me - due to holidays - more time to get our trunk into a releasable state. Or so it must have looked from the outside. What I should have communicated is that I'd like to set our bi-monthly release planning to release around the first weekend of the odd-numbered months. Why? Well, it allows the release schedule to continue without suffering from most of the holidays: summer holidays (at least mine) are mostly during august; Christmas isn't around a first weekend and October holidays are rarely close to the first weekend of November. This means that February holidays may interfere with our release planning, but if I look at school holidays, these are mostly in the middle or the end of February. This plan would also mean that a release should be cut around next weekend. Then there's the issue of me being the only one with the right access to SourceForge to upload ABCL release files. We can do two things about that: 1. Give others upload access to SourceForge 2. Host our downloads on c-l.net The second option would make any committer a release manager, if we decided to do that [because we would commit the releases to our repository, which automatically updates our site]. The reason our downloads are currently hosted on SF is that they provide nice download stats. If we decide we don't need those or that we can create those stats ourselves on c-l.net, there's no reason to continue with SF. What are your comments? Bye, Erik.

2 2

[armedbear-devel] BLOCK-NODE -> specific node types done; let's go for a release
by Erik Huelsmann 31 Aug '09

31 Aug '09

Hi everybody, Over the past weeks, I've been rewriting our generic use of BLOCK-NODE structures to specific structures for every node type. There are TAGBODY-NODEs, CATCH-NODEs, etc. Although I introduced that change for a reason: I want to be able to analyse the nesting set up by pass1 from outer to inner blocks. This gives us some opportunities in preventing functions from becoming closures. Currently every function which is enclosed in a block which defines a closure, becomes a closure. That's too generic and doesn't need to be. Well, as I said, even though I want to introduce changes to that extent, I'm done changing the existing structure. This means I think we're in shape to go for a release. Everybody please run your test suite and your own applications to see if we still meet the quality levels we've delivered in the past releases (or, preferably: better!) I've updated changes, but there are probably a lot more tasks to be addressed. Just to name one: did 'build.xml' get fixed for releasing with Java 1.5? Or should I use 1.6 next time? Bye, Erik.

3 5

[armedbear-devel] BUG: problem with THE type checks in compiler
by eric.marsden＠free.fr 31 Aug '09

31 Aug '09

Hi, There is a problem in the recent improvements to the way the compiler handles THE type checking. File-compiling the code below leads to an error "The value 6 is not of type FOO". (defconstant +foo-OK+ (the (integer 0 10) 5)) (deftype foo () '(integer 0 10)) (defmacro assemble-foo (x) `(the foo ,x)) (defconstant +foo+ (assemble-foo 6)) Eric

3 4