March 2018 - armedbear-devel - mailman3.common-lisp.net

available encodings symbols strangeness; was: How to determine allowable external formats at run time
by Robert Dodier 02 Apr '18

02 Apr '18

Hi, I've made some progress, but ran into a problem. I see that SYSTEM:AVAILABLE-ENCODINGS returns a list of symbols which represent the various encodings that are discovered via Charset.availableCharsets. That's great, but the symbols are apparently not interned in the keyword package, so it makes it somewhat confusing to determine whether a given encoding is on the list. e.g. (member :utf-16 (system:available-encodings)) returns NIL, although looking at the list, you can see that :UTF-16 is there. Confusingly, (mapcar #'symbol-package (system:available-encodings)) shows that all symbols are in the keyword package, but (find-symbol "UTF-16" :keyword) returns NIL. I believe the problem is that availableEncodings in src/org/armedbear/lisp/Stream.java says, in part, (at line 399 in the current version) new Symbol(charset, PACKAGE_KEYWORD) but I think more appropriate would be PACKAGE_KEYWORD.intern(charset) or, at any rate, something to intern the charset symbol in the package, since the Symbol constructor doesn't have that effect. What does anyone think about that? I tried to sign in to Trac to enter a bug report about this but wasn't able. (1) "Login" wants a common-lisp.net user id -- I don't see a method to obtain a user id on the common-lisp.net website. (2) "OpenId login", clicked on Google, got: "Error in discovery: HTTP Response status from identity URL host is not 200. Got status 404". Hope this helps, and all the best. Robert Dodier

2 3

Re: available encodings symbols strangeness; was: How to determine allowable external formats at run time
by Robert Dodier 27 Mar '18

27 Mar '18

On Sun, Mar 25, 2018 at 8:41 PM, Alan Ruttenberg <alanruttenberg(a)gmail.com> wrote: > You can submit a bug report at https://github.com/armedbear/abcl/issues if > you would like. Thanks Alan, I have reported the bug as: https://github.com/armedbear/abcl/issues/82 Perhaps it would be helpful to mention this bug tracker elsewhere. I wasn't able to find it by looking at the common-lisp.net project info, I only found Trac. best, Robert Dodier

2 1

How to determine allowable external formats at run time
by Robert Dodier 25 Mar '18

25 Mar '18

Hi, I'm wondering if there is a way to get the list of allowable values of the :EXTERNAL-FORMAT keyword for OPEN. Is there a way to determine the possible values at run time? I looked into the source code for MAKE-FILE-STREAM but I got a little lost ... perhaps someone can point me in the right direction. What I'm trying to do is to verify whether a given format is acceptable. I am thinking that I can inspect a list to see if the given format is on the list. Another workable solution would be a function which can report whether the given format is acceptable, if such a function exists. Thanks for your help, Robert Dodier

2 1

[CfPart] European Lisp Symposium 2018, 16-17 April, Marbella, Spain
by Didier Verna 14 Mar '18

14 Mar '18

ELS'18 - 11th European Lisp Symposium Centro Cultural Cortijo de Miraflores Marbella, Málaga Spain April 16-17 2018 Sponsored by Brunner Software GmbH, Franz Inc., LispWorks Ltd., and EPITA http://www.european-lisp-symposium.org/ Recent news: - Invited speakers announced (see below) - Registration open. Don't miss the March 25 deadline for the early bird discount! The purpose of the European Lisp Symposium is to provide a forum for the discussion and dissemination of all aspects of design, implementation and application of any of the Lisp and Lisp-inspired dialects, including Common Lisp, Scheme, Emacs Lisp, AutoLisp, ISLISP, Dylan, Clojure, ACL2, ECMAScript, Racket, SKILL, Hop and so on. We encourage everyone interested in Lisp to participate. Keynote speakers: - Vsevolod Domkin -- Lisp in a Startup: the Good, the Bad, and the Ugly - R. Matthew Emerson -- This Old Lisp (a look at Clozure CL) Important dates: - 25 Mar 2018 Early registration deadline - 16-17 Apr 2018 Symposium Programme chair: David Cooper, Genworks Intl., USA Local chairs: Andrew Lawson, Ravenpack, Spain Programme committee: Andy Wingo, Igalia, Spain Christophe Rhodes, Goldsmiths University, UK Christopher Wellons Null Program Canada Irène Durand, LaBRI, Université de Bordeaux, France Julian Padget, University of Bath, UK Ludovic Courtès, Inria, France Michael Sperber, DeinProgramm, Germany Nicolas Neuss, FAU Erlangen-Nürnberg, Germany Olin Shivers Northeastern University USA Philipp Marek, Austria Robert Strandh, LaBRI, Université de Bordeaux, France Sacha Chua Living an Awesome Life Canada Scott McKay, Future Fuel, USA Search Keywords: #els2018, ELS 2018, ELS '18, European Lisp Symposium 2018, European Lisp Symposium '18, 10th ELS, 10th European Lisp Symposium, European Lisp Conference 2018, European Lisp Conference '18 -- Resistance is futile. You will be jazzimilated. Lisp, Jazz, Aïkido: http://www.didierverna.info

1 0

interrupting and destroying threads [internals, help wanted]
by Alan Ruttenberg 02 Mar '18

02 Mar '18

TL;DR; Doesn't seem to work reliably. Interrupts, for example in slime, don't reliably get acted on. Sometimes threads are killed instead of interrupted. Some effort to amend this is discussed below. Background: There are two interrupt systems in ABCL, one defined on org.armedbear.lisp.Lisp, and one on org.armedbear.lisp.LispThread. It is unclear why there are two parallel systems. The interrupt defined on org.armedbear.lisp.Lisp works as follows: Call (interrupt-lisp) A object variable "interrupted" is set. At several places in the interpreter (eval, tagbody) the variable is checked. When compiling code - typically at branch points - code is emitted to check the variable, and if set, handleInterrupts is called. handleInterrupts starts a break loop. It's not clear which thread that will happen in, since there is a delay between when an interrupt is signaled and when the break is called, so you might be in a different thread than the one in which interrupt-lisp is called. This interrupt system is *not* the one used in slime or bordeaux threads. The second interrupt system, which *is* used by slime and bordeaux uses an instance variable on LispThread called "threadInterrupted". One calls (interrupt-thread thread function &rest args) to interrupt. At that point the variable threadInterrupted is set to true and the function is queued for eventual (hopefully prompt) execution. Then the java thread built-in interrupt call is made, which also sets some state in the thread (JVM internal) indicating a request for interrupt. Java's interrupt is checked by Java in some set of internals that wait, such as when sleep is called, or presumably when there is blocking IO, or by explicit checks by user code. When detected by java an InterruptedException is thrown. When doing user checked the user has to throw the exception. It is intended that there are exception handlers to process the interrupts. In several places ABCL explicitly catches the exception and calls processThreadInterrupts which executes the queued set of interrupt functions and should then proceed in whatever it was doing. Presumably the issue with threads dying is when an Interrupted Exception is not handled. Handling of InterruptException happens at only a few points - thread-join, sleep, object-wait. Unlike as with lisp-interrupt the compiler does *not* generate code to check. As a consequence if a thread is doing anything else it will not notice the exception. This leads to a poor interactive experience - slime's control-c often does not work in a timely manner. Handling the interrupt can be tricky if called at the wrong time as there is no guarantee that other lisp state is consistent. Related to this the implementation of destroy thread is suboptimal. Bordeaux threads documents that it is implementation-defined whether unwind-protected forms are handled on destroy. Practically it seems many implementations do, as the package lparallel (highly recommended!) depends on that and implements the ability to kill worker threads on all supported platforms *except* than ABCL. Thread destroying is implemented in a similar manner to interrupts. There is a variable that is set when the destruction is requested, with a single place where it is checked an acted on (beginning of eval). When detected a ThreadDestroyed exception is thrown. ThreadDestroyed is caught at the top of the thread's execution, which seems to explain why unwind protects are not handled. That the only check is in eval means that if you are executing only compiled code the thread will not actually be destroyed. --- As you can see, this is something of a mess. I've made some initial attempts to remedy it but am not confident enough that they are the correct way of doing things, or whether they will work reliably. The changes are: 1) Whenever Lisp interrupts are checked, also check for thread interrupts. There are a few places in the java code that do this and I simply add a check for thread interrupts at those points as well. In addition I modify the code generation so that when checks for lisp interrupts are generated as part of compiled code, I also generate a check for thread interrupts. 2) Don't call JVMs thread.interrupt. The benefit of not calling it is that you remove the possibility that the lisp will be in an inconsistent state when it handles the interrupt. The disadvantage is that you won't be able to interrupt anything that's not in lisp code. 3) Have destroy-thread use interrupt-thread to throw to a new catch tag which is set around the thread run function. There is already a provision for defining a wrapper around a thread's run function - a lisp function called (unsurprisingly) THREAD-FUNCTION-WRAPPER. Currently it has an abort restart handler. However destroying a thread is not necessarily the same as abort. Since the throw works correctly wrt unwind protect and other lisp state, behavior of destroy is predictable - active unwind-protections are run before the thread exits. Note: the ThreadDestroyed exception is never called now. --- The above seem to work "ok" but haven't been extensively tested. Responsiveness to slime's control-c is often fast, and lparallel can kill worker threads. So the first question I have is: Is there something I've missed, or does it seem like the strategy above should work. Is anybody doing thread-heavy work that you could test this against. Having done the above and verified that it passes the smoke test, I'd like to enable using java's thread.interrupt(), so that interrupts can happen even when JVM or foreign java code is running. I've enabled it to see what would happen and things work ok in some cases, but there are gaps. For example, in one case lparallel code complained about working with a lock. So the second question is: What needs to be done to have thread.interrupt be reliable. My current speculation is that the way to handle it is in code generation. The code generation for unwind protect already uses exception catching to for unwind protect and some other cases. One thought is to, effectively, automatically add more cleanup code wherever the compiler generates and (JVM Exception) catch. However the compiler is complicated and I don't understand enough of how it works yet. The differences, as they stand now, can be seen at https://github.com/armedbear/abcl/compare/master...alanruttenberg:thread-in… Comments, suggestions, help, would very much be welcomed. Regards, Alan Ruttenberg

1 0