pro

pro@common-lisp.net

344 discussions

Erlang style processes in Common Lisp
by shenanigans＠sonic.net 27 Sep '15

27 Sep '15

Creators of Erlang have a Lisp background, and one feature of the Erlang VM (BEAM) that I'd like back-ported into Common Lisp is their process. An Erlang "process" is cheap to create, cheap to destroy, cheap when blocked, and upon exit performs bulk gc of its allocated memory; e.g., munmap(). Handling tens of thousands of requests per second per node isn't uncommon, and these often have *several* workers per request or connection: hundreds of thousands of processes. Under such scenarios, anything less than this approach to lightweight processes might suffer from stalls during long gc runs that would be avoided or significantly reduced under Erlang's model. How might we get equivalent cheap ephemeral processes into a contemporary Common Lisp implementation? For those unfamiliar with it, what Erlang means by "process" stems from an implementation of Actor Model with caveats. Each process is more co-routine than thread, and many may run within the same OS thread managed by the Erlang scheduler. The BEAM VM pre-empts processes via "reduction" counting, which may be understood as unit of VM time. Their famed tag line, "Let it crash," may be loosely understood in CL terms as an implicit HANDLER-CASE. The open question here is to address a stated non-goal of CL-MUPROC, "we rely on the CL implementation's MP system" and "considerably heavier than Erlang processes". [See presentation link from https://common-lisp.net/project/cl-muproc/ ] Some Erlang-on-CL packages use OS threads or in the case of Erlang-in-Lisp, fork(). While easier for a library author to implement, these contradict the definition of cheap here. Other such packages punt the issue altogether and instead focus only on pattern-matching aspects of Erlang the language. Then there's Lisp-Flavoured-Erlang, and Elixir offers hygienic macros that manipulate the full AST properly-- all play nice on same Erlang VM at runtime-- for people simply looking to avoid Erlang syntax or semantics. But let's focus on cheap Erlang style processes in Common Lisp here, please. 1. Semantics of library elements are straight-forward enough (e.g., SPAWN, EXIT) and may be borrowed wholesale and safely named within their own package. 2. Memory allocation & garbage collection: Erlang BEAM VM doesn't begin to reclaim memory until very late, so an initial implementation here might assume ephemeral worker processes and omit gc until exit of a worker. However, some processes are long-lived in practice. One compromise might be acceptable: one nursery per process, but anything promoted to higher generations gets handled however your gc does it now. This states nothing about use of shared versus multiple heaps across OS threads, so such matters may continue to be implementation-dependent. 3. Co-routines: For something similar to Erlang's reductions, there would need to be a measure of runtime complexity per process. However, I've used single thread co-routines for near realtime systems in C and CL with a loop of function pointers and ensuring that each referenced function executes under some threshold determined through experimentation and confirmed via test cases. No pre-empting needed. While fragile and not easily portable across different hardware (including same architecture), this may be acceptable for a preliminary draft. Using CL-MUPROC routines as an example and extending MUPROCN: perhaps its body becomes this top-level list of functions from which interleaving across processes may occur. Then add variations for playing well with DO or LOOP semantics. 4. Message-passing: SBCL's sb-concurrency extension with Queue and Mailbox (implementation of "Optimistic FIFO Queue") can be the solution here too. More aligned with CL principles, allowing for multiple mailboxes-- and therefore priority messaging-- would be a welcome advancement beyond Erlang. (Erlang allows only one mailbox per process: sequential but with pattern matching, nothing out-of-band...) An important implementation detail that simplifies gc and increases cache locality across NUMA nodes: messages get duplicated when delivered-- each process only references its own copy! 5. (Almost) nothing shared: Erlang enforces strict prohibition against shared memory, and common practice is to use an in-memory database (ETS) as key/value store in lieu of globals. Scala allows but discourages shared memory. A CL-inspired take on this might be to use SBCL's approach with thread creation: upon creating a new process, you get: A) global special values, but modify at own risk... B) LET bindings are local to each process; C) threads don't inherit dynamic bindings from parent. i.e., http://sbcl.org/manual/index.html#Special-Variables 6. Scheduling processes on OS threads: This is delicate in Erlang, and I've experienced specialized use cases interfering with their scheduler when under heavy load. Instead for our purposes, let the programmer handle the mapping of processes to OS threads. Less is more here. 7. Finally, gen_server & OTP fiends may be implemented as their own packages... or skipped entirely! Thoughts on feasibility? Thanks, -Daniel

11 20

Conditions and *PRINT-READABLY*
by Edi Weitz 07 Sep '15

07 Sep '15

Hi everybody! With the code at the end of this mail I would have expected an outcome similar to this one R: T , E: T - [REFUSED TO PRINT] R: T , E: NIL - [REFUSED TO PRINT] R: NIL, E: T - #<MY-CONDITION 20098B07> R: NIL, E: NIL - My condition occured. because the standard says that "if *PRINT-READABLY* is true, printing proceeds as if *PRINT-ESCAPE* were also true." This would mean that the first two lines should look the same because the value of *PRINT-ESCAPE* is irrelevant in this case. However, while this is what happens for example with LispWorks, AllegroCL, and CLISP, in some other Lisps like SBCL, CCL, and ECL the second line looks like the fourth. Isn't that wrong? Or is their interpretation of the standard that the report function is responsible for obeying *PRINT-READABLY*? But how does that fit with the possibility of using strings for :REPORT? Thanks, Edi. (define-condition my-condition () () (:report "My condition occured.")) (defun test () (let ((error (make-condition 'my-condition))) (dolist (readably '(t nil)) (dolist (escape '(t nil)) (format t "~&R: ~3A, E: ~3A - " readably escape) (handler-case (write error :readably readably :escape escape) (print-not-readable () (write-string "[REFUSED TO PRINT]")))))))

3 2

PRINT-OBJECT for built-in classes
by Edi Weitz 29 Aug '15

29 Aug '15

The following "works" in three different CL implementations I tried while in three others the way complex numbers are printed doesn't change. ? (defmethod print-object ((obj complex) stream) (format stream "#< ~A + ~A * I >" (realpart obj) (imagpart obj))) #<STANDARD-METHOD PRINT-OBJECT (COMPLEX T)> ? #c(1 1) #< 1 + 1 * I > My understanding of 22.1.2 of the standard is that each Lisp MUST have a PRINT-OBJECT method for complex numbers. The question then is whether I'm allowed to redefine it like above (I think I am) and/or whether an implementation is allowed to accept this redefinition without a warning but then to ignore it (which, as I said, is what happens in three respectable Lisps). Thanks, Edi. [Note: There's no point in talking me out of this as I don't actually want to do it anyway. It's just an example and I'm only interested in what exactly is governed by the standard. Complex numbers are also just an example. I'm interested in PRINT-OBJECT for built-in classes in general.]

5 5

Do symbols need to be EQ?
by Edi Weitz 16 Aug '15

16 Aug '15

Just out of curiosity and without any relevance in practise: Is there one place in the standard where it is explicitly said that two symbols which are the "same" symbol must be "identical"? I know that there are a couple of examples where this is implied, but formally the examples aren't part of the standard, right? The EQ dictionary entry for example shows this example: (eq 'a 'a) => true and then it continues with this note (emphasis mine): "Symbols that print the same USUALLY are EQ to each other because of the use of the INTERN function." And the entry for INTERN is actually the closest I could find in terms of clarification because it says that if a symbol of a specified name is already accessible, _IT_ is returned -- which sounds like object identity to me. But how does this fit into the picture? CL-USER 1 > (defparameter *s* 'foo) *S* CL-USER 2 > (unintern 'foo) T CL-USER 3 > (defparameter *s2* 'foo) *S2* CL-USER 4 > (eq *s* *s2*) NIL *S* has lost its home package and is thus not EQ to *S2*, sure, but how do we explain this in terms of object identity? Has the UNINTERN operation changed the identity of *S* which once was the one and only CL-USER::FOO but can't be anymore because this role is now occupied by *S2*? Did I miss some clarifying words in the standard? Did I just manage to confuse myself? Thanks, Edi. PS: The UNINTERN entry warns about side effects which could harm consistency, so maybe this is what they meant?

12 24

Proper behavior of slot-initforms in defstruct?
by Jean-Claude Beaudoin 04 Aug '15

04 Aug '15

Please consider the following code: (defparameter init-a 1) (let ((init-a 42) (serial-no 0)) (defstruct foo (a init-a) (b (incf serial-no))) (defun get-foo-serial-no () serial-no)) (defstruct (bar (:include foo)) (c 33) d) When one loads the above and then try to call #'make-bar the result varies widely from one lisp implementation (clisp) to another. clisp: (make-bar) --> #S(BAR :A 1 :B 1 :C 33 :D NIL) ccl: (make-bar) --> <enter the debugger saying: "Unbound variable: SERIAL-NO"> lispworks, allegro and sbcl also behave more or less like ccl. What is the proper ANSI-CL behavior in this case here? Is clisp right in evaluating the slot initform in its "proper" lexical context? Or is the correct behavior to replicate the slot initform verbatim in the sub-structure constructor regardless of its original lexical context like the others do? I guess that this question has probably been asked before, in a somewhat distant past, but my google skills have not been sharp enough to find it, sorry.

5 9

Caching of effective methods in custom method combinations
by Edi Weitz 21 Jul '15

21 Jul '15

Hi everybody, Here's some behavior I see on SBCL as well as LispWorks, so I'm assuming for now they're right in doing so. My question would be why they are. I start with the following code which I compile and load: ;;;;;;;;;;;;;;;;;;;;;;;; (defvar *current-qualifiers* '(:a :b :c)) (define-method-combination test () ((methods *)) (let ((selected-methods (loop for method in methods when (intersection (method-qualifiers method) *current-qualifiers*) collect method))) `(call-method ,(first selected-methods) ,(rest selected-methods)))) (defgeneric foo (thing) (:method-combination test)) (defmethod foo :a (thing) '(:red)) (defmethod foo :b (thing) '(:blue)) ;;;;;;;;;;;;;;;;;;;;;;;;;; Now, in the REPL I do the following: CL-USER> (foo 42) (:BLUE) CL-USER> (setq *current-qualifiers* '(:a :c)) (:A :C) CL-USER> (foo 42) (:BLUE) I almost expected this. The effective method obviously isn't computed anew but was cached. But even if I now re-evaluate the DEFINE-METHOD-COMBINATION form, (FOO 42) will still return (:BLUE). Only if I re-evaluate the DEFGENERIC form will the return value change to (:RED). My question is if the standard somewhere allows this caching to happen. That would for example mean that any kind of "dynamic method combination" (for example based on the time of the day, just for grins) is impossible. Or am I missing something and my interpretation is wrong? Thanks, Edi.

3 4

Should GC hooks be used to help broken software?
by Elias Mårtenson 06 Jul '15

06 Jul '15

The following question was raised during my development of an asynchronous library <https://github.com/lokedhs/cl-rabbit-async> I'm currently building for RabbitMQ. In summary, the library allows you to create an object of type ASYNCH-CONNECTION, from which instances of ASYNC-CHANNEL can be retrieved. A connection holds a reference to all channels that it uses, and each channel holds a reference to its connection. The connection object has a pointer to a native CFFI object that for the underlying connection (my library is built on the RabbitMQ C API). My question is: Should I use trivial-garbage to create a GC hook for the connection object so that if the user of the library forgets to close the connection, it will get closed eventually once it's GC'ed? I can see arguments for both behaviours: - It's a bad idea, since losing the reference to the connection object means that the program is broken, and silently cleaning up the underlying connection might hide the fact there is a bug (if the GC doesn't run often enough I might have hundreds or even thousands of lingering connections) - On the other hand, it might be a good idea since Lisp developers often use the REPL to experiment, so it's easy to accidentally lose a reference to an object during testing. Thus, using the GC hook will improve the stability of one's development environment. To me, there is no strictly correct answer to the question, which is why I'm asking for suggestions from you guys. Regards, Elias (loke on #lisp)

6 9

cl-ppcre:split question
by Mark H. David 02 Jul '15

02 Jul '15

The examples for cl-ppcre:split here http://weitz.de/cl-ppcre/#split include this tempting recipe: (split "\\s+" "foo bar baz frob") => ("foo" "bar" "baz" "frob") However, (split "\\s+" " foo bar baz frob") => ("" "foo" "bar" "baz" "frob") I.e., the result list includes an empty string as the first element if there are one or more whitespace characters at the beginning of the string. How can I get rid of the empty string at the beginning? Is there a regexp match for this that would work with split? Thanks, Mark

2 1

Heads up on Graphics Ports
by David McClain 08 May '15

08 May '15

Hi all, I just ran into a glitch in the new LW7 for Mac and Windows. Starting some time ago, there is an additional parameter in GRAPHICS-STATE called SCALE-THICKNESS. Up to now, it had defaulted to NIL, and that’s why i never noticed when it was introduced. Starting in LWM7.0 and LWW7.0 it defaults to T. Not sure when scaling thickness to transformed coordinates would make sense. I have scaled things in the past according to zoom state: if you double the window pane size, then thicknesses of lines also doubles in my plotting package. I’m hard pressed to imagine a situation in which I’d ever want line thickness to scale according to the affine transform in effect. And how would that work, when the domain covers (-1 1) while a range covers (10 1000)? At any rate, just a heads up to everyone. Not broken, just different in the new version of LW. 73 de Dave, N7AIG David McClain N7AIG(a)arrl.net

1 0

[CfPart] ELS 2015, April 20-21 2015, London
by Didier Verna 01 Apr '15

01 Apr '15

ELS'15 - 8th European Lisp Symposium Goldsmiths College, London, UK April 20-21, 2015 http://www.european-lisp-symposium.org/ Sponsored by EPITA, Goldsmiths University of London, Franz Inc., Lispworks Ltd., Clozure Associates and Google Recent news: - A few seats left, still time to register! - Programme now online (schedule may still change a little) - Invited speakers announced: Zach Beane, Bodil Stokke, Martin Cracauer The purpose of the European Lisp Symposium is to provide a forum for the discussion and dissemination of all aspects of design, implementation and application of any of the Lisp and Lisp-inspired dialects, including Common Lisp, Scheme, Emacs Lisp, AutoLisp, ISLISP, Dylan, Clojure, ACL2, ECMAScript, Racket, SKILL, Hop and so on. We encourage everyone interested in Lisp to participate. The 8th European Lisp Symposium features 3 invited talks, one tutorial, 4 technical sessions and 2 lightning talks slots. The full programme is now available on the website: http://www.european-lisp-symposium.org/. Programme chair: Julian Padget, University of Bath, UK Local chair: Christophe Rhodes, Goldsmiths, University of London, UK Programme committee: Sacha Chua — Toronto, Canada Edmund Weitz — University of Applied Scicences, Hamburg, Germany Rainer Joswig — Hamburg, Germany Henry Lieberman — MIT, USA Matthew Flatt — University of Utah, USA Christian Queinnec — University Pierre et Marie Curie, Paris 6, France Giuseppe Attardi — University of Pisa, Italy Marc Feeley — University of Montreal, Canada Stephen Eglen — University of Cambridge, UK Robert Strandh — University of Bordeaux, France Nick Levine — RavenPack, Spain Search Keywords: #els2015, ELS 2015, ELS '15, European Lisp Symposium 2015, European Lisp Symposium '15, 8th ELS, 8th European Lisp Symposium, European Lisp Conference 2015, European Lisp Conference '15 -- My new Jazz CD entitled "Roots and Leaves" is out! Check it out: http://didierverna.com/records/roots-and-leaves.php Lisp, Jazz, Aïkido: http://www.didierverna.info

1 0

← Newer
1
...
11
12
13
14
15
16
17
...
35
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

pro