May 2012 - pro - mailman3.common-lisp.net

[pro] style question about types, coercion, expectations for function parameters
by Ryan Davis 01 Jun '12

01 Jun '12

Pro-cl, I'd like to sanity check some of the lisp idioms my shop has (re)invented. Some background: We do a lot of web programming, shuffling data to and fro, occasionally doing interesting calculations, but mostly pushing bits around and displaying information in ways humans can use. We have very few CPU bound operations, and most of those are complicated SQL queries, not lisp operations. Most of the applications we create have few users. As such, we haven't put much time into optimizing our lisp code, and rely on dynamism to support rapid development. We have very few type declarations, liberal use of CLOS, not too concerned with consing, etc. Basically using the simplest code we can get away with. It works well for our purposes, and we complicate code for speed when we can't get away with it. We came to lisp from C#, so have a distaste for that style of static typing. I'd like some more opinions on a pattern that has cropped up. One of the problems we were having was quickly determining what a function expected for it's arguments. As a somewhat contrived example, SLIME helpfully would tell me that #'send-email wanted (to from subject body), but then it was left to me to guess what values I should pass in. In real code this was frequently non-trivial, and we'd be hand-tracing to figure out where the parameter was used to figure out what it should be. Should "to" be a string, a CLOS Client object, or the database ID of a Client? The answer we arrived at was "yes": (defun send-email (to from subject body) (let ((to (etypecase to (string to) ((integer 0) (email (fetch-client to))) (client (email to)) ))) ;; ... more code )) The "to" parameter can be anythings that can be mapped to an email address. It is send-email's job to send email, and it will figure it out based on whatever you provide. If it can't do it, it'll tell you. Usually the etypecase is pulled into it's own function, and we have something like: (defun send-email (to from subject body) (let ((to (coerce-email to))) ;; ... more code )) And also usually wrapped into a setf-generating macro and we have: (defun send-email (to from subject body) (coerce-email! to) ;; ... more code ) This is very easy to follow, and when figuring out what to pass for "to", it's trivial to M-. a few times and see what are the allowed values. In rare cases we want this typecase to be extensible, for example to allow another package to add new mappings. When this happens we end up with a generic method: (defmethod coerce-email (to) ;;same typecase as before ) ;;elsewhere (defmethod coerce-email ((to server)) (administrator-email to)) Method dispatch takes care of the rest. Sometimes the coerce-* function is a cond or typecase, and strays outside the flexibility offered by standard method combination, things like using Access's [1] generic interface. In practice, this is approach is mostly used for converting database CLOS objects to integer IDs and vice versa, depending on whether the function wants an integer ID or an instantiated object. Some alternatives we tried: * using defmethods, but it didn't seem to offer any advantage over a simple typecase, it was just more typing, more code to read later, and less flexibility if the business wanted something weird. * using check-type or assert to constrain the input options, like C#/Java style static-typing, but that just resulted in duplicate code at call sites to convert from whatever you had to whatever send-email wanted. * having multiple similarly named functions where the function name indicated what type was expected (eg: send-email, send-email-to-client, send-email-to-client-id), this also didn't seem to offer any advantage over a simple typecase * naming parameters after what can be accepted (client-id, email-string, client-or-id, etc) ended up hard to keep up to date. There are a lot of tradeoffs with this approach, and obviously this wouldn't work for anything with high performance requirements. Has anyone else run into a similar problem and come up with a different/same/better solution? Any problems I'm missing? [1] https://github.com/AccelerationNet/access Thanks, -- Ryan Davis Acceleration.net Director of Programming Services 2831 NW 41st street, suite B Gainesville, FL 32606 Office: 352-335-6500 x 124 Fax: 352-335-6506

5 4

[pro] ANNOUNCE: Docbrowser (and WAT - Whatever Active Templates)
by Elias Martenson 30 May '12

30 May '12

I'm making this post here because my project, Docbrowser, was included in the latest update of Quicklisp, and I wanted to give a heads-up to anyone who may be interested in the project. Docbrowser is a documentation browser for Common Lisp. It uses Hunchentoot to provide a simple way of getting an overview of the functions, classes and variables). The source code is available at http://code.google.com/p/docbrowser/, or you could just install it via Quicklisp. The project also contains a template system that generates Lisp code that is subsequently compiled. It's quite effective, and while it's not an "official" project on its own right (yet), I just wanted to get the word out in case anyone is interested in looking at it. Regards, Elias

1 0

[pro] Default behavior of standard output streams
by Didier Verna 26 May '12

26 May '12

Hello, there is a discrepancy amongst compilers on how the default output streams behave in some situations. Consider the following test case: (format *standard-output* "This goes to standard output.~%") (format *error-output* "This goes to error output.~%") (format *query-io* "This goes to query io.~%") Now, launch you favorite Lisp on a terminal and load this file at startup in 3 different ways: myfavoritelisp --the-load-option test.lisp myfavoritelisp --the-load-option test.lisp > log myfavoritelisp --the-load-option test.lisp > log 2>&1 SBCL and CMUCL behave in a way which I think is right: case #1: everything is printed on the terminal case #2: stdout goes in the file case #3: stdout and stderr go in the file (*query-io* stays on the tty) ECL redirects the output of *query-io* in the file for cases #2 and #3. The behavior of stdout and stderr is the same as in SBCL and CMUCL. Finally, LW, ACL, CLISP, CCL and ABCL redirect everything in the file in cases #2 and #3. I'm ready to fill in bug reports for the last 6 compilers, but are these really bugs? I'm convinced that it is for stderr, but it may be arguable for *query-io*. WDYT? -- Resistance is futile. You will be jazzimilated. Scientific site: http://www.lrde.epita.fr/~didier Music (Jazz) site: http://www.didierverna.com

7 6

[pro] CDR proposal: compiled-file-p
by Sam Steingold 15 May '12

15 May '12

In 2004 I submitted a CLRFI which was then accepted as CLRFI-2. It specified a new function COMPILED-FILE-P which was supposed to be useful for system construction utilities, e.g., asdf: Function (compiled-file-p file-name) ==> valid-p Returns true if the file appears to be a valid compiled file (i.e., exists, is readable, and the contents appears to be valid for this implementation), false otherwise. Implementations are required to inspect the contents (e.g., checking just the pathname type is not sufficient). Although the completeness of the inspection is not required, this function should be able to detect, e.g., file format changes between versions. I re-submitted the spec to the CDR editors a week ago. No response so far. The detailed html spec is attached. -- Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000 http://www.childpsy.net/ http://mideasttruth.com http://ffii.org http://www.memritv.org http://jihadwatch.org http://camera.org http://memri.org You cannot fire me. Slaves are not fired. Slaves are sold.

3 7

[pro] #; comments...
by Pascal Costanza 15 May '12

15 May '12

Hi, I'm thinking about writing up a CDR on #;-style comments. Currently, if you want to comment out a form from CL code, there is no standard way of doing this. Many people use something like #+nil, or similar tricks, but this is not 100% reliable. R6RS Scheme introduced #; as a solution, and this seems to become somewhat popular in CL as well. Here is a suggested wording. ------------- snip ----------------- #; is used to comment out single expressions; the syntax is /#; expression/. This textual notation is treated as whitespace; that is, it is as if the ``#; expression'' did not appear and only a space appeared in its place. #; operates by skipping over the form. Skipping over the form is accomplished by binding *read-suppress* to true and then calling read. ------------- snip ----------------- The text is based on 2.4.8.17 of the HyperSpec. Yes, this would be a fairly short CDR document. Any comments or ideas about this? Best, Pascal -- Pascal Costanza

20 33

[pro] return-with-dynamic-extent...
by Pascal Costanza 13 May '12

13 May '12

Hi, Here is a description of what I think is a feature missing in Common Lisp, both the standard and current implementations, namely a more complete way to ensure stack allocation of objects. Based on some recent experience with translating some C++ benchmarks to Common Lisp - more specifically the fluid animate benchmark from the PARSEC benchmark suite - being able to allocate objects on the stack can be a major performance win. Unfortunately, this is harder than necessary to achieve in Common Lisp. Here is a simplified example. Assume you have the following class definition in C++: struct vec { int x; int y; int z; vec(int x, int y, int z): x(x), y(y), z(z) {} }; Then you can define the following operations on such a class: vec vplus(vec v1, vec v2) { vec v(v1.x+v2.x, v1.y+v2.y, v1.z+v2.z); return vec; } In such a function definition, the local variable v is stack allocated, plus the return value is returned by value, not by reference. This holds for any intermediate values as well, so if you have other operations like vminus, vmul and vdiv, etc., defined accordingly, you can write expressions like this: vplus(v1, vmul(v2, v3)); …and rest assured that all intermediate values are allocated on the stack as well. (C++ loses some performance in some cases because it can happen that it relies too much on copying of data structures, but this has been largely resolved in C++11, where move operations are supported really well.) In Common Lisp, you can declare local variables with dynamic extent: (let ((v (make-vec :x 1 :y 2 :z 3))) (declare (dynamic-extent v)) …) This has the effect that (in Common Lisp that support this) the struct that v refers to is stack allocated. Unfortunately, Common Lisp implementations can only do this for data structure constructors that the implementation knows about - there is no way for users to add additional operations to the set of known dynamic extent 'allocators.' For example, if you say the following: (defun vplus (v1 v2) (let ((v (make-vec :x (+ (vec-x v1) (vec-x v2)) :y (+ (vec-y v1) (vec-y v2)) :z (+ (vec-z v1) (vec-z v2))))) (declare (dynamic-extent v)) v)) …you can be very sure that the resulting program will fail, because the reference to the stack allocated data structure is returned, but the data will be gone afterwards. This had the consequence that in our attempts at translating the fluid animate benchmark, we had to introduce all intermediate results explicitly in separate variables, and be very explicit about all the necessary computational steps. This was very tedious and the result is not nice to look at. What would be nice is, if we could instead have some form of saying that we want to return a data structure on the stack: (defun vplus (v1 v2) (let ((v (make-vec :x (+ (vec-x v1) (vec-x v2)) :y (+ (vec-y v1) (vec-y v2)) :z (+ (vec-z v1) (vec-z v2))))) (declare (dynamic-extent v)) (return-with-dynamic-extent v)) The idea is that now, the data that v refers to is returned by value on the stack. If the call site receives the value with a matching dynamic-extent declaration, then v can just stay on the stack. Same (hopefully) for intermediate results in more complex expressions. Just like dynamic-extent declarations, an implementation should be allowed to ignore return-with-dynamic-extent and replace it with a mere return form. To be able to support move operations on the stack, it may be interesting to specify that eq and eql are not reliable anymore for data that is return by return-with-dynamic-extent. I presented the idea at ELS'12 in Zadar, and Christophe Rhodes commented that it may also be necessary to have some form of global declaim that states that a function will always return something with return-with-dynamic-extent, so it's easier for the compiler to know what to do with a return value. I don't claim that I have completely thought this through, but I think this should be doable and shouldn't have too many negative consequences for the rest of a system that doesn't use dynamic extents. I won't be able to implement this, but I just wanted to make sure that the idea is on the table, so someone who is interested may pick it up… Pascal -- Pascal Costanza The views expressed in this email are my own, and not those of my employer.

3 6

Re: [pro] pro Digest, Vol 20, Issue 2
by Devon Sean McCullough 07 May '12

07 May '12

> From: Steve Haflich <shaflich(a)gmail.com> > ... #+never #+notyet #+nomore or #+bug456 I myself have long used, e.g., #+| When bug 456 is fixed | Peace --Devon P.S. #+nil to unconditionally disable code would be an affront to NIL, the New Implementation of Lisp, which runs on the ancient DEC Vax VMS OS. My friend GSB was lead implementer.

1 0