September 2010 - pro - mailman3.common-lisp.net

[pro] The Best Examples of "Code is Data"
by Kazimir Majorinc 28 Sep '10

28 Sep '10

As first, good luck with this list! I'm in search for best examples of "code is data" paradigm in Common Lisp. For most CL-ers, it probably means "macros", but eval, backquote, anything that processes the code as data is of interest. As "best" I think on the most surprising, powerful, sophisticated examples, not necessarily of a pedagogical value. Imagine that someone invited you to write the presentation "Five best CL macros ... I seen" or "Five best CL macros ... I wrote." What would you chose and why? Kazimir Majorinc

26 60

Re: [pro] Fwd: The Best Examples of "Code is Data"
by Vladimir Sedach 28 Sep '10

28 Sep '10

Kazimir Majorinc: > formulas. So far so good. However, some essential operations on formulas > (substitution, unification, rules or inference ...) require analysis of > the formula. > > Similar analysis of the function could be done only by transforming it > back into formula with (caddr (function-lambda-expression Si)), and it > is not simpler, even if it work, what is not guaranteed according to > Hyperspec. This example is confusion about phase separation (http://axisofeval.blogspot.com/2010/07/whats-phase-separation-and-when-do-y…) but more relevantly it presents a false dichotomy for the solution space and perfectly demonstrates the power of macros. There is no need to choose between EVAL and lambdas - make the formula operators be macros and the formulas will know how to analyze (macro-expansion time) *and* evaluate (result of macro-expansion) themselves - the formula interpreter falls out "for free" from doing a COMPILE on a lambda that binds the formula free variables. As a bonus, now the code is factored such that all the analysis code is grouped with the definition of the relevant operators, instead of being scattered around inside an interpreter. A similar example came up in an ll1 mailing list discussion on macros vs closures (http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02060.html - recommended reading as there is also discussion of good macros), but using SQL query languages as the domain. CLSQL, for example, does it with macros, which enable all kinds of compile[macro-expansion]-time analysis and optimization. Smalltalkers do the same thing with closures and using #doesNotUnderstand: (which is sort of like re-binding APPLY): http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02096.html I wrote about how this technique might be used in Common Lisp: http://carcaddar.blogspot.com/2009/04/closure-oriented-metaprogramming-via.… I think the macro approach to queries is more succinct and powerful (you are no longer limited to function calling syntax for your query DSL). The query DSL is something that a lot of people seem to like - LINQ in .NET attempts to do the same thing, but a much higher price (an new syntax for expressing queries, a new protocol for query consumers (database interfaces), and a horribly complicated implementation). The other really powerful example pointed out in the ll1 discussion was SETF. I think SETF is one of the best examples of macros around. It takes the idea of accessors, and turns them into a generic idea of "places" that can be modified in the same way they are read. This extends the language with an entirely new domain concept that works transparently everywhere, with absolutely no changes needed to any of its other parts. There are ways to fake "places" with closures (ex: Oleg Kiselyov's "pointers as closures" trick, or the way dynamic variables work in SRFI 39), but they work by completely changing the accessor protocol, introducing a second protocol for updating places (funcall the closure with new value), and because the "directions" to the place are not reified they are not composable (ie - you can't do (setf (gethash foo (aref bar)) baz) with closures), and cannot be implemented efficiently. Now think about what it would take to do this in Java - you'd need a whole new DSL implemented on top of the Interpreter pattern, a Factory to help you produce the grammar of that DSL, all with a Strategy component so you can define new places. I bet it would be really pleasant to use as well. Vladimir

4 3

[pro] Anaphoric macros (Was: The Best Examples of "Code is Data")
by Daniel Gackle 28 Sep '10

28 Sep '10

Attila's experience matches mine: anaphoric macros are useful because they enhance readability. (I mean, of course, when they're used properly, but that's such a tiresome caveat to have to add and I'm hoping one benefit of this list is that we don't have to jump through *all* the usual hoops.) Given this experience I find it hard to fathom the critique of anaphoric macros that they are bad for readability. Plausible reasons for disagreement are (a) we've had different experiences; (b) we have different tastes; or (c) we're talking about different things. Perhaps the best approach in every case is to make our terms explicit, so at risk of overspecifying I'm going to be as explicit as I can. "Anaphoric macros" for me nearly always means AWHEN or AIF, very occasionally ACOND and that's about it. When I say these "improve readability" I mean that they make code shorter in an easily grokkable way. "Used properly" means "used in a small snippet": when their scope grows beyond the nearly trivial, their benefit -- concision -- tends to get diluted to the point of why bother. Worse, the meaning of IT gets exponentially harder to grok the further IT strays from ITs anaphoric introduction. Here are two simple examples chosen nearly at random from the Skysheet code (Skysheet is the web-based spreadsheet startup I'm working at): (awhen (search "Version/" str) (after-slash it)) (awhen (getf sh :col-deltas) (alist->hash it)) AWHEN makes these forms slightly more trivial than they would be without it. I like that: trivial is second only to non-existent as a good thing for code to be. Here are two more. These are about as complex as I'd ever like an anaphoric macro to get: (awhen (next-as-binary-op) (when (>= (precedence it) level) (consume) it)) (aif (implicit arg) (cond ((eq it :column) +abs-row+) ((eq it :row) +abs-col+) (t +abs-abs+)) (aim-rank (getf arg :aim))) (That COND would be better as a CASE, but it has to run in both CL and JS via Parenscript, and CASE doesn't map to an expression in the latter.) In all four examples I prefer AWHEN/AIF to an explicit binding because it makes the code shorter and more idiomatic. I guess you'll have to take my word for it that the concept temporarily denoted IT is in each case obvious from context and would not get any obviouser with an explicit name. A couple of general reflections. Readability is hard to talk about because it's so in the eye of the beholder. My own views on what's readable have changed a lot. They even vary per-program. In the end I find the definition is a cultural thing that has to get hammered out on a per-project basis by people who are working together (and needs a certain amount of re-hammering with each new person). Finally, I hesitate to post the above because anaphoric macros are kind of a bike-shedding topic: not that interesting but easy to have an opinion about, so they get argued about more than they deserve. But I think it's a useful exercise occasionally to get right down to the threading on the screws. Too often in abstract discussions about programming I find myself wondering what people really mean; this experience gets worse as the subject matter gets more meta. Perhaps anaphoric macros are worth talking about *because* they are so simple. They're a chance to practice talking concretely about abstraction. Dan Gackle On Mon, Sep 27, 2010 at 3:56 AM, Attila Lendvai <attila.lendvai(a)gmail.com>wrote: > > macros that are useful and should be allowed. It is just that > > anaphoric macros, despite all their cuteness, tax code readers > > unnecessarily hard and thus have not been adopted as acceptable > > practice in multi-programmer environments. > > > well, it's one opinion. > > there's at least one counter example in our team of 4. and i > personally do like reasonable usage of anaphoric macros, even when > done by my colleagues. and the reason is exactly that it *rises* code > readability for us. > > our practice includes special coloring for "it" and avoids any usage > which is not blatantly trivial. > > -- > attila > > PS: argh, i so much didn't want to join such a subjective > discussion... i need to meditate on this a bit more to successfully > resist next time. the first exercise will be to let go my opinion of > with-nesting... :) > > _______________________________________________ > pro mailing list > pro(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/pro >

1 0

[pro] Fwd: The Best Examples of "Code is Data"
by Raymond Wiker 25 Sep '10

25 Sep '10

---------- Forwarded message ---------- From: Raymond Wiker <rwiker(a)gmail.com> Date: Fri, Sep 24, 2010 at 2:51 PM Subject: Re: [pro] The Best Examples of "Code is Data" To: Kazimir Majorinc <kazimir(a)chem.pmf.hr> I do not think that this is a case where eval is needed, and it is not a case where eval is likely to be either the best or the simplest solution. Instead of using list-construction operations to construct a "function" to be eval'd, you can use the function-construction operator (lambda, that is). In addition to being more conventional (simply by not using eval), it also gives you a mechanism for passing parameters into your functions without resorting to specials. Two mechanisms, in fact - you can use ordinary parameters and closed-over variables. On Thu, Sep 23, 2010 at 9:53 PM, Kazimir Majorinc <kazimir(a)chem.pmf.hr> wrote: > OK, but both of you paid more attention to the additional niceties or > advantages, unspecified in the problem, than to confirm or reject the > claim that eval is the best solution for problem under specified > criteria "simpler is better." > > It looks to me as implicit claim "Yes, eval is the best solution of that > problem, but if the problem is developed further, other approaches (like > few mentioned) will (soon?) prevail." > > Is it right?

2 1

[pro] substitution macro lisp extension?
by Mark H. David 22 Sep '10

22 Sep '10

Is there any well-known implementation of a macro similar to old Lisp Machine's defsubst, but not as fancy, one that simply emits a macro after doing a reasonable, if less than perfect, attempt to do substitutions and guarantee first-to-last arg once-and-once-only evaluation? I realize that most people just use declare inline for this kind of thing. Or define a macro like defsubst just to emit a defun with a declare-inline declaration. I would like something that's different: something that always emits a macro def given a function-like semantics for args and body. It's a pretty simple thing, but I was wondering if it's sitting around in a library somewhere. What's out there? Thanks, Mark

7 8

Re: [pro] The Best Examples of "Code is Data"
by Daniel Weinreb 09 Sep '10

09 Sep '10

Scott L. Burson wrote: > On Thu, Sep 9, 2010 at 8:26 AM, Daniel Weinreb <dlw(a)itasoftware.com> wrote: > >> One of the very first things you want when >> you start writing macros is for auto-indentation >> to do the right thing. The IDE can't just >> know this for a macro. >> > > Huh. I always thought Zmacs did an adequate job of this. Just know > what &BODY means and give me COM-INDENT-DIFFERENTLY for special cases, > and I'm pretty happy. > Yes, Zmacs knew about &rest, as does our stuff at ITA. That automatically notes that the symbol name is of the form (a b c ... . rest), usually just (a .rest). > (Dan will of course know what I'm referring to, but for everyone else, > the Zmacs command COM-INDENT-DIFFERENTLY -- bound to C-Tab, if memory > serves -- could be used to manually cycle through a series of > alternate indentation points for the current line. Oh, I forgot about that one! I probably didn't write it. There are a few gems in the Symbolics code that would be nice to extract. > I'm not aware of > an equivalent command having been implemented for GNU Emacs.) > > -- Scott >

1 0

[pro] [LibCL-devel] A brave new world
by Daniel Herring 07 Sep '10

07 Sep '10

[Most discussion should take place on the libcl-devel list.] Hi all, It has been several months since LibCL saw significant progress. This was a period of small experiments, reflection, and numerous "real-world" distractions. This past week, I had a bit of free time; and recent rumors of quicklisp focused the need to sieze the moment and put thoughts into action. [Thanks Zach. Honestly, with everything in my head, I couldn't have handled volunteers anyway.] Here's the result: a revised plan for LibCL, and a skeleton code base ready for contributors. http://libcl.com/ [main website, as yet unchanged] http://gitorious.org/libcl/world [new git repo] * User experience The user experience will see relatively few changes; just more stuff will be available. ASDF 1 to ASDF 2 is the biggest configuration change. * Developer experience This is much improved. No more tediously maintaining two git repos for each package. No more harrowing forced conversions of upstream sources into git. No more massive packages.lisp with a metadata defstruct. Now, each package is defined by a simple package.lisp file. This contains a single plist of information. Many of the keys are inherited from the old packages.lisp file; others are defined in the new metadata.txt. Key new metadata types are :maintainers, :tags, :todo, and :source. Now a single file describes everything needed to obtain and describe a (nearly) arbitrary upstream package. Helper scripts standardize the process. A simple framework structures the steps needed for building a release. It also applies local patches until they can be merged upstream. Names aren't finalized, but see workflow.txt. The html index will have a mark denoting local patches, except for patches following a naming convention indicating "trivial" operations like renaming directories and building documentation. * Expanded vision History: LibCL began after yet-another-computer/OS-upgrade cycle left me spending months reconstructing my libraries. The current systems just didn't work for me. I started with a bunch of things; but realized that many packages needed to be dropped because they were hard to support (most notably, anything using FFI). Then I moved towards the idea of creating a "core standard library", with a team of developers, etc. That never gelled. People were asking for more; I even found myself needing one-off, non-portable documentation tools to build the docs for LibCL... LibCL now aims to be a 1-stop shop. If a library isn't in LibCL, please send a metadata file to this mailing list, with a subject line like "[metadata] awesome-package.lisp". Or use a merge request on the gitorious repository. Same thing goes for applications. If/when the downloads start bloating, we can offer variants as needed. How are non-portable tools handled? Through tags. Only portable, pure CL libraries may be tagged with :pure. Only :pure libraries seeing widespread use may be tagged with :core. The default invocation of compile-libcl will only compile :core libraries. Calling compile-libcl with other tags allow people to compile things specific to their interests. Selecting a package by name will compile it and all its dependencies. Selecting T will compile everything. * Upcoming feature PJB and others had a great idea. After "(compile-libcl t)", walk the packages and generate a master index of all available symbols. Link this index to docstrings, and eventually add links to the full documentation. Good for searching. This will be included in the next LibCL release. ------------- Call for help The current libcl-world is a rough skeleton; but key pieces are already in place. Any help would be appreciated; please coordinate on the libcl-devel list; here are some suggested places to start. Easy tasks: - add metadata for new packages - create/add tags to packages - add maintainer information to packages Harder tasks: - coordinate metadata and/or the tarball cache with quicklisp? - fix package :todos, and add more as needed - update configure-libcl for ASDF 2 - website stuff - create a cache system for package tarballs and log files - create a comment/voting system for packages - sort through the old upstream/local repos for patches which never made it upstream - design a "fetch on demand" layer a la quicklisp or MiKTeX - hook "fetch on demand" into ASDF - design a system for caching .so and .dll files for FFI bindings Most changes may be handled through git merge requests. Metadata changes may be sent to libcl-devel with a subject like "[metadata] package.lisp". Contact me directly if you need a shell account for web changes (the current website should be available on gitorious). Thanks for reading this far. This will be a busy week personally, and I should be offline during September 12-14; but feel free to make progress without me. Let's see where this thing goes. Later, Daniel

2 1

[pro] Introducing the list
by Hans Hübner 03 Sep '10

03 Sep '10

Hi folks, this list has been created in response to an email by Dan Weinreb, addressed to a bunch of Common Lisp programmers that he knows, who wondered where to turn to for some advice on the existence of certain Common Lisp libraries. comp.lang.lisp was deemed not to be the right place, because it is full of spam and requires sophisticated special-purpose software and regular care taking to be readable. Thus, we came up with the idea of a mailing list targeted at people who already know and use Common Lisp and who don't want to discuss the merits of it or how other languages are worse or better. This is an implementation-agnostic list. Discussing any Common Lisp implementation is on-topic, although implementation specific questions are probably better discussed on the respective mailing lists. We generally also know why we chose one implementation over another, so "my implementation/your implementation"-type discussions should not be started unless they are technical (as opposed to license-oriented). Be inclusive and friendly, accept commercial and non-commercial developers. Let's not use the word "free" here, unless it is about beer. :) -Hans

1 0

[pro] Greetings
by Peter Seibel 03 Sep '10

03 Sep '10

Greetings fellow Lisp pros. Heh, this list should have been called "pros" and then when someone starts a list for beginners (i.e. one to help grow the list of Lisp programmers) it could have been called "cons". ;-) -Peter -- Peter Seibel http://www.codequarterly.com/

2 1

Re: [pro] How to get useful Lisp info?
by Hans Hübner 03 Sep '10

03 Sep '10

Daniel, On Fri, Sep 3, 2010 at 16:18, <dherring(a)tentpost.com> wrote: > If pro(a)cl.net will be an implementation-agnostic place for serious CL > users, it needs an accompanying place for new user questions. Is that > place lispforum.com? a new mailing list? I'm a bit sceptical regarding a new mailing list for CL beginners. For one, it would further support fragmentation. For another, starting the two lists at the same time will likely lead to confusion (where do I post this?). I think we should start get pro@ going and then maybe consider adding a new list for beginners. -Hans

2 1