On Wed, Jul 20, 2011 at 3:32 PM, Didier Verna didier@lrde.epita.fr wrote:
Dear friends,
I'm starting to write a chapter for an upcoming book on domain specific languages. The chapter is called (tentatively):
Extensible languages -- blurring the distinction between DSLs and GPLs
GPL meaning General Purpose Language in this context ;-)
My intention is to demonstrate how the task of implementing a DSL is made easier when it boils down to an extension or subset of your original GPL (hence reusing its infrastructure), instead of being a totally different language, only written on top of the other.
Obviously, I'm going to illustrate this with Common Lisp, and I intend to speak of dynamicity (not only dynamic typing, but in general all things that can be deferred to the run-time), introspection, intersession, structural or procedural reflexivity, meta-object protocols (not sure about this one), macro systems and JIT-compilation. Also, more specifically to Lisp, reader macros (compiler macros maybe?), the condition system (and its ability to *not* unwind) and restarts.
Right now, I would like to know if any of you have DSL "pearls", nice examples of DSLs that you have written in Lisp by using some of its features in a clever or elegant way. I would also gladly accept any point of view or comment on what's important to mention, in terms of design principle or anything else, things that I may have missed in the list above.
Not really "pearls", but lately I found myself using more and more of Lisp at macroexpansion time, which seems to be what you're looking for. Two examples:
- for my DO+ (doplus) iteration macro - a somewhat iterate-like construct with a simpler implementation that does not use a code walker - I used structures quite heavily. do+ controls iteration via a series of clauses that are macros (either built-in or written by users) that have a peculiar aspect: they don't expand to proper Lisp code, but return structures or lists of structures [1]. do+ macroexpands its clauses (recursively when needed) and uses the returned structures to drive the building of its own expansion. Basically I explicitly used macros as compile-time functions, defining my own limited "AST" (not really a tree but a flat list of structures) which is not Lisp's but is later translated to Lisp, so it's one extra indirection than what is typically found in macros. I think this shows nicely how the distinction between code and data is blurry in Lisp, and as a consequence the distinction between DSL and GPL is blurry as well. This allows the implementation to remain simple: do+ is a little less than 350 lines of code, but only roughly half of them are the actual macro and its supporting code, the rest are built-in clause macros and other minor stuff.
- I also have sketched a HTML generation library - tentatively called tag-pool - where I used CLOS generic functions at macroexpansion time to drive the expansion of tag macros to HTML-outputting code. Contrary to the popular approach, used by CL-WHO among others, to represent HTML as a limited sexp-based DSL where tag names are keywords and a single macro does all the translation work, I have one macro per tag. For CLOS dispatch purposes, to each tag is also associated a class, but it's used only at expansion time, the output code does not instantiate any CLOS object. Tag macros follow a CLOS-based protocol to emit the various bits of HTML like attributes, body, and so on. Users can specialize methods that are part of the protocol to finely control how tag macros are translated, for example providing defaults for certain attributes, or post-processing some attribute's value, etc. As an example, the SCRIPT macro translates its Lisp body to JavaScript using Parenscript, by specializing a single generic function. The body code can itself contain HTML-generating macros and those are translated to HTML-emitting JavaScript as well. Again, a key feature of Lisp (CLOS and generic function dispatch) is used at macroexpansion time to make the implementation of the DSL simpler and more easily extensible.
You can find both examples here: http://code.google.com/p/tapulli/ although only do+ is documented and reasonably complete.
[1] ok, technically a structure *is* proper Lisp code since it is a self-evaluating object and it can be externalized...
Best, Alessio