Dear mailing-list member,

There is no doubt that the most exciting thing that has happened since the previous progress report is that Climacs now has a syntax module for Common Lisp.

A group of four students: Nada Ayad, Julien Cazaban, Pascal Fong Kye, and Bruno Mery, have been working for the past two months or so to get this working. The first commits were made last week (just in time for me to announce it at the Amsterdam Lisp meeting), and we have seen a steady stream of improvements since then. In theory, the project is due tomorrow, so there is definitely the possibility that this stream will stop after that. So feel free to play around with the features, submit patches, add new features, etc.

I am discovering the features myself now, so I am sure I do not yet have the full picture. Yesterday, I tested some of the syntax highlighting features, and I am already impressed. For instance, trying to use a digit that is not valid for the radix in the CL #nrxxx construct makes the token change colors.

There are still some problems, of course, that may or may not be fixed by tomorrow. I advice against major modifications on your part before that.

Other improvements since the previous progress report:

* Improvements to Prolog syntax. (thanks to Christophe Rhodes)

* Improvements to HTML syntax (organize the grammar around concepts in the standard, more elements, performance improvements).

* Performance improvements of the Earley Parser in the form of precomputed prediction rules and a faster test for whether a rule has been predicted. Also avoid generic-function dispatch on functions that are called frequently. (thanks in part to Christophe Rhodes)

The next speed improvement to the parsing framework might be to allow for a "manual" test to determine whether a rule is valid for prediction. To illustrate this idea for CL syntax: currently, whenever an expression is expected, all rules that handle the dispatch macro character ## will be predicted. However, most often, the next lexeme is not ##, so these predictions will not be used further. They must, however, be traversed when the completer searches this state for valid incomplete items. All this is costly. A simple test that makes it possible to test the next lexeme when a rule is about to be predicted could cut down on this work considerably. A similar phenomenon will occur in HTML mode when each inline element will be predicted for each word in an ordinary paragraph of text.

Keep up the good work.