Here are my opinions on the issues discussed in the recent mail.
Nikodemus's points are excellent. For this effort to succeed, it must start small. I also agree that it should be bottom-up. It should start with existing, accepted libraries, and avoid innovation: experience has shown that it's usually unwise to standardize things that haven't been tried out pretty thoroughly.
I'm sympathetic with what Peter Denno says, but I've been though things like this before, and trying to "do the right thing" from the beginning, in a project such as this one, often ends up stopping the whole thing from happening. We should start pulling things together, and then try to move towards the more careful, professional practices that Peter describes.
Jochen brings up the question of whether we want to standardize on official interfaces only, or on implementations as well. CLtL itself attempted to describe all the functions, macros, etc. without saying anything about their implementation. Here, there may be some conceptual tension between whether we are trying to define (more or) a langauge, establish a set of API's to libraries without saying anything about their implementation, or gather a useful common set of libraries.
The Manifesto talks about "language extensions" at one point, and a "standard library" later on, which may lead to some confusion about this point. On the other hand, if you look at CLtL, you can see that it does not separate a "core language" from a bunch of functions and macros that can be build on that core language. (That's a big reason that Common Lisp is criticized as being a language that's "too large". If you consider Java plus all the standard Java libraries, it looks big too!) We had originally wanted to define CLtL that way, but it was too difficult, for many practical reasons. So Common Lisp is left with a legacy of not distinguishing clearly between what is the "language" and what are "already-loaded standard libraries".
This leads into the question of scope. There are some things that are so basic, and so widely needed, and so small, that it's completely clear that they belong: a library for portable sockets (usocket), for example. A way to write one's own streams is clearly a language extension, and one of the various stream packages should certainly be included. It would be great to be able to have a standard way to deal with Unicode strings, although there are some thing that made that hard. Thread manipulation is another obvious candidate.
Then you can move up to thinks that are quite clearly libraries, yet considered something that any system should not be without, such as regular expressions. Then there are XML libraries, an HTTP client, an HTTP server. There are some things where it's not clear what approach is best, such as generation of web pages and interfacing to relational databases, but we could pick one.
What I, myself, would like to see is easy ways for programmers to do all the normal things that application programs and servers can do today. How much of that belongs in the scope of the CLtL3 process is yet to be determined. We might agree that there should be easy and common ways to do an HTTP server, but also agree that CLtL3 isn't the place to put them.
Sequences, along the lines that Christope Rhodes has started to do, would be great. On the other hand, I think what he has so far isn't ideal and needs more development, and I'd be hesitant about declaring it a standard at this point.
New ideas on packages and modules, while welcome, are something I'd consider too innovative to put into a "standard" library. These thing are hard to invent and take a long time to get experience with. I was one of the people who designed packges (yes, mea culpa) and I've been involved more than once in trying to come up with better modularity constructs, and I can assure you it's not obvious or easy. Let's stick to what's well-understood for the foreseeable future.
-- Dan
Am 16.05.2009 um 04:55 schrieb Daniel Weinreb:
The Manifesto talks about "language extensions" at one point, and a "standard library" later on, which may lead to some confusion about this point. On the other hand, if you look at CLtL, you can see that it does not separate a "core language" from a bunch of functions and macros that can be build on that core language. (That's a big reason that Common Lisp is criticized as being a language that's "too large". If you consider Java plus all the standard Java libraries, it looks big too!) We had originally wanted to define CLtL that way, but it was too difficult, for many practical reasons. So Common Lisp is left with a legacy of not distinguishing clearly between what is the "language" and what are "already-loaded standard libraries".
Yes thats true - but isn't this an even bigger reason to be more careful for CLtL3? I think it might be interesting to split the effort into a CLtL3 Core and CLtL3 Extensions. In CLtL3 Extensions many more things are allowed, it is a place for experimentation. Proven things may later get included into the CLtL3 Core. At this stage we can start collecting things and start picking pearls when there is enough on the table.
This leads into the question of scope. There are some things that are so basic, and so widely needed, and so small, that it's completely clear that they belong: a library for portable sockets (usocket), for example. A way to write one's own streams is clearly a language extension, and one of the various stream packages should certainly be included. It would be great to be able to have a standard way to deal with Unicode strings, although there are some thing that made that hard. Thread manipulation is another obvious candidate.
The real problem is that for nearly all things like threads or sockets there is a huge difference in what people demand. Some people want strict Unix semantics others just want something simple. Some features may not be easily done in a platform independent way. The problem is here to how to decide what we want to settle on. There needs to be a clear process.
Language extensions are another problem. While every language user can write an XML-Parser to a specified interface, it isn't that easy to extend a closed vendors lisp with things like e.g. user extensible streams. How do we want to get vendors involved? Without vendor support CLtL3 will only be just another paper tiger.
Then you can move up to thinks that are quite clearly libraries, yet considered something that any system should not be without, such as regular expressions. Then there are XML libraries, an HTTP client, an HTTP server. There are some things where it's not clear what approach is best, such as generation of web pages and interfacing to relational databases, but we could pick one.
I think concentrating on this would scatter the effort too much at this point. While I find things like XML-Libraries (cxml) or regular expressions (cl-ppcre) relatively straightforward to decide - how would we want to choose a HTTP Server? CL-HTTP? AllegroServe? Hunchentoot? UCW? Araneida? I think there is nothing wrong collecting data about available options and evaluate them.
What I, myself, would like to see is easy ways for programmers to do all the normal things that application programs and servers can do today. How much of that belongs in the scope of the CLtL3 process is yet to be determined. We might agree that there should be easy and common ways to do an HTTP server, but also agree that CLtL3 isn't the place to put them.
Yes
Sequences, along the lines that Christope Rhodes has started to do, would be great. On the other hand, I think what he has so far isn't ideal and needs more development, and I'd be hesitant about declaring it a standard at this point.
Sequences have a long way to go - I don't know if there is any reference implementation yet. If there is one it exists probably only for one implementation. But I still think it is a perfect candidate for CLtL3 Extensions.
New ideas on packages and modules, while welcome, are something I'd consider too innovative to put into a "standard" library. These thing are hard to invent and take a long time to get experience with. I was one of the people who designed packges (yes, mea culpa) and I've been involved more than once in trying to come up with better modularity constructs, and I can assure you it's not obvious or easy. Let's stick to what's well-understood for the foreseeable future.
I guess you mean CL packages and not ASDF, DEFSYSTEM a.s.o. Hierarchical packages do exist for quite some time on many different implementations. I personally do like the idea of Tim Bradshaws conduits package better; particularily per-package aliases. With the latter you can specify nicknames for other packages in your own packages without changing the package names or nicknames globally. The conduits system is severals years old and I think of it as a proven thing. I think there should be some discussion if something of this could make it into CLtL3.
Tim Bradshaw's Conduits: http://www.tfeb.org/lisp/hax.html#CONDUITS
ciao, Jochen
-- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9 , 90408 Nuremberg
Fon +49 (0)911 517 999 82 Fax +49 (0)911 517 999 83
mailto:info@crispylogics.com http://www.crispylogics.com
2009/5/15 Jochen Schmidt js@crispylogics.com:
Am 16.05.2009 um 04:55 schrieb Daniel Weinreb:
The Manifesto talks about "language extensions" at one point, and a "standard library" later on, which may lead to some confusion about this point. On the other hand, if you look at CLtL, you can see that it does not separate a "core language" from a bunch of functions and macros that can be build on that core language. (That's a big reason that Common Lisp is criticized as being a language that's "too large". If you consider Java plus all the standard Java libraries, it looks big too!) We had originally wanted to define CLtL that way, but it was too difficult, for many practical reasons. So Common Lisp is left with a legacy of not distinguishing clearly between what is the "language" and what are "already-loaded standard libraries".
Yes thats true - but isn't this an even bigger reason to be more careful for CLtL3? I think it might be interesting to split the effort into a CLtL3 Core and CLtL3 Extensions. In CLtL3 Extensions many more things are allowed, it is a place for experimentation. Proven things may later get included into the CLtL3 Core. At this stage we can start collecting things and start picking pearls when there is enough on the table.
This is in line with my thoughts as well.
This leads into the question of scope. There are some things that are so basic, and so widely needed, and so small, that it's completely clear that they belong: a library for portable sockets (usocket), for example. A way to write one's own streams is clearly a language extension, and one of the various stream packages should certainly be included. It would be great to be able to have a standard way to deal with Unicode strings, although there are some thing that made that hard. Thread manipulation is another obvious candidate.
The real problem is that for nearly all things like threads or sockets there is a huge difference in what people demand. Some people want strict Unix semantics others just want something simple. Some features may not be easily done in a platform independent way. The problem is here to how to decide what we want to settle on. There needs to be a clear process.
Indeed .. deciding what to 'settle on', as you put it, is going to be the biggest problem we face initially. My hope is that we, as a group, can reach a reasonable consensus after some (likely heated) discussion.
In the end, just choosing something and going for it will be significantly more productive then waiting for the perfect solution. CLtL3 will not be frozen in time and impossible to update like ANSI, it will be a fluid proccess and a continually growing language. I believe this is what the original intent behind the original CL efforts, and i'd very much like to see modern CL in line with the original design goals.
Worse is better, especially when the other option is 'nothing'. :)
Language extensions are another problem. While every language user can write an XML-Parser to a specified interface, it isn't that easy to extend a closed vendors lisp with things like e.g. user extensible streams. How do we want to get vendors involved? Without vendor support CLtL3 will only be just another paper tiger.
By beginning with (mostly) portable language extensions, we don't initially have to care about vendors. If a certain lisp doesn't support a feature we need, then that lisp is not CLtL3 compatible... this is not a big deal, it's not like we're ANSI. I don't expect the CLtL3 project to even be a blip on the radar of a commercial implementor, as most of them they already ship a CLtL3-like lisp. If their customers start demanding CLtL3 compatibility, we've already won.
As for the open source lisps... well, the solution is obvious. We write code.
Then you can move up to thinks that are quite clearly libraries, yet considered something that any system should not be without, such as regular expressions. Then there are XML libraries, an HTTP client, an HTTP server. There are some things where it's not clear what approach is best, such as generation of web pages and interfacing to relational databases, but we could pick one.
I think concentrating on this would scatter the effort too much at this point. While I find things like XML-Libraries (cxml) or regular expressions (cl-ppcre) relatively straightforward to decide -
I'd just like to point out that Edi himself would be against 'standardising' CL-PPCRE. The design goals of CL-PPCRE were not to create a most excellent lispy implementation of regexps, but to try and do a better PERL-compatible regexp engine than Perl.
Sequences, along the lines that Christope Rhodes has started to do, would be great. On the other hand, I think what he has so far isn't ideal and needs more development, and I'd be hesitant about declaring it a standard at this point.
Sequences have a long way to go - I don't know if there is any reference implementation yet. If there is one it exists probably only for one implementation. But I still think it is a perfect candidate for CLtL3 Extensions.
SB-SEQUENCE is the reference implementations, and i agree with your 'extensions' proposal.
New ideas on packages and modules, while welcome, are something I'd consider too innovative to put into a "standard" library. These thing are hard to invent and take a long time to get experience with. I was one of the people who designed packges (yes, mea culpa) and I've been involved more than once in trying to come up with better modularity constructs, and I can assure you it's not obvious or easy. Let's stick to what's well-understood for the foreseeable future.
I guess you mean CL packages and not ASDF, DEFSYSTEM a.s.o. Hierarchical packages do exist for quite some time on many different implementations. I personally do like the idea of Tim Bradshaws conduits package better; particularily per-package aliases. With the latter you can specify nicknames for other packages in your own packages without changing the package names or nicknames globally. The conduits system is severals years old and I think of it as a proven thing. I think there should be some discussion if something of this could make it into CLtL3.
I like conduits as well, and a few lisps also have hierarchal packages now as well. I think that the package system is worth re-visiting (sorry dan!), and simple extensions like conduits are a good start.. I can see uses for cloned packages and the like within CLtL3 itself.
Keep the great ideas and discussion coming!
Cheers,
drewc
Tim Bradshaw's Conduits: http://www.tfeb.org/lisp/hax.html#CONDUITS
ciao, Jochen
-- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9 , 90408 Nuremberg
Fon +49 (0)911 517 999 82 Fax +49 (0)911 517 999 83
mailto:info@crispylogics.com http://www.crispylogics.com
cltl3-devel mailing list cltl3-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/cltl3-devel
-
Daniel Weinreb -
Drew Crampsie -
Jochen Schmidt