Re: [cltl3-devel] RFC: CLtL3 Charter
It's awesome that the standard is getting revisited. In my view there are four main problem areas. I think it's most important that the broken or lacking core language features be addressed! Other languages have groups continually going over these things; Common Lisp needs one too. (1) Missing or broken language features that should be added or fixed Crummy type-system (at least parameterised types please)! As Levente also brought up: also access to type inference, etc. This would allow: Efficient generic functions dispatched to inline expansions at compile time. Value semantics. More help for code walkers and transformers (what does a declare form refer to?, etc.) Limited(?) serialisation of closures. Ability to locally alias a symbol and package (2) Brokenness in the current Lisp standard Different facilities should be separated and put into different packages. Pathnames are overly complicated and do not handle symlinks well. Extensible sequences, gray streams, etc. Hash-tables should be generalised like extensible sequences allow user implementation of other datastructures. eq, eql, equal, equalp are not enough; nothing should restrict itself to these comparisons. Generally all functions like +, -, equal etc. should be overridable for user's custom types with the efficient compile-time dispatched generic function mechanism. This would allow (explicit) laziness! Possibly kill the many bizarrely named or strangely parameterised and hardly used functions like `get'. (3) Stuff all implementations provide but is not standard Unsafe machine pointers Weak pointers Details from room GC Stack frame inspection etc. (4) Libraries that basically already exist but would be useful as part of the standard cl-ppcre, local-time, osicat, cffi, etc.
Am 06.09.2009 um 09:35 schrieb John Fremlin:
It's awesome that the standard is getting revisited.
CLtL3 will only be a specification, not a standard. This is a big difference. Between CLtL2 and ANSI CL are significant differences and creating a standard like ANSI CL was a complex and costly task. We hope that the result of CLtL3 will be a specification that gets accepted as solid progress for Common Lisp as a programming language - not Common Lisp as a standard. I think time will show if there will be benefits if the standard itself will get revised. The fact that there exists an established standard (ANSI CL) will have a stake on what we can do with CLtL3. It doesn't make any sense to me, to break ANSI CL support; a future CLtL3 capable Common Lisp should also work with programs written for ANSI CL.
In my view there are four main problem areas. I think it's most important that the broken or lacking core language features be addressed! Other languages have groups continually going over these things; Common Lisp needs one too.
(1) Missing or broken language features that should be added or fixed
Crummy type-system (at least parameterised types please)!
As others already outlined extending the type system could probably the most efficient way to drive CLtL3 against some wall. It's not impossible to implement your own type systems within Common Lisp without touching the CL type system at all. I think I remember reading Thomas Burdick talk about this on c.l.l before; maybe it was someone else. On the other side: I would be the last who would say NO! if someone can show how to extend the CL type system in a way that doesn't break ANSI CL code, is sound and (most important) is useful. I just think that this is maybe one of those parts we will not tackle with CLtL3.
As Levente also brought up: also access to type inference, etc. This would allow:
Efficient generic functions dispatched to inline expansions at compile time.
Generic functions do not dispatch on types, but on classes (and on EQL values). There exists some classes which correspond to types like numbers, strings a.s.o. This means you can already do something like: (class-direct-subclasses (find-class 'number)) => (#<BUILT-IN-CLASS REAL 20945223> #<BUILT-IN-CLASS COMPLEX 20944833>) The consequence is, that you cannot dispatch on all types expressible in CL. Allowing dispatch on arbitrary types is probably not easy and perhaps not very efficient. Though: Extensions to the dispatch system are a very interesting topic for CLtL3. AFAIK Pascal Costanza is working on some ideas to realize predicate dispatch within CLOS.
Value semantics.
Value semantics? I would like to here some more about why you think this is needed for CL and how it should work.
More help for code walkers and transformers (what does a declare form refer to?, etc.)
defining access to environments - I would like to see something like that too.
Limited(?) serialisation of closures.
Would be somewhat cool. More important to me would be compiling and loading code from a stream.
Ability to locally alias a symbol and package
Package-local aliases, nicknames for packages that are only valid within the package in which they are defined are quite possible. Perhaps ideas from Tim Bradshaws conduits package should be taken into account. I fear locally aliasing a symbol is more difficult. You cannot mean creating a new symbol with the same name shadowing the first, because thats already how the package system works. So I think you actually mean accessing the _same_ symbol using a different name within a particular package. SYMBOL-NAME currently is a property of a symbol which is independent from the concept of a package. Following your idea SYMBOL-NAME would give different names depending on the value of *package* in the current dynamic extent. IMPORT is just defined on symbols and does not associate names with which the imported symbols are imported (because IMPORT gets them using the global symbol property SYMBOL-NAME); so one would need another function (e. g. called IMPORT-ALIASED) to associate a symbol with a particular name in the chosen package. A problem is though, that in ANSI CL there is this constraint were the symbol-name of a symbol is always the same - regardless of the current package - this could lead to problems with ANSI CL programms.
(2) Brokenness in the current Lisp standard
Different facilities should be separated and put into different packages.
More: New facilities defined in CLtL3 should be clearly modularised and - of course - get their own packages were applicable. You should recognize a CLtL3 using program easily by its usage of CLtL3 packages. There could be a common prefix like "CLTL3." for all those packages. Perhaps it would be interesting to expose existing ANSI CL symbols in different packages, but they will all have to be available in "CL".
Pathnames are overly complicated and do not handle symlinks well.
Windows XP doesn't handle symlinks well ;-) - The situation in common lisp is somewhat similar to that of java 2-3 years ago. They solved it in JSR 203. CL Pathnames are actually better than their fame. Im not sure what the best idea would be - more clearly define how pathnames should behave or just orthogonally implement URIs and make all functions who accept pathname designators let accept uris too (URIs as pathname designators). Either one would restrict those to file-URIs or one could establish a protocol to implement support for other schemes.
Extensible sequences, gray streams, etc.
Of course!
Hash-tables should be generalised like extensible sequences allow user implementation of other datastructures.
yes
eq, eql, equal, equalp are not enough; nothing should restrict itself to these comparisons.
http://www.nhplace.com/kent/PS/EQUAL.html
Generally all functions like +, -, equal etc. should be overridable for user's custom types with the efficient compile-time dispatched generic function mechanism.
This is already possible to some extend using symbol shadowing. I would not want to allow "operator overloading" like in C++ (shudder)
This would allow (explicit) laziness!
Possibly kill the many bizarrely named or strangely parameterised and hardly used functions like `get'.
"Kill" gets to far - this would break ANSI CL compatibility. We could deprecate their use and don't import them in the CLtL3 package(s) - so one could use CL:GET if one wants to.
(3) Stuff all implementations provide but is not standard
Unsafe machine pointers
Weak pointers
Details from room
GC
Stack frame inspection
Usefulness is king.
etc.
(4) Libraries that basically already exist but would be useful as part of the standard
cl-ppcre, local-time, osicat, cffi, etc.
Thats actually not a good idea (as discussed on the list before). ciao, Jochen -- Jochen Schmidt CRISPYLOGICS Uhlandstr. 9, 90408 Nuremberg Fon +49 (0)911 517 999 82 Fax +49 (0)911 517 999 83 mailto:(format nil "~(~36r@~36r.~36r~)" 870180 1680085828711918828 16438) http://www.crispylogics.com
Jochen Schmidt <js@crispylogics.com> writes: [...]
The fact that there exists an established standard (ANSI CL) will have a stake on what we can do with CLtL3. It doesn't make any sense to me, to break ANSI CL support; a future CLtL3 capable Common Lisp should also work with programs written for ANSI CL.
Possibly, but I hope you understand that the ANSI CL standard was not perfect, is not implemented perfectly, and if a small change or clarification is proposed then you agree that it should be judged on its individual merits (as affecting existing programs or not, etc.) and not summarily rejected. [...]
Generic functions do not dispatch on types, but on classes (and on EQL values).
Yes, I am painfully aware of this and find it very unfortunate that there are three kinds of valid type in CL: types you can give to declare or the, types you can give to typep, and classes. [...]
The consequence is, that you cannot dispatch on all types expressible in CL. Allowing dispatch on arbitrary types is probably not easy and perhaps not very efficient. Though: Extensions to the dispatch system are a very interesting topic for CLtL3. AFAIK Pascal Costanza is working on some ideas to realize predicate dispatch within CLOS.
To digress into this tangent, it might great if generic functions could also dispatch on types. If it makes it inefficient for those cases which use this functionality (e.g. satisfies), fair enough, but it would make things less complicated to explain. What I was talking about before was (1) a facility for macros to inspect the declared or derived types in the lexical environment, (2) a new auto-sealing generic function system built on this. Namely, if the type is known and matches a defined method then that method may be inlined at the compiler's discretion; if it is later redefined (defsealedmethod) or a more specific method is defined then that may not affect compiled code (in the same vein as inline functions at the moment). [...]
Value semantics? I would like to here some more about why you think this is needed for CL and how it should work.
It would be nice if it were possible to get decent code out of a CL compiler for things like bytemap or manardb, where the ability to pass around small structures instead of horribly packing things into integers would be great. For another example, an array of structures rather than an array of pointers to structures has hugely different performance. [...]
defining access to environments - I would like to see something like that too.
To be clear one of the messiest areas of the CL standard is the business of declarations; for example, the distinction between bound or unbound declarations. How can a code-walker/transformer decide what a declaration refers to if the declaration is not standard? [...]
Would be somewhat cool. More important to me would be compiling and loading code from a stream.
You can load code from a stream in ANSI (but not Allegro, which assumes its a fasl or not depending on the element-type of the stream -- another messy bug they are afraid to touch). [...]
Package-local aliases, nicknames for packages that are only valid within the package in which they are defined are quite possible. Perhaps ideas from Tim Bradshaws conduits package should be taken into account.
Or Ron Garret's ideas. [...]
the chosen package. A problem is though, that in ANSI CL there is this constraint were the symbol-name of a symbol is always the same - regardless of the current package - this could lead to problems with ANSI CL programms.
You could munge the reader to have a local alias table that it consults, it wouldn't affect anything that didn't touch this table. Having package versioning so multiple versions of the same package could be simultaneously loaded would make some of the dependency nuisances go away, and allow for somewhat smoother live updates. [...]
eq, eql, equal, equalp are not enough; nothing should restrict itself to these comparisons.
What's that got to do with it? I was complaining about the limitations of the standard make-hash-table :test. (Is there another place where the set of test functions is restricted?) [...]
This is already possible to some extend using symbol shadowing. I would not want to allow "operator overloading" like in C++ (shudder)
Why not?
This would allow (explicit) laziness!
Possibly kill the many bizarrely named or strangely parameterised and hardly used functions like `get'.
"Kill" gets to far - this would break ANSI CL compatibility. We could deprecate their use and don't import them in the CLtL3 package(s) - so one could use CL:GET if one wants to.
Yes. [...]
Stack frame inspection
Usefulness is king.
? Backtraces are badly implemented by many projects and would be used by more if they weren't so tricky. [...]
2009/9/6 John Fremlin <john@fremlin.org>:
Jochen Schmidt <js@crispylogics.com> writes: [...]
The fact that there exists an established standard (ANSI CL) will have a stake on what we can do with CLtL3. It doesn't make any sense to me, to break ANSI CL support; a future CLtL3 capable Common Lisp should also work with programs written for ANSI CL.
Possibly, but I hope you understand that the ANSI CL standard was not perfect, is not implemented perfectly, and if a small change or clarification is proposed then you agree that it should be judged on its individual merits (as affecting existing programs or not, etc.) and not summarily rejected.
CLtL3 will make every effort to avoid breaking compatibility with ANSI. While a non-compatible change may be worth looking at, it likely will not be a part of this spec unless there is an extremely good reason and little chance of breaking ANSI CL code.
[...]
The consequence is, that you cannot dispatch on all types expressible in CL. Allowing dispatch on arbitrary types is probably not easy and perhaps not very efficient. Though: Extensions to the dispatch system are a very interesting topic for CLtL3. AFAIK Pascal Costanza is working on some ideas to realize predicate dispatch within CLOS.
To digress into this tangent, it might great if generic functions could also dispatch on types. If it makes it inefficient for those cases which use this functionality (e.g. satisfies), fair enough, but it would make things less complicated to explain
It would make things significantly more complex to implement (read: impossible in many cases to determine the effective method), and with little gain.. one can easily hack something up with typecase and a hash table if one wants to dispatch on CL types. How would it make things less complicated to explain? AMOP is fairly simple (or it at leasts builds on simple concepts).. adding types to the GF dispatch would complicate things significantly. The goals of CLtL3 are not to make CL 'simpler', 'more understandable' or 'easy for newbies'.
What I was talking about before was (1) a facility for macros to inspect the declared or derived types in the lexical environment,
Something like this may make it as part of the environments module. But, we won't _require_ any compile-time type inference.
auto-sealing generic function system built on this. Namely, if the type is known and matches a defined method then that method may be inlined at the compiler's discretion; if it is later redefined (defsealedmethod) or a more specific method is defined then that may not affect compiled code (in the same vein as inline functions at the moment).
Something like this is _way_ out of scope. I'd like to offer the tools to enable someone to build it portably, but it's not going into the description.
[...]
Value semantics? I would like to here some more about why you think this is needed for CL and how it should work.
It would be nice if it were possible to get decent code out of a CL compiler for things like bytemap or manardb, where the ability to pass around small structures instead of horribly packing things into integers would be great. For another example, an array of structures rather than an array of pointers to structures has hugely different performance.
Out of scope as well.
[...]
defining access to environments - I would like to see something like that too.
To be clear one of the messiest areas of the CL standard is the business of declarations; for example, the distinction between bound or unbound declarations. How can a code-walker/transformer decide what a declaration refers to if the declaration is not standard?
Environments are worth looking into, yes.
[...]
Package-local aliases, nicknames for packages that are only valid within the package in which they are defined are quite possible. Perhaps ideas from Tim Bradshaws conduits package should be taken into account.
Or Ron Garret's ideas.
Conduits, and hierarchal packages, are something we'll have to look at further, I can see uses within CLtL3 itself.
Having package versioning so multiple versions of the same package could be simultaneously loaded would make some of the dependency nuisances go away, and allow for somewhat smoother live updates.
I'd need to see some code for this, it's a neat idea, but the implementation is likely hairy. Can you point to an existing implementation?
[...]
eq, eql, equal, equalp are not enough; nothing should restrict itself to these comparisons.
What's that got to do with it? I was complaining about the limitations of the standard make-hash-table :test. (Is there another place where the set of test functions is restricted?)
Extensible hash tables are covered by a CDR, and so are on the list of things we will consider.
[...]
This is already possible to some extend using symbol shadowing. I would not want to allow "operator overloading" like in C++ (shudder)
Why not?
Because it's crap, and we have SHADOW.
Possibly kill the many bizarrely named or strangely parameterised and hardly used functions like `get'.
Not going to happen.. ever. I used GET just yesterday, and we will not be 'killing' anything. If you don't like it, don't use it, but removing functionality and CL compatibility is not worth it, and not a goal of CLtL3.
"Kill" gets to far - this would break ANSI CL compatibility. We could deprecate their use and don't import them in the CLtL3 package(s) - so one could use CL:GET if one wants to. ,
We will also _not_ be deprecating things from CL because they are bizarrely name or parameterized . If anything i'd like to un-deprecate a few things ANSI sent down (*-if-not, i',m looking at you).
Yes.
[...]
Stack frame inspection
Usefulness is king.
?
Backtraces are badly implemented by many projects and would be used by more if they weren't so tricky.
Trivial-backtrace exists, so there is some need for this. It's covered under 'Editing and Introspection' in the charter. Cheers, drewc
[...]
_______________________________________________ cltl3-devel mailing list cltl3-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/cltl3-devel
I have a suggestion concerning what Cltl3 will not do (specially after the recent discussions with John Fremlin). Nothing personal, John Fremlin, but we must do things in a very conservative way, and this was already discussed in this topic. This or something similar should be added to the charter: "The group don't intend to include things incompatibly with the ANSI specification, nor to include features that can be easily included with tools provided by ANSI CL like symbol shadowing, packages and read tables, or with the other features included in this description. Making CL more readable or understandable is not one of Cltl3's goals. It will not change function or macro names nor create new names or aliases for them in some other package, even if the new names are more descriptive or acceptable. This description is intended to be as easy to implement as possible and features difficult to implement are only going to be adopted if they are strictly necessary to achieve other Cltl3's goals. In particular, This description will not enforce implementations to do anything at all. The group, however, expects this new description to provide features and and advantages over the current ANSI specification so that implementations choose to adopt it."
Gustavo <gugamilare@gmail.com> writes:
It will not change function or macro names nor create new names or aliases for them in some other package, even if the new names are more descriptive or acceptable.
That seems unnecessarily restrictive. What's wrong with a COMMON-LISP-3 package which has more orthogonal names? Heck, it could even take advantage of some theoretical versioned-package functionality. -- Robert A. Uhl I take great delight at jeering at overly healthy types and telling them that they're going to feel really stupid one day, lying in a bed dying of nothing. --GB
2009/9/6 Robert Uhl <eadmund42@gmail.com>
Gustavo <gugamilare@gmail.com> writes:
It will not change function or macro names nor create new names or aliases for them in some other package, even if the new names are more descriptive or acceptable.
That seems unnecessarily restrictive. What's wrong with a COMMON-LISP-3 package which has more orthogonal names? Heck, it could even take advantage of some theoretical versioned-package functionality.
I based this sentence on what Drew Crampsie said: "We will also _not_ be deprecating things from CL because they are bizarrely name or parameterized." Just renaming functions or creating a new package with more "orthogonal names" (whatever that means) does not include any new feature nor functionality in the language and it is something you can do yourself. It falls into what was already said: that it is not one of Cltl3's goal to make CL more readable or easier for newbies or whatever. It also depends too much on personal taste. Having one function with different names makes more confusion than clarification. Such a thing pollutes the packages with names you are not going to use. For instance, I might not like some naming conventions in COMMON-LISP-3 or I might have some code that already uses CL's standard names, so I decide to import from both packages COMMON-LISP-3 and COMMON-LISP. Then I will have to manually shadow all symbols that I don't want to use, or I will be obligated not to use those names in my package. Not to mention that doing this will make it look like that we are trying to force people to program in a different language, learn new names or new parameter orders. This is very different from including extensions - you may safely and easily just ignore such extensions. Maybe in the course of discussion we end up creating functions that are generalizations of other functions already in ANSI CL, then it will be ok to deprecate those functions, but this is very different from just creating aliases. And, as a mater of fact, I can't imagine how to implement versioned packages in a way that its cost is below its benefits. I can only imaging that something in this direction will need big changes in implementations for only providing a small benefit, or to occupy more space (many versions of the same functions). I will be glad if someone proves me wrong. Conduit packages look much more useful than versioned packages.
-- Robert A. Uhl I take great delight at jeering at overly healthy types and telling them that they're going to feel really stupid one day, lying in a bed dying of nothing. --GB
Drew Crampsie wrote:
What I was talking about before was (1) a facility for macros to inspect the declared or derived types in the lexical environment,
Something like this may make it as part of the environments module. But, we won't _require_ any compile-time type inference.
Fair enough.
auto-sealing generic function system built on this. Namely, if the type is known and matches a defined method then that method may be inlined at the compiler's discretion; if it is later redefined (defsealedmethod) or a more specific method is defined then that may not affect compiled code (in the same vein as inline functions at the moment).
Something like this is _way_ out of scope. I'd like to offer the tools to enable someone to build it portably, but it's not going into the description.
Well, if the tools were there that would be an important start. I think that at least supporting this sort of extension should be considered.
I'd need to see some code for this, it's a neat idea, but the implementation is likely hairy. Can you point to an existing implementation?
No :-)
This is already possible to some extend using symbol shadowing. I would not want to allow "operator overloading" like in C++ (shudder) Why not?
Because it's crap, and we have SHADOW.
Well, that's not an argument; I like it. I think it's great. How does this essentially differ from the generalized sequences proposal? If you called it generalized numbers would you be happier?
Possibly kill the many bizarrely named or strangely parameterised and hardly used functions like `get'.
Not going to happen.. ever. I used GET just yesterday, and we will not be 'killing' anything. If you don't like it, don't use it, but removing functionality and CL compatibility is not worth it, and not a goal of CLtL3.
"Kill" gets to far - this would break ANSI CL compatibility. We could deprecate their use and don't import them in the CLtL3 package(s) - so one could use CL:GET if one wants to. ,
We will also _not_ be deprecating things from CL because they are bizarrely name or parameterized . If anything i'd like to un-deprecate a few things ANSI sent down (*-if-not, i',m looking at you).
I meant that using the CLTL3 package should not mean that the more unfortunate functions from CL be automatically imported. You cannot bind (flet ((get (...))) for example, and while I have indeed used get (unfortunately, more than a week ago so not on a par with your mastery of it), I think it is unfortunate that a specific functionality managed to grab a generic three letter name. I guess we will have to wait for Attila Lendvai to complete his redesign of the Lisp namespace and CLTL3 will be for other things ;-)
Backtraces are badly implemented by many projects and would be used by more if they weren't so tricky.
Trivial-backtrace exists, so there is some need for this. It's covered under 'Editing and Introspection' in the charter.
Trivial backtrace and the portable backtrace functionality in it is just a start. Backtraces are (badly) implemented in many server projects. I didn't understand that you'd gone so far to fixing on your charter. Sorry for butting in; have fun!
participants (5)
-
Drew Crampsie -
Gustavo -
Jochen Schmidt -
John Fremlin -
Robert Uhl