On 04/09/2012 10:37 AM, Faré wrote:
On Sun, Apr 8, 2012 at 15:28, Nikodemus Siivola nikodemus@random-state.net wrote:
On 8 April 2012 17:36, Faré fahree@gmail.com wrote:
I think requiring a few marginal hackers doing weird things to specifiy :encoding :default is a small price to pay for everyone to be able to specify
I disagree. Consider this:
X has a system that used to be in, say, LATIN-9. He uses latin-9 at home, and everything works fine. His users either use it as well, or at least another single-byte encoding.
ASDF is updated, and X's user reports breakage. Everything works fine for X, because he didn't update ASDF yet. So he updates ASDF, and X updates his system to specify :LATIN-9 (or :DEFAULT, or whatever).
Now another of his users reports breakage, because /they/ didn't update ASDF yet -- and their ASDF doesn't support :ENCODING, so things break. They update ASDF, which in turn breaks another :LATIN-N system they were using.
The potential cost is non-trivial, and I really don't pretend to know eg. how many Japanese hackers user non-UTF-encodings in their source.
IMO encouraging people to add :encoding :utf-8 is much saner.
I agree that transition costs must be considered.
Let's examine the two scenarios, where the default is :default vs where the default is :utf-8.
In both cases, crucial points follow: (a) currently :encoding is NOT supported by ASDF. (b) therefore, whenever anyone modifies his defsystem to use :encoding, his system will NOT be backward compatible anymore. (c) we want to make most code as backward-compatible as possible. (d) the application programmer controls what version of ASDF is installed, the library developer doesn't.
If the default is :utf-8 (my recommendation), then
- A few programmers of non-UTF-8 applications may hit a snag upgrading ASDF;
- then they can either use asdf-encodings or use :encoding :default.
- Their code is then not compatible with older ASDFs anymore, but
- as application programmers, they fully control which ASDF they use, and
- even if they need to support old CL implementations,
ASDF still supports them (the exception being GCL, that looks quite dead).
- Meanwhile, library authors can already start migrating to UTF-8,
and everyone who upgrades ASDF can reliably enjoy now the benefits of non-ASCII, while preserving backwards-compatibility.
Won't library authors need to wait until their user base has upgraded ASDF before they can start migrating to UTF-8? The external-format support helps write portable libraries using non-ASCII characters and is only available after an upgrade.
I do see a concern that if developers are required to change their definitions to add :encoding :default then they will be forced to also make sure their user base has upgraded now. Further if their users do upgrade ASDF then it breaks again - there is no migration path for them.
If the default is :default (your recommendation), then
- library developers can't ensure their code use a predictable encoding;
- this makes any attempt to actually use of non-ASCII characters unreliable.
- Sure, they might be tempted to use :encoding :utf-8, but then
their libraries will be gratuitously incompatible with anyone who hasn't upgraded his ASDF, which is a pain to users.
Perhaps the difference is that portable UTF-8 source is new source and requires an upgrade of ASDF anyway, whereas making the default :utf-8 forces :encoding :default on current users and affects legacy code that is already written without a migration path.
- thus, library developers can do nothing but wait for EVERYONE
to be using a recent ASDF before they can do anything.
Wouldn't this be the reality for portable libraries no matter which default is chosen?
- Therefore, noone will enjoy any benefit of :encoding for a year,
and when we do, it will cause massive backward incompatibility.
I don't appreciate the 'massive backward incompatibility' so perhaps do not understand your perspective? I see that future projects using UTF-8 source would need to declare this in the system definition, but this would not seem to qualify.
Choosing :default would seem to cause the least backward incompatibility as this is the current behaviour, and offers a migration path to get ASDF upgrades in place.
Admittedly, in either case, library developers could use such conditional reading as in #+asdf-unicode #:asdf-unicode :encoding :utf-8 or #+asdf-unicode :encoding #:asdf-unicode :latin1 to make their libraries safer in a backwards-compatible way.
It would be great if some suggestions like this could be offered to ease the transition.
In both cases, library developers are encouraged to use UTF-8, which already most people do, if only that tends to be the default these days for users of SBCL and they send bug reports to library authors.
A default of :default allows a few odd non-UTF8 application developers to continue using unportable hacks for a few months, while forcing everyone else to wait to do anything and bringing massive backwards-incompatibility of libraries in the end.
What 'massive backwards-incompatibility' would be caused by making :default the default?
Most portable libraries are ASCII, and there would be some benefit in libraries needing UTF-8 support to declare this in the system definition.
A default of :utf-8 forces these few odd non-UTF8 application developers to do a documented step before they continue doing what they are doing, at their own upgrade pace (they control when to upgrade ASDF); they can then replace a lot of non-portable hacks with a portable :encoding. Meanwhile, everyone starts enjoying reliable non-ASCII today.
There may be a concern that their users would have to upgrade ASDF now.
How can everyone enjoy reliable non-ASCII today, without the user base having upgraded ASDF?
I agree there's no solution that makes everyone happy. I believe that a default of :utf-8 doesn't actually make anyone terribly unhappy, and empowers everyone to make the changes they need, without requiring for anyone to wait for other people to make changes (except indeed for a few stray libraries). And so my plans are unchanged for now (but of course, please keep sending the complaints if you think it's wrongheaded; it's still time to not do it).
NB: I'd especially like the opinion of people who actually develop non-ASCII and non-UTF8 libraries or applications.
PS: I just made asdf-encodings much less dumb. I added good support for: sbcl, clozure, clisp, ecl, cmu I added dubious support for: abcl, allegro, scl, lispworks I think these will remain 8-bit only: cormanlisp, gcl, genera, rmcl, xcl. Precious little testing so far, except that it doesn't break everything on SBCL. Help welcome to test and expand it. ssh://common-lisp.net/project/asdf/git/asdf-encodings.git git://common-lisp.net/projects/asdf/asdf-encodings.git http://common-lisp.net/gitweb?p=projects/asdf/asdf-encodings.git
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Every major horror of history was committed in the name of an altruistic motive. — Ayn Rand