This topic came up at the Boston Lisp meeting last night. Here was my takeaway from the discussion.
ASDF does not need to support arbitrary encodings in system definition files. UTF-8 is the simplest and most portable solution for non-ASCII characters in system definitions.
Thus ASDF can simply declare that all system definitions are read using UTF-8. On systems where unicode support is not available, system definitions will be read as "extended ASCII", and the non-ASCII characters will simply print incorrectly. Filenames should only be specified using ASCII characters to avoid portability issues.
Authors do need a way to support arbitrary encodings in other source files, but this should be restricted to cases where there is an actual need to use a non-UTF-8 character set. Example use cases are locale-specific libraries, a toolchain that converts between character sets, or possibly a character-set-detection library. Again, other character sets can be a major portability issue.
There are only a handful of libraries that use something other than ASCII or UTF-8. These should be easy to fix, given approval from their authors.
Hopefully that's an accurate record of the general consensus and this writeup will be helpful to others in some way.
- Daniel