Dear Lispers,
The most significant bit for this email is: DO YOU HAVE A source-registry.conf ON WINDOWS? or do you know someone who has? If so, please contact me, either on this list or in private, because I'm considering a backward-incompatible change in its location.
(If you use or don't use source-registry.conf on Linux, that can be an interesting data point, too, for comparison of impact.)
If not, then there will be no negative impact to the proposed change.
I'm having a hard time finding an affected Windows user who cares, which may be a sign that there is none and I can proceed, or that I'm simply disconnected from the user base.
Right now, ASDF tries to do things kind-of-the-windows-way, where "common-lisp/" replaces the vendor/app/ path component. Thus, it puts its configuration in %LOCALAPPDATA%/common-lisp/config/ its cache in %LOCALAPPDATA%/common-lisp/cache/ and for source code in %LOCALAPPDATA%/common-lisp/source/
I'm considering dropping the Windows special-casing, and offering the XDG model on all platforms, including Windows. That would mean the configuration would instead go in %LOCALAPPDATA%/Configuration/common-lisp/ and the cache in %LOCALAPPDATA%/Cache/common-lisp/ while source code remains in %LOCALAPPDATA%/common-lisp/source/ with these the conceptual mapping XDG_CONFIG_HOME=%LOCALAPPDATA%/Configuration instead of ~/.config/ and XDG_DATA_HOME=%LOCALAPPDATA%/ instead of ~/.local/share/ and XDG_CACHE_HOME=%LOCALAPPDATA%/Cache/ instead of ~/.cache/
The advantage of the change would be that ASDF would offer to Lisp programs a uniform XDG view on all platforms, which has some costs in terms of being slightly unfamiliar to Windows users and a transition issue for some of them, but going forward allows for: * all files can be redirected on all platforms using the XDG_CONFIG_HOME environment variable and its siblings. * the same directory concepts on all platforms. * slightly simpler code for the ASDF maintainers, and much simpler documentation: no need to explain vast subtle discrepancies from one platform to the next. * the ability to handle backup of configuration and skipping of backup of cache in a simpler way than with the usual Windows semi-convention. * a more uniform API for the Lisp programmer
The main disadvantage is that this setup is somewhat different from current conventions on Windows. On the other hand, conventions on Unix were similar to those on Windows some ten years back, and have improved thanks to XDG -- maybe they can evolve for the better on Windows, too, in the next ten years.
To me, the big issue is with transition for people who currently use a source-registry.conf on Windows. Since ASDF will start to look for it at a different location, their setup will need to be amended when they upgrade ASDF.
What do you think? Does anyone care at all? Does anyone actually have a common-lisp/config/source-registry.conf file and be confused if the location changes?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Corollaries to the Law of Bitur-Camember: The political process destroys the value of all known resources that are up for grabs. The socialist process of systematically denying legitimacy to property rights applies the political process universally and destroys the value of all available resources.
On 2015/4/29 07:13, Faré wrote:
Dear Lispers,
The most significant bit for this email is: DO YOU HAVE A source-registry.conf ON WINDOWS? or do you know someone who has? If so, please contact me, either on this list or in private, because I'm considering a backward-incompatible change in its location.
[…]
For systems that build production code under Windows with ABCL running asdf-3.1.4, we use the the user configuration directory file:~/.config/common-lisp/source-registry.conf.d/ (it always seemed conceptually easier to have many small files that may be easily moved than placing all the forms in a single file:~/.config/common-lisp/source-registry.conf.) At production runtime, we explicitly "neuter" the ASDF configuration strategy via the appropriate ASDF API, as we are running from within a Java web application archive (to be described in a long overdue ABCL-DEV blog post).
As I understand your proposal, even though we don't use the "source-registry.conf" file, we would still be effected by your proposed change, as the value of XDG_CONFIG_HOME would be based on %LOCALAPPDATA%, not %USERPROFILE% as is currently the case.
We would probably not use the XDG_CONFIG_HOME environment variable, instead adapting to ASDF defaults, as it would be additional complexity to describe to operations and one more thing that could go wrong.
As long as you make your proposed transition across a minor revision (aka as ASDF 3.2.x), I do not have a problem with your proposal. For the record, although none of your listed benefits of this change would give us any immediate advantage that I can determine, I support the drive towards greater cross-platform uniformity.
Dear Mark,
~/.config/common-lisp/source-registry.conf.d/ is being used because you're compiling on Unix. If you were using Windows, it would be using something else based on LOCALAPPDATA, not USERPROFILE.
For reliable builds, especially ones that create cross-platform objects, we recommend you don't use either, but explicitly pass a proper (or null) overriding configuration to the initialization functions initialize-source-registry and initialize-output-translations.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I've never understood why it is 'greed' to want to keep the money you've earned but not greed to want to take somebody else's money. — Thomas Sowell
On Tue, Apr 28, 2015 at 11:34 PM, Mark Evenson evenson@panix.com wrote:
On 2015/4/29 07:13, Faré wrote:
Dear Lispers,
The most significant bit for this email is: DO YOU HAVE A source-registry.conf ON WINDOWS? or do you know someone who has? If so, please contact me, either on this list or in private, because I'm considering a backward-incompatible change in its location.
[…]
For systems that build production code under Windows with ABCL running asdf-3.1.4, we use the the user configuration directory file:~/.config/common-lisp/source-registry.conf.d/ (it always seemed conceptually easier to have many small files that may be easily moved than placing all the forms in a single <file:~/.co
ASDF configuration and cache
Inbox x
Faré fahree@gmail.com
10:13 PM (2 hours ago) to ASDF-devel Dear Lispers,
The most significant bit for this email is: DO YOU HAVE A source-registry.conf ON WINDOWS? or do you know someone who has? If so, please contact me, either on this list or in private, because I'm considering a backward-incompatible change in its location.
(If you use or don't use source-registry.conf on Linux, that can be an interesting data point, too, for comparison of impact.)
If not, then there will be no negative impact to the proposed change.
I'm having a hard time finding an affected Windows user who cares, which may be a sign that there is none and I can proceed, or that I'm simply disconnected from the user base.
Right now, ASDF tries to do things kind-of-the-windows-way, where "common-lisp/" replaces the vendor/app/ path component. Thus, it puts its configuration in %LOCALAPPDATA%/common-lisp/config/ its cache in %LOCALAPPDATA%/common-lisp/cache/ and for source code in %LOCALAPPDATA%/common-lisp/source/
I'm considering dropping the Windows special-casing, and offering the XDG model on all platforms, including Windows. That would mean the configuration would instead go in %LOCALAPPDATA%/Configuration/common-lisp/ and the cache in %LOCALAPPDATA%/Cache/common-lisp/ while source code remains in %LOCALAPPDATA%/common-lisp/source/ with these the conceptual mapping XDG_CONFIG_HOME=%LOCALAPPDATA%/Configuration instead of ~/.config/ and XDG_DATA_HOME=%LOCALAPPDATA%/ instead of ~/.local/share/ and XDG_CACHE_HOME=%LOCALAPPDATA%/Cache/ instead of ~/.cache/
The advantage of the change would be that ASDF would offer to Lisp programs a uniform XDG view on all platforms, which has some costs in terms of being slightly unfamiliar to Windows users and a transition issue for some of them, but going forward allows for: * all files can be redirected on all platforms using the XDG_CONFIG_HOME environment variable and its siblings. * the same directory concepts on all platforms. * slightly simpler code for the ASDF maintainers, and much simpler documentation: no need to explain vast subtle discrepancies from one platform to the next. * the ability to handle backup of configuration and skipping of backup of cache in a simpler way than with the usual Windows semi-convention. * a more uniform API for the Lisp programmer
The main disadvantage is that this setup is somewhat different from current conventions on Windows. On the other hand, conventions on Unix were similar to those on Windows some ten years back, and have improved thanks to XDG -- maybe they can evolve for the better on Windows, too, in the next ten years.
To me, the big issue is with transition for people who currently use a source-registry.conf on Windows. Since ASDF will start to look for it at a different location, their setup will need to be amended when they upgrade ASDF.
What do you think? Does anyone care at all? Does anyone actually have a common-lisp/config/source-registry.conf file and be confused if the location changes?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Corollaries to the Law of Bitur-Camember: The political process destroys the value of all known resources that are up for grabs. The socialist process of systematically denying legitimacy to property rights applies the political process universally and destroys the value of all available resources.
Mark Evenson via common-lisp.net
11:34 PM (1 hour ago) to asdf-devel On 2015/4/29 07:13, Faré wrote:
Dear Lispers,
The most significant bit for this email is: DO YOU HAVE A source-registry.conf ON WINDOWS? or do you know someone who has? If so, please contact me, either on this list or in private, because I'm considering a backward-incompatible change in its location.
[…]
For systems that build production code under Windows with ABCL running asdf-3.1.4, we use the the user configuration directory file:~/.config/common-lisp/source-registry.conf.d/ (it always seemed conceptually easier to have many small files that may be easily moved than placing all the forms in a single file:~/.config/common-lisp/source-registry.conf.) At production runtime, we explicitly "neuter" the ASDF configuration strategy via the appropriate ASDF API, as we are running from within a Java web application archive (to be described in a long overdue ABCL-DEV blog post).
As I understand your proposal, even though we don't use the "source-registry.conf" file, we would still be effected by your proposed change, as the value of XDG_CONFIG_HOME would be based on %LOCALAPPDATA%, not %USERPROFILE% as is currently the case.
We would probably not use the XDG_CONFIG_HOME environment variable, instead adapting to ASDF defaults, as it would be additional complexity to describe to operations and one more thing that could go wrong.
As long as you make your proposed transition across a minor revision (aka as ASDF 3.2.x), I do not have a problem with your proposal. For the record, although none of your listed benefits of this change would give us any immediate advantage that I can determine, I support the drive towards greater cross-platform uniformity. Mark Evenson (panix.com), ASDF-devel (asdf-devel@common-lisp.net) Dear Mark,
Saved 9.96 GB (66%) of 15 GB used Manage Last account activity: 1 hour ago Details nfig/common-lisp/source-registry.conf>.) At production
runtime, we explicitly "neuter" the ASDF configuration strategy via the appropriate ASDF API, as we are running from within a Java web application archive (to be described in a long overdue ABCL-DEV blog post).
As I understand your proposal, even though we don't use the "source-registry.conf" file, we would still be effected by your proposed change, as the value of XDG_CONFIG_HOME would be based on %LOCALAPPDATA%, not %USERPROFILE% as is currently the case.
We would probably not use the XDG_CONFIG_HOME environment variable, instead adapting to ASDF defaults, as it would be additional complexity to describe to operations and one more thing that could go wrong.
As long as you make your proposed transition across a minor revision (aka as ASDF 3.2.x), I do not have a problem with your proposal. For the record, although none of your listed benefits of this change would give us any immediate advantage that I can determine, I support the drive towards greater cross-platform uniformity.
-- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On 2015/4/29 10:03, Faré wrote:
Dear Mark,
~/.config/common-lisp/source-registry.conf.d/ is being used because you're compiling on Unix. If you were using Windows, it would be using something else based on LOCALAPPDATA, not USERPROFILE.
Hmmm. Do you mean "compiling [ABCL] on Unix"? That is not the case, as the ABCL being used has been compiled on the Windows host. And the creation of the WAR artifact for production is also happening on the Windows host.
While running on Windows, everything *is* being compiled "under" Cygwin. Maybe that is the cause?
For reliable builds, especially ones that create cross-platform objects, we recommend you don't use either, but explicitly pass a proper (or null) overriding configuration to the initialization functions initialize-source-registry and initialize-output-translations.
I'm a little unsure what you are suggesting here, perhaps because I am using a strategy outside of the "normal" use of ASDF. This strategy can roughly be described as follows: At build time for the WAR, I need to locate all the ASDF systems in a running ABCL image. This allows me to package both Quicklisp available systems with local ones in a consistent manner that freezes their versions, and doesn't rely on access to the network at production time. Then I iterate across all the dependencies performing an rsync of all the source artifacts into a designated build directory that is subsequently packaged in the WAR deployment artifact. Here, I assume that the ".asd" file lies at the top of the source artifact file hierarchy, an assumption which seems to largely work with the ASDF systems I am utilizing at the penalty of including sub ".asd" systems twice (since they are the same system, this inefficiency doesn't really matter although it should be cleaned up). At runtime, I use ASDF:INITIALIZE-SOURCE-REGISTRY to explicitly refer to the directory containing the source artifacts, which is now located within the WAR artifact. ABCL pathnames are able to specify locations within the zip file which constitutes the WAR artifact. After this manual configuration of the source registry, the systems are available by the usual ASDF:LOAD-SYSTEM mechanism.
Under this scheme is there an application of "proper (or null) overriding configuration" that you recommend using?
Best, Mark
On Wed, Apr 29, 2015 at 2:00 AM, Mark Evenson evenson@panix.com wrote:
On 2015/4/29 10:03, Faré wrote:
Dear Mark,
~/.config/common-lisp/source-registry.conf.d/ is being used because you're compiling on Unix. If you were using Windows, it would be using something else based on LOCALAPPDATA, not USERPROFILE.
Hmmm. Do you mean "compiling [ABCL] on Unix"? That is not the case, as the ABCL being used has been compiled on the Windows host. And the creation of the WAR artifact for production is also happening on the Windows host.
While running on Windows, everything *is* being compiled "under" Cygwin. Maybe that is the cause?
If you have :unix in your *features*, then ASDF decides that ABCL is running on Unix. Is that not a correct criteria? Are you enabling :unix because of cygwin?
Looking at the code, I realize that it was misinterpreting the XDG standard in consulting ~/.config/ in addition to the XDG_CONFIG_HOME (when defined) rather than instead of it as a fallback as undefined — fixing this bug originally reported by Jason Miller was the initial reason for this upheaval of configuration paths. This made that path work under Windows when it wasn't actually meant to work. Ouch. On Windows, I intended only the local-appdata and else appdata to be consulted. Ouch. So fixing the code means we'll be backward-incompatible, anyway. Ouch.
Well, we might as well do the Right Thing™ if we're going to break compatibility anyway. What is the Right Thing?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If all values are relative, then cannibalism is a matter of taste. — Leo Strauss
For reliable builds, especially ones that create cross-platform objects, we recommend you don't use either, but explicitly pass a proper (or null) overriding configuration to the initialization functions initialize-source-registry and initialize-output-translations.
I'm a little unsure what you are suggesting here, perhaps because I am using a strategy outside of the "normal" use of ASDF. This strategy can roughly be described as follows: At build time for the WAR, I need to locate all the ASDF systems in a running ABCL image. This allows me to package both Quicklisp available systems with local ones in a consistent manner that freezes their versions, and doesn't rely on access to the network at production time. Then I iterate across all the dependencies performing an rsync of all the source artifacts into a designated build directory that is subsequently packaged in the WAR deployment artifact. Here, I assume that the ".asd" file lies at the top of the source artifact file hierarchy, an assumption which seems to largely work with the ASDF systems I am utilizing at the penalty of including sub ".asd" systems twice (since they are the same system, this inefficiency doesn't really matter although it should be cleaned up). At runtime, I use ASDF:INITIALIZE-SOURCE-REGISTRY to explicitly refer to the directory containing the source artifacts, which is now located within the WAR artifact. ABCL pathnames are able to specify locations within the zip file which constitutes the WAR artifact. After this manual configuration of the source registry, the systems are available by the usual ASDF:LOAD-SYSTEM mechanism.
Under this scheme is there an application of "proper (or null) overriding configuration" that you recommend using?
Best, Mark
-- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On 2015/4/29 11:24, Faré wrote:
On Wed, Apr 29, 2015 at 2:00 AM, Mark Evenson evenson@panix.com wrote:
On 2015/4/29 10:03, Faré wrote:
Dear Mark,
~/.config/common-lisp/source-registry.conf.d/ is being used because you're compiling on Unix. If you were using Windows, it would be using something else based on LOCALAPPDATA, not USERPROFILE.
Hmmm. Do you mean "compiling [ABCL] on Unix"? That is not the case, as the ABCL being used has been compiled on the Windows host. And the creation of the WAR artifact for production is also happening on the Windows host.
While running on Windows, everything *is* being compiled "under" Cygwin. Maybe that is the cause?
If you have :unix in your *features*, then ASDF decides that ABCL is running on Unix. Is that not a correct criteria? Are you enabling :unix because of cygwin?
Please re-explain your reasons for thinking that ABCL is mistakenly using file:%USERPROFILE%/.config/ as we're not pushing :UNIX to *FEATURES* as far as I can see:
CIMS-dev-01:~/work/abcl$ uname -a CYGWIN_NT-6.1 CIMS-dev-01 1.7.35(0.287/5/3) 2015-03-04 12:09 x86_64 Cygwin CIMS-dev-01:~/work/abcl$ ./abcl.bat VM settings: Max. Heap Size (Estimated): 4.00G Ergonomics Machine Class: client Using VM: Java HotSpot(TM) 64-Bit Server VM
Armed Bear Common Lisp 1.4.0-dev Java 1.8.0_40 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.742 seconds. Startup completed in 5.988 seconds. ; Loading jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/abcl-contrib.abcl ... ; Loading jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/asdf.abcl ... ; Loaded jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/asdf.abcl (7.577 seconds) Using probed value of abcl-contrib: 'C:/Users/markeven/work/abcl/dist/abcl-contrib.jar'. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/quicklisp/ to ASDF. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/mvn/ to ASDF. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/jss/ to ASDF. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/jfli/ to ASDF. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/asdf-jar/ to ASDF. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/asdf-install/ to ASDF. Added jar:file:C:/Users/markeven/work/abcl/dist/abcl-contrib.jar!/abcl-asdf/ to ASDF. ; Loaded jar:file:C:/Users/markeven/work/abcl/dist/abcl.jar!/org/armedbear/lisp/abcl-contrib.abcl (7.694 seconds) Loading C:\Users\markeven.abclrc completed in 7.697 seconds. Type ":help" for a list of available commands. CL-USER(1): (find :unix *features*) NIL CL-USER(2): *features* (:ASDF-PACKAGE-SYSTEM :ASDF3.1 :ASDF3 :ASDF2 :ASDF :OS-WINDOWS :ASDF-UNICODE :ABCL-BUNDLE-OP-SUPPORTED :DRAKMA-NO-SSL :HUNCHENTOOT-NO-SSL :X86-64 :WINDOWS :ARMEDBEAR :ABCL :COMMON-LISP :ANSI-CL :CDR6 :MOP :PACKAGE-LOCAL-NICKNAMES)
On Wed, Apr 29, 2015 at 2:38 AM, Mark Evenson evenson@panix.com wrote:
~/.config/common-lisp/source-registry.conf.d/ is being used because you're compiling on Unix. If you were using Windows, it would be using something else based on LOCALAPPDATA, not USERPROFILE.
Please re-explain your reasons for thinking that ABCL is mistakenly using file:%USERPROFILE%/.config/ as we're not pushing :UNIX to *FEATURES* as far as I can see:
Indeed that was a bug in ASDF, that will be fixed in next release. To preserve backward compatibility with this bug on Windows, you'll have to export XDG_CONFIG_HOME=%USERPROFILE%/.config/ Also, if you think that it's OK to import this Unixism to the Windows world, we could make that the default instead of %LOCALAPPDATA%/Configuration/ Problem being most users might want to ATTRIB +h %USERPROFILE%/.config/ which ASDF won't do for you.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org No one can make you feel inferior without your consent — Eleanor Roosevelt But you're only fooling yourself if you can't recognize your superiors — #f
-
Faré -
Mark Evenson