Bug report: COMPILE-FILE, COMPILE-FILE-PATHNAME disagree on output dir
COMPILE-FILE-PATHNAME is supposed to return "the pathname that COMPILE-FILE would write into, if given the same arguments" but in the example below, COMPILE-FILE-PATHNAME returns "src/obj/foo.abcl" while the file that is actually written into is "obj/foo.abcl". Looking at the spec, my opinion is that COMPILE-FILE is doing the right thing, and COMPILE-FILE-PATHNAME should be changed to match it. I suppose some might disagree with me about which behavior is correct, but no one can disagree that the two functions' behavior should match. $ mkdir src obj $ echo '(format t "~&Hello, world!~%")' > src/foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.179 seconds. Startup completed in 0.828 seconds. Type ":help" for a list of available commands. CL-USER(1): (defvar *infile* (make-pathname :directory '(:relative "src") :name "foo" :type "lisp")) *INFILE* CL-USER(2): (defvar *outfile* (make-pathname :directory '(:relative "obj") :type "abcl" :defaults *infile*)) *OUTFILE* CL-USER(3): (compile-file *infile* :output-file *outfile*) ; Compiling /tmp/foo/src/foo.lisp ... ; (FORMAT T ...) ; Wrote /tmp/foo/obj/foo.abcl (0.014 seconds) #P"/tmp/foo/obj/foo.abcl" NIL NIL CL-USER(4): (compile-file-pathname *infile* :output-file *outfile*) #P"/tmp/foo/src/obj/foo.abcl" CL-USER(5):
I wrote:
Looking at the spec, my opinion is that COMPILE-FILE is doing the right thing, and COMPILE-FILE-PATHNAME should be changed to match it.
After spending some more time reading the spec, I now believe that I was wrong above, and that COMPILE-FILE behaved incorrectly in my test. I now believe that the pathname that COMPILE-FILE should write into is the one that the code below returns. (defun compile-file-pathname (input-file &key output-file &allow-other-keys) (let ((defaults (make-pathname :type "abcl" :defaults (merge-pathnames input-file)))) (cond ((null output-file) defaults) (t (merge-pathnames output-file defaults))))) The function above may be helpful in fixing "Wrong file type returned by COMPILE-FILE-PATHNAME" (ticket #476) and "COMPILE-FILE, COMPILE-FILE-PATHNAME disagree on output dir" (not yet ticketed). It is my own original work, and I release it into the public domain, no rights reserved.
On Nov 3, 2020, at 03:27, Robert Munyer <2433647181@munyer.com> wrote:
[…]
The function above may be helpful in fixing "Wrong file type returned by COMPILE-FILE-PATHNAME" (ticket #476) and "COMPILE-FILE, COMPILE-FILE-PATHNAME disagree on output dir" (not yet ticketed). It is my own original work, and I release it into the public domain, no rights reserved.
Your implementation of COMPILE-FILE-PATHNAME has now been merged into [abcl-1.8.1-dev][1][2]. Thanks for your analysis, thinking, and implementation here. This will be part of abcl-1.8.1 which will be released in the next couple weeks, pending further shakedown of abcl-1.8.0 issues. Until then, you will need to build abcl from source to get a version which includes your fix. I have closed [ticket 476][476] as your implementation addresses the problems you uncovered, and continues to pass the CI tests at the same level as without. [476]: https://abcl.org/trac/ticket/476 [1]: https://abcl.org/trac/changeset/15480 [2]: https://github.com/armedbear/abcl/commit/d6e3b2f3e02807dc39a05571897f77a0588... -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."
Your implementation of COMPILE-FILE-PATHNAME has now been merged into [abcl-1.8.1-dev][1][2].
Thanks for your analysis, thinking, and implementation here.
This will be part of abcl-1.8.1 which will be released in the next couple weeks, pending further shakedown of abcl-1.8.0 issues. Until then, you will need to build abcl from source to get a version which includes your fix.
I have closed [ticket 476][476] as your implementation addresses the problems you uncovered, and continues to pass the CI tests at the same level as without.
COMPILE-FILE-PATHNAME seems good now, but I'm still finding pathname- related bugs in COMPILE-FILE itself. The latest is one where it throws a "file ... does not exist" error while trying to compress the temporary files that it has just created into an archive. I'll paste a demo of that bug at the end of this message. If you remove this assignment statement from the source of COMPILE-FILE: (setf output-file (make-pathname :defaults (if output-file (merge-pathnames output-file *default-pathname-defaults*) (compile-file-pathname input-file)) :version nil)) , and replace it with this one: (setf output-file (compile-file-pathname input-file :output-file output-file)) , then the pathname-related bugs that I've been noticing go away. I see that the existing code is going out of its way to force the pathname version number to NIL. I don't know if there is a good reason for it to be doing that. If there is a good reason for it, it should be done in COMPILE-FILE-PATHNAME, not in COMPILE-FILE. Here is a COMPILE-FILE-PATHNAME that forces the version number; use this if forcing the version number is actually desirable. (defun compile-file-pathname (input-file &key output-file &allow-other-keys) (let ((defaults (make-pathname :type *compile-file-type* :defaults (merge-pathnames input-file)))) (make-pathname :version nil :defaults (cond ((null output-file) defaults) (t (merge-pathnames output-file defaults)))))) The 2nd and 3rd of the three code snippets above are my own original work, and I release them into the public domain, no rights reserved. Here's the report of the "file ... does not exist" error: The code below works in CCL, CLISP and SBCL, but not in ABCL. It _does_ work in ABCL if you replace the first code snippet shown above with the second, the one that's only two lines long. $ echo '(format t "~&Hello, world!~%")' > bar.lisp $ java -jar abcl-231e00ef.jar Armed Bear Common Lisp 1.8.1-dev Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.363 seconds. Startup completed in 1.35 seconds. Type ":help" for a list of available commands. CL-USER(1): (compile-file (make-pathname :name "bar" :type "lisp") :output-file (make-pathname :name "bar-baz")) ; Compiling /tmp/foo/bar.lisp ... ; (FORMAT T ...) #<THREAD "interpreter" {430133BA}>: Debugger invoked on condition of type FILE-ERROR The file /tmp/foo/bar-baz does not exist. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):
On Nov 8, 2020, at 22:52, Robert Munyer <2433647181@munyer.com> wrote:
[…]
FILE-PATHNAME seems good now, but I'm still finding pathname- related bugs in COMPILE-FILE itself. The latest is one where it throws a "file ... does not exist" error while trying to compress the temporary files that it has just created into an archive. I'll paste a demo of that bug at the end of this message.
If you remove this assignment statement from the source of COMPILE-FILE:
(setf output-file (make-pathname :defaults (if output-file (merge-pathnames output-file *default-pathname-defaults*) (compile-file-pathname input-file)) :version nil))
, and replace it with this one:
(setf output-file (compile-file-pathname input-file :output-file output-file))
, then the pathname-related bugs that I've been noticing go away.
It makes a lot of sense to use COMPILE-FILE-PATHNAME now that it does what we want. Applied as [2][] in github.com pull [355][]. Thanks for the code. [355]: <https://github.com/armedbear/abcl/pull/355> [2]: <https://github.com/armedbear/abcl/pull/355/commits/bbf8321f7cb58cc92148b2c8382b3c8832eeeb64>
I see that the existing code is going out of its way to force the pathname version number to NIL. I don't know if there is a good reason for it to be doing that. If there is a good reason for it, it should be done in COMPILE-FILE-PATHNAME, not in COMPILE-FILE.
Agreed to ignore the PATHNAME-VERSION complications, until someone argues for restoring them. I can’t think of any current filesystems that the Bear encounters where a version would have an unambiguous meaning which would be portable to the ones I commonly use. best, Mark -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."
On Nov 11, 2020, at 16:51, Mark Evenson <evenson@panix.com> wrote:
On Nov 8, 2020, at 22:52, Robert Munyer <2433647181@munyer.com> wrote:
[…]
FILE-PATHNAME seems good now, but I'm still finding pathname- related bugs in COMPILE-FILE itself. The latest is one where it throws a "file ... does not exist" error while trying to compress the temporary files that it has just created into an archive. I'll paste a demo of that bug at the end of this message.
Your patch has been [now applied in 1.8.1-dev][1]. [1]: <https://gitlab.common-lisp.net/abcl/abcl/-/commit/344b4f667f7fc6721a95f5f8f6a2adef4165bfe0> Please let me know if this doesn’t (finally) fix your problems. -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."
participants (2)
-
Mark Evenson -
Robert Munyer