In 2004 I submitted a CLRFI which was then accepted as CLRFI-2. It specified a new function COMPILED-FILE-P which was supposed to be useful for system construction utilities, e.g., asdf:
Function (compiled-file-p file-name) ==> valid-p
Returns true if the file appears to be a valid compiled file (i.e., exists, is readable, and the contents appears to be valid for this implementation), false otherwise.
Implementations are required to inspect the contents (e.g., checking just the pathname type is not sufficient). Although the completeness of the inspection is not required, this function should be able to detect, e.g., file format changes between versions.
I re-submitted the spec to the CDR editors a week ago. No response so far.
The detailed html spec is attached.
Note that for the sake of being able to simultaneously support several implementations on the same computer (or network), asdf has taken the route of trying to segregate fasl files and any other build output (e.g. C files and Lisp files generated by CFFI) to directories the name of which depend on the implementation, its version, the operating system and architecture, etc.
XCVB hopes to eventually go further and identify build output by cryptographic digest of input, including underlying build tools as well as input dependencies.
This segregation makes such output-inspection function much less useful.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org You think war is economically beneficial? Then why share those benefits with dirty foreigners? Let's have a civil war — A war we're sure to win!
On Tue, May 15, 2012 at 11:54 AM, Sam Steingold sds@gnu.org wrote:
In 2004 I submitted a CLRFI which was then accepted as CLRFI-2. It specified a new function COMPILED-FILE-P which was supposed to be useful for system construction utilities, e.g., asdf:
Function (compiled-file-p file-name) ==> valid-p
Returns true if the file appears to be a valid compiled file (i.e., exists, is readable, and the contents appears to be valid for this implementation), false otherwise.
Implementations are required to inspect the contents (e.g., checking just the pathname type is not sufficient). Although the completeness of the inspection is not required, this function should be able to detect, e.g., file format changes between versions.
I re-submitted the spec to the CDR editors a week ago. No response so far.
The detailed html spec is attached.
-- Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000 http://www.childpsy.net/ http://mideasttruth.com http://ffii.org http://www.memritv.org http://jihadwatch.org http://camera.org http://memri.org You cannot fire me. Slaves are not fired. Slaves are sold.
pro mailing list pro@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro
- Faré snuerr@tznvy.pbz [2012-05-15 12:58:29 -0400]:
Note that for the sake of being able to simultaneously support several implementations on the same computer (or network), asdf has taken the route of trying to segregate fasl files and any other build output (e.g. C files and Lisp files generated by CFFI) to directories the name of which depend on the implementation, its version, the operating system and architecture, etc.
I know, and I am not sure this is TRT. This approach leads to a propagation of directories and files which might uselessly duplicate each other on one hand, while not providing the appropriate flexibility on the other. E.g., CLISP compiles to platform-independent byte-code whose format changes relatively rarely, so having separate directories for clisp-ver-arch-os is a total waste, which also gives the users an impression that they cannot distribute clisp-fas files to different arches.
I just added abi-version to the spec to address this issue.
On Tue, May 15, 2012 at 4:26 PM, Sam Steingold sds@gnu.org wrote:
- Faré snuerr@tznvy.pbz [2012-05-15 12:58:29 -0400]:
Note that for the sake of being able to simultaneously support several implementations on the same computer (or network), asdf has taken the route of trying to segregate fasl files and any other build output (e.g. C files and Lisp files generated by CFFI) to directories the name of which depend on the implementation, its version, the operating system and architecture, etc.
I know, and I am not sure this is TRT. This approach leads to a propagation of directories and files which might uselessly duplicate each other on one hand, while not providing the appropriate flexibility on the other.
1) there is no fully reliable way to detect beforehand whether the file will be actual portable bytecode, or something that has platform-specific information, not only because of magic escapes (if they exist), but also because of cffi-extracted constants or other clever read-time, macroexpansion-time or compile-time evaluations.
2) I don't know what you mean by "not providing the appropriate flexibility". ASDF (and XCVB) make it fully user-configurable, and seggregate according to a wide variety of criteria that can be further compounded with e.g. hostname.
E.g., CLISP compiles to platform-independent byte-code whose format changes relatively rarely, so having separate directories for clisp-ver-arch-os is a total waste, which also gives the users an impression that they cannot distribute clisp-fas files to different arches.
How do you avoid the issues with embedded cffi constants? Certainly, a clever enough system could track that no non-portable operations happened during a compilation and validate fasls as portable after-the-fact. That's more clever than either CLISP or ASDF is. Or you could be optimistic until things break badly. Which is frankly a dumb way to save disk space -- but ASDF allows you to configure your system that way if you desire.
If you want some non-trivial feature, that would be more useful: a way to track whether a file was compiled portably, and/or a way to declare that it should and to trap it if it doesn't.
As for namespace issues. Should CDR's be available through (require :cdr3) with their contents in (find-package :cdr3) ?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org It's possible to program a computer in English. It's also possible to make an airplane controlled by reins and spurs. — John McCarthy
Hi,
I am one of the CDR editors, but I haven't seen your email. I have double-checked the email forwarding just now, and it seems to work. I cannot check anymore whether your email accidentally landed in my spam folder, because it has a retention period of a week.
Could you try to resend your CDR proposal, and then send a notification to my private address as well? If I then still don't receive your proposal via the editors email address, I have to dig deeper to try to see where the problem is.
In any case, I will process the CDR proposal next weekend.
Thanks a lot, Pascal
On 15 May 2012, at 17:54, Sam Steingold wrote:
In 2004 I submitted a CLRFI which was then accepted as CLRFI-2. It specified a new function COMPILED-FILE-P which was supposed to be useful for system construction utilities, e.g., asdf:
Function (compiled-file-p file-name) ==> valid-p
Returns true if the file appears to be a valid compiled file (i.e., exists, is readable, and the contents appears to be valid for this implementation), false otherwise.
Implementations are required to inspect the contents (e.g., checking just the pathname type is not sufficient). Although the completeness of the inspection is not required, this function should be able to detect, e.g., file format changes between versions.
I re-submitted the spec to the CDR editors a week ago. No response so far.
The detailed html spec is attached.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head> <title>Function COMPILED-FILE-P</title> <meta name="author" value="Sam Steingold"/> </head> <body>
<h1>Function <code>COMPILED-FILE-P</code></h1>
<h2>Author</h2>
<p>Sam Steingold</p>
<h2>Related</h2>
<p>ANSI Common Lisp standard function <code>compile-file</code>.</p>
<h2>Abstract</h2>
<p>A facility to determine whether a file is a valid compiled file for the specific implementation.</p>
<h2>Rationale</h2>
<p>Build tools, like <code>defsystem</code> or <code>asdf</code>, have to determine whether a file needs to be recompiled.</p> <p>Obviously, when the compiled file is older than the source file, recompilation is in order.</p> <p>Alas, there are other situations when this might be necessary, e.g., when the implementation changes the compiled file format or when two implementations use the same name for their compiled files (<code>.fasl</code> is used by both <code>SBCL</code> and <code>ACL</code>). </p>
<h3>Current Practice</h3>
<p>Implementation-dependent.</p>
<h3>Cost of adoption</h3>
<p>Probably tiny: an implementation must be able to check for compiled file validity, so all it takes is to export the necessary functionality, e.g.:</p>
<pre id="compiled-file-p-clisp"> #+clisp (defun compiled-file-p (file-name) (with-open-file (in file-name :direction :input :if-does-not-exist nil) (and in (char= #\( (peek-char nil in)) (let ((form (ignore-errors (read in nil nil)))) (and (consp form) (eq (car form) 'SYSTEM::VERSION) (null (nth-value 1 (ignore-errors (eval form))))))))) </pre>
<h3>Cost of non-adoption</h3>
<p>Users will suffer random errors when trying to load invalid binary files.</p>
<h2>Specification</h2>
<p>Function</p><pre> (compiled-file-p file-name) ==> valid-p </pre>
<p>Returns</p><dl> <dt><code>true</code></dt><dd>if the file appears to be a valid compiled file (i.e., exists, is readable, and the contents appears to be valid for this implementation),</dd> <dt><code>false</code></dt><dd>otherwise.</dd></dl>
<p>Implementations are required to inspect the contents (e.g., checking just the pathname type is not sufficient). Although the completeness of the inspection is not required, this function should be able to detect, e.g., file format changes between versions.</p>
<h3>Exceptional situations</h3> <ul> <li>signals an error of type <code>type-error</code> when the argument is not a <em>pathname designator</em>.</li> </ul>
<h3>Examples</h3>
<pre> (compiled-file-p "foo.lisp") ==> NIL (compiled-file-p (compile-file "foo.lisp")) ==> T </pre>
<h2>Reference Implementation</h2>
<p>See <a href="#compiled-file-p-clisp">above</a>.</p>
<h2>History</h2>
<p>This used to be <strong>CLRFI-2</strong> (in 2004).</p>
<h2>Notes</h2>
<p>The trivial implementation:</p> <pre> (defun compiled-file-p (file-name) (not (nth-value 1 (ignore-errors (load file-name))))) </pre> <p>is wrong because,</p> <ol> <li><code>load</code> may fail even though the file is valid: even when <code>foo.lisp</code> contains calls to <code>error</code>,<pre> (compiled-file-p (compile-file "foo.lisp")) </pre>should still return <code>T</code>.</li> <li>this is not side-effect-free, i.e., this may define new functions and macros (or, worse yet, redefine some existing functions and macros or execute some malicious code).</li></ol> </body> </html>
-- Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000 http://www.childpsy.net/ http://mideasttruth.com http://ffii.org http://www.memritv.org http://jihadwatch.org http://camera.org http://memri.org You cannot fire me. Slaves are not fired. Slaves are sold. _______________________________________________ pro mailing list pro@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/pro
-- Pascal Costanza
Hi, my e-mail to which you replied was CCed to editors@cdr.eurolisp.org. why doesn't it qualify as the resubmission?
On 15 May 2012, at 21:29, Sam Steingold wrote:
Hi, my e-mail to which you replied was CCed to editors@cdr.eurolisp.org. why doesn't it qualify as the resubmission?
It does qualify. As I said, I will process the CDR submission in any case. However, I also see the resubmission and your last email only through the Pro mailing list. Could you send an email just to editors@cdr.eurolisp.org?
Pascal
-- Pascal Costanza
Could you send an email just to editors-dmfRLhNpelYzOtNqt8oIng@public.gmane.org?
done (twice, with different attachments)
-
Faré -
Pascal Costanza -
Sam Steingold