Hi,
Some time ago Mark Evenson created ASDF-JAR on ABCL. It has some issues packaging ASD files which uses #. syntax to read e.g. version files (version.lisp-sexp) such as bordeaux-threads does.
We know ECL started hooking into the ASDF/BUNDLE machinery for ASDF to overcome problems like these. Could you help me/us out and explain how we can plug ABCL into the same machinery? From what I've seen, it's most efficient if the resulting code be committed to ASDF. That's fine by me. At this point, I'd just be happy with an explanation on how to do the hooking up: the new ASDF/BUNDLE functionality doesn't seem to be documented in the manual yet.
Thanks for any help you may be able to provide!
Bye,
Erik.
Hi Fare,
On Sun, Mar 31, 2013 at 9:39 PM, Erik Huelsmann ehuels@gmail.com wrote:
Hi,
Some time ago Mark Evenson created ASDF-JAR on ABCL. It has some issues packaging ASD files which uses #. syntax to read e.g. version files (version.lisp-sexp) such as bordeaux-threads does.
We know ECL started hooking into the ASDF/BUNDLE machinery for ASDF to overcome problems like these. Could you help me/us out and explain how we can plug ABCL into the same machinery? From what I've seen, it's most efficient if the resulting code be committed to ASDF. That's fine by me. At this point, I'd just be happy with an explanation on how to do the hooking up: the new ASDF/BUNDLE functionality doesn't seem to be documented in the manual yet.
Thanks for any help you may be able to provide!
In the mean time, I've been digging around in ASDF/BUNDLE's code. It seems ABCL is missing in the conditionals in BUNDLABLE-FILE-P, resulting in that function always returning NIL no ABCL.
If I add abcl to the third conditional in that function, I'm being thrown a much more useful error: ABCL does not support UIOP/LISP-BUILD:COMBINE-FASLS which seems a much better error to run into.
[A few hours later] Ok, so I've hacked together a COMBINE-FASLS function and I'm able to generate a combined FASL. Case in point: bordeaux-threads--system.fasl. So far so good.
The next step seems to be one of these:
1. Figure out how to generate system definition files for prebuilt systems (with concatenated fasls) 2. Figure out how to unpack (packaged) fasls and package them into larger combinations (--all-systems fasls)
To be honest, I think the former is more important than the latter as the former seems to be the way to prevent ASDF to go looking for sources.
Still thanks for any help you may be able to provide!
Bye,
Erik.
PS: All I had to do to get to where I'm now is to create the ASDF-2.32-based patch:
Index: src/org/armedbear/lisp/asdf.lisp =================================================================== --- src/org/armedbear/lisp/asdf.lisp (revision 14453) +++ src/org/armedbear/lisp/asdf.lisp (working copy) @@ -4753,11 +4753,20 @@ ;;; Links FASLs together (with-upgradability () (defun combine-fasls (inputs output) - #-(or allegro clisp clozure cmu lispworks sbcl scl xcl) + #-(or abcl allegro clisp clozure cmu lispworks sbcl scl xcl) (error "~A does not support ~S~%inputs ~S~%output ~S" (implementation-type) 'combine-fasls inputs output) #+clozure (ccl:fasl-concatenate output inputs :if-exists :supersede) #+(or allegro clisp cmu sbcl scl xcl) (concatenate-files inputs output) + #+abcl + (let ((fasls (make-hash-table :test #'equal))) + (loop :for fasl :in inputs + :do (setf (gethash fasl fasls) + (make-pathname :host nil + :device nil + :directory nil + :defaults fasl))) + (sys:zip output fasls)) #+lispworks (let (fasls) (unwind-protect @@ -8684,7 +8693,7 @@ (or #+ecl (or (equalp type (compile-file-type :type :object)) (equalp type (compile-file-type :type :static-library))) #+mkcl (equalp type (compile-file-type :fasl-p nil)) - #+(or allegro clisp clozure cmu lispworks sbcl scl xcl) (equalp type (compile-file-type))))) + #+(or abcl allegro clisp clozure cmu lispworks sbcl scl xcl) (equalp type (compile-file-type)))))
(defgeneric* (trivial-system-p) (component))
I admit I had never looked at how ACBL-JAR works before. I see it's there: http://trac.common-lisp.net/armedbear/browser/trunk/abcl/contrib/asdf-jar/as...
IIUC, the gist is that you somehow traverse the system to gather what files to zip, and the tricky issue is that since the output-translations function will be different in the build environment and the deployment environment, you have to build a map of build-time pathnames in the filesystem to deploy-time pathnames in the jar file. Because ASDF 1 or 2 didn't have a good way to do that, ABCL currently does it in a clunky way that doesn't account for ASDF extensions. Doing it correctly is possible using exported functionality of ASDF3, but requires intercepting (with-asdf-cache (:override t) ...) and *output-translation-functions* to build a reverse translation table around a call to (traverse 'load-op system).
Your second email suggests you're looking into combining fasls as an alternative to building a jar. Sounds great. Regarding how to build a system that loads the concatenated fasl, see in asdf/bundle.lisp the (largely untested, stolen from asdf-ecl.lisp) defmethod perform ((o binary-op) (s system)) Or search the same file for "Example use:"
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Justice is not concerned with the results of the various transactions but only with whether the transactions themselves are fair. — F.A. Hayek, "Law, Legislation and Liberty", I.6.j
On Sun, Mar 31, 2013 at 3:39 PM, Erik Huelsmann ehuels@gmail.com wrote:
Hi,
Some time ago Mark Evenson created ASDF-JAR on ABCL. It has some issues packaging ASD files which uses #. syntax to read e.g. version files (version.lisp-sexp) such as bordeaux-threads does.
We know ECL started hooking into the ASDF/BUNDLE machinery for ASDF to overcome problems like these. Could you help me/us out and explain how we can plug ABCL into the same machinery? From what I've seen, it's most efficient if the resulting code be committed to ASDF. That's fine by me. At this point, I'd just be happy with an explanation on how to do the hooking up: the new ASDF/BUNDLE functionality doesn't seem to be documented in the manual yet.
Thanks for any help you may be able to provide!
Bye,
Erik.
Hi Faré,
On Mon, Apr 1, 2013 at 12:04 AM, Faré fahree@gmail.com wrote:
I admit I had never looked at how ACBL-JAR works before. I see it's there:
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/contrib/asdf-jar/as...
IIUC, the gist is that you somehow traverse the system to gather what files to zip, and the tricky issue is that since the output-translations function will be different in the build environment and the deployment environment, you have to build a map of build-time pathnames in the filesystem to deploy-time pathnames in the jar file. Because ASDF 1 or 2 didn't have a good way to do that, ABCL currently does it in a clunky way that doesn't account for ASDF extensions. Doing it correctly is possible using exported functionality of ASDF3, but requires intercepting (with-asdf-cache (:override t) ...) and *output-translation-functions* to build a reverse translation table around a call to (traverse 'load-op system).
As we discussed on IRC, I think it's best to hook into ASDF's facilities: This is simply the only way to leverage cross-implementation efforts. If ASD files need to be fixed or changed to support these kinds of hooks, I'm quite sure chances of that happening increase if people on all implementations are bound to run into them.
Your second email suggests you're looking into combining fasls as an alternative to building a jar. Sounds great. Regarding how to build a system that loads the concatenated fasl, see in asdf/bundle.lisp the (largely untested, stolen from asdf-ecl.lisp) defmethod perform ((o binary-op) (s system)) Or search the same file for "Example use:"
Yes, I'm trying to come up with the best way to support deployment. Both the JAR solution as well as the "concatenated FASL" solution aim at that goal. If the concatenated FASL solution helps to get there, then that's what I'll aim at.
Bye,
Erik.
armedbear-devel@common-lisp.net
-
Erik Huelsmann -
Faré