There are many things to clarify about logical pathnames and the use thereof or not, but I don't think this should block a release of ASDF 2. Should it? And so I'd like to declare it an ASDF 2.1 or ASDF 3 issue, and invite you to focus on blocking issues for ASDF 2. (Of course, if someone has a working, portable, solution that makes everyone happy, I'll gladly merge and commit it.)
I think the asdf-bundle extension is more important, because underlying it is the structure of how we deliver extensions, and how we play well with ECL, especially as it might entail some slight backwards incompatibility for ECL that I'd rather not happen *after* we release ASDF 2.
Can we focus on getting ASDF 2 out before we add new features?
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Superstition brings bad luck. — Paul Carvel
On 31 March 2010 14:00, Juan Jose Garcia-Ripoll juanjose.garciaripoll@googlemail.com wrote:
On Wed, Mar 31, 2010 at 4:39 PM, james anderson james.anderson@setf.de wrote:
i simply don't understand your requirements. on one hand both with and without asdf, but then it is objected, that a mechanism which requires asdf does not work without it. please explain.
What I do not understand is what YOU do not understand.
In an standalone system there is no ASDF. That is all.
I want to be able to take an ASDF system describing my program and its dependencies on other libraries and compile it and produce a standalone executable. That program will not have ASDF inside, it will not have the notion of a central registry it will have nothing, but it still has to work.
Please understand that the program I am talking about is not a dumped lisp image that contains everything needed to build the system and even some extra stuff that was there (database of ASDF systems and other trash). The programs I produce are just a set of compiled files, one per ASDF system, linked together with a core library that is the Common Lisp environment ECL offers.
When the program starts, the linked files are "loaded" one after another, as if LOAD was used but actually much faster. But I insist again ASDF is not part of the things that is loaded simply because there is no need.
The only thing that one might need ASDF for is to locate the data associated to the program, but if we provide logical pathname translations (one host per library), then a small piece of code can be inserted at the beginning of the program which locates the data and sets up the appropriate translations.
if the requirement is that exact same exact formulation be used with and without asdf, then a) the additional system initialization argument is excluded b) the second option in my earlier message provides this path.
I only find "b" acceptable because I still do not understand how "a" is going to work without a central registry and a lot of complicated logic that has to propagate to those distributed binaries.
Having "b", which I recall was one logical host name per system, one may simply prepend to the compiled set of files a tiny bit of lisp that looks as follows
(defvar *program-data-location* #+windows ;; Data and binaries live together (program-location) #+unix ;; Data is installed somewhere standard "/usr/share/program/" #+osx ;; Assume this is an application bundle (merge-pathnames "Content/Resources/" (program-location)))
(defun set-host-translation (root extra) ...)
(set-host-translation "cl-ppcre" "cl-ppcre/") (set-host-translation "cl-unicode" cl-unicode/")
With this simple, automatically generated code I can pack the given systems and let them work anywhere, even though ASDF is not present.
The same or a similar procedure can be used to link many systems into a single distributable binary fasl + resources, install it in a standard location that can be found by certain lisps and prepend a simple logic for finding out its data... Without using ASDF in the file itself.
There is even a third model in which installation paths are known beforehand. Say that a package maintainer from Debian has the role of generating binaries for 10 libraries out there and they have complex inter-dependencies. Libraries can be compiled in the machine of the maintainer and then installed by ASDF in their standard locations. ASDF can, in the process, hard-code the pathname translations when installing the files.
For instance, if we produce binaries for cl-ppcre-1.0.1 and cl-unicode-2.0.1 and the latter depends on the former, and we know that they are going to live in /usr/lib/common-lisp/cl-ppcre-1.0.1 /usr/lib/common-lisp/cl-unicode-2.0.1 when "asdf-build" compiles the files and "asdf-installs" them we may get
;; Stripped down version of a system definition ;; which only contains the precompiled files ;; and which can be searched and used by ASDF /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.asd /usr/lib/common-lisp/cl-unicode-1.0.1/cl-unicode.asd
;; The binaries themselves /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre-1.0.1.fas /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode-1.0.1.fas
;; Proxy binaries that can be loaded using "require" ;; even if ASDF is not present. These files define the logical ;; pathname translations based on hardcoded locations. /user/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.fas /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.fas
;; Data used by the library /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.dat
;; Symbolic link to the last version /usr/lib/common-lisp/cl-ppcre -> cl-ppcre-1.0.1
The FASL without version numbers files can explicitly load the required versions of the libraries because they know the hardcoded paths of their files. In doing so they can also set up the appropriate pathname translations that are used by the other files.
I have listed just two examples with different paradigms on how to build and distribute software, and which go beyond current use in which the user has to set up a collection of registries, in your case you even define pathname translations and have magic to make ASDF understand them, sources are always present, things are recompiled...
But there are other possibilities. ASDF can be used to produce programs and libraries that can be distributed, ASDF can itself be used without people noticing it (see the automatically generated FASL files), it can integrate with other build processes (via some simple scripts asdf-build, asdf-install which I may offer to write), we can achieve better system descriptions, and a lot of things that make people's life easier because they can forget that they use ASDF at all -- just like when writing C I am free to forget that I am using "make" at some point.
Juanjo
On Wed, Mar 31, 2010 at 8:11 PM, Faré fahree@gmail.com wrote:
There are many things to clarify about logical pathnames and the use thereof or not, but I don't think this should block a release of ASDF 2. Should it? And so I'd like to declare it an ASDF 2.1 or ASDF 3 issue, and invite you to focus on blocking issues for ASDF 2. (Of course, if someone has a working, portable, solution that makes everyone happy, I'll gladly merge and commit it.)
I have uploaded a newer version of the feature that allows two different syntaxes
:logical-host "MY-HOST" :logical-pathname "MY-HOST:MY-BASE"
The three pathnames that are defined are "xxx;SRC;*.*.*", "xxx;DATA;*.*.*" and "xxx;OUTPUT-FILES;*.*.*" where the first two map to the source directory of the system and the second one to the translated directory for FASL and other generated files. The "xxx" here denotes the argument provided by the user. Note that the code will complain if there are wildcards.
I think the asdf-bundle extension is more important, because underlying it is the structure of how we deliver extensions, and how we play well with ECL, especially as it might entail some slight backwards incompatibility for ECL that I'd rather not happen *after* we release ASDF 2.
Actually if you plan an immediate release I would leave that for 2.1 because I would rather sort out that thing of the incompatibility later, and also because I believe there is room for improvement in the patch, including support for other implementations.
For the record, even if the ASDF crew decided to leave the bundle out of the core and thus expel ECL's extensions from it as well, I would continue to ship everything packed together with ECL, so that a single REQUIRE is needed.
Juanjo
On Wed, Mar 31, 2010 at 11:16 PM, Juan Jose Garcia-Ripoll < juanjose.garciaripoll@googlemail.com> wrote:
I have uploaded a newer version of the feature that allows two different syntaxes
:logical-host "MY-HOST" :logical-pathname "MY-HOST:MY-BASE"
The three pathnames that are defined are "xxx;SRC;*.*.*", "xxx;DATA;*.*.*"
I have just realized that the :logical-pathname case is indeed much more complicated than what I had assumed: my implementation just set the given translations for the given host, overwriting previous translations.
A real implementation of arbitrary logical pathname translations would require a much more complicated code, probably based on something like james anderson's utilities, requiring: 1) Analysis of the argument to :logical-pathname 2) Analysis and comparison of existing previous translations 3) Producing an intersection set, with the new paths 4) Sorting out conflicts.
But this indeed makes it more questionable the possibility of arbitrary paths vs. a simple host controlled by ASDF.
Why? Because we need more complicated functions, and because these functions have to be present in the code that is shipped with these translations, in other words, having an implicit dependency on a functionality provided by ASDF.
So I go back to the original proposal of :logical-host, which I insist is just simple, verifiable, does not imply complicated logic and can be exported to compiled code and distributed files.
Juanjo
-
Faré -
Juan Jose Garcia-Ripoll