I made a moderation error, so I'm forwarding this message manually. -Hans
---------- Forwarded message ---------- From: Tamas Papp tkpapp@gmail.com To: pro@common-lisp.net Date: Tue, 28 Jun 2011 12:26:24 +0000 (UTC) Subject: library configuration Hi,
A user of my LLA library asked for more flexibility in specifying which foreign library to load. For example, BLAS/LAPACK come in so many variants that it is impossible to cover them with the :or feature of cffi:define-foreign-library, especially because users have their own preferences when more than one is available.
I am looking for a portable way to specify configuration for a library before/during loading with ASDF2. It would be sufficient if the user could specify an ALIST (or something similar) somewhere and I could read/ use it when the library is loaded. Does ASDF2 have a "standard" mechanism for library config files (eg where they are, how to handle them etc), or should I use something else? I thought of using *features*, but I need actual values instead of merely checking for the presence of symbols.
Any suggestions are welcome.
Thanks,
Tamas
I am looking for a portable way to specify configuration for a library before/during loading with ASDF2. It would be sufficient if the user could specify an ALIST (or something similar) somewhere and I could read/ use it when the library is loaded. Does ASDF2 have a "standard" mechanism for library config files (eg where they are, how to handle them etc), or should I use something else? I thought of using *features*, but I need actual values instead of merely checking for the presence of symbols.
ASDF (1 or 2) does not handle C libraries. Extensions to ASDF do. In this case, you might want to ask on the CFFI mailing-list.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Faré fahree@gmail.com writes:
I am looking for a portable way to specify configuration for a library before/during loading with ASDF2. It would be sufficient if the user could specify an ALIST (or something similar) somewhere and I could read/ use it when the library is loaded. Does ASDF2 have a "standard" mechanism for library config files (eg where they are, how to handle them etc), or should I use something else? I thought of using *features*, but I need actual values instead of merely checking for the presence of symbols.
ASDF (1 or 2) does not handle C libraries. Extensions to ASDF do.
The question is not about C libraries, but how to express and manage configuration options that must be initialized before the library is built.
I think the answer for ASDF is "there is no way to express or manage any library configuration decisions."
Hunchentoot also has this issue regarding building with SSL, and manages it by checking for :hunchentoot-no-ssl in cl:*features*. For a non-boolean option, that's not a workable approach.
Zach
On 28 Jun 2011, at 17:49, Zach Beane wrote:
Faré fahree@gmail.com writes:
I am looking for a portable way to specify configuration for a library before/during loading with ASDF2. It would be sufficient if the user could specify an ALIST (or something similar) somewhere and I could read/ use it when the library is loaded. Does ASDF2 have a "standard" mechanism for library config files (eg where they are, how to handle them etc), or should I use something else? I thought of using *features*, but I need actual values instead of merely checking for the presence of symbols.
ASDF (1 or 2) does not handle C libraries. Extensions to ASDF do.
The question is not about C libraries, but how to express and manage configuration options that must be initialized before the library is built.
I think the answer for ASDF is "there is no way to express or manage any library configuration decisions."
Hunchentoot also has this issue regarding building with SSL, and manages it by checking for :hunchentoot-no-ssl in cl:*features*. For a non-boolean option, that's not a workable approach.
One could use a global variable that stores one's own configuration option, for example as a plist or alist. This would have to be in the common-lisp-user package, and theoretically clashes with other such variables. However, this is not substantially worse than potential clashes of keyword symbols in *features*. Just use a name that is unlikely to be used by others (for example, by using the ASDF/package name with an -OPTIONS suffix, or some such).
boundp is your friend to check whether the variable is actually defined, which could either yield a warning, or assume default options.
Pascal
-- Pascal Costanza The views expressed in this email are my own, and not those of my employer.
On 28 June 2011 13:38, Pascal Costanza pc@p-cos.net wrote:
On 28 Jun 2011, at 17:49, Zach Beane wrote:
Faré fahree@gmail.com writes:
I am looking for a portable way to specify configuration for a library before/during loading with ASDF2. It would be sufficient if the user could specify an ALIST (or something similar) somewhere and I could read/ use it when the library is loaded. Does ASDF2 have a "standard" mechanism for library config files (eg where they are, how to handle them etc), or should I use something else? I thought of using *features*, but I need actual values instead of merely checking for the presence of symbols.
ASDF (1 or 2) does not handle C libraries. Extensions to ASDF do.
The question is not about C libraries, but how to express and manage configuration options that must be initialized before the library is built.
I think the answer for ASDF is "there is no way to express or manage any library configuration decisions."
Hunchentoot also has this issue regarding building with SSL, and manages it by checking for :hunchentoot-no-ssl in cl:*features*. For a non-boolean option, that's not a workable approach.
One could use a global variable that stores one's own configuration option, for example as a plist or alist. This would have to be in the common-lisp-user package, and theoretically clashes with other such variables. However, this is not substantially worse than potential clashes of keyword symbols in *features*. Just use a name that is unlikely to be used by others (for example, by using the ASDF/package name with an -OPTIONS suffix, or some such).
boundp is your friend to check whether the variable is actually defined, which could either yield a warning, or assume default options.
Pascal
What's so special about Lisp here? How do other languages do it?
If it's about libraries that are shared by everyone using the standard C ABI, then shouldn't it all "just work", using the LD_LIBRARY_PATH and other standard configuration files, and/or paths wired into your object files?
If it's about wrapper libraries created by cffi-grovel, then ASDF will load them from wherever the fasl cache is defined. This doesn't play very well with distribution of binaries (as opposed to distribution of source), but at least it should "do the right thing", and the basic configuration knobs are there if you want to do better.
Or maybe what you want is some way to specify C compiler flags for cffi-grovel? This is currently done through the cffi-grovel::*cc-flags* special variable.
Now, if you want to somehow have per-user configuration, overridable with an environment variable and/or an explicit programmer parameter, I recommend that you use the same conventions as currently used by ASDF 2 and read configuration from a file in the same directory. You may copy/paste code from ASDF 2, and/or I will happily refactor the code so that it may be used by ASDF extensions as well as by ASDF itself.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Faré fahree@gmail.com writes:
What's so special about Lisp here? How do other languages do it?
./configure --with-foo=/bar/baz
Zach
Regarding the original question, take a look at cl-pdf.asd. CL projects needing configuration capabilities have devised various approaches; but the current standard is to have a user-editable asd file, and cl-pdf is one of the more sophisticated examples.
On Tue, 28 Jun 2011, Zach Beane wrote:
Faré fahree@gmail.com writes:
What's so special about Lisp here? How do other languages do it?
./configure --with-foo=/bar/baz
The specialness about CL is that people want an all-in-one solution. Download, configure, build, install, and load all in one step.
Unfortunately, this doesn't leave natural breaks for humans to oversee the process.
In a way, user-editable asd files remind me of pre-autoconf makefiles...
C users have download, configure, make, make install, compile with headers, link with objects, and run with loader all as separate steps. Sometimes tedious, but remarkably flexible and robust. Distributions of precompiled code hide the tedium from end users.
A while ago, I was toying with a more decoupled build system for CL. It consisted of three components based on experience with the GNU autotools. (Yes, they're built on a terrible foundation of m4 and portable shell and portable make, but they're light years ahead of the competition in terms of good engineering.)
coli - configure lisp (including user inputs and automated probes) buli - build and install lisp (and trace what was built/required) loli - load lisp (a well-defined REQUIRE with hooks for versioning)
The idea was that the components are designed to work together, but there are public APIs so they can be swapped out or used individually. Put hooks and restarts in loli so buli can (re)compile as needed. Another user may disable these hooks and delete the sources after each install. Add hooks in buli to (re)run coli. With a public API, loli's hooks could be redirected to booli which could emulate buli's hook points -- people could rewrite and swap pieces without forking the whole build system. etc.
Like so many things, this was shelved due to a perceived lack of community interest and a definite shunting of personal energy.
- Daniel
On 28 Jun 2011, at 19:52, Faré wrote:
On 28 June 2011 13:38, Pascal Costanza pc@p-cos.net wrote:
On 28 Jun 2011, at 17:49, Zach Beane wrote:
Faré fahree@gmail.com writes:
I am looking for a portable way to specify configuration for a library before/during loading with ASDF2. It would be sufficient if the user could specify an ALIST (or something similar) somewhere and I could read/ use it when the library is loaded. Does ASDF2 have a "standard" mechanism for library config files (eg where they are, how to handle them etc), or should I use something else? I thought of using *features*, but I need actual values instead of merely checking for the presence of symbols.
ASDF (1 or 2) does not handle C libraries. Extensions to ASDF do.
The question is not about C libraries, but how to express and manage configuration options that must be initialized before the library is built.
I think the answer for ASDF is "there is no way to express or manage any library configuration decisions."
Hunchentoot also has this issue regarding building with SSL, and manages it by checking for :hunchentoot-no-ssl in cl:*features*. For a non-boolean option, that's not a workable approach.
One could use a global variable that stores one's own configuration option, for example as a plist or alist. This would have to be in the common-lisp-user package, and theoretically clashes with other such variables. However, this is not substantially worse than potential clashes of keyword symbols in *features*. Just use a name that is unlikely to be used by others (for example, by using the ASDF/package name with an -OPTIONS suffix, or some such).
boundp is your friend to check whether the variable is actually defined, which could either yield a warning, or assume default options.
Pascal
What's so special about Lisp here? How do other languages do it?
If it's about libraries that are shared by everyone using the standard C ABI, then shouldn't it all "just work", using the LD_LIBRARY_PATH and other standard configuration files, and/or paths wired into your object files?
If it's about wrapper libraries created by cffi-grovel, then ASDF will load them from wherever the fasl cache is defined. This doesn't play very well with distribution of binaries (as opposed to distribution of source), but at least it should "do the right thing", and the basic configuration knobs are there if you want to do better.
Or maybe what you want is some way to specify C compiler flags for cffi-grovel? This is currently done through the cffi-grovel::*cc-flags* special variable.
Now, if you want to somehow have per-user configuration, overridable with an environment variable and/or an explicit programmer parameter, I recommend that you use the same conventions as currently used by ASDF 2 and read configuration from a file in the same directory. You may copy/paste code from ASDF 2, and/or I will happily refactor the code so that it may be used by ASDF extensions as well as by ASDF itself.
Configuration files are for languages where there is a strong distinction between programs and data. Lisp doesn't need no stinking configuration files. ;)
You may want configuration options that change which parts of your code are read and how, and how certain macros are expanded. Such configuration options need to be specified before code is read or macro-expanded.
Pascal
-- Pascal Costanza The views expressed in this email are my own, and not those of my employer.
Pascal Costanza wrote:
One could use a global variable that stores one's own configuration option, for example as a plist or alist. This would have to be in the common-lisp-user package, and theoretically clashes with other such variables.
Yes. One thing I do for this is to use CL-USER variables that are named after the package involved, so as to avoid name clashes. For example, Clon uses a variable named cl-user:*com.dvlsoft.clon.dump*.
I don't think this will ever clash with anyone ;-)
-
Daniel Herring -
Didier Verna -
Faré -
Hans Hübner -
Pascal Costanza -
Zach Beane