Hi Faré, thanks for all the answers.
I'm starting to see where the mismatch lies between my thinking and what is needed. However like with Stelian's, this email is excellent and I'd like to go through it fully. I'll do it in an odd order though.
If it's anyone but Integrators, then mucking with implementation identifiers is wrong-headed … you're not using it the intended way, either:
- There's a bug in ASDF, so go fix ASDF, or
- There's a bug in your code, so go fix your code.
…
True, and I'm fine with the idea that my approach was poor, I would love a better one. The static link capabilities did look really interested but SBCL is on an older ASDF (I guess for the reasons you mentioned). Maybe that is part of ASDF I can start reading with the hope I can contribute at some point.
But please don't introduce towers of complexity that defeat the design of ASDF.
I can only ask for your trust that this isnt what I don't want more complexity or to defeat the design on ASDF (even if I achieved both in the current PR). I'm just looking to ship useful things for lispers to use, publicly making these kind of mistakes isn't a problem to me if it gets me closer to making something that is useful and works well with the existing tooling (ASDF being a huge part of that for me).
may I suggest you give a look at bazelisp
Thanks, I'll have a peek. But I really like ASDF and want to be useful on as many lisp implementations as is feasible. So I'd rather stay in this pond (and help if I can). I just need to understand more deeply where certain things should be done.
1- Author, who writes a given system. 2- Referrer, who uses the system from another system. 3- Integrator, who assembles a bunch of systems into an application. 4- (End) User, who uses the application and shouldn't even have to know it's in Lisp.
What problem are you trying to solve for people in which role? How is it not already solved by the ASDF cache?
I'm trying to allow 2 & 3 to consume the lisp library without having to worry about the implementation decisions 1 made (unless they want to).
Let's say I am 1 in this case for example, and that I'm making a GL based UI for people to use. I decide to wrap an existing GL UI system written in C and make a nice lisp interface on top. I would argue that the fact my implementation uses a C library is irrelevant to the consumer of my library. They wanted a GL UI.
Of course this shouldn't stop them from digging into this if they want to, but the standard experience should be as standard a lisp experience as possible. load-system and start being productive.
So my closing question is: If I want to be able to use shared-object files with my lisp libraries what is the correct asdf way for this to be done? Given that: - The shared-object file is not available through any package manager - people should be able to just quickload the project and start working - I need to support osx/linux/windows so I will be including a selection of .so/dll/dylib files. - Requiring every user to have a C compiler set up on their machine is not acceptable.
Note that I don't mind some people having to build it themselves as it's not feasible include precompiled objects for all platforms. But I want to cover a basic subset
Thanks again for the time you are taking to go through this.
Regards Chris
On 6 October 2016 at 01:34, Faré fahree@gmail.com wrote:
On Wed, Oct 5, 2016 at 3:26 AM, Chris Bagley chris.bagley@gmail.com wrote:
I started this feature after finding it a pain to use osicat as neither Windows or OSX ship with a C compiler installed. When the groveler failed the output was very unhelpful, and as I wasn't a developer on the groveler I had no idea what it was or how I should be interacting with it. I was just a consumer who saw value in osicat.
I like to distinguish 4 roles: 1- Author, who writes a given system. 2- Referrer, who uses the system from another system. 3- Integrator, who assembles a bunch of systems into an application. 4- (End) User, who uses the application and shouldn't even have to know it's in Lisp.
Only the Integrator should have to worry about matching together Lisp systems, Lisp implementations, C compilers, etc. For others, it should "just work". By default.
What problem are you trying to solve for people in which role? How is it not already solved by the ASDF cache?
If it's anyone but Integrators, then mucking with implementation identifiers is wrong-headed. If you're trying to write something for Integrators, then look at the ASDF bundle operations as a good start.
And if you're not satisfied with ASDF, may I suggest you give a look at bazelisp? Sure, it only works on Linux amd64 using SBCL at this point, but it might be easier to port for much better results than would be trying to abuse ASDF and try to go against the design of ASDF.
Where can I find details on these 'internal invariants'? This sounds like a problem with the currently implementation and not the concept of avoiding duplication of work (caching).
The invariants of ASDF are mainly that you should be using the ASDF cache and it should just work. If you're not using it the intended way, either:
- There's a bug in ASDF, so go fix ASDF, or
- There's a bug in your code, so go fix your code.
I really would like to understand this as I attempt to ship games in lisp and dealing with C dependencies from projects using CFFI is by far the single biggest issue of shipping.
That's an application delivery issue, for Integrators. For that, please use ASDF (monolithic) bundle operations, GDL famously does just that.
See also the cffi-toolchain static-link capabilities, that could use some love. (SBCL notably reverted my patch to generate self-contained static linked binaries, because it was broken on MacOS and Windows.)
Or use Bazel (which patches SBCL).
But please don't introduce towers of complexity that defeat the design of ASDF.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If you're a quiet, law-abiding citizen most of the time but occasionally cut someone up and bury them in your backyard, you're a bad guy. — Paul Graham, "The Power of the Marginal"