how to distinguish public vs private (entry-point vs internal) systems
When creating an lisp application I usually have one (or several) what I call top-level asdf systems which advertise the public interface to the application, and I may have several internal systems which are used but not intended for public use. What is the convention with asdf to distinguish entry-point systems from internal/private systems?
On 6 Feb 2019, at 2:22, Jim Newton wrote:
When creating an lisp application I usually have one (or several) what I call top-level asdf systems which advertise the public interface to the application, and I may have several internal systems which are used but not intended for public use.
What is the convention with asdf to distinguish entry-point systems from internal/private systems?
I generally try to use either Faré's "slashy" systems (like "shop2/common") in my work. When I can, it's even better to use a `:module` which isn't visible at all. I think what you are really asking is "how should I name a system that the user should never load *directly*?" I don't have a great answer to this question.
I’m both asking how they should be named, and how to advertise them for programmatic consumption. For example, and automatic testing program such as that included in quicklisp, should not try to stand-alone load systems which are not designed to work stand-alone. We have to work around this by artificially making all systems “work” in standalone enough to fool quicklisp. quickref is another tool which tries to publish documentation extracted from packages, but quickref would like to skip packages which are not part of the public API, such as test case packages which may require other non-public testing frameworks. It would be nice if asdf had some declarative way of specifying which systems are intended as entry points. That would also avoid different people relying on non-standard naming conventions to encode declarative information.
On 06 Feb 2019, at 15:36, Robert Goldman <rpgoldman@sift.info> wrote:
On 6 Feb 2019, at 2:22, Jim Newton wrote:
When creating an lisp application I usually have one (or several) what I call top-level asdf systems which advertise the public interface to the application, and I may have several internal systems which are used but not intended for public use.
What is the convention with asdf to distinguish entry-point systems from internal/private systems?
I generally try to use either Faré's "slashy" systems (like "shop2/common") in my work. When I can, it's even better to use a :module which isn't visible at all.
I think what you are really asking is "how should I name a system that the user should never load directly?" I don't have a great answer to this question.
On 6 Feb 2019, at 9:02, Jim Newton wrote:
I’m both asking how they should be named, and how to advertise them for programmatic consumption. For example, and automatic testing program such as that included in quicklisp, should not try to stand-alone load systems which are not designed to work stand-alone. We have to work around this by artificially making all systems “work” in standalone enough to fool quicklisp.
Can you explain the quicklisp constraint? How does it find all systems? One simple expedient for *this* quicklisp issue -- if I understand it correctly -- would be to have a `test-op` default `perform` method for all systems that simply succeeds. It should probably by default issue a warning that no "real" test method exists, and that warning should have a particular type so that it can be muffled by quicklisp. Probably also we should allow the programmer of the original system to make a `test-op` no-op method that emits no warning (because the system is intended not to be testable).
quickref is another tool which tries to publish documentation extracted from packages, but quickref would like to skip packages which are not part of the public API, such as test case packages which may require other non-public testing frameworks.
I'm not sure that ASDF can do anything about this one -- packages are not an artifact that it really understands or takes responsibility for. Perhaps it would be better to extend the `package` object so that it can hold a slot that can designate it as `internal` or `external`. Quicker could provide a trivial system that would export this package extension. I think it would be useful for other documentation systems, as well.
It would be nice if asdf had some declarative way of specifying which systems are intended as entry points. That would also avoid different people relying on non-standard naming conventions to encode declarative information.
The trivial solution here would be to create a class inheriting from `asdf:system` that would be something like `internal-system`. Harder would be getting people to adopt it. But if there's interest, I could try to make one or, given the limited amount of time I have to work on ASDF these days, I would be happy to accept a pull request. Best, Robert
On 06 Feb 2019, at 15:36, Robert Goldman <rpgoldman@sift.info> wrote:
On 6 Feb 2019, at 2:22, Jim Newton wrote:
When creating an lisp application I usually have one (or several) what I call top-level asdf systems which advertise the public interface to the application, and I may have several internal systems which are used but not intended for public use.
What is the convention with asdf to distinguish entry-point systems from internal/private systems?
I generally try to use either Faré's "slashy" systems (like "shop2/common") in my work. When I can, it's even better to use a :module which isn't visible at all.
I think what you are really asking is "how should I name a system that the user should never load directly?" I don't have a great answer to this question.
quickref is another tool which tries to publish documentation extracted from packages, but quickref would like to skip packages which are not part of the public API, such as test case packages which may require other non-public testing frameworks.
I'm not sure that ASDF can do anything about this one -- packages are not an artifact that it really understands or takes responsibility for. Perhaps it would be better to extend the package object so that it can hold a slot that can designate it as internal or external. Quicker could provide a trivial system that would export this package extension. I think it would be useful for other documentation systems, as well.
Ooops, I didn’t mean to say package, I meant to say system. Sorry for the confusion.
I’m both asking how they should be named, and how to advertise them for programmatic consumption. For example, and automatic testing program such as that included in quicklisp, should not try to stand-alone load systems which are not designed to work stand-alone. We have to work around this by artificially making all systems “work” in standalone enough to fool quicklisp.
Can you explain the quicklisp constraint? How does it find all systems?
One simple expedient for this quicklisp issue -- if I understand it correctly -- would be to have a test-op default perform method for all systems that simply succeeds. It should probably by default issue a warning that no "real" test method exists, and that warning should have a particular type so that it can be muffled by quicklisp. Probably also we should allow the programmer of the original system to make a test-op no-op method that emits no warning (because the system is intended not to be testable).
As I understand quicklisp, it ties to compile each system in a top-level sbcl, and asserts that that works. As far as I know that is the only test it does. I don’t believe it does anything special with test-op.
Hi, when I read your post, I instantly came up with the system that may not be the target of OPERATE like ;; foo.asd ;; https://gist.github.com/privet-kitty/84350b73d528533ac8e19e5bba6aa333 (defpackage :foo.asdf (:use :cl :asdf :uiop) (:export #:hideable-system)) (in-package :foo.asdf) (eval-when (:compile-toplevel :load-toplevel :execute) (defclass hideable-system (system) ((private :initform nil :initarg :private :reader private-p)) (:documentation "If PRIVATE is true, ASDF signals an error when one tries to OPERATE this system.")) (defmethod operate :around (operation (component hideable-system) &key &allow-other-keys) (if (private-p component) (error "Private system ~S cannot be directly operated." (component-name component)) (call-next-method)))) (defsystem "foo" :license "public domain" :depends-on ("foo/private") :components ((:file "main"))) (defsystem "foo/private" :pathname "private" :class "FOO.ASDF:HIDEABLE-SYSTEM" :private t :components ((:file "package"))) Here FOO/PRIVATE cannot be OPERATEd (but can be PERFORMed and then loaded, compiled, tested etc. via FOO). It seems to work at least superficially, though ASDF Best Practices states:
You SHOULD NOT define methods on asdf:operate --- most of the time it's totally the wrong thing because users would not be "operating" on your system, but on their system that depends on it. Instead you SHOULD define methods on asdf:perform, asdf:component-depends-on, etc. https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md
Anyway, I don't have a sufficient knowledge on ASDF's internal.
I think the approach of disabling operate is not a good solution to this problem because, for example (this is one I have actually encountered), one might have multiple test systems that one wishes to test separately. E.g., "Foo/input-test" "Foo/processing-test" ... and so on. So I think it's better to handle this in a way that focuses on Quicklisp, and indexing, rather than attempting to change the core functions of ASDF. It's just as easy to create a system class that simply says "I am a system that Quicklisp should not build." Unfortunately, there has been enough bad blood between Xach and some members of the ASDF-devel community that this discussion won't be reaching him. Jim, Hugo, either of you want to try to be a conduit? Best, R On 6 Feb 2019, at 10:52, Hugo Ishimaru wrote:
Hi,
when I read your post, I instantly came up with the system that may not be the target of OPERATE like ;; foo.asd ;; https://gist.github.com/privet-kitty/84350b73d528533ac8e19e5bba6aa333 (defpackage :foo.asdf (:use :cl :asdf :uiop) (:export #:hideable-system)) (in-package :foo.asdf)
(eval-when (:compile-toplevel :load-toplevel :execute) (defclass hideable-system (system) ((private :initform nil :initarg :private :reader private-p)) (:documentation "If PRIVATE is true, ASDF signals an error when one tries to OPERATE this system."))
(defmethod operate :around (operation (component hideable-system) &key &allow-other-keys) (if (private-p component) (error "Private system ~S cannot be directly operated." (component-name component)) (call-next-method))))
(defsystem "foo" :license "public domain" :depends-on ("foo/private") :components ((:file "main")))
(defsystem "foo/private" :pathname "private" :class "FOO.ASDF:HIDEABLE-SYSTEM" :private t :components ((:file "package")))
Here FOO/PRIVATE cannot be OPERATEd (but can be PERFORMed and then loaded, compiled, tested etc. via FOO). It seems to work at least superficially, though ASDF Best Practices states:
You SHOULD NOT define methods on asdf:operate --- most of the time it's totally the wrong thing because users would not be "operating" on your system, but on their system that depends on it. Instead you SHOULD define methods on asdf:perform, asdf:component-depends-on, etc. https://gitlab.common-lisp.net/asdf/asdf/blob/master/doc/best_practices.md
Anyway, I don't have a sufficient knowledge on ASDF's internal.
Robert P. Goldman Research Fellow Smart Information Flow Technologies (d/b/a SIFT, LLC) 319 N. First Ave., Suite 400 Minneapolis, MN 55401 Voice: (612) 326-3934 Email: rpgoldman@SIFT.net
participants (3)
-
Hugo Ishimaru -
Jim Newton -
Robert Goldman