I've seen flat namespaces and one-package-per-file and everything in-between and beyond. All style can be done well or be done poorly. What matters most is that namespaces must be curated to be livable. Having someone responsible to maintain it, make it good, make it consistent, keep it good, fix the bugs, improve the interfaces, etc. — That's the important and hard part.

That said, I admit I've grown to like the approach of ASDF and LIL, of one-package-per-file internally (also each file a system, as for LIL), plus special package files that (automatically) import and reexport all symbols in a directory. This style imposes a somewhat onerous discipline, but in exchange you have a powerful framework to keep you code well-organized: each file by construction can only refer to symbols from files that it depends on, you have think of your dependencies clearly. Any forward reference requires special provisions. Without this discipline, ASDF could not have grown up from the 4516 line mess that ASDF had become as of 2.6, to the well-organized 13525 line robust and portable software that it is as of 3.3.2.9. It does so much more under the hood, and is portable to so many more platforms, yet at the same time the code is so much more readable, and the underlying algorithm, while much more elaborate, is also much more understandable. One of the few things that survives of my work at ITA :-(

As for LIL, it also uses packages for punning: there are distinct but related symbols stateful:insert pure:insert classy:insert and posh:insert for inserting a key / value pair in a map in all combinations of stateful interface-passing-style, pure interface-passing-style, classic imperative object oriented style, and purely functional object oriented style — with macros that can automatically wrap one into the others. That's incredibly powerful and liberating: being able to use parametric polymorphism to bootstrap elaborate pure functional data structures like Haskell typeclasses, then exporting classic imperative object-oriented hierarchies for users who don't want to know.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Trillions of fossils can't be wrong!

On Sat, Aug 25, 2018 at 8:32 PM Scott McKay swmckay@gmail.com wrote:

+1

I think we did OK in CLIM: an API package, and an internal package to implement in.

--Scott

On Aug 25, 2018, at 7:53 PM, Ken Tilton kentilton@gmail.com wrote:

Packages are massively overrated. This is not Java where every frickin source file is a namespace. There is a certain obsessive compulsiveness about packages that does nothing but slow developers down. Well, right, they are a palliative for the OCD disease. But it *is* a disease, so that does not count.

What part of agile do we not understand? Fences, boxes, categories, types all invented for their own sake let us bask in our taxonomicity while getting no code written, and god help the sucker who tries to use our OCD mess forever battling package issues.

Stop. Wrong way. Go back.

On Tue, Aug 21, 2018 at 6:36 PM, Daniel Pezely daniel@pezely.com wrote:

...

I posted this to Stack Overflow but was hoping for more input.

https://stackoverflow.com/questions/51638864/common-lisp-style-multiple-pack...

May I get recommendations or links to representative code repositories with good style for multiple related Common Lisp packages, please?

For instance, consider a high-level workflow library with accompanying lower-level API, each in its own CL package but same git repo due to synchronized releases.

Each system (*.asd file) isolates tests and may be invoked using:

(asdf:test-system foo :force t)

Separate systems may be built via make, which definitely helps isolate SBCL code-coverage reports.

Some users of the library may only want to load the lower-level API. For simplifying dependencies for those using higher-level API, it seems best to keep everything bundled in one repo. Any revision to one library would likely require updating all for the same release.

I currently have a single directory tree with a subdirectory for each CL package. There's a top-level Makefile plus one in each subdirectory that the library maintain would use. The top-level also contains symbolic links for .asd files pointing into relevant subdirectories. (It's a library deeply dependent upon POSIX calls via uiop-posix, so it's only applicable on an OS with sym-links.)

This seems to be an issue at large considering issue #1 for Quicklisp-docs [0].

Found nothing relevant in Google's CL style guide [1], State of the Common Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4]. Browsing repos seem to have quite a mix of styles.

[0] https://github.com/rudolfochrist/quicklisp-docs/issues/1 [1] https://google.github.io/styleguide/lispguide.xml [2] https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-... [3] http://weitz.de/cl-recipes/ [4] http://lisp-lang.org/learn/writing-libraries but see also their section on Continuous Integration

Repo to be fixed: https://gitlab.com/dpezely/cl-mmap (commit a23bd88d of 2018-07-14; release will be tagged when fixed)

Thanks, -Daniel

-- Kenneth Tilton http://tiltontec.com/

Perhaps this presents an opportunity for maintainers of Quicklisp (perhaps in collaboration with Quickdocs) to introduce a style guide for making their lives easier.

I edited the Google Common Lisp Style Guide (well, at least the version that was published a few years back; I know it has evolved internally since, but no updates were published.) I also published an ASDF "best practices" document.

There are many acceptable yet mutually incompatible ways of using Common Lisp packages. You must pick one and stick to it within any given system. Maybe you can describe them (or at least, one) in a blog post and/or some other document that you write.

Then again, there are also proposed alternatives to packages. Or you may embrace modernity and embrace Racket modules, or their reimplementation by Gerbil Scheme.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org It is paradoxical, yet true, to say, that the more we know, the more ignorant we become in the absolute sense, for it is only through enlightenment that we become conscious of our limitations. Precisely one of the most gratifying results of intellectual evolution is the continuous opening up of new and greater prospects. — Nikola Tesla

On 2018-08-25 05:29 PM, Scott McKay wrote:

+1

I think we did OK in CLIM: an API package, and an internal package to implement in.

--Scott

On Aug 25, 2018, at 7:53 PM, Ken Tilton <kentilton@gmail.com mailto:kentilton@gmail.com> wrote:

...

Packages are massively overrated.  [...]

Scott,

Are you referring to the earlier CLIM from CMU [1] or McCLIM [2] or another specific implementation?  My understanding is that "CLIM" today usually implies the spec, but I've yet to use any of these packages.

What did you or others appreciate most about its package organization?

McCLIM seems to separate API from internals and uses various systems named `mcclim/foo` and packages named `mcclim-foo` invoked by those systems.  This is just from glancing at various *.asd files and package.lisp.

The older CMU version seems to be more fine-grained by separating platform-specific bits under `impl_dep` subdirectory.

[1] http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/gui/clim/0.h...   [2] https://github.com/McCLIM/McCLIM

Thanks,   -Daniel

Hi!

ASDF does not "build on packages". ASDF loads files, in some of which usually packages happen to be defined, but ASDF doesn't depend on that. You could define an entire system only of symbols in the cl-user (or even keyword) package, and neither ASDF nor Quicklisp would care.

The only concern of packages is organizing symbols. The question is how that should correspond to the file structure (and how that should look like). There is a continuum there, and I guess that it would help to have some resource showing the usual patterns.

Yours aye

Svante

Am 27. August 2018 04:47:28 MESZ, schrieb Daniel Pezely daniel@pezely.com:

On 2018-08-25 04:53 PM, Ken Tilton wrote:

...

Packages are massively overrated. This is not Java where every

frickin

...

source file is a namespace. There is a certain obsessive compulsiveness about packages that does nothing but slow developers down. Well, right, they are a palliative for the OCD disease. But it *is* a disease, so that does not count.

What part of agile do we not understand? Fences, boxes, categories, types all invented for their own sake let us bask in our taxonomicity

...

while getting no code written, and god help the sucker who tries to use our OCD mess forever battling package issues.

Stop. Wrong way. Go back.

Ken,

How might we use this criticism constructively?

For the intention of having other people use the source code and possibly being picked up by Quicklisp, what do you propose instead, if packages are overrated?  My understanding: Quicklisp relies upon ASDF systems, which in turn builds upon packages by convention. What have I missed there?

This is separate from being agile.  I see agile methodologies as how to

attain a "release" most effectively while criteria or requirements continue evolving.

The question of the original post is ultimately about the release as a discrete artifact.

In my case and for many others, the release means playing nice with other code fetched by Quicklisp as well as minimizing barriers for other people to comprehend and reason about the code without involving original authors or current maintainers.  Using conventional mechanism such as packages and ASDF systems serves that end.

When you say, "Go back", go back to what?

The days of downloading a tar file-- or worse, a single source file-- from assorted locations in hopes of having it all work together thankfully ended for the most part.  I've happily traded that era for repeatable, reliable builds with optional version-pinning and am forever thankful to Xach for bringing this to contemporary CL users.

Or have I misunderstood your points?

Thanks,   -Daniel

Woo hoo, "not abhorrent"! ;-)

I don't know precisely what McCLIM did because I had been CLIM'ing for ages and could not bring myself to do it again.

Here was my thinking: - Two packages per library - The first is the API package; it exports the names that all ordinary users will use. - The second is the internal API packages. It exports the names that people who will *extend* the library will use, and can also serve as an implementation package. It :USEs the first package. - One library per "big orthogonal thing" – whatever that is.

In hindsight, I would have had fewer CLIM libraries, probably just a utilities/geometry one, a windows one, and an output recording/presentation/application/commands one.

On Mon, Aug 27, 2018 at 9:27 AM, Ken Tilton kentilton@gmail.com wrote:

On Sun, Aug 26, 2018 at 10:47 PM Daniel Pezely daniel@pezely.com wrote:

...

On 2018-08-25 04:53 PM, Ken Tilton wrote:

Packages are massively overrated. This is not Java where every frickin source file is a namespace. There is a certain obsessive compulsiveness about packages that does nothing but slow developers down. Well, right, they are a palliative for the OCD disease. But it *is* a disease, so that does not count.

What part of agile do we not understand? Fences, boxes, categories, types all invented for their own sake let us bask in our taxonomicity while getting no code written, and god help the sucker who tries to use our OCD mess forever battling package issues.

Stop. Wrong way. Go back.

Ken,

How might we use this criticism constructively?

Use one package per library. If you want to document your library, document the public bits.

btw, I did not find the McCLIM idea of a second package for internals abhorrent.

And the day I let a disaster like ASDF dictate my coding is the day I sign up for bartender school.

We had an 80kloc CL app that was well divided into packages and it was an endless source of pain. It took a week but I flattened everything into one package and we had one bug related to symbol confusion. But then we used the old 2-3-character prefixing scheme for readabilty and disambiguation.

-kt

Kenneth Tilton http://tiltontec.com/

Yep, a test suite is fine, and making it a separate project as well.

-hk

On Mon, Aug 27, 2018 at 11:08 AM Don Morrison dfm@ringing.org wrote:

On Mon, Aug 27, 2018 at 9:29 AM Ken Tilton kentilton@gmail.com wrote:

...

Use one package per library. If you want to document your library,

document the public bits.

While by and large this does seem to work well, there is often value in having a second package for the tests, that uses the library’s main package. Otherwise you can accidentally use internal symbols in your tests in ways not publicly exposed to real users of your library; on those occasions when you really do need access to internal symbols in your tests it seems better to have to explicitly import or qualify them.

-- Don Morrison dfm@ringing.org “Each of these views has been held by an important segment of the cognitive science community, and the two segments do not often communicate with each other, except sometimes to quarrel.” – Herbert Simon, /Models of My Life/