[slime-devel] [RfC] Reorganization of source
Hi, I'd like to suggest a reorganization of the Slime sources. Some not-so-long time ago, there has been some vague consensus of factoring some parts of Slime out into a contrib/ directory. The intention of this mail is to straighten out the vagueness while building upon the consensus. I propose: a) to introduce a new file `swank-utils.lisp' (and perhaps also `slime-utils.el') and collect common utility functions there in. Gripe: `swank.lisp' is currently almost 6000 lines long. As a result, it's very hard to get an overview of what's there already, and consequently what could be reused in other (or new!) parts. Also there's some stuff that can be very useful for foreign projects that perhaps would like to take advantage, but don't want all of SWANK (the arglist stuff comes to mind.) b) to introduce a new directory `modules/' that contain self-contained subsystems. For instance: modules/ inspector/ fuzzy-compl/ These subdirectories contain both Common Lisp and Elisp code. It's subject to further discussion what's the best way of loading these subsystems. Are there special requirements? The purpose is to reduce the line count of `swank.lisp' while aiding understanding by explicitely decoupling of independent subsystems. Also to establish the ground work for a plugin architecture if this deems useful. c) to introduce a new directory `backends/' and place all the `swank-sbcl.lisp', `swank-clisp.lisp' &c files there. This is merely for aesthetical reasons (less clutter in the main source directory) and hence rather optionally. -T.
[sending this posting is against my better judgement :)] * "Tobias C. Rittweiler" <87ps1u2qvj.fsf@freebits.de> : | I'd like to suggest a reorganization of the Slime sources. Some | not-so-long time ago, there has been some vague consensus of factoring | some parts of Slime out into a contrib/ directory. The intention of this | mail is to straighten out the vagueness while building upon the | consensus. | | a) to introduce a new file `swank-utils.lisp' (and perhaps also | `slime-utils.el') and collect common utility functions there in. | | Gripe: `swank.lisp' is currently almost 6000 lines long. As a | result, it's very hard to get an overview of what's there already, | and consequently what could be reused in other (or new!) parts. slime.el is over 11k lines long. I think byte compiling slime.el in a new emacs and fixing all statically detected errors would be a start. Splitting up slime.el into bunch of files along lines of functionality would give insights into how the common lisp code would need to be split up. Unless you want to do this as a CVS branch, it may be better to plan for reorganization than decide directory structure upfront. i.e. do the splitting up in iterations and reorg bits at a time, Personally I'd prefer the reorg done on a git repo outside of official CVS and synched back (on a branch perhaps) | Also there's some stuff that can be very useful for foreign projects | that perhaps would like to take advantage, but don't want all of | SWANK (the arglist stuff comes to mind.) | | | b) to introduce a new directory `modules/' that contain self-contained | subsystems. For instance: | | modules/ | inspector/ | | fuzzy-compl/ | | These subdirectories contain both Common Lisp and Elisp code. It's | subject to further discussion what's the best way of loading these | subsystems. Are there special requirements? My personal taste would be (at least initially) to put common lisp and emacs lisp in different directories: emacs-lisp/slime-{inspector,fuzzy-completions}.el lisp/{inspector,fuzzy-completions}.lisp And keep exactly one file per functional unit. As an initial-cut the [backend]/swank-{cmucl,lw,...}.lisp would cut across these functional boundaries and have all the code for that lisp platform. Rationale: You'd want to add one directory to load-path. There is no equivalent to load-path in cl but the same aesthetic applies: swank.lisp would look in one directory for "plugin" files -- viz lisp/ which has the lisp/{inspector,fuzzy-completions,...}.lisp files. | The purpose is to reduce the line count of `swank.lisp' while | aiding understanding by explicitely decoupling of independent | subsystems. Also to establish the ground work for a plugin | architecture if this deems useful. | | | c) to introduce a new directory `backends/' and place all the | `swank-sbcl.lisp', `swank-clisp.lisp' &c files there. This is | merely for aesthetical reasons (less clutter in the main source | directory) and hence rather optionally. -- Regards Madhu
Madhu <enometh@meer.net> writes:
[sending this posting is against my better judgement :)]
Thanks, I appreciated it. I do intend to do the clean up on a branch. As your points cover details, I'll come back to them when their time is due. -T.
* Tobias C. Rittweiler [2007-08-11 14:32+0200] writes:
Hi,
I'd like to suggest a reorganization of the Slime sources. Some not-so-long time ago, there has been some vague consensus of factoring some parts of Slime out into a contrib/ directory. The intention of this mail is to straighten out the vagueness while building upon the consensus.
I propose:
a) to introduce a new file `swank-utils.lisp' (and perhaps also `slime-utils.el') and collect common utility functions there in.
Gripe: `swank.lisp' is currently almost 6000 lines long. As a result, it's very hard to get an overview of what's there already, and consequently what could be reused in other (or new!) parts.
Also there's some stuff that can be very useful for foreign projects that perhaps would like to take advantage, but don't want all of SWANK (the arglist stuff comes to mind.)
No consensus here. The basic structure, a few big files, should stay as it is. It's also not our business to support foreign projects. If other projects depend on Slime internals, it's their problem.
b) to introduce a new directory `modules/' that contain self-contained subsystems. For instance:
modules/ inspector/
fuzzy-compl/
These subdirectories contain both Common Lisp and Elisp code. It's subject to further discussion what's the best way of loading these subsystems. Are there special requirements?
One requirement is that Slime, in the default configuration, should also work without the modules directory.
The purpose is to reduce the line count of `swank.lisp' while aiding understanding by explicitely decoupling of independent subsystems. Also to establish the ground work for a plugin architecture if this deems useful.
The purpose is to remove (bloated) features from Slime. Putting bloated code in it's own directory is a diplomatic way to remove a feature. The code in the modules directory is maintained (or not) by those people who care about it; not necessarily the Slime maintainers. Conceptually, the stuff in the module directory isn't part of Slime. "Plugin architecture" sounds to me like "over engineered" :-) We want something simple and stupid, like require.
c) to introduce a new directory `backends/' and place all the `swank-sbcl.lisp', `swank-clisp.lisp' &c files there. This is merely for aesthetical reasons (less clutter in the main source directory) and hence rather optionally.
Similar to a). Overall, I think Slime's file organization as it is now is quite ok. The problem is that some features take way to much code. Reorganizing files alone doesn't reduce the amount of code. Removing features or replacing them with simpler versions does, but that doesn't seem to be popular in the Lisp world. Helmut.
Helmut Eller <heller@common-lisp.net> writes:
I propose:
a) to introduce a new file `swank-utils.lisp' (and perhaps also `slime-utils.el') and collect common utility functions there in. [...]
No consensus here. The basic structure, a few big files, should stay as it is.
What's the benefit?
b) to introduce a new directory `modules/' that contain self-contained subsystems. [...]
One requirement is that Slime, in the default configuration, should also work without the modules directory.
...
The purpose is to remove (bloated) features from Slime. Putting bloated code in it's own directory is a diplomatic way to remove a feature. The code in the modules directory is maintained (or not) by those people who care about it; not necessarily the Slime maintainers. Conceptually, the stuff in the module directory isn't part of Slime.
I deliberately widened that view to also include core features -- especially the inspector which you once classified as a such. The reason is simple: The INSPECT-FOR-EMACS stuff spans almost 1000 lines and is mostly uninteresting, not-really-pretty looking code. However, one has to remark at this point that both these properties are somewhat inherent to the purpose of the code, as it describes layout and presentation. It's necessary cruft so to speak. I'd really like to see this stuff swapped out into another file.
[...] Overall, I think Slime's file organization as it is now is quite ok. The problem is that some features take way to much code. Reorganizing files alone doesn't reduce the amount of code.
While reorganizing files alone doesn't reduce the amount of code per se, it _does_ reduce the amount of conceptually coherent code that I have to keep into my head at once. Or rather it helps in ignoring the bits that are uninteresting to the current problem. (Of course, there's buffer folding. But I don't think that it works that well with `M-.' --- and it doesn't allow to leverage Common Lisp's package system, assuming multiple package definitions within one source file is a big no-no.) -T.
* Tobias C. Rittweiler [2007-08-12 12:51+0200] writes:
Helmut Eller <heller@common-lisp.net> writes:
No consensus here. The basic structure, a few big files, should stay as it is.
What's the benefit?
I like it that way.
The reason is simple: The INSPECT-FOR-EMACS stuff spans almost 1000 lines and is mostly uninteresting, not-really-pretty looking code. However, one has to remark at this point that both these properties are somewhat inherent to the purpose of the code, as it describes layout and presentation. It's necessary cruft so to speak.
I'd really like to see this stuff swapped out into another file.
The original inspector was much much smaller. The ugliness is mostly due the misuse of the inspector as class browser. If you can remove the CLOS stuff, more power to you. Removing the inspector entirely is not an option.
While reorganizing files alone doesn't reduce the amount of code per se, it _does_ reduce the amount of conceptually coherent code that I have to keep into my head at once. Or rather it helps in ignoring the bits that are uninteresting to the current problem.
(Of course, there's buffer folding. But I don't think that it works that well with `M-.' --- and it doesn't allow to leverage Common Lisp's package system, assuming multiple package definitions within one source file is a big no-no.)
C-x n p works well with Slime. Helmut.
Helmut Eller <heller@common-lisp.net> writes:
* Tobias C. Rittweiler [2007-08-12 12:51+0200] writes:
Helmut Eller <heller@common-lisp.net> writes:
No consensus here. The basic structure, a few big files, should stay as it is.
What's the benefit?
I like it that way.
What's the reason of your liking then? :)
I'd really like to see this [inspector] stuff swapped out into another file.
The original inspector was much much smaller. The ugliness is mostly due the misuse of the inspector as class browser. If you can remove the CLOS stuff, more power to you. Removing the inspector entirely is not an option.
Oh, I didn't say the inspector should be removed. Merely that it should be swapped out, while still belonging to the core of Slime. Hence I moved from the idea of a contrib/ directory to the idea of a modules/ directory where even core features can be placed.
(Of course, there's buffer folding. But I don't think that it works that well with `M-.' --- and it doesn't allow to leverage Common Lisp's package system, assuming multiple package definitions within one source file is a big no-no.)
C-x n p works well with Slime.
While it's nice, it doesn't seem to work with `M-.' on definitions outside of the currently narrowed page. Also this doesn't address my point of leveraging the package systems (which would allow dropping the Elisp naming style of prepending the package name (or the name of the current logical entity respectively) onto symbols in some parts of the code.) -T.
* Tobias C. Rittweiler [2007-08-12 13:57+0200] writes:
What's the reason of your liking then? :)
Good taste needs no reason :)
Hence I moved from the idea of a contrib/ directory to the idea of a modules/ directory where even core features can be placed.
That's exactly what I'd like to avoid. I don't want to have more core features to support. The reorganization thing sounds like an excuse for introducting more features. Keeping the current organization increases the pressure to actually simplify the existing code instead of just moving bad code to other files. Helmut.
Helmut Eller <heller@common-lisp.net> writes:
Hence I moved from the idea of a contrib/ directory to the idea of a modules/ directory where even core features can be placed.
That's exactly what I'd like to avoid. I don't want to have more core features to support. The reorganization thing sounds like an excuse for introducting more features.
It is, partially. I want a way to introduce new features that are clearly cut apart from the core. What do you understand under "features to support"? And what do you understand under "support" in this context? (Do you mean accomodating code from features you do not care about while hacking on some other code?)
Keeping the current organization increases the pressure to actually simplify the existing code instead of just moving bad code to other files.
I can understand that point. But it does not seem hold. New features seem to be piled up (and want to be piled up), but without much care about the rest of the code base, because -- in my eyes -- no one is able to keep an overview over the code base anymore. And the weight of the existing code base seems to prevent any large-scale simplification. I for one try to keep my nose out of slime.el. My intention is to remedy exactly this by an incremental approach. -T.
* Tobias C. Rittweiler [2007-08-12 16:20+0200] writes:
It is, partially. I want a way to introduce new features that are clearly cut apart from the core.
Well, I think that a way to introduce new features is uninteresting as long as we don't have a way to remove features (from the core).
What do you understand under "features to support"? And what do you understand under "support" in this context?
(Do you mean accomodating code from features you do not care about while hacking on some other code?)
Supporting a feature means to ensure that the intended functionality actually works and continues to work in the future. It means also to read the bug reports and to fix the code in question; or to read the CVS commits to undo the introduces bugs; and also writing the related test cases and documentation.
I can understand that point. But it does not seem hold. New features seem to be piled up (and want to be piled up), but without much care about the rest of the code base, because -- in my eyes -- no one is able to keep an overview over the code base anymore.
I consider it a good thing that no new features are added.
And the weight of the existing code base seems to prevent any large-scale simplification. I for one try to keep my nose out of slime.el.
My intention is to remedy exactly this by an incremental approach.
The only thing that prevents simplification is that nobody has the guts to remove features. Helmut.
"Tobias C. Rittweiler" <tcr@freebits.de> writes:
Helmut Eller <heller@common-lisp.net> writes:
C-x n p works well with Slime.
While it's nice, it doesn't seem to work with `M-.' on definitions outside of the currently narrowed page.
This is a bug that should be fixed by using (widen) whenever the definition lies outside the narrowing region. Fixing bugs and limitations like this one is, in my opinion, a more useful thing for improving SLIME than attempting a grand re-organization of code. Matthias -- Matthias Köppe -- http://www.math.uni-magdeburg.de/~mkoeppe (currently @math.ucdavis.edu)
Matthias Koeppe <mkoeppe+slime@mail.math.uni-magdeburg.de> writes:
Fixing bugs and limitations like this one is, in my opinion, a more useful thing for improving SLIME than attempting a grand re-organization of code.
And it's precisely what I've been doing over the last months. It's just that it sometimes feels like a bug hydra: everytime one fixes a bug, three others pop up (not necessarily related to the changes on made, but simply things found on the course of debugging the other problem.) -T.
participants (4)
-
Helmut Eller -
Madhu -
Matthias Koeppe -
Tobias C. Rittweiler