On 2016-01-02 00:33 CET, Robert Goldman rpgoldman@sift.net wrote:
On 1/1/16 Jan 1 -6:18 AM, Kambiz Darabi wrote:
On 2016-01-01 09:59 CET, Kambiz Darabi darabi@m-creations.com wrote:
Happy New Year,
On 2015-12-31 16:12 CET, Robert Goldman rpgoldman@sift.net wrote:
On 12/31/15 Dec 31 -5:46 AM, Kambiz Darabi wrote:
On 2015-12-29 01:58 CET, Robert Goldman rpgoldman@sift.net wrote:
The problem of having clones not get auto-populated seems not worse than needing to do a magical incantation when you merge stuff back and forth at the cost of garbling your repo.
The complete magical incantation reads:
git subtree add --prefix=ext/alexandria --squash https://gitlab.common-lisp.net/alexandria/alexandria.git master
and to update it to the lastest master, replace 'add' with 'pull'.
The submodules cause annoying quiet failures, but even when the annoying quiet failures happen, you don't end up mangling your repo.
The problem with submodules is that every single person who checks out the repo is concerned with it, whereas git subtree has to be performed only by one single person and for all others it just works without them even knowing that there is a 'subtree' being used somewhere.
Can you explain that stuff about needing to squash? I read the subtree tutorial and that discussion just seemed like gibberish to me. Very far from clear. Maybe it's easier than the discussion made it seem. As I read it, it seemed to be saying that if you didn't remember to do --squash at the right times you would end up with a repo clogged with duplicate commits.
If that's the case...ugh. But perhaps it was just a bad explanation? Or does your subtree add expression above ensure that the squashing becomes the default?
My command above does contain the '--squash' option. The git-subtree author just chose the wrong default: usually, you are not interested in seeing each and every commit in the external repo.
But again, if you do the wrong thing, it's just a matter of:
git reset --hard HEAD~2 git subtree add/pull --squash ...
If we could figure this out maybe subtrees would be easier. But if you have to remember to squash every time, forget it....
IMO such commands should be automated with a makefile entry, so you wouldn't have to remember the --squash option.
Unfortunately, right now the makefile is in the worst condition of the whole project, because of the recent merge of minimakefile, which by its nature was largely untestable (since exercising its functionality would change the repository, and even the upstream repository).
AFAICS there are only two places where a remote repo ist changed:
https://gitlab.common-lisp.net/asdf/asdf/blob/master/tools/git.lisp#L18 https://gitlab.common-lisp.net/asdf/asdf/blob/master/tools/git.lisp#L44
So testing makefile changes is only a matter of renaming the remotes 'origin', 'cl.net', and 'github'.
One can even git clone locally, add the local 'remotes', name them appropriately and test the changes.
Indeed, just recently, I wasn't able to use the makefile to fix my problems with the addition of new ext/ submodules because..... the makefile didn't work until the submodules were added.
This is also a point in favor of subtree, as the dependencies are then part of the repo.
These 9 lines were necessary to implement the new behaviour:
https://gitlab.common-lisp.net/darabi/asdf/commit/fc24dae0a7ba8ccbe7633584da...
I found it quite easy to find the correct place to modify the code although I'm not at all familiar with the code base.
I pushed a branch to my repo which contains the necessary changes:
https://gitlab.common-lisp.net/darabi/asdf/tree/ext-subtree
The specification of what to get where and which version is in ext/dependencies.lisp-expr
and
git checkout ext-subtree make ext
does the job.
I'm happy to consider switching to using git-subtree. It may indeed be better.
But not until it gets some better documentation. The discussion in the Pro-Git book, to be honest, just makes my head hurt. See https://git-scm.com/book/en/v1/Git-Tools-Subtree-Merging
This is the documentation for the concept of subtree merging, not the 'git subtree' command. You don't have to deal with the subtree merge yourself, the command 'git subtree' does it for you.
I apologise for not having provided the pointer to the docs in the first place (and will give a short example below):
man git subtree
should give you this:
http://manpages.ubuntu.com/manpages/trusty/man1/git-subtree.1.html
So, the remote repository is added as a remote for the *containing* repository? ASDF would have the cl-scripting as an upstream remote for ASDF, not for a contained thing?
No, the commits in cl-scripting (leading to HEAD) are imported into a local branch (but you don't have to deal with that branch in any way).
Let's go through the process, after you pull my branch ext-subtree:
after Faré's last commit 'tools: Simplify a ,@ into a ,', there is 'Remove git submodules in ext' which deletes .gitmodules and all ext subdirs:
https://gitlab.common-lisp.net/darabi/asdf/commit/d028e6d2b9d729a47a7380b05e...
then, I performed
git subtree add --squash --prefix=ext/alexandria <alexandria-repo-url> master
which creates two commits, look at the first one with 'git log --name-status 915257':
Squashed 'ext/alexandria/' content from commit 5a17c07
git-subtree-dir: ext/alexandria git-subtree-split: 5a17c072970cf50213f7f896c40e6e640638391f
A .boring A .gitignore A AUTHORS A LICENCE A README A alexandria-tests.asd A alexandria.asd ...
As one can see, this is the content of the alexandria repo. Hidden behind the git-subtree-split property, is the branch which contains the history of alexandria which led to master:
git log --oneline 5a17c07
5a17c07 fix hygiene and multiple evaluation in ENSURE-GETHASH b1c6ee0 Fix default ARRAY-INDEX and ARRAY-LENGTH. a019eda Extended DELETE-FROM-PLIST.1 test to test the removal of duplicate keys. 4c4361c Use the original DELETE-FROM-PLIST replacement.
Now back to the second commit which was performed by 'git subtree add':
commit 62116639895d8c2f537b0cb8f26314716edf3900 Merge: d028e6d 9152572 Author: Kambiz Darabi darabi@m-creations.net Date: Fri Jan 1 12:53:03 2016 +0100
Add ext/alexandria as git subtree
git subtree add --prefix=ext/alexandria --squash https://gitlab.common-lisp.net/alexandria/alexandria.git master
This commit is a merge and doesn't add any specific files, but if you look at ext/alexandria, you can see that is was populated with the content of "Squashed 'ext/alexandria/'".
That's it. From now on, Alexandria's commit 5a17c07 will live in our repo at ext/alexandria.
If one day you decide that it's time to update to the latest version of alexandria, you perform:
git subtree pull --squash --prefix=ext/alexandria <alexandria-repo-url> master
and the same thing happens again.
Now to the makefile automation part:
The file ext/dependencies.lisp-expr
https://gitlab.common-lisp.net/darabi/asdf/commit/fc24dae0a7ba8ccbe7633584da...
currently contains:
(setf asdf-tools::*ext-dependencies* '(("alexandria" "master" "https://gitlab.common-lisp.net/alexandria/alexandria.git") ... ("optima" "master" "https://github.com/m2ym/optima.git")))
performing 'make ext' leads to a 'git subtree pull --squash' as described above. So, as a developer, you can decide at which point in time you would like to upgrade to newer versions of the dependencies.
If I was the maintainer, I would edit dependencies.lisp-expr and replace 'master' with the latest release tag of the corresponding repos or a specific commit if the repo doesn't have release tags. This would lead to clearly defined dependencies which would make build and testing reproducible.
But this is of course your decision.
HTH
Kambiz