Proposal: project pages deployment through GitLab
Hi all, As you all have noticed, I've been working to clean up common-lisp.net and simplifying the administration process. One of the things we have done is that we implemented a deployment pipeline for the common-lisp.net main site using GitLab. It's been a great joy to work with it so far and made deployment of new site content easier than ever before. *My proposal is to set up a GitLab CI based deployment pipeline* for *all common-lisp.net <http://common-lisp.net> projects*. Meaning that I'm proposing to import the current project pages (/project/*/public_html) into GitLab repositories (<project-name>-site) with a gitlab-ci file which causes the content to be published. The approach above will mean simple import of the existing static content. However, after import, the static output can be replaced by different input and a static content generator, just like we did with the common-lisp.net site. Eager to hear your thoughts, Erik.
That sounds like a natural next step to me. Maybe we'll finally set up a Project Page for Gendl in that case! On Fri, Oct 26, 2018 at 11:24 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi all,
As you all have noticed, I've been working to clean up common-lisp.net and simplifying the administration process.
One of the things we have done is that we implemented a deployment pipeline for the common-lisp.net main site using GitLab. It's been a great joy to work with it so far and made deployment of new site content easier than ever before.
*My proposal is to set up a GitLab CI based deployment pipeline* for *all common-lisp.net <http://common-lisp.net> projects*. Meaning that I'm proposing to import the current project pages (/project/*/public_html) into GitLab repositories (<project-name>-site) with a gitlab-ci file which causes the content to be published.
The approach above will mean simple import of the existing static content. However, after import, the static output can be replaced by different input and a static content generator, just like we did with the common-lisp.net site.
Eager to hear your thoughts,
Erik.
-- My Best, Dave Cooper, david.cooper@gen.works genworks.com, gendl.org +1 248-330-2979
Hi Dave, On Fri, Oct 26, 2018 at 6:53 PM Dave Cooper <david.cooper@genworks.com> wrote:
That sounds like a natural next step to me.
Ok. Good. It's probably good to note that in my plans this type of project page deployment will be the *only* supported way for project page deployment once introduced. (Given the little feedback I received last time I pitched the idea, I thought it was a good idea to mention this fact.) I've analysed the content of some of the largest /project/*/public_html directories to prepare for this move. While we made a huge step in reducing the content of public_html/ directories which wasn't really "site content" (yet, rather Git or Darcs repositories), some of the largest ones are using the public_html/ directory for binary artifact distribution (mostly release binaries). I'll have a look at what we can do to support hosting that kind of content (maybe with an artifact repository of some kind?) instead of depending on public_html/ for it.
Maybe we'll finally set up a Project Page for Gendl in that case!
I'll also check if I can make this a phased approach where we can move the projects over which aren't hosting (release/non-site) artifacts and work on the other projects when we have a solution for the binary artifact content. Regards, Erik.
On Fri, Oct 26, 2018 at 11:24 AM Erik Huelsmann < ehuelsmann@common-lisp.net> wrote:
Hi all,
As you all have noticed, I've been working to clean up common-lisp.net and simplifying the administration process.
One of the things we have done is that we implemented a deployment pipeline for the common-lisp.net main site using GitLab. It's been a great joy to work with it so far and made deployment of new site content easier than ever before.
*My proposal is to set up a GitLab CI based deployment pipeline* for *all common-lisp.net <http://common-lisp.net> projects*. Meaning that I'm proposing to import the current project pages (/project/*/public_html) into GitLab repositories (<project-name>-site) with a gitlab-ci file which causes the content to be published.
The approach above will mean simple import of the existing static content. However, after import, the static output can be replaced by different input and a static content generator, just like we did with the common-lisp.net site.
Eager to hear your thoughts,
Erik.
-- My Best,
Dave Cooper, david.cooper@gen.works genworks.com, gendl.org +1 248-330-2979
Ok. Good. It's probably good to note that in my plans this type of project page deployment will be the *only* supported way for project page deployment once introduced. (Given the little feedback I received last time I pitched the idea, I thought it was a good idea to mention this fact.)
I've analysed the content of some of the largest /project/*/public_html directories to prepare for this move. While we made a huge step in reducing the content of public_html/ directories which wasn't really "site content" (yet, rather Git or Darcs repositories), some of the largest ones are using the public_html/ directory for binary artifact distribution (mostly release binaries). I'll have a look at what we can do to support hosting that kind of content (maybe with an artifact repository of some kind?) instead of depending on public_html/ for it.
I'm not following. What exactly does "importing the current project pages into GitLab repositories" entail, and why is that a problem for release tarballs? Is it not just putting the static site files into source control? One other thing that should be preserved is hosting for user pages, like https://common-lisp.net/~vsedach/ Vladimir
Heh. I noticed recently that gitlab was finally adding project pages like github gh-pages. I was gong to ask if this was available on c-l.net. IIRC, these are essentiall static (mostly) html pages for your site so you can host documentation and such. I'd be interested in this since cmucl hosts its html page elsewhere, and it'a somewhat of a pain to update them. I'm interested! And again, thanks for all your hard work on this! On Fri, Oct 26, 2018 at 8:24 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi all,
As you all have noticed, I've been working to clean up common-lisp.net and simplifying the administration process.
One of the things we have done is that we implemented a deployment pipeline for the common-lisp.net main site using GitLab. It's been a great joy to work with it so far and made deployment of new site content easier than ever before.
*My proposal is to set up a GitLab CI based deployment pipeline* for *all common-lisp.net <http://common-lisp.net> projects*. Meaning that I'm proposing to import the current project pages (/project/*/public_html) into GitLab repositories (<project-name>-site) with a gitlab-ci file which causes the content to be published.
The approach above will mean simple import of the existing static content. However, after import, the static output can be replaced by different input and a static content generator, just like we did with the common-lisp.net site.
Eager to hear your thoughts,
Erik.
-- Ray
Hi Raymond, Thanks for your reaction! On Sun, Oct 28, 2018 at 6:52 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Heh. I noticed recently that gitlab was finally adding project pages like github gh-pages. I was gong to ask if this was available on c-l.net. IIRC, these are essentiall static (mostly) html pages for your site so you can host documentation and such. I'd be interested in this since cmucl hosts its html page elsewhere, and it'a somewhat of a pain to update them.
I'm interested! And again, thanks for all your hard work on this!
You're welcome. Just for my clear understanding, since you explicitly reference "github pages": I was planning to map the static content to the "regular" project pages space: https://common-lisp.net/project/*/ . Is that sufficient? Or were you thinking of using your own domain? Or are you looking for something like "https://cmucl.common-lisp.io/" ? Regards, Erik.
I haven't given a great deal of thought about this because I just learned about gitlab pages. Right now just having thing https://common-lisp.net/project/ <https://common-lisp.net/project/*/> is ok, because cmucl just copies it's web pages to public_html/mirror (or something) and www.cons.org (maybe still?) and www.cmucl.org just rsyncs that directory periodically. I think cmucl.common-lisp.net.io would be nice as well since that's how I manage my various github pages projects. It would be convenient I think if my cmucl-www repo could have html pages (via markdown or whatever) automatically get displayed an cmucl-www.common-lisp.io (or something) without me having to do a bit of manual work that I have to do today. (Changes are infrequent now because I'm just a lazy guy.) Again, I haven't given a lot of thought about this, but on the face of it, it seems nice for people use to github. But I also understand that this is done by volunteers so I don't definitely don't want to overtax the very generous volunteers working on this! There's certainly no rush. (And I noticed a bit ago that c-l.net's gitlab wiki now supports kaTeX in markdown pages which is really great to see! I wish it were Mathjax instead, but kaTeX is certainly better than none!) On Sun, Oct 28, 2018 at 11:24 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi Raymond,
Thanks for your reaction!
On Sun, Oct 28, 2018 at 6:52 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Heh. I noticed recently that gitlab was finally adding project pages like github gh-pages. I was gong to ask if this was available on c-l.net. IIRC, these are essentiall static (mostly) html pages for your site so you can host documentation and such. I'd be interested in this since cmucl hosts its html page elsewhere, and it'a somewhat of a pain to update them.
I'm interested! And again, thanks for all your hard work on this!
You're welcome.
Just for my clear understanding, since you explicitly reference "github pages": I was planning to map the static content to the "regular" project pages space: https://common-lisp.net/project/*/ . Is that sufficient? Or were you thinking of using your own domain? Or are you looking for something like "https://cmucl.common-lisp.io/" ?
Regards,
Erik.
-- Ray
Hi Raymond, On Sat, Nov 3, 2018 at 12:12 AM Raymond Toy <toy.raymond@gmail.com> wrote:
I haven't given a great deal of thought about this because I just learned about gitlab pages. Right now just having thing https://common-lisp.net/project/ <https://common-lisp.net/project/*/> is ok, because cmucl just copies it's web pages to public_html/mirror (or something) and www.cons.org (maybe still?) and www.cmucl.org just rsyncs that directory periodically.
The way I'm using GitLab Pages on common-lisp.net is a "light" (or modified) version of GitLab Pages. However, when this all turns out to be a success (and when I migrate the VM to a machine which has spare IP addresses), we can go for the "full" GitLab Pages version, which includes you being able to run the cmucl.org domain on our server and directly deploy to that.
I think cmucl.common-lisp.net.io would be nice as well since that's how I manage my various github pages projects.
Ok. That's something to work on. Would be nice to gather feedback and see if people prefer this over common-lisp.net/project/* pages.
It would be convenient I think if my cmucl-www repo could have html pages (via markdown or whatever) automatically get displayed an cmucl-www.common-lisp.io (or something) without me having to do a bit of manual work that I have to do today. (Changes are infrequent now because I'm just a lazy guy.)
Right. That's what this change is about: if you set up a GitLab project named "cmucl/cmucl-site" (and appropriate deployment instructions using .gitlab-ci.yml, as per my earlier e-mail), your pages will be automatically deployed after you push to the repository.
Again, I haven't given a lot of thought about this, but on the face of it, it seems nice for people used to github.
But I also understand that this is done by volunteers so I don't definitely don't want to overtax the very generous volunteers working on this! There's certainly no rush.
Yea. Actually I think this change cuts both ways though: it'll help you deploy your site *and* it helps me simplify as well as modernize the ' common-lisp.net' services.
(And I noticed a bit ago that c-l.net's gitlab wiki now supports kaTeX in markdown pages which is really great to see! I wish it were Mathjax instead, but kaTeX is certainly better than none!)
As for Mathjax, you could put a plea in with the GitLab developers? :-) Anyway, if you have any questions, feel free to ask here on the list and I'm happy to help! Regards, Erik.
On Sun, Oct 28, 2018 at 11:24 AM Erik Huelsmann < ehuelsmann@common-lisp.net> wrote:
Hi Raymond,
Thanks for your reaction!
On Sun, Oct 28, 2018 at 6:52 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Heh. I noticed recently that gitlab was finally adding project pages like github gh-pages. I was gong to ask if this was available on c-l.net. IIRC, these are essentiall static (mostly) html pages for your site so you can host documentation and such. I'd be interested in this since cmucl hosts its html page elsewhere, and it'a somewhat of a pain to update them.
I'm interested! And again, thanks for all your hard work on this!
You're welcome.
Just for my clear understanding, since you explicitly reference "github pages": I was planning to map the static content to the "regular" project pages space: https://common-lisp.net/project/*/ . Is that sufficient? Or were you thinking of using your own domain? Or are you looking for something like "https://cmucl.common-lisp.io/" ?
Regards,
Erik.
-- Ray
On Oct 26, 2018, at 17:23, Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi all,
As you all have noticed, I've been working to clean up common-lisp.net and simplifying the administration process.
One of the things we have done is that we implemented a deployment pipeline for the common-lisp.net main site using GitLab. It's been a great joy to work with it so far and made deployment of new site content easier than ever before.
My proposal is to set up a GitLab CI based deployment pipeline for all common-lisp.net projects. Meaning that I'm proposing to import the current project pages (/project/*/public_html) into GitLab repositories (<project-name>-site) with a gitlab-ci file which causes the content to be published.
The approach above will mean simple import of the existing static content. However, after import, the static output can be replaced by different input and a static content generator, just like we did with the common-lisp.net site.
[Sorry about the late response here. I think I pinged you on IRC about this, but then I am repeating myself here for public record.] Would it be possible to blacklist a given project from this process? I would have some difficulties migrating the ‘armedbear’ project to this scheme: 1. The ‘armedbear’ project uses a single svn repository for its website content as well as its source code. The website content also contains all the binary releases for the project. While not huge by contemporary standards, the idea of importing all these binary releases into a git repository seems like the wrong path for future-proofing. 2. Using gitlab pages would impact the current release engineering process in unknown ways. I wouldn’t anticipate that there would be too much of an impact in adapting, but change, you know? sincerely, Mark -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."
participants (5)
-
Dave Cooper -
Erik Huelsmann -
Mark Evenson -
Raymond Toy -
Vladimir Sedach