November 2018 - clo-devel - mailman3.common-lisp.net

Proposal: project pages deployment through GitLab
by Erik Huelsmann 18 Nov '20

18 Nov '20

Hi all, As you all have noticed, I've been working to clean up common-lisp.net and simplifying the administration process. One of the things we have done is that we implemented a deployment pipeline for the common-lisp.net main site using GitLab. It's been a great joy to work with it so far and made deployment of new site content easier than ever before. *My proposal is to set up a GitLab CI based deployment pipeline* for *all common-lisp.net <http://common-lisp.net> projects*. Meaning that I'm proposing to import the current project pages (/project/*/public_html) into GitLab repositories (<project-name>-site) with a gitlab-ci file which causes the content to be published. The approach above will mean simple import of the existing static content. However, after import, the static output can be replaced by different input and a static content generator, just like we did with the common-lisp.net site. Eager to hear your thoughts, Erik.

5 8

How to deploy your project pages through GitLab
by Erik Huelsmann 19 Dec '18

19 Dec '18

Hi, As per my prior mail, I'm sending instructions as to how to start using GitLab to deploy your project pages (i.e. how to deploy what you would normally copy to /project/*/public_html using GitLab automation). First, the delpoyment through GitLab uses GitLab Pages which means that everything you can find about GitLab Pages based deployment of webpages (e.g. in the documentation: https://docs.gitlab.com/ee/user/project/pages/#how-it-works) applies here as well -- except for the URL on which your site will be published. Another important note is that if the directory /project/<your-project>/public_html exists, your pages will be served from that. If it doesn't, your project pages will be served from GitLab Pages, if that doesn't exist, your project pages are redirected to GitLab itself. This means that you can experiment with GitLab pages based deployment by moving your public_html directory out of the way. If your tests fail, simply move it back into place and your existing pages will be restored. *What do you need to do to use GitLab deployment?* 1. Create a GitLab repository called <project>-site in the group <project> 2. Add the content to be deployed to the repository (either static or to be built) 3. Add a '.gitlab-ci.yml' file to the repository 4. Rename the /project/<project>/public_html directory For the simplest of deployments (taking what's in the repository "as-is") see the cl-couch pages deployment repository and .gitlab-ci.yml at https://gitlab.common-lisp.net/cl-couch/cl-couch-site/. For an intermediary level deployment processing Markdown to HTML and wrapping the result in a template, see the common-lisp.net homepage generator (which we'd like to abstract into a generic tool; we can use a hand there!) at https://gitlab.common-lisp.net/clo/cl-site/. For the most complex we currently have - using Docker - see the Quickref build at https://gitlab.common-lisp.net/quickref/quickref/tree/continuous-integration ; note that this is currently on its "continuous-integration" branch. I hope it'll be adopted soon to be on the master branch. I'll add similar instructions to the c-l.net FAQ. Please feel free to follow-up with any questions you might have when you start playing around with this new feature. Regards, Erik.

2 13

ABCL tracker broken
by Michał "phoe" Herda 07 Nov '18

07 Nov '18

Via https://www.reddit.com/r/Common_Lisp/comments/9ur9fp/abcl_bug_tracker_and_m… https://abcl.org/trac/report/1 Trac Error TracError: No Trac environment found at /srv/project/armedbear/trac [Errno 2] No such file or directory: '/srv/project/armedbear/trac/VERSION'

3 2

ABCL Trac instance is down
by Mark Evenson 07 Nov '18

07 Nov '18

The ABCL trac is down due to missing the <file://common-lisp.net/srv/project/armedbear/trac/> directory. Any idea what happened? Can we please restore?

1 0

Moving Subversion repositories without OS-level owner into Gitlab
by Erik Huelsmann 03 Nov '18

03 Nov '18

Hi, On common-lisp.net, we have 46 projects which use Subversion as their version control system. Of those, there are 24 projects which have no OS-level users to commit to these projects. I'm planning to move these projects (listed below) to Gitlab. That way, hopefully, they become more easily accessible for "outside" contribution again. * advanced-readtable * asdf-addons * caleb * cffi-objects * clarity * cl-cairo2 * cl-colors * cl-darcs * cl-heap * cl-libtai * cl-monad-macros * cl-ncurses * cl-table * fomus * funds * graphic-forms * ht-ajax * lisplab * nio * nxtlisp * plain-odbc * python-on-lisp * rdnzl * the-feebs-war Regards, Erik.

1 0

Cleanup of empty projects
by Erik Huelsmann 03 Nov '18

03 Nov '18

Hi, Last week I cleaned up a number of still-born projects: projects which hadn't changed their placeholder pages and didn't have any commits in version control. Apparently, this missed the fact that some projects don't have their placeholder page, nor any other form of project page(s) and also don't have any commits in version control. This week, I'm working on setting up deployment of project pages through GitLab. That is to say, I'm looking at the projects which don't have any OS users left to deploy them. These seem really good candidates to move deployment to Gitlab for: that would enable deployment without the need to re-introduce new OS-level users. So, my immediate finding is that the following projects are actually completely empty (not even left the original placeholder page in place -- which is what I looked at last week -- but even removed *that*): * cl-blog * cl-component These projects don't have a repository, webpage or even OS-level group members (meaning the members which are there in GitLab were last active *before* the current instance was created...). Regards, Erik.

1 0

First projects deploying through GitLab
by Erik Huelsmann 03 Nov '18

03 Nov '18

Hi all, Based on earlier discussion, I've decided to support deployment through GitLab while not (yet) removing the ability to deploy through other methods and have proceeded to move some projects without OS-level members to deploy from GitLab. So far, I've started with (yes, I'm aware cl-dwim is nothing but a redirect): * cl-couch * submarine * cl-dwim * bdb Reasons to start with these: 1. Their current sites are deployed using Darcs (now converted to Git) and imported 2. They are among the smallest public_html directories we host 3. They don't have any remaining OS-level users who can access and deploy the project As a result, the associated OS-level groups as well as the associated /project/* directory hierarchy has been decommissioned as there was nothing that remained to be stored. With the conversion of these projects the proof of concept (that we can run from /project/*/public_html as well as GitLab deployments mixed across projects) has been realised. The above means other projects can start deploying their https://common-lisp.net/project/*/ pages through GitLab as well. I'll send a new mail how to set this up for your projects. Regards, Erik.

1 0

Re: Proposal: project pages deployment through GitLab
by Erik Huelsmann 01 Nov '18

01 Nov '18

Hi Vladimir, On Tue, Oct 30, 2018 at 8:15 AM Vladimir Sedach <vas(a)oneofus.la> wrote: > > While you're probably correct about the move from SourceForge to Trac, > > there's definitely not something like that going on in the current > > proposal: you use TRAMP to build your static assets and you will keep > doing > > that, even after deployment moves to GitLab. GitLab Pages is - from a > user > > perspective - nothing more than a pipeline which specifies where the > > generated output is. > > TRAMP is not a static site generator, it is an Emacs virtual > file system for editing files and running commands remotely, that > works over SSH. I do not use a site generator for c-l.net projects. > Ah. Ok. My bad. I assumed TRAMP was a site generator. > > That pipeline/CI job builds your static content using > > your own preferred static site generator (in case of c-l.net, that's > some > > amalgamation of cl-markdown, plump, cl-who, etc. and in your case that's > > TRAMP). Building in this context can be as simple as "take the pre-built > > pages stored in a Git repository and declare those to be the build > output". > > The additional bit that Pages does is to take the output from a CI build > > and deploy it on the Pages host. This last step simply means the pages > > become available as https://common-lisp.net/project/<some-project>/... > > My concern is that this requires you to use git, Yes. Another assumption: everybody keeps their (site/software) assets in source control/version control these days. Isn't your common-lisp.net site versioned? > and then requires you to use Gitlab Sure. That's where you send your version controlled assets anyway, right? On my part (as a maintainer of the site) this isn't much more effort since everything required is already in place (from my perspective). > (and it sounds like you also need to configure > Gitlab Pages to know about files in your git repository), Well, you'll be given that by me upon the initial creation/import of your site into GitLab. All you need to do (when you want to go without a static site builder) is to edit your pages and push them to your repo. all so that Gitlab can copy some files to > https://common-lisp.net/project/<some-project>/. That is a lot of > dependencies if you only want to copy some files. I understand what you're saying and I appreciate the feedback. I also understand you would be annoyed if the only way to deploy would be to go through GitLab. Would you go so far as to say that you would move your project pages elsewhere, though? (Not threatening or anything, really just trying to understand your situation.) As you have encountered already, this approach has problems. Yea. I would assume every option has problems. The existing way of working has problems as well in the sense that they put a maintenance effort on the common-lisp.net admin team and I can fix issues from my mobile phone when I need to deal with GitLab issues whereas I really need a computer with terminal (I know there are terminals for phones, but they don't work for me) to fix issues with any other part of the setup. Also, people expect certain applications to be installed when logins are provided. GitLab defines the service level pretty well and all that in a single installation package. > It is a nice option, but I do not think it should be the only option. > That's clear. Thanks a bunch for the feedback! I'll check what I can do to allow multiple deployment methods to co-exist. I'm eager to hear from others regarding this topic as well, though! Regards, Erik.

3 3