First, thanks for looking into this.
On Thu, Dec 13, 2018 at 1:29 PM Erik Huelsmann ehuelsmann@common-lisp.net wrote:
Hi,
On Thu, Dec 13, 2018 at 1:54 AM Raymond Toy toy.raymond@gmail.com wrote:
With your help, cmucl's web site can now be generated using gitlab pages. This works great, once I move public_html out of the way.
However, the public_html directory had directories holding other artifacts like the pdf and html versions of the user manual and other documentation and, more importantly, the binaries of the various snapshots and releases.
How would this be handled? I could check these items into the repo, but huge binary blobs in the repo that would just be copied out seems like not a good idea.
Correct. As in: adding the artifacts to a Git repository isn't really a great idea. I have 2 solutions: One using GitLab (preferred solution) and the other one based on the existing hosting setup on the filesystem.
The preferred solution is to upload the files into GitLab using the following command:
$ curl --request POST --header "PRIVATE-TOKEN: <YOUR_PRIVATE_TOKEN>" --form "file=@/project/cmucl/public_html/downloads/release/21d/cmucl-src-21d.tar.bz2" https://gitlab.common-lisp.net/api/v4/projects/201/uploads
You can find the "201" in the URL above on the project homepage (https://gitlab.common-lisp.net/cmucl/cmucl-site) which lists the "Project ID". The private token can be created on the account profile page at https://gitlab.common-lisp.net/profile/personal_access_tokens (if you need to create on: you need to give it "API" scope; the others aren't required).
This adds a bit of burden to anyone who is creating binaries for cmucl. I currently have a cron job that runs rsync to grab public_html to my local machine for my own backup purposes. How would that work in this scenario?
The above is based on the info available from https://docs.gitlab.com/ee/api/projects.html#upload-a-file
The other option is that we map /project/*/downloads to https://common-lisp.net/project/*/downloads where the default path /project/*/downloads is a symlink to /project/*/public_html/downloads in order to make sure that all content in existing "downloads" directories remains correctly served. This option isn't my preference because it still requires people to have SSH access, whereas the first option means we can simply re-use existing GitLab authentication mechanisms (from where I stand as an admin, that is; as a user, you need to create the private token you didn't need before).
As someone who already has ssh access, this is easiest because it preserves what already on disk. And the cmucl mirrors, this probably preserves most of existing behavior.
But wouldn't we need to move public_html out of the way so gitlab pages can serve the pages? But if that's done, then you can't just point your browser to http://common-lisp.net/project/cmucl/downloads/ to see all of the files, right?
I do agree with you that using uploaded files to gitlab makes a lot of sense. If I were starting from scratch without any existing mirrors and my rsync backup, that seem to be the way to go.
Any suggestions? Maybe we shouldn't really use gitlab pages? (That would be kind of sad because automatically generating the site would make me more motivated to keep it a bit more up-to-date.)
What's your take?
Regards,
Erik.
On Sat, Nov 3, 2018 at 8:37 AM Erik Huelsmann ehuelsmann@common-lisp.net wrote:
Hi,
As per my prior mail, I'm sending instructions as to how to start using GitLab to deploy your project pages (i.e. how to deploy what you would normally copy to /project/*/public_html using GitLab automation).
First, the delpoyment through GitLab uses GitLab Pages which means that everything you can find about GitLab Pages based deployment of webpages (e.g. in the documentation: https://docs.gitlab.com/ee/user/project/pages/#how-it-works) applies here as well -- except for the URL on which your site will be published.
Another important note is that if the directory /project/<your-project>/public_html exists, your pages will be served from that. If it doesn't, your project pages will be served from GitLab Pages, if that doesn't exist, your project pages are redirected to GitLab itself. This means that you can experiment with GitLab pages based deployment by moving your public_html directory out of the way. If your tests fail, simply move it back into place and your existing pages will be restored.
What do you need to do to use GitLab deployment?
- Create a GitLab repository called <project>-site in the group <project>
- Add the content to be deployed to the repository (either static or to be built)
- Add a '.gitlab-ci.yml' file to the repository
- Rename the /project/<project>/public_html directory
For the simplest of deployments (taking what's in the repository "as-is") see the cl-couch pages deployment repository and .gitlab-ci.yml at https://gitlab.common-lisp.net/cl-couch/cl-couch-site/.
For an intermediary level deployment processing Markdown to HTML and wrapping the result in a template, see the common-lisp.net homepage generator (which we'd like to abstract into a generic tool; we can use a hand there!) at https://gitlab.common-lisp.net/clo/cl-site/.
For the most complex we currently have - using Docker - see the Quickref build at https://gitlab.common-lisp.net/quickref/quickref/tree/continuous-integration ; note that this is currently on its "continuous-integration" branch. I hope it'll be adopted soon to be on the master branch.
I'll add similar instructions to the c-l.net FAQ.
Please feel free to follow-up with any questions you might have when you start playing around with this new feature.
Regards,
Erik.
-- Ray