How to deploy your project pages through GitLab
Hi, As per my prior mail, I'm sending instructions as to how to start using GitLab to deploy your project pages (i.e. how to deploy what you would normally copy to /project/*/public_html using GitLab automation). First, the delpoyment through GitLab uses GitLab Pages which means that everything you can find about GitLab Pages based deployment of webpages (e.g. in the documentation: https://docs.gitlab.com/ee/user/project/pages/#how-it-works) applies here as well -- except for the URL on which your site will be published. Another important note is that if the directory /project/<your-project>/public_html exists, your pages will be served from that. If it doesn't, your project pages will be served from GitLab Pages, if that doesn't exist, your project pages are redirected to GitLab itself. This means that you can experiment with GitLab pages based deployment by moving your public_html directory out of the way. If your tests fail, simply move it back into place and your existing pages will be restored. *What do you need to do to use GitLab deployment?* 1. Create a GitLab repository called <project>-site in the group <project> 2. Add the content to be deployed to the repository (either static or to be built) 3. Add a '.gitlab-ci.yml' file to the repository 4. Rename the /project/<project>/public_html directory For the simplest of deployments (taking what's in the repository "as-is") see the cl-couch pages deployment repository and .gitlab-ci.yml at https://gitlab.common-lisp.net/cl-couch/cl-couch-site/. For an intermediary level deployment processing Markdown to HTML and wrapping the result in a template, see the common-lisp.net homepage generator (which we'd like to abstract into a generic tool; we can use a hand there!) at https://gitlab.common-lisp.net/clo/cl-site/. For the most complex we currently have - using Docker - see the Quickref build at https://gitlab.common-lisp.net/quickref/quickref/tree/continuous-integration ; note that this is currently on its "continuous-integration" branch. I hope it'll be adopted soon to be on the master branch. I'll add similar instructions to the c-l.net FAQ. Please feel free to follow-up with any questions you might have when you start playing around with this new feature. Regards, Erik.
With your help, cmucl's web site can now be generated using gitlab pages. This works great, once I move public_html out of the way. However, the public_html directory had directories holding other artifacts like the pdf and html versions of the user manual and other documentation and, more importantly, the binaries of the various snapshots and releases. How would this be handled? I could check these items into the repo, but huge binary blobs in the repo that would just be copied out seems like not a good idea. Any suggestions? Maybe we shouldn't really use gitlab pages? (That would be kind of sad because automatically generating the site would make me more motivated to keep it a bit more up-to-date.) On Sat, Nov 3, 2018 at 8:37 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi,
As per my prior mail, I'm sending instructions as to how to start using GitLab to deploy your project pages (i.e. how to deploy what you would normally copy to /project/*/public_html using GitLab automation).
First, the delpoyment through GitLab uses GitLab Pages which means that everything you can find about GitLab Pages based deployment of webpages (e.g. in the documentation: https://docs.gitlab.com/ee/user/project/pages/#how-it-works) applies here as well -- except for the URL on which your site will be published.
Another important note is that if the directory /project/<your-project>/public_html exists, your pages will be served from that. If it doesn't, your project pages will be served from GitLab Pages, if that doesn't exist, your project pages are redirected to GitLab itself. This means that you can experiment with GitLab pages based deployment by moving your public_html directory out of the way. If your tests fail, simply move it back into place and your existing pages will be restored.
What do you need to do to use GitLab deployment?
1. Create a GitLab repository called <project>-site in the group <project> 2. Add the content to be deployed to the repository (either static or to be built) 3. Add a '.gitlab-ci.yml' file to the repository 4. Rename the /project/<project>/public_html directory
For the simplest of deployments (taking what's in the repository "as-is") see the cl-couch pages deployment repository and .gitlab-ci.yml at https://gitlab.common-lisp.net/cl-couch/cl-couch-site/.
For an intermediary level deployment processing Markdown to HTML and wrapping the result in a template, see the common-lisp.net homepage generator (which we'd like to abstract into a generic tool; we can use a hand there!) at https://gitlab.common-lisp.net/clo/cl-site/.
For the most complex we currently have - using Docker - see the Quickref build at https://gitlab.common-lisp.net/quickref/quickref/tree/continuous-integration ; note that this is currently on its "continuous-integration" branch. I hope it'll be adopted soon to be on the master branch.
I'll add similar instructions to the c-l.net FAQ.
Please feel free to follow-up with any questions you might have when you start playing around with this new feature.
Regards,
Erik.
-- Ray
Hi, On Thu, Dec 13, 2018 at 1:54 AM Raymond Toy <toy.raymond@gmail.com> wrote:
With your help, cmucl's web site can now be generated using gitlab pages. This works great, once I move public_html out of the way.
However, the public_html directory had directories holding other artifacts like the pdf and html versions of the user manual and other documentation and, more importantly, the binaries of the various snapshots and releases.
How would this be handled? I could check these items into the repo, but huge binary blobs in the repo that would just be copied out seems like not a good idea.
Correct. As in: adding the artifacts to a Git repository isn't really a great idea. I have 2 solutions: One using GitLab (preferred solution) and the other one based on the existing hosting setup on the filesystem. The preferred solution is to upload the files into GitLab using the following command: $ curl --request POST --header "PRIVATE-TOKEN: <YOUR_PRIVATE_TOKEN>" --form "file=@/project/cmucl/public_html/downloads/release/21d/cmucl-src-21d.tar.bz2" https://gitlab.common-lisp.net/api/v4/projects/*201*/uploads You can find the "201" in the URL above on the project homepage ( https://gitlab.common-lisp.net/cmucl/cmucl-site) which lists the "Project ID". The private token can be created on the account profile page at https://gitlab.common-lisp.net/profile/personal_access_tokens (if you need to create on: you need to give it "API" scope; the others aren't required). The above request will return something like: { "alt": "dk", "url": "/uploads/66dbcd21ec5d24ed6ea225176098d52b/cmucl-src-21d.tar.bz2", "markdown": "" } The URL is relative to the project's URL, so the one returned above would expand to https://gitlab.common-lisp.net/cmucl/cmucl-site/uploads/66dbcd21ec5d24ed6ea2... The above is based on the info available from https://docs.gitlab.com/ee/api/projects.html#upload-a-file The other option is that we map /project/*/downloads to https://common-lisp.net/project/*/downloads where the default path /project/*/downloads is a symlink to /project/*/public_html/downloads in order to make sure that all content in existing "downloads" directories remains correctly served. This option isn't my preference because it still requires people to have SSH access, whereas the first option means we can simply re-use existing GitLab authentication mechanisms (from where I stand as an admin, that is; as a user, you need to create the private token you didn't need before). Any suggestions? Maybe we shouldn't really use
gitlab pages? (That would be kind of sad because automatically generating the site would make me more motivated to keep it a bit more up-to-date.)
What's your take? Regards, Erik.
On Sat, Nov 3, 2018 at 8:37 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi,
As per my prior mail, I'm sending instructions as to how to start using
GitLab to deploy your project pages (i.e. how to deploy what you would normally copy to /project/*/public_html using GitLab automation).
First, the delpoyment through GitLab uses GitLab Pages which means that
everything you can find about GitLab Pages based deployment of webpages (e.g. in the documentation: https://docs.gitlab.com/ee/user/project/pages/#how-it-works) applies here as well -- except for the URL on which your site will be published.
Another important note is that if the directory
/project/<your-project>/public_html exists, your pages will be served from that. If it doesn't, your project pages will be served from GitLab Pages, if that doesn't exist, your project pages are redirected to GitLab itself. This means that you can experiment with GitLab pages based deployment by moving your public_html directory out of the way. If your tests fail, simply move it back into place and your existing pages will be restored.
What do you need to do to use GitLab deployment?
1. Create a GitLab repository called <project>-site in the group
<project>
2. Add the content to be deployed to the repository (either static or to be built) 3. Add a '.gitlab-ci.yml' file to the repository 4. Rename the /project/<project>/public_html directory
For the simplest of deployments (taking what's in the repository "as-is") see the cl-couch pages deployment repository and .gitlab-ci.yml at https://gitlab.common-lisp.net/cl-couch/cl-couch-site/.
For an intermediary level deployment processing Markdown to HTML and wrapping the result in a template, see the common-lisp.net homepage generator (which we'd like to abstract into a generic tool; we can use a hand there!) at https://gitlab.common-lisp.net/clo/cl-site/.
For the most complex we currently have - using Docker - see the Quickref build at https://gitlab.common-lisp.net/quickref/quickref/tree/continuous-integration ; note that this is currently on its "continuous-integration" branch. I hope it'll be adopted soon to be on the master branch.
I'll add similar instructions to the c-l.net FAQ.
Please feel free to follow-up with any questions you might have when you start playing around with this new feature.
Regards,
Erik.
-- Ray
First, thanks for looking into this. On Thu, Dec 13, 2018 at 1:29 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi,
On Thu, Dec 13, 2018 at 1:54 AM Raymond Toy <toy.raymond@gmail.com> wrote:
With your help, cmucl's web site can now be generated using gitlab pages. This works great, once I move public_html out of the way.
However, the public_html directory had directories holding other artifacts like the pdf and html versions of the user manual and other documentation and, more importantly, the binaries of the various snapshots and releases.
How would this be handled? I could check these items into the repo, but huge binary blobs in the repo that would just be copied out seems like not a good idea.
Correct. As in: adding the artifacts to a Git repository isn't really a great idea. I have 2 solutions: One using GitLab (preferred solution) and the other one based on the existing hosting setup on the filesystem.
The preferred solution is to upload the files into GitLab using the following command:
$ curl --request POST --header "PRIVATE-TOKEN: <YOUR_PRIVATE_TOKEN>" --form "file=@/project/cmucl/public_html/downloads/release/21d/cmucl-src-21d.tar.bz2" https://gitlab.common-lisp.net/api/v4/projects/201/uploads
You can find the "201" in the URL above on the project homepage (https://gitlab.common-lisp.net/cmucl/cmucl-site) which lists the "Project ID". The private token can be created on the account profile page at https://gitlab.common-lisp.net/profile/personal_access_tokens (if you need to create on: you need to give it "API" scope; the others aren't required).
This adds a bit of burden to anyone who is creating binaries for cmucl. I currently have a cron job that runs rsync to grab public_html to my local machine for my own backup purposes. How would that work in this scenario?
The above is based on the info available from https://docs.gitlab.com/ee/api/projects.html#upload-a-file
The other option is that we map /project/*/downloads to https://common-lisp.net/project/*/downloads where the default path /project/*/downloads is a symlink to /project/*/public_html/downloads in order to make sure that all content in existing "downloads" directories remains correctly served. This option isn't my preference because it still requires people to have SSH access, whereas the first option means we can simply re-use existing GitLab authentication mechanisms (from where I stand as an admin, that is; as a user, you need to create the private token you didn't need before).
As someone who already has ssh access, this is easiest because it preserves what already on disk. And the cmucl mirrors, this probably preserves most of existing behavior. But wouldn't we need to move public_html out of the way so gitlab pages can serve the pages? But if that's done, then you can't just point your browser to http://common-lisp.net/project/cmucl/downloads/ to see all of the files, right? I do agree with you that using uploaded files to gitlab makes a lot of sense. If I were starting from scratch without any existing mirrors and my rsync backup, that seem to be the way to go.
Any suggestions? Maybe we shouldn't really use gitlab pages? (That would be kind of sad because automatically generating the site would make me more motivated to keep it a bit more up-to-date.)
What's your take?
Regards,
Erik.
On Sat, Nov 3, 2018 at 8:37 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi,
As per my prior mail, I'm sending instructions as to how to start using GitLab to deploy your project pages (i.e. how to deploy what you would normally copy to /project/*/public_html using GitLab automation).
First, the delpoyment through GitLab uses GitLab Pages which means that everything you can find about GitLab Pages based deployment of webpages (e.g. in the documentation: https://docs.gitlab.com/ee/user/project/pages/#how-it-works) applies here as well -- except for the URL on which your site will be published.
Another important note is that if the directory /project/<your-project>/public_html exists, your pages will be served from that. If it doesn't, your project pages will be served from GitLab Pages, if that doesn't exist, your project pages are redirected to GitLab itself. This means that you can experiment with GitLab pages based deployment by moving your public_html directory out of the way. If your tests fail, simply move it back into place and your existing pages will be restored.
What do you need to do to use GitLab deployment?
1. Create a GitLab repository called <project>-site in the group <project> 2. Add the content to be deployed to the repository (either static or to be built) 3. Add a '.gitlab-ci.yml' file to the repository 4. Rename the /project/<project>/public_html directory
For the simplest of deployments (taking what's in the repository "as-is") see the cl-couch pages deployment repository and .gitlab-ci.yml at https://gitlab.common-lisp.net/cl-couch/cl-couch-site/.
For an intermediary level deployment processing Markdown to HTML and wrapping the result in a template, see the common-lisp.net homepage generator (which we'd like to abstract into a generic tool; we can use a hand there!) at https://gitlab.common-lisp.net/clo/cl-site/.
For the most complex we currently have - using Docker - see the Quickref build at https://gitlab.common-lisp.net/quickref/quickref/tree/continuous-integration ; note that this is currently on its "continuous-integration" branch. I hope it'll be adopted soon to be on the master branch.
I'll add similar instructions to the c-l.net FAQ.
Please feel free to follow-up with any questions you might have when you start playing around with this new feature.
Regards,
Erik.
-- Ray
-- Ray
Hi, On Thu, Dec 13, 2018 at 10:51 PM Raymond Toy <toy.raymond@gmail.com> wrote:
First, thanks for looking into this.
No problem. Thanks for the discussion. It sharpens the final solution.
the public_html directory had directories holding other artifacts like the pdf and html versions of the user manual and other documentation and, more importantly, the binaries of the various snapshots and releases.
How would this be handled? I could check these items into the repo, but huge binary blobs in the repo that would just be copied out seems like not a good idea.
Correct. As in: adding the artifacts to a Git repository isn't really a great idea. I have 2 solutions: One using GitLab (preferred solution) and the other one based on the existing hosting setup on the filesystem.
The preferred solution is to upload the files into GitLab using the following command:
$ curl --request POST --header "PRIVATE-TOKEN: <YOUR_PRIVATE_TOKEN>" --form "file=@/project/cmucl/public_html/downloads/release/21d/cmucl-src-21d.tar.bz2" https://gitlab.common-lisp.net/api/v4/projects/201/uploads
You can find the "201" in the URL above on the project homepage ( https://gitlab.common-lisp.net/cmucl/cmucl-site) which lists the "Project ID". The private token can be created on the account profile page at https://gitlab.common-lisp.net/profile/personal_access_tokens (if you need to create on: you need to give it "API" scope; the others aren't required).
This adds a bit of burden to anyone who is creating binaries for cmucl. I currently have a cron job that runs rsync to grab public_html to my local machine for my own backup purposes. How would that work in this scenario?
It probably doesn't work too well: it's neither in the /project directories nor in the same directory from which GitLab pages hosts your website. Regardless, I can provide you with the paths to sync, but the mirror will still point to the GitLab instance for the hosted binaries.
The above is based on the info available from
https://docs.gitlab.com/ee/api/projects.html#upload-a-file
The other option is that we map /project/*/downloads to
https://common-lisp.net/project/*/downloads where the default path /project/*/downloads is a symlink to /project/*/public_html/downloads in order to make sure that all content in existing "downloads" directories remains correctly served. This option isn't my preference because it still requires people to have SSH access, whereas the first option means we can simply re-use existing GitLab authentication mechanisms (from where I stand as an admin, that is; as a user, you need to create the private token you didn't need before).
As someone who already has ssh access, this is easiest because it preserves what already on disk. And the cmucl mirrors, this probably preserves most of existing behavior.
But wouldn't we need to move public_html out of the way so gitlab pages can serve the pages?
Yes. There's no avoiding moving public_html out of the way.
But if that's done, then you can't just point your browser to http://common-lisp.net/project/cmucl/downloads/ to see all of the files, right?
Well, I was planning to adjust the Apache configuration to pick up the existence of /project/cmucl/downloads/ to serve files on the URL https://common-lisp.net/project/cmucl/downloads/ ; that way we can move public_html, but keep what used to be served from public_html/downloads/.
I do agree with you that using uploaded files to gitlab makes a lot of sense. If I were starting from scratch without any existing mirrors and my rsync backup, that seem to be the way to go.
True. I think I'll document my preferred solution in the FAQ while you can use the solution which depends on the slightly adjusted Apache config I need to put in place. Would that slightly adjusted Apache config work for you for the uploads while still depending on GitLab to build your site? Regards, Erik.
On Thu, Dec 13, 2018 at 2:22 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I do agree with you that using uploaded files to gitlab makes a lot of sense. If I were starting from scratch without any existing mirrors and my rsync backup, that seem to be the way to go.
True. I think I'll document my preferred solution in the FAQ while you can use the solution which depends on the slightly adjusted Apache config I need to put in place. Would that slightly adjusted Apache config work for you for the uploads while still depending on GitLab to build your site?
I agree with your idea for the FAQ. And I appreciate having an adjusted Apache config to serve up the files. Just let me know what the name of the directory will be and I'll test it out soon. I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is. Thanks for your help! -- Ray
Hi Ray, On Fri, Dec 14, 2018 at 4:52 AM Raymond Toy <toy.raymond@gmail.com> wrote:
On Thu, Dec 13, 2018 at 2:22 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I do agree with you that using uploaded files to gitlab makes a lot of sense. If I were starting from scratch without any existing mirrors and my rsync backup, that seem to be the way to go.
True. I think I'll document my preferred solution in the FAQ while you can use the solution which depends on the slightly adjusted Apache config I need to put in place. Would that slightly adjusted Apache config work for you for the uploads while still depending on GitLab to build your site?
I agree with your idea for the FAQ. And I appreciate having an adjusted Apache config to serve up the files. Just let me know what the name of the directory will be and I'll test it out soon.
I've adjusted the Apache config (and moved the /project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with the downloads: (a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public (b) /srv/project/cmucl/downloads For those reading this discussion, I've implemented another tool to help you establish a nice URL space while using the upload facility from GitLab's API as referred to in one of the earlier posts: You can create a file called ".download-map" in your project's public_html directory which maps URLs to GitLab's upload facility through temporary URL redirects. I'll need to write a FAQ about it, but in case anybody wants to play with it, I'm here to help! Thanks for your help!
-- Ray
Regards, Erik.
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I've adjusted the Apache config (and moved the /project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good. And even better, I don't have to change any of the existing links from either the site or the cmucl wiki. Awesome! Thank you so much!
I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with the downloads:
(a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public (b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.) And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads path? Again, I'm really grateful for you help with this. -- Ray
Friendly ping on what the best way would be to get the html files. It's preferable to be able to use rsync so that we don't needlessly download files that haven't changed. On Sat, Dec 15, 2018 at 11:49 AM Raymond Toy <toy.raymond@gmail.com> wrote:
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I've adjusted the Apache config (and moved the /project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good.
And even better, I don't have to change any of the existing links from either the site or the cmucl wiki. Awesome!
Thank you so much!
I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with the downloads:
(a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public (b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.)
And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads path?
Again, I'm really grateful for you help with this.
-- Ray
-- Ray
Hi Ray, I'm thinking we don't store any surprises in /var/opt/gitlab/gitlab-rails/shared/pages, so I could "open up" permissions of that directory. Then you could use rsync to sync /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/. I've opened up this path: /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public So, you should be able to access it using rsync now. Let me know if you have further questions or issues to get the sync up and running. Regards, Erik. On Tue, Dec 18, 2018 at 10:44 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Friendly ping on what the best way would be to get the html files. It's preferable to be able to use rsync so that we don't needlessly download files that haven't changed.
On Sat, Dec 15, 2018 at 11:49 AM Raymond Toy <toy.raymond@gmail.com> wrote:
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I've adjusted the Apache config (and moved the
/project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good.
And even better, I don't have to change any of the existing links from either the site or the cmucl wiki. Awesome!
Thank you so much!
I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with the
downloads:
(a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public (b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.)
And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads path?
Again, I'm really grateful for you help with this.
-- Ray
-- Ray
Thanks. This should work out just fine, but I'll ask Carl since he mirrors this to cmucl.org. I'll let you know how it goes but I don't expect any issues. On Tue, Dec 18, 2018 at 2:00 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi Ray,
I'm thinking we don't store any surprises in /var/opt/gitlab/gitlab-rails/shared/pages, so I could "open up" permissions of that directory. Then you could use rsync to sync /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/.
I've opened up this path: /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public
So, you should be able to access it using rsync now. Let me know if you have further questions or issues to get the sync up and running.
Regards,
Erik.
On Tue, Dec 18, 2018 at 10:44 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Friendly ping on what the best way would be to get the html files. It's preferable to be able to use rsync so that we don't needlessly download files that haven't changed.
On Sat, Dec 15, 2018 at 11:49 AM Raymond Toy <toy.raymond@gmail.com> wrote:
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I've adjusted the Apache config (and moved the /project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good.
And even better, I don't have to change any of the existing links from either the site or the cmucl wiki. Awesome!
Thank you so much!
I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with the downloads:
(a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public (b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.)
And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads path?
Again, I'm really grateful for you help with this.
-- Ray
-- Ray
-- Ray
Does there need to be some configuration for rsync? I can rsync from /project/cmucl/downloads, but I cannot do rsync -av rsync://common-lisp.net/var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/ html @ERROR: Unknown module 'var' On Tue, Dec 18, 2018 at 2:08 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Thanks. This should work out just fine, but I'll ask Carl since he mirrors this to cmucl.org. I'll let you know how it goes but I don't expect any issues.
On Tue, Dec 18, 2018 at 2:00 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi Ray,
I'm thinking we don't store any surprises in /var/opt/gitlab/gitlab-rails/shared/pages, so I could "open up" permissions of that directory. Then you could use rsync to sync /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/.
I've opened up this path: /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public
So, you should be able to access it using rsync now. Let me know if you have further questions or issues to get the sync up and running.
Regards,
Erik.
On Tue, Dec 18, 2018 at 10:44 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Friendly ping on what the best way would be to get the html files. It's preferable to be able to use rsync so that we don't needlessly download files that haven't changed.
On Sat, Dec 15, 2018 at 11:49 AM Raymond Toy <toy.raymond@gmail.com> wrote:
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I've adjusted the Apache config (and moved the /project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good.
And even better, I don't have to change any of the existing links from either the site or the cmucl wiki. Awesome!
Thank you so much!
I will have to update the site and all the wiki pages to point to the right place but this should be a very simple search and replace. And let people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with the downloads:
(a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public (b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.)
And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads path?
Again, I'm really grateful for you help with this.
-- Ray
-- Ray
-- Ray
-- Ray
Hi Ray, On Wed, Dec 19, 2018 at 7:20 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Does there need to be some configuration for rsync? I can rsync from /project/cmucl/downloads, but I cannot do
rsync -av rsync:// common-lisp.net/var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/ html @ERROR: Unknown module 'var'
Hmm. It seems I needed to adjust the rsync config. I'm now able to run this command successfully: $ rsync -av rsync://common-lisp.net/pages/cmucl/cmucl-site/public/ html Does it work for you too? Regards, Erik.
On Tue, Dec 18, 2018 at 2:08 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Thanks. This should work out just fine, but I'll ask Carl since he mirrors this to cmucl.org. I'll let you know how it goes but I don't expect any issues.
On Tue, Dec 18, 2018 at 2:00 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi Ray,
I'm thinking we don't store any surprises in
I've opened up this path:
/var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public
So, you should be able to access it using rsync now. Let me know if
you have further questions or issues to get the sync up and running.
Regards,
Erik.
On Tue, Dec 18, 2018 at 10:44 PM Raymond Toy <toy.raymond@gmail.com>
wrote:
Friendly ping on what the best way would be to get the html files. It's preferable to be able to use rsync so that we don't needlessly download files that haven't changed.
On Sat, Dec 15, 2018 at 11:49 AM Raymond Toy <toy.raymond@gmail.com>
wrote:
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
I've adjusted the Apache config (and moved the
/project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good.
And even better, I don't have to change any of the existing links
from
either the site or the cmucl wiki. Awesome!
Thank you so much!
> > I will > have to update the site and all the wiki pages to point to the
right
> place but this should be a very simple search and replace. And let > people who mirror the site know what the new path is.
There are 2 new paths to be mirrored, (a) with the site; (b) with
(a)
/var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public
(b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.)
And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads
/var/opt/gitlab/gitlab-rails/shared/pages, so I could "open up" permissions of that directory. Then you could use rsync to sync /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/. the downloads: path?
Again, I'm really grateful for you help with this.
-- Ray
-- Ray
-- Ray
-- Ray
That's great! Everything works. Thanks! On Wed, Dec 19, 2018 at 11:56 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi Ray,
On Wed, Dec 19, 2018 at 7:20 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Does there need to be some configuration for rsync? I can rsync from /project/cmucl/downloads, but I cannot do
rsync -av rsync://common-lisp.net/var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/ html @ERROR: Unknown module 'var'
Hmm. It seems I needed to adjust the rsync config. I'm now able to run this command successfully:
$ rsync -av rsync://common-lisp.net/pages/cmucl/cmucl-site/public/ html
Does it work for you too?
Regards,
Erik.
On Tue, Dec 18, 2018 at 2:08 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Thanks. This should work out just fine, but I'll ask Carl since he mirrors this to cmucl.org. I'll let you know how it goes but I don't expect any issues.
On Tue, Dec 18, 2018 at 2:00 PM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote:
Hi Ray,
I'm thinking we don't store any surprises in /var/opt/gitlab/gitlab-rails/shared/pages, so I could "open up" permissions of that directory. Then you could use rsync to sync /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public/.
I've opened up this path: /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public
So, you should be able to access it using rsync now. Let me know if you have further questions or issues to get the sync up and running.
Regards,
Erik.
On Tue, Dec 18, 2018 at 10:44 PM Raymond Toy <toy.raymond@gmail.com> wrote:
Friendly ping on what the best way would be to get the html files. It's preferable to be able to use rsync so that we don't needlessly download files that haven't changed.
On Sat, Dec 15, 2018 at 11:49 AM Raymond Toy <toy.raymond@gmail.com> wrote:
On Sat, Dec 15, 2018 at 10:49 AM Erik Huelsmann <ehuelsmann@common-lisp.net> wrote: > > I've adjusted the Apache config (and moved the /project/cmucl/public_html/downloads directory to /project/cmucl/downloads to test). You can now use the GitLab site deployment through GitLab Pages and expect the directory /project/*/downloads to be mapped into the site at https://common-lisp.net/project/*/downloads.
I moved public_html out of the way, and everything looks good.
And even better, I don't have to change any of the existing links from either the site or the cmucl wiki. Awesome!
Thank you so much! > >> >> I will >> have to update the site and all the wiki pages to point to the right >> place but this should be a very simple search and replace. And let >> people who mirror the site know what the new path is. > > > There are 2 new paths to be mirrored, (a) with the site; (b) with the downloads: > > (a) /var/opt/gitlab/gitlab-rails/shared/pages/cmucl/cmucl-site/public > (b) /srv/project/cmucl/downloads
I'm a bit confused on what these two paths mean. Is (a) where the actual html pages live? Previously, the html pages were stored in public_html/mirror. (Also, I don't seem to have access to that directory.)
And (b) is the path I can use to rsync all the downloads? I'm guessing the site generator doesn't have access to the downloads path?
Again, I'm really grateful for you help with this.
-- Ray
-- Ray
-- Ray
-- Ray
-- Ray
participants (2)
-
Erik Huelsmann -
Raymond Toy