Simplifying hosting services (aka "breaking up the monolith")
Hi, In 2021, common-lisp.net has had two types of service interruptions: (1) mailing lists were not functioning correctly [while regular mail traffic seemed to be] an (2) deployment of project sites wasn't working. The latter only came to light recently, but seems to have existed for some time. Since I can remember (probably since the creation of the service) has common-lisp.net hosted its project webpages on its main domain in a sub directory, which at times has been called /project/, or /projects/ or even /p/. This setup mixes project hosting with hosting for the main site itself and restricts the tooling we can use to host the projects' websites. The reason for the sites not deploying well is that I've implemented a workaround in the past to be able to deploy projects using GitLab Pages while our deployment model (deploying to the /project/ subdirectory on the main domain) is out of line with what GitLab Pages were designed for. At the same time, our configuration is running with extensive sets of rewrite rules to keep historic URLs "working" and redirected to (hopefully) existing current URLs, which also extremely complicates our setup. In order to simplify our setup (and eliminate the deployment problems we're experiencing) and at the same time add support for requests like those from Marco who wants to be able to deploy sites for multiple projects under the same umbrella, I've decided I want to move projects' sites to their own (sub)domains. In the past I thought this would need to be subdomains along the lines of https://<project>.project.common-lisp.net/. Although that's a deployment model that would work with GitLab Pages, I've decided against it. I'll move the hosted projects to: https://<project>.common-lisp.dev/ (I acquired common-lisp.dev for this purpose this morning.) The setup I'm proposing is that we have a good look at the tons of rewrite rules we have currently in place and clean up the rewrite rules that we don't need anymore. Then, we create rewrite rules for the current project namespaces at https://common-lisp.net/project/<project>/ to map to https://<project>.common-lisp.dev/. There are a few projects which aren't using GitLab Pages to deploy their websites yet, mostly because they have no active maintainers. These projects will keep being served from their current locations on the filesystem of the common-lisp.net host, but their content will be available through the new <project>.common-lisp.dev URL space. Further change proposals in order to separate the mail flow for the mailing lists from the regular mail flow (and thereby further reducing integration between components) are upcoming, but I'll need to address one thing at a time (due to time constraints). Are there any comments, remarks, additions, things you want me to take into account with respect to the above? -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.
[...]
In order to simplify our setup (and eliminate the deployment problems we're experiencing) and at the same time add support for requests like those from Marco who wants to be able to deploy sites for multiple projects under the same umbrella, I've decided I want to move projects' sites to their own (sub)domains. In the past I thought this would need to be subdomains along the lines of https://<project>.project.common-lisp.net/. Although that's a deployment model that would work with GitLab Pages, I've decided against it. I'll move the hosted projects to:
https://<project>.common-lisp.dev/
(I acquired common-lisp.dev for this purpose this morning.)
Very good idea, especially for security considerations: it's better that user-generated content be hosted under a different TLD.
The setup I'm proposing is that we have a good look at the tons of rewrite rules we have currently in place and clean up the rewrite rules that we don't need anymore. Then, we create rewrite rules for the current project namespaces at https://common-lisp.net/project/<project>/ to map to https://<project>.common-lisp.dev/. There are a few projects which aren't using GitLab Pages to deploy their websites yet, mostly because they have no active maintainers. These projects will keep being served from their current locations on the filesystem of the common-lisp.net host, but their content will be available through the new <project>.common-lisp.dev URL space.
+1 -- Stelian Ionescu
Very on board with this. Gitlab pages continues to get improvements. So the more we can switch to a vanilla pages setup, the more features we get for free and the less documentation we have to write and maintain. Since Gitlab pages works for users as well, I'm personally excited to have etimmons.common-lisp.dev set up at some point :). -Eric
I don't think I have much voting power on the matter but I like the idea and I'm excited to see it happen - thanks for the initiative and the realization! ~phoe
Hi I have no objections about the new setup (although, the switch may confuse someone). But, call me an old curmudgeon, I will want to be able to deploy my static web pages and NOT use some non-lisp page generation tool (if I understood correctly what it would entail) (*). Hence I would respectfully ask to be able to produce the-old-curmudgeon.common-lisp.X with possibly old age and quaint, "handcrafted" HTML. All the best MA (*) Plus, a deployment via an invocation of http://helambda.sf.net should be a must (yes; I will eventually migrate it to common-lisp.net). On Tue, Jan 11, 2022 at 9:56 PM Eric Timmons <etimmons@mit.edu> wrote:
Very on board with this. Gitlab pages continues to get improvements. So the more we can switch to a vanilla pages setup, the more features we get for free and the less documentation we have to write and maintain.
Since Gitlab pages works for users as well, I'm personally excited to have etimmons.common-lisp.dev set up at some point :).
-Eric
-- Marco Antoniotti, Professor tel. +39 - 02 64 48 79 01 DISCo, Università Milano Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
Hi Marco,
I have no objections about the new setup (although, the switch may confuse someone).
But, call me an old curmudgeon, I will want to be able to deploy my static web pages and NOT use some non-lisp page generation tool (if I understood correctly what it would entail) (*).
Fortunately, you're misunderstanding how this works: you can still use a lisp-based page generation tool to create the static content. When you get to the point where you need help to get that going, please drop the clo-devel@ mailing list a line and I'm sure we can help you there.
Hence I would respectfully ask to be able to produce the-old-curmudgeon.common-lisp.X with possibly old age and quaint, "handcrafted" HTML.
That too is totally fine. It will remain fully supported. The only thing that changes is that the content of your git repository is taken from git and automagically pushed into the hosted site. (which basically saves you an rsync step from a local directory to a remote directory and saves me from having to support multiple methods of opening the server up for upload access.) If you have further questions, don't hesitate to ask! Regards, Erik PS:
(*) Plus, a deployment via an invocation of http://helambda.sf.net should be a must (yes; I will eventually migrate it to common-lisp.net).
The domain you're referring to there isn't available on sourceforge, at least: clicking it got me a 404-Not-Found. -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.
Hi Erik thanks for the reassurances. All the best Marco PS the correct link is http://helambdap.sf.net On Tue, Jan 11, 2022 at 10:56 PM Erik Huelsmann <ehuels@gmail.com> wrote:
Hi Marco,
I have no objections about the new setup (although, the switch may confuse someone).
But, call me an old curmudgeon, I will want to be able to deploy my static web pages and NOT use some non-lisp page generation tool (if I understood correctly what it would entail) (*).
Fortunately, you're misunderstanding how this works: you can still use a lisp-based page generation tool to create the static content. When you get to the point where you need help to get that going, please drop the clo-devel@ mailing list a line and I'm sure we can help you there.
Hence I would respectfully ask to be able to produce the-old-curmudgeon.common-lisp.X with possibly old age and quaint, "handcrafted" HTML.
That too is totally fine. It will remain fully supported. The only thing that changes is that the content of your git repository is taken from git and automagically pushed into the hosted site. (which basically saves you an rsync step from a local directory to a remote directory and saves me from having to support multiple methods of opening the server up for upload access.)
If you have further questions, don't hesitate to ask!
Regards, Erik
PS:
(*) Plus, a deployment via an invocation of http://helambda.sf.net should be a must (yes; I will eventually migrate it to common-lisp.net).
The domain you're referring to there isn't available on sourceforge, at least: clicking it got me a 404-Not-Found.
-- Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.
-- Marco Antoniotti, Professor tel. +39 - 02 64 48 79 01 DISCo, Università Milano Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
I support whatever changes you want to make that make your life easier. I'll do whatever I need to do to make sure my stuff works with the new scheme. Just let me know (and give me a bit of time if it requires some non-trivial effort on my part.) On Tue, Jan 11, 2022 at 11:25 AM Erik Huelsmann <ehuels@gmail.com> wrote:
Hi,
In 2021, common-lisp.net has had two types of service interruptions: (1) mailing lists were not functioning correctly [while regular mail traffic seemed to be] an (2) deployment of project sites wasn't working. The latter only came to light recently, but seems to have existed for some time.
Since I can remember (probably since the creation of the service) has common-lisp.net hosted its project webpages on its main domain in a sub directory, which at times has been called /project/, or /projects/ or even /p/. This setup mixes project hosting with hosting for the main site itself and restricts the tooling we can use to host the projects' websites. The reason for the sites not deploying well is that I've implemented a workaround in the past to be able to deploy projects using GitLab Pages while our deployment model (deploying to the /project/ subdirectory on the main domain) is out of line with what GitLab Pages were designed for. At the same time, our configuration is running with extensive sets of rewrite rules to keep historic URLs "working" and redirected to (hopefully) existing current URLs, which also extremely complicates our setup.
In order to simplify our setup (and eliminate the deployment problems we're experiencing) and at the same time add support for requests like those from Marco who wants to be able to deploy sites for multiple projects under the same umbrella, I've decided I want to move projects' sites to their own (sub)domains. In the past I thought this would need to be subdomains along the lines of https://<project>.project.common-lisp.net/. Although that's a deployment model that would work with GitLab Pages, I've decided against it. I'll move the hosted projects to:
https://<project>.common-lisp.dev/
(I acquired common-lisp.dev for this purpose this morning.)
The setup I'm proposing is that we have a good look at the tons of rewrite rules we have currently in place and clean up the rewrite rules that we don't need anymore. Then, we create rewrite rules for the current project namespaces at https://common-lisp.net/project/<project>/ to map to https://<project>.common-lisp.dev/. There are a few projects which aren't using GitLab Pages to deploy their websites yet, mostly because they have no active maintainers. These projects will keep being served from their current locations on the filesystem of the common-lisp.net host, but their content will be available through the new <project>.common-lisp.dev URL space.
Further change proposals in order to separate the mail flow for the mailing lists from the regular mail flow (and thereby further reducing integration between components) are upcoming, but I'll need to address one thing at a time (due to time constraints).
Are there any comments, remarks, additions, things you want me to take into account with respect to the above?
-- Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.
-- Ray
Hi all,
I'll move the hosted projects to:
https://<project>.common-lisp.dev/
(I acquired common-lisp.dev for this purpose this morning.)
Yesterday, i succeeded to set up common-lisp.dev with a wildcard cert from Letsencrypt (thanks EFF!) based on DNS authentication. Today, I've succeeded to change the GitLab Pages setup to serve pages under *.common-lisp.dev instead of *.pages.example.com (which is what I set up on Monday to fix the pages deployment problems). The above means that everybody can create personal and project sites under "common-lisp.dev" as of today. Projects that are currently using a *-site repository need to have their repository renamed to "*.common-lisp.dev" to be able to benefit from this new infrastructure. An example is the cl-tar/cl-tar-site project that Eric Timmons created recently for his CL-TAR project. I've used it (with his permission) as a guinea pig. The repository has been renamed (it's now https://gitlab.common-lisp.net/cl-tar/cl-tar.common-lisp.dev); you can find its contents at https://cl-tar.common-lisp.dev/. Next steps will be: * Renaming existing *-site repositories to benefit from the new common-lisp.dev infrastructure --> I'll do this myself, because there are only a few repositories that need renaming and it reduces the period where the site is "in flux". * Serve the remaining content from the filesystem in /project/*/public_html/ from *.common-lisp.dev/ * Add pointers to the GitLab Pages documentation on how to set up project pages, group pages and personal pages --> I can use a hand here: anybody who can review our current site and add articles on how to use GitLab Pages for each of the three mentioned types of sites, *and* explain how to do so in a lispy way, would be urgently requested to help out. * Move websites of projects that currently aren't using gitlab pages, into gitlab pages. * Do generic cleanup on the host's filesystem In summary: over the coming days, your repository may be renamed. I'll be the cause of that. This only applies to repositories which are named <group_name>/<group_name>-site. In the mean time, you can play around and create your own -- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.
participants (7)
-
Eric Timmons -
Erik Huelsmann -
Marco Antoniotti -
Michał "phoe" Herda -
Philipp Marek -
Raymond Toy -
Stelian Ionescu