I suspect it'll be easier and more robust to proxy to the pages daemon with appropriate rewrite rules than to use the Gitlab API.

If you do go via the API you'd need to ensure that every *-site repo uses the same branch name (I think) and ensure that the artifacts for the latest run are saved forever. For instance, it looks like the latest pages job at https://gitlab.common-lisp.net/cl-docker-images/cl-docker-images-site/-/jobs... has already had its artifacts purged.

-Eric

I suspect it'll be easier and more robust to proxy to the pages daemon with appropriate rewrite rules than to use the Gitlab API.

More robust probably -- but not easier.

AFAICS the pages daemon just proxies to the other gitlab instance - without knowing the last job ID I can't say Apache which URL to rewrite to.

If you do go via the API you'd need to ensure that every *-site repo uses the same branch name (I think) and ensure that the artifacts for the latest run are saved forever. For instance, it looks like the latest pages job at https://gitlab.common-lisp.net/cl-docker-images/cl-docker-images-site/-/jobs... has already had its artifacts purged.

Well, that would be another problem then, right.

Unfortunately, I can't easily tell when this last worked. Maybe Apr 2021? But I've also changed my ci config a bit. The biggest change was not needing to generate the pages with emacs; that change worked. I've made some recent changes to see if my config was broken in some way and removed the constraint on deploying only from the master branch. This didn't work.

If there's something I can do to help, please let me know.

Looking around some more, I found /var/log/gitlab/gitlab-pages/@4000000061d899ba05d2a384.s:

... { "content_type": "text/html; charset=utf-8", "correlation_id": "01FRT0HT3ARTVAQQA14X18PF78", "duration_ms": 83, "error": "domain does not exist", "host": "cmucl.pages.example.com", "level": "info", "method": "GET", "msg": "access", "pages_host": "cmucl.pages.example.com", "pages_https": false, "proto": "HTTP/1.1", "referrer": "", "remote_addr": "127.0.0.1:49094", "remote_ip": "127.0.0.1", "status": 200, "system": "http", "time": "2022-01-07T10:15:06Z", "ttfb_ms": 78, "uri": "/-/cmucl-site/-/jobs/26695/artifacts/public/doc/index.html", "user_agent": "curl/7.74.0", "written_bytes": 17807 }

Perhaps we "just" need some job to extract the new artifact at the right location?

Reading the gitlab documentation (at [1]) makes me think that local serving from a normal filesystem is no longer wanted...

Ad 1: https://docs.gitlab.com/ee/administration/pages/#prepare-gitlab-pages-for-14...

Hi!

...

...

I suspect it'll be easier and more robust to proxy to the pages daemon with appropriate rewrite rules than to use the Gitlab API.

More robust probably -- but not easier.

AFAICS the pages daemon just proxies to the other gitlab instance - without knowing the last job ID I can't say Apache which URL to rewrite to.

This surprises me. Perhaps the Gitlab setup on this server is significantly different than I expect, though.

If we assume the following:

1. Gitlab Pages daemon is configured to listen on port 9000
2. The pages domain is configured to be $PAGES_DOMAIN (currently appears

to be pages.example.com).

Then Apache /should/ just need to take a request URL like https://common-lisp.net/project/$PROJECT/$PATH, overwrite the Host header to be $PROJECT.$PAGES_DOMAIN, and proxy it to http://localhost:9000/$PROJECT-site/$PATH

Yes. That should be an option. I have no idea why I didn't come up with that before, instead using the somewhat complex construct of resorting to disc access. Ah. I see. It's this bit of the apache configuration which is the cause (caused by trying to support Mark E's fetish for maintaining old URLs along the line of thought of "Good URLs don't change"):

[snip: config sent in private]

Hi

since we are at this.

I noticed that the .yml has a 'mv' as a command to deploy the pages.

This is the minimal setup I have for with-context-site

pages: stage: deploy script: mv html public artifacts: paths: - public tags: - site-gen only: - master

should it be 'cp' instead of 'mv'?

I am asking, because it is faster 😏

All the best

Marco

On Tue, Jan 11, 2022 at 12:04 AM Erik Huelsmann ehuels@gmail.com wrote:

Hoi Raymond,

Today i was able to implement a measure which at least allows us to serve the latest deployed content. The solution is a bit rough around the edges: redirects don't work well yet, but content does serve correctly.

Regards,

Erik.

On Fri, Jan 7, 2022, 01:34 Raymond Toy toy.raymond@gmail.com wrote:

...

Not sure what happened but the pages for cmucl-site aren't actually deploying.

For example, the pipeline https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5701 says the pages were deployed. If you download the artifacts and look at public/doc/index.html, you'll see that there's an entry for Hemlock docs.

But when I visit https://common-lisp.net/project/cmucl/doc/index.html, the entry for the Hemlock docs is not there.

This used to work, but I'm not sure what's changed.

Also, a side note. If you browse the archive and navigate to public/doc and click on index.html, the URL is http://cmucl.pages.example.com/-/cmucl-site/-/jobs/26695/artifacts/public/do..., which doesn't exist. What can I do to make this work?

-- Ray

Sorry for the delay.

I tried to look for the deployed pages, and https://common-lisp.net/project/cmucl no longer exists (404), and the new site cmucl.common-lisp.dev doesn't exist either.

Gitlab CI thinks everything is working though. I don't remember if this is the way it always was or not, but if you visit https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5765, the deploy stage has two steps: "pages" and "pages:deploy". I can rerun "pages", but not "pages:deploy".

On Mon, Jan 10, 2022 at 3:04 PM Erik Huelsmann ehuels@gmail.com wrote:

Hoi Raymond,

Today i was able to implement a measure which at least allows us to serve the latest deployed content. The solution is a bit rough around the edges: redirects don't work well yet, but content does serve correctly.

Regards,

Erik.

On Fri, Jan 7, 2022, 01:34 Raymond Toy toy.raymond@gmail.com wrote:

...

Not sure what happened but the pages for cmucl-site aren't actually deploying.

For example, the pipeline https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5701 says the pages were deployed. If you download the artifacts and look at public/doc/index.html, you'll see that there's an entry for Hemlock docs.

But when I visit https://common-lisp.net/project/cmucl/doc/index.html, the entry for the Hemlock docs is not there.

This used to work, but I'm not sure what's changed.

Also, a side note. If you browse the archive and navigate to public/doc and click on index.html, the URL is http://cmucl.pages.example.com/-/cmucl-site/-/jobs/26695/artifacts/public/do..., which doesn't exist. What can I do to make this work?

-- Ray

On Sat, Jan 15, 2022 at 10:21 AM Erik Huelsmann ehuels@gmail.com wrote:

Hi Ray,

...

Sorry for the delay.

No problem!

...

I tried to look for the deployed pages, and

https://common-lisp.net/project/cmucl no longer exists (404), and the new site cmucl.common-lisp.dev doesn't exist either.

The issue here is that the site correctly renders when you use https://common-lisp.net/project/cmucl/ but not without the trailing forward-slash. This is one of the reasons to want to implement the pages daemon and run the domains on common-lisp.dev.

Yes, when I use the trailing slash, I see the expected results. Thanks so much for fixing this!

...

Gitlab CI thinks everything is working though. I don't remember if this

is the way it always was or not, but if you visit https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5765, the deploy stage has two steps: "pages" and "pages:deploy". I can rerun "pages", but not "pages:deploy".

I think the step "pages:deploy" is a built-in step in GitLab CI which you're not supposed to be able to re-run. However, it'll rerun when the last job gets rerun (or so I believe).

Looks that way. I just reran the pipeline an hour or so ago, and once I used the correct link, the expected changes are there.

Thanks!

-- Bye,

Erik.

http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.

There seems to be something wrong with the Gitlab Pages hosted sites. It looks like the path is being dropped when the request is forwarded to the Pages daemon.

Compare https://cl-tar.common-lisp.dev/, https://cl-tar.common-lisp.dev/cl-tar/v0.2.1/manual/, and https://cl-tar.common-lisp.dev/theme.css

-Eric

On 1/18/22 18:33, Erik Huelsmann wrote:

Today i renamed all repositories that deploy a site, such as antiek/antik-site so that it deploys its site on antiek.common-lisp.dev http://antiek.common-lisp.dev.

All sites that used to be hosted under common-lisp.net/project/ http://common-lisp.net/project/ are now forwarded to common-lisp.dev http://common-lisp.dev too.

It would be absolutely great if we could get contributions from people who are comfortable with gitlab pages describing how to use it with CL site builder software as content for common-lisp.net http://common-lisp.net

Regards,

Erik

On Sat, Jan 15, 2022, 19:37 Raymond Toy <toy.raymond@gmail.com mailto:toy.raymond@gmail.com> wrote:

On Sat, Jan 15, 2022 at 10:21 AM Erik Huelsmann <ehuels@gmail.com
<mailto:ehuels@gmail.com>> wrote:

    Hi Ray,

     > Sorry for the delay.
    No problem!

     > I tried to look for the deployed pages, and
    https://common-lisp.net/project/cmucl
    <https://common-lisp.net/project/cmucl> no longer exists (404),
    and the new site cmucl.common-lisp.dev
    <http://cmucl.common-lisp.dev> doesn't exist either.

    The issue here is that the site correctly renders when you use
    https://common-lisp.net/project/cmucl/
    <https://common-lisp.net/project/cmucl/> but not without the
    trailing
    forward-slash. This is one of the reasons to want to implement the
    pages daemon and run the domains on common-lisp.dev
    <http://common-lisp.dev>.


Yes, when I use the trailing slash, I see the expected results. 
Thanks so much for fixing this!


     > Gitlab CI thinks everything is working though.  I don't
    remember if this is the way it always was or not, but if you
    visit
    https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5765
    <https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5765>,
    the deploy stage has two steps: "pages" and "pages:deploy".  I
    can rerun "pages", but not "pages:deploy".

    I think the step "pages:deploy" is a built-in step in GitLab CI
    which
    you're not supposed to be able to re-run. However, it'll rerun when
    the last job gets rerun (or so I believe).


Looks that way.  I just reran the pipeline an hour or so ago, and
once I used the correct link, the expected changes are there.

Thanks!





    -- 
    Bye,

    Erik.

    http://efficito.com <http://efficito.com> -- Hosted accounting
    and ERP.
    Robust and Flexible. No vendor lock-in.



-- 
Ray

Hi Ray,

Fillip Marek checked out your problem report this morning. There was a configuration issue and https://cmucl.common-lisp.dev renders correctly now.

Regards,

Erik.

On Wed, Jan 19, 2022 at 5:57 AM Raymond Toy toy.raymond@gmail.com wrote:

Yeah, I've noticed that common-lisp.dev is doing something different now. cmucl.common-lisp.dev is not loading the JS files. Chrome console says the server is responding with a MIME type of "text/html" for the file js/common.js. Without JS, I can't navigate to the various pages.

Before it was moved, common-lisp.net/project/cmucl/ worked fine and loaded the JS files without problems.

On Tue, Jan 18, 2022 at 8:19 PM Eric Timmons etimmons@mit.edu wrote:

...

There seems to be something wrong with the Gitlab Pages hosted sites. It looks like the path is being dropped when the request is forwarded to the Pages daemon.

Compare https://cl-tar.common-lisp.dev/, https://cl-tar.common-lisp.dev/cl-tar/v0.2.1/manual/, and https://cl-tar.common-lisp.dev/theme.css

-Eric

On 1/18/22 18:33, Erik Huelsmann wrote:

...

Today i renamed all repositories that deploy a site, such as antiek/antik-site so that it deploys its site on antiek.common-lisp.dev http://antiek.common-lisp.dev.

All sites that used to be hosted under common-lisp.net/project/ http://common-lisp.net/project/ are now forwarded to common-lisp.dev http://common-lisp.dev too.

It would be absolutely great if we could get contributions from people who are comfortable with gitlab pages describing how to use it with CL site builder software as content for common-lisp.net http://common-lisp.net

Regards,

Erik

On Sat, Jan 15, 2022, 19:37 Raymond Toy <toy.raymond@gmail.com mailto:toy.raymond@gmail.com> wrote:

On Sat, Jan 15, 2022 at 10:21 AM Erik Huelsmann <ehuels@gmail.com
<mailto:ehuels@gmail.com>> wrote:

    Hi Ray,

     > Sorry for the delay.
    No problem!

     > I tried to look for the deployed pages, and
    https://common-lisp.net/project/cmucl
    <https://common-lisp.net/project/cmucl> no longer exists (404),
    and the new site cmucl.common-lisp.dev
    <http://cmucl.common-lisp.dev> doesn't exist either.

    The issue here is that the site correctly renders when you use
    https://common-lisp.net/project/cmucl/
    <https://common-lisp.net/project/cmucl/> but not without the
    trailing
    forward-slash. This is one of the reasons to want to implement the
    pages daemon and run the domains on common-lisp.dev
    <http://common-lisp.dev>.


Yes, when I use the trailing slash, I see the expected results.
Thanks so much for fixing this!


     > Gitlab CI thinks everything is working though.  I don't
    remember if this is the way it always was or not, but if you
    visit
    https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5765
    <https://gitlab.common-lisp.net/cmucl/cmucl-site/-/pipelines/5765>,
    the deploy stage has two steps: "pages" and "pages:deploy".  I
    can rerun "pages", but not "pages:deploy".

    I think the step "pages:deploy" is a built-in step in GitLab CI
    which
    you're not supposed to be able to re-run. However, it'll rerun when
    the last job gets rerun (or so I believe).


Looks that way.  I just reran the pipeline an hour or so ago, and
once I used the correct link, the expected changes are there.

Thanks!





    --
    Bye,

    Erik.

    http://efficito.com <http://efficito.com> -- Hosted accounting
    and ERP.
    Robust and Flexible. No vendor lock-in.



--
Ray

-- Ray