Dear common-lisp.net administrators, We had been relying on rsync to transfer artifacts for the cmucl.org website from common-lisp.net. The script, which did this periodically rsync -aqvz rsync://common-lisp.net/project/cmucl/downloads/ . now times out. I tried referencing the host gitlab.common-lisp.net but that seems to have no effect. Is there a new host I should? Kindly, Carl
common-lisp.net is now pointing to the same IP addresses as gitlab.common-lisp.net (which is the same as future.common-lisp.net). What is that rsync URL protocol? Is that a service we need to set up on the new host? If so, Carl, would you consider setting that up for us if I provide credentials? Dave Cooper ---- On Mon, 10 Mar 2025 17:37:36 -0400 Carl Shapiro <cshapiro@panix.com> wrote --- Dear common-lisp.net administrators, We had been relying on rsync to transfer artifacts for the cmucl.org website from common-lisp.net. The script, which did this periodically rsync -aqvz rsync://common-lisp.net/project/cmucl/downloads/ . now times out. I tried referencing the host gitlab.common-lisp.net but that seems to have no effect. Is there a new host I should? Kindly, Carl
It appears the script assumes an rsync daemon listening on TCP port 873. Carl, is there some reason to not use ssh instead? Rsync can support this trivially with much improved security properties. — jb On Mar 10, 2025 at 18:45 -0400, David Cooper <david.cooper@genworks.com>, wrote:
common-lisp.net is now pointing to the same IP addresses as gitlab.common-lisp.net (which is the same as future.common-lisp.net).
What is that rsync URL protocol? Is that a service we need to set up on the new host?
If so, Carl, would you consider setting that up for us if I provide credentials?
Dave Cooper
---- On Mon, 10 Mar 2025 17:37:36 -0400 Carl Shapiro <cshapiro@panix.com> wrote ---
Dear common-lisp.net administrators,
We had been relying on rsync to transfer artifacts for the cmucl.org website from common-lisp.net. The script, which did this periodically
rsync -aqvz rsync://common-lisp.net/project/cmucl/downloads/ .
now times out. I tried referencing the host gitlab.common-lisp.net but that seems to have no effect. Is there a new host I should?
Kindly,
Carl
Jon Boone <ipmonger@delamancha.org> writes:
It appears the script assumes an rsync daemon listening on TCP port 873. Carl, is there some reason to not use ssh instead? Rsync can support this trivially with much improved security properties.
Thanks for the suggestion. I set this up many years ago so I cannot be absolutely certain but I suspect that I used rsync the way I did because that is the method that was documented at the time. https://web.archive.org/web/20040606094613/http://common-lisp.net/rsync.shtm... I am open to doing something different if there are viable options available. As for ssh in particular, my personal ssh credentials for c-l.net stopped working a while ago. Though, for the purpose of a cron job, a account with read-only access to project data would feel more appropriate. Kindly, Carl
Jon Boone<ipmonger@delamancha.org> writes:
It appears the script assumes an rsync daemon listening on TCP port 873. Carl, is there some reason to not use ssh instead? Rsync can support this trivially with much improved security properties. Thanks for the suggestion.
I set this up many years ago so I cannot be absolutely certain but I suspect that I used rsync the way I did because that is the method that was documented at the time. Likewise, I set up a monthly cronjob to backup cmucl and other things I have on c-l.net. Probably been doing this ever since rsync was available. https://web.archive.org/web/20040606094613/http://common-lisp.net/rsync.shtm...
I am open to doing something different if there are viable options available. Me too, as long as I don’t have to provide my c-l.net credentials. Some
On 3/10/25 5:57 PM, Carl Shapiro wrote: passwordless ssh would be fine. Or other scheme.
Yep. I just noted that I cannot ssh to common-lisp.net anymore (offending key). Can you point me to the instructions to fix this? Moreover: I have already set up gitlab.common-lisp.net with 2FA, but now, it gets me to the "Explore/Projects". Would it be possible to point to the personal pages instead? Thanks MA On Tue, Mar 11, 2025 at 3:01 PM Raymond Toy <toy.raymond@gmail.com> wrote:
On 3/10/25 5:57 PM, Carl Shapiro wrote:
Jon Boone <ipmonger@delamancha.org> <ipmonger@delamancha.org> writes:
It appears the script assumes an rsync daemon listening on TCP port 873. Carl, is there some reason to not use ssh instead? Rsync can support this trivially with much improved security properties.
Thanks for the suggestion.
I set this up many years ago so I cannot be absolutely certain but I suspect that I used rsync the way I did because that is the method that was documented at the time.
Likewise, I set up a monthly cronjob to backup cmucl and other things I have on c-l.net. Probably been doing this ever since rsync was available.
https://web.archive.org/web/20040606094613/http://common-lisp.net/rsync.shtm...
I am open to doing something different if there are viable options available.
Me too, as long as I don’t have to provide my c-l.net credentials. Some passwordless ssh would be fine. Or other scheme.
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY CSCE 2025 - csce.lakecomoschool.org
Marco, Thanks for the report. Your account did and does exist on the new host, but your report unmasked a missing `/custom` directory which was on the old host and not on the new, and upon which your .bash_profile indirectly relied. So I synced that `/custom` root directory from the old host to new, and I think you should be able to log in now. Please confirm your ability to log in and let us know what else is missing! Dave Cooper P.S. (And this goes for Everyone using ssh or otherwise with active home directories on the legacy clnet host) as I've mentioned, you should be able to access the old host via any unused subdomain e.g. legacy.common-lisp.net (you may have to update your .ssh/config accordingly). Most of these home directories were already synced to the new host in late December 2024 so anyone with home directories on legacy.common-lisp.net may want to use rsync at least with --dry-run from legacy, to pick up any lagging modifications between that and current common-lisp.net. Only a few home directories are very active so I assume you know who you are. I apologize for insufficient communication regarding migration of the home directories. P.P.S. There are still some legacy home directories which are not transferred to the new host because they appeared inactive by [some metric] and look like they would just be taking up space. Please let us know if you see any legacy home directories which you feel ought to be synced. And of course let us know of any other missing services or weirdness. ---- On Tue, 11 Mar 2025 10:11:39 -0400 Marco Antoniotti <marco.antoniotti@unimib.it> wrote --- Yep. I just noted that I cannot ssh to http://common-lisp.net anymore (offending key). Can you point me to the instructions to fix this? Moreover: I have already set up http://gitlab.common-lisp.net with 2FA, but now, it gets me to the "Explore/Projects". Would it be possible to point to the personal pages instead? Thanks MA On Tue, Mar 11, 2025 at 3:01 PM Raymond Toy <mailto:toy.raymond@gmail.com> wrote: -- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY CSCE 2025 - http://csce.lakecomoschool.org On 3/10/25 5:57 PM, Carl Shapiro wrote: Jon Boone mailto:ipmonger@delamancha.org writes: It appears the script assumes an rsync daemon listening on TCP port 873. Carl, is there some reason to not use ssh instead? Rsync can support this trivially with much improved security properties. Thanks for the suggestion. I set this up many years ago so I cannot be absolutely certain but I suspect that I used rsync the way I did because that is the method that was documented at the time. Likewise, I set up a monthly cronjob to backup cmucl and other things I have onhttp://c-l.net. Probably been doing this ever since rsync was available. https://web.archive.org/web/20040606094613/http://common-lisp.net/rsync.shtm... I am open to doing something different if there are viable options available. Me too, as long as I don’t have to provide myhttp://c-l.net credentials. Some passwordless ssh would be fine. Or other scheme.
Moreover: I have already set up http://gitlab.common-lisp.net with 2FA, but now, it gets me to the "Explore/Projects". Would it be possible to point to the personal pages instead? Please try it now (with being logged in).
Marco (and anyone else with ssh access): You may need an updated ssh keypair which is stronger or more modern - whatever is needed by default by current latest debian bookworm. If your logins are still not working apparently because of invalid key type, please send me a new public key which uses a current encryption method such as e.g. ed25519 and I will add to your .ssh/authorized_keys. Dave Cooper ---- On Tue, 11 Mar 2025 10:11:39 -0400 Marco Antoniotti <marco.antoniotti@unimib.it> wrote --- Yep. I just noted that I cannot ssh to http://common-lisp.net anymore (offending key). Can you point me to the instructions to fix this? Moreover: I have already set up http://gitlab.common-lisp.net with 2FA, but now, it gets me to the "Explore/Projects". Would it be possible to point to the personal pages instead? Thanks MA On Tue, Mar 11, 2025 at 3:01 PM Raymond Toy <mailto:toy.raymond@gmail.com> wrote: -- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY CSCE 2025 - http://csce.lakecomoschool.org On 3/10/25 5:57 PM, Carl Shapiro wrote: Jon Boone mailto:ipmonger@delamancha.org writes: It appears the script assumes an rsync daemon listening on TCP port 873. Carl, is there some reason to not use ssh instead? Rsync can support this trivially with much improved security properties. Thanks for the suggestion. I set this up many years ago so I cannot be absolutely certain but I suspect that I used rsync the way I did because that is the method that was documented at the time. Likewise, I set up a monthly cronjob to backup cmucl and other things I have onhttp://c-l.net. Probably been doing this ever since rsync was available. https://web.archive.org/web/20040606094613/http://common-lisp.net/rsync.shtm... I am open to doing something different if there are viable options available. Me too, as long as I don’t have to provide myhttp://c-l.net credentials. Some passwordless ssh would be fine. Or other scheme.
On 11/03/2025 19:25, David Cooper wrote:
Marco (and anyone else with ssh access): You may need an updated ssh keypair which is stronger or more modern - whatever is needed by default by current latest debian bookworm.
If your logins are still not working apparently because of invalid key type, please send me a new public key which uses a current encryption method such as e.g. ed25519 and I will add to your .ssh/authorized_keys.
Dave Cooper
It's actually a bit more strict than default debian bookworm; I applied the server config recommendations from https://github.com/jtesta/ssh-audit, see /etc/ssh/sshd_config.d/local.conf This means you can't use ECDSA keys (RSA, ED25519 or ED25519-SK are all OK) and must use sufficiently modern ciphers, MAC and key-exchange algorithms. Georgiy
Hi So. Bottom line: can we use RSA, ED25519 or ED25519-SK? Some of us (me, at a minimum; sorry) do not have all the time to RTFM for the latest and greatest encryption. If ssh-keygen works with any of the schemes above, please let me know, and also post a note on the main website. All the best Marco On Tue, Mar 11, 2025 at 8:29 PM Georgiy Tugai <georgiy@tugai.id.au> wrote:
On 11/03/2025 19:25, David Cooper wrote:
Marco (and anyone else with ssh access): You may need an updated ssh keypair which is stronger or more modern - whatever is needed by default by current latest debian bookworm.
If your logins are still not working apparently because of invalid key type, please send me a new public key which uses a current encryption method such as e.g. ed25519 and I will add to your .ssh/authorized_keys.
Dave Cooper
It's actually a bit more strict than default debian bookworm; I applied the server config recommendations from https://github.com/jtesta/ssh-audit, see /etc/ssh/sshd_config.d/local.conf
This means you can't use ECDSA keys (RSA, ED25519 or ED25519-SK are all OK) and must use sufficiently modern ciphers, MAC and key-exchange algorithms.
Georgiy
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY CSCE 2025 - csce.lakecomoschool.org
On 2025-03-12 10:10, Marco Antoniotti wrote:
Hi
So. Bottom line: can we use RSA, ED25519 or ED25519-SK? Some of us (me, at a minimum; sorry) do not have all the time to RTFM for the latest and greatest encryption. If ssh-keygen works with any of the schemes above, please let me know, and also post a note on the main website
Bottom line: RSA, ED25519, ED25519-SK, all of which work with ssh-keygen. The other stuff comes in if you're using a (much) older, or limited (third-party) client. Regards, Georgiy
Hi So. Bottom line: can we use RSA, ED25519 or ED25519-SK? Some of us (me, at a minimum; sorry) do not have all the time to RTFM for the latest and greatest encryption. If ssh-keygen works with any of the schemes above, please let me know, and also post a note on the main website. All the best Marco Hi Marco, Thanks for the feedback! Relevant news items have been added at https://common-lisp.nett Apologies for the ungraceful switching of the toplevel domain without enough communication to our dear ssh users - our hand was kind of forced by an SSL cert expiration (I mean, we could have extended it on the old host, but I really didn't want to kick that can down the road again) and somehow our quiet but active ssh users were a blind spot for me. Dave
David Cooper <david.cooper@genworks.com> writes:
common-lisp.net is now pointing to the same IP addresses as gitlab.common-lisp.net (which is the same as future.common-lisp.net).
Good to know, thank you. I tried that on the suggestion of the top news item under "Latest Common-Lisp.net news" on the home page.
What is that rsync URL protocol? Is that a service we need to set up on the new host?
An efficient file synchronization protocol. Since the early days, you could access all project data through rsync. See this link https://web.archive.org/web/20040606094613/http://www.common-lisp.net/rsync.... This was a reasonable feature back then. Even now, two decades later, rsync is still a preferred way for sites to have their data mirrored in bulk.
If so, Carl, would you consider setting that up for us if I provide credentials?
I would certainly consider helping out. But, if there is an alternate way to get this feature without standing up a new service that could add to the maintenance burden, it would seem worthwhile to try it out first. Kindly, Carl
Hi Carl, I apologize that your account was somehow missed in the host migration. Why didn't you say something sooner? I just synchronized your account and its home directory contents, replete with its .ssh/authorized_keys, to the current host, so you should be able to ssh to current common-lisp.net now. If not, please let me know. And, I added you to sudo group so you can add an additional user with restricted access for doing the rsync's, if you like. And feel free to look into installing the needed rsync service, and go ahead and install it if you can, if that is running on the old host. Our initial goal is to replicate as many of the legacy services as possible. Ever since I started using rsync heavily (maybe 15-20 years ago), it always seems to go by default over ssh, so I'm not familiar with the actual rsync transfer protocol or service per se. Maybe I should be, if the server I'm purporting to administer has documentation which assumes that... Dave ---- On Mon, 10 Mar 2025 21:09:09 -0400 Carl Shapiro <cshapiro@panix.com> wrote --- David Cooper <mailto:david.cooper@genworks.com> writes:
common-lisp.net is now pointing to the same IP addresses as gitlab.common-lisp.net (which is the same as future.common-lisp.net).
Good to know, thank you. I tried that on the suggestion of the top news item under "Latest Common-Lisp.net news" on the home page.
What is that rsync URL protocol? Is that a service we need to set up on the new host?
An efficient file synchronization protocol. Since the early days, you could access all project data through rsync. See this link https://web.archive.org/web/20040606094613/http://www.common-lisp.net/rsync.... This was a reasonable feature back then. Even now, two decades later, rsync is still a preferred way for sites to have their data mirrored in bulk.
If so, Carl, would you consider setting that up for us if I provide credentials?
I would certainly consider helping out. But, if there is an alternate way to get this feature without standing up a new service that could add to the maintenance burden, it would seem worthwhile to try it out first. Kindly, Carl
David Cooper <david.cooper@genworks.com> writes:
I apologize that your account was somehow missed in the host migration. Why didn't you say something sooner?
Good question. I think the problem I had predated the host migration so the host migration did not change the status quo for me.
I just synchronized your account and its home directory contents, replete with its .ssh/authorized_keys, to the current host, so you should be able to ssh to current common-lisp.net now. If not, please let me know.
Thank you. Will do.
And, I added you to sudo group so you can add an additional user with restricted access for doing the rsync's, if you like. And feel free to look into installing the needed rsync service, and go ahead and install it if you can, if that is running on the old host. Our initial goal is to replicate as many of the legacy services as possible. Ever since I started using rsync heavily (maybe 15-20 years ago), it always seems to go by default over ssh, so I'm not familiar with the actual rsync transfer protocol or service per se. Maybe I should be, if the server I'm purporting to administer has documentation which assumes that...
For whatever it's worth, this inspired me to try and point the rsync to the legacy host which does seem to accept connections suggesting that the service is still running. I think rsync uses ssh when you specify a source or destination that includes the name of a user and a host. For anonymous rsync that talks directly to rsyncd, ssh is not used. Anyway, sounds like the next step is for me to dust off my old private key and get myself logged-in again. Once that happens, I can scope out what is needed to restore anonymous rsync access. If I can make it work, I hope others can benefit from it, too. Kindly, Carl
participants (6)
-
Carl Shapiro -
David Cooper -
Georgiy Tugai -
Jon Boone -
Marco Antoniotti -
Raymond Toy