Since this mail, I've been quietly working on setting up the VPS we need anyway, while waiting for people to get learn about the move and hopefully join the mailing list.
On Tue, Mar 29, 2011 at 11:35 PM, Erik Huelsmann ehuels@gmail.com wrote:
The server running common-lisp.net has been running in the current configuration for a number of years now. Its Debian distribution is still Etch, which has been moved to the archives June 20th last year. As a consequence, we're locked out of even the most basic of maintenance needs: security updates.
Bottom line: common-lisp.net as it is today can't be maintained anymore. We need to upgrade the system.
We have two options for the system upgrade:
- Copy the VPS and run dist-upgrade; then fix whatever falls over.
- Create a new VPS, and make a clean start, meaning that we identify
all services currently running on cl-net and we arrange for the same or equivalent services to be running on the new system.
Stelian has indicated his preference is for option (2), as too much has been customized or is installed in custom locations to make a standard dist-upgrade work. I concur.
Well, the new VPS is up and running. The technical configuration isn't quite to my liking, but we can definitely work with it.
Another remark from Stelian is that we could migrate one service at a time, creating subdomains (svn.common-lisp.net, git.common-lisp.net, trac...., etc) which might help future migrations.
As far as the domains go: I've set up 'new.common-lisp.net' to point to the new VPS; all the other domains will come when we're ready to go live with the different services.
So, from where we stand now, we need to:
- Identify all services, users and data we need to migrate
- Find people who want to help migrate the services
- Per component, identify a migration plan - we can start migrating
services with complete migration plans 4) Test the new clean setups as well as the combination of the new setup and the old data
While at it, I've aptitude installed most of the services mentioned below.
Services that I know we're running [without looking at the box]:
* Mail [MTA]
- postfix, postfix-gld
* Web
- apache2
* Ticket tracker (RT - internal use)
- otrs2
* Mailing lists
- mailman
* Version control
- subversion, git, hg, darcs, cvs
* Repository browsing - Subversion - Git - Darcs - CVS (do we still want to support CVS ??) * Trac
- trac, trac-mercurial, trac-xmlrpc, trac-git
Does anybody have experience with trac-mastertickets, trac-spamfilter, trac-email2trac or trac-accountmanager? How would they fit with the cl-net site setup? They all sound interesting.
* spam detection
- spamassasin; although postfix integration still needs setting up
* Git deamon * rsync (what for?)
- rsync
* sshd
- sshd
* ftpd
--- Actually, no. The service is running but unused on current cl-net; I think we can use http for downloads and use scp for uploads.
* lisppaste (on sbcl-0.9.7??)
-- to be done
* fast-cgi
- libapache2-mod-fastcgi
* custom administration scripts
-- to be done
With the status above, we're getting closer to the point where we actually have to do something with the data and services on the current machine.
Before we get that far, though, it would be nice if someone could configure the OTRS instance I set up on the new box for cl-net use. [I'll help, of course.]
I've been thinking about the next steps. The easy steps would be:
1. copy /project data to the new server 2. nfs-mount /project on the old box (if we use NFSv3, both Subversion and CVS should be NFS safe, I understand) 3. migrate the anonymous read services for version control to the new box 4. copy and test the custom scripts, including installation of cron jobs etc 5. migrate trac.common-lisp.net to the new server
and from there it requires a bit more thought to get things right. Remaining questions: * how to migrate the users password and other secure data * related: how to we bridge the gap between the time of the migration of the user account data (/etc/passwd and /etc/group) and the migration of the last service * which services need account (login) data [I know about Subversion commit, trac administration, cvs commit, website update; others?]
Well, more questions and ideas how to proceed are *very* welcome.
Bye,
Erik.
Erik,
first off, thank you for doing the upgrade.
You write that you installed otrs2 - We are running rt for request tracking and while it is not particularily convenient, I have gotten used to it and as I am the person handling the majority of the user requests, I would not want to change to another request tracker unless there is a reason that convinces me or someone else steps up to handle user requests. I am absolutely not saying that I insist on being the first level supporter forever.
On Sat, Apr 2, 2011 at 10:48 PM, Erik Huelsmann ehuels@gmail.com wrote:
Services that I know we're running [without looking at the box]:
* Mail [MTA]
- postfix, postfix-gld
[...]
* Mailing lists
- mailman
We are running exim and greylistd. I can continue to support that configuration, but I can't support postfix. If someone else volunteers to do that, I'll gladly pass the task.
The one particular thing to note here is that the exim configuration has an automatic integration with mailman, i.e. there are no explicit aliases for mailman lists. This must be taken care of if the mail configuration is changed.
-Hans
Hi Hans,
Thanks for reviewing the setup so far.
You write that you installed otrs2 - We are running rt for request tracking and while it is not particularily convenient, I have gotten used to it and as I am the person handling the majority of the user requests, I would not want to change to another request tracker unless there is a reason that convinces me or someone else steps up to handle user requests.
The reason I set up OTRS is that Drew and I started using OTRS for the Tech Coop recently. He was surprised how easy and self-explanatory the tool is. Another reason was that we'd need to support only a single tool instead of building up experience for management of multiple tools. Talking about myself, I don't have any RT experience and won't be able to maintain that; however if cl.net is running it, someone must have experience at it. Any idea who?
Additionally, OTRS features a FAQ module which helps creating and publishing FAQ items while you support customers [I've installed that module too.], which seemed like a good idea - the more FAQ items, the less real support calls, was my thought.
Based on Drew's positive reaction to OTRS, would you mind having a look at OTRS? It's up and running (except for the incoming mail interface); you can look around and see if you like it.
I am absolutely not saying that I insist on being the first level supporter forever.
Right. Although not related to the technical migration, I do think you have an important issue there: cl.net maintenance - imo - is taken too much for granted by the community. I'd love to see fresh blood stepping in to help out with the maintenance and expansion of the site.
On Sat, Apr 2, 2011 at 10:48 PM, Erik Huelsmann ehuels@gmail.com wrote:
Services that I know we're running [without looking at the box]:
* Mail [MTA]
- postfix, postfix-gld
[...]
* Mailing lists
- mailman
We are running exim and greylistd. I can continue to support that configuration, but I can't support postfix. If someone else volunteers to do that, I'll gladly pass the task.
Ah. Same thing as with OTRS: I've installed it because we use it on other Tech Coop run sites.
The one particular thing to note here is that the exim configuration has an automatic integration with mailman, i.e. there are no explicit aliases for mailman lists. This must be taken care of if the mail configuration is changed.
There's an integration between postfix and mailman like that too. Can you elaborate on the maintenance you had to do for exim? I mean, was there other maintenance than creating and deleting mailing lists (technical issues)? I'm trying to understand the type of maintenance you had to perform, just so I know what I'm getting myself into (see below).
It'd be too bad if you were to hand in part of your maintenance tasks only because of a technical migration. It does seem healthy for c-l.net though if there were more people to share the load with. Maybe we should take the opportunity to put the maintainers of cl.net more into the spotlight. Having more people to share the load with, would also allow the website to be updated and maybe it would even allow development of an easier maintenance interface. Although not related to the technical migration itself, I do think some of these things could be subject of discussion after we complete the migration.
I'll take over support of the mail subsystem. That addresses part of my concern of being with too few people in the maintenance team and at the same time allows us to run the same MTA in as many sites as possible.
Thanks again for your review of the list of installed software!
Bye,
Erik.
Hans and I discussed the mailer setup off-list. So that's solved now. However, nobody commented on this section of the mail below. I'd like to have some feedback/thoughts on the "this bit needs more thought" part directly below the numbered list.
With the status above, we're getting closer to the point where we actually have to do something with the data and services on the current machine.
Before we get that far, though, it would be nice if someone could configure the OTRS instance I set up on the new box for cl-net use. [I'll help, of course.]
I've been thinking about the next steps. The easy steps would be:
- copy /project data to the new server
- nfs-mount /project on the old box (if we use NFSv3, both Subversion
and CVS should be NFS safe, I understand) 3. migrate the anonymous read services for version control to the new box 4. copy and test the custom scripts, including installation of cron jobs etc 5. migrate trac.common-lisp.net to the new server
and from there it requires a bit more thought to get things right. Remaining questions:
- how to migrate the users password and other secure data
- related: how to we bridge the gap between the time of the migration
of the user account data (/etc/passwd and /etc/group) and the migration of the last service
- which services need account (login) data [I know about Subversion
commit, trac administration, cvs commit, website update; others?]
Well, more questions and ideas how to proceed are *very* welcome.
Anybody?
Thanks in advance!
Bye,
Erik.
On Fri, Apr 8, 2011 at 4:45 PM, Erik Huelsmann ehuels@gmail.com wrote:
Remaining questions:
- how to migrate the users password and other secure data
I would assume that the password file formats have not changed, so I would assume that the files can just be copied over.
- related: how to we bridge the gap between the time of the migration
of the user account data (/etc/passwd and /etc/group) and the migration of the last service
You could make the password related files immutable on one box and have password changes only be done on the other, then synchronize the password files from the master to the slave frequently. Frankly, I'd avoid that issue by telling users that passwords cannot be changed during the migration and make the migration phase short.
-Hans
-
Erik Huelsmann -
Hans Hübner