Importing mailing list archives

This page describes the process by which Savannah hackers can import mailing list archives. If you are a user who would like an archive imported, please open a support request with the relevant information.

Check MailSystem for general information on list archives.

The format of the mail archive to import may vary, but we need mbox files. Anything else has to be converted. Check for spurious "From " lines.

Public mailing lists

Log in to lists.gnu.org as user list; log in as mharc:

 lists:~$ ssh -l mharc lists.gnu.org

You'll need an archive in one big mailbox. You probably can cat several mailboxes together if needed (but be sure there is a blank line at the end of each file first, to avoid running messages together):

 cat 1.mbox 2.mbox >>mybig.mailbox

Split it monthly:

 ~mharc/bin/mbox-month-pack archive.mbox

Put the resulting mailboxes in ~mharc/mbox/.

If the mailman-maintained .mbox file will suffice (which it should, unless you are in some unusual situation), then the above can be reduced to:

 mkdir -m 755 ~mharc/mbox/$listname
 cd ~mharc/mbox/$listname
 ~mharc/bin/mbox-month-pack /var/lib/mailman/archives/private/$listname.mbox/$listname.mbox

In any case, you'll then need to regenerate the web archives:

 ~mharc/bin/web-archive -rebuild $listname >/tmp/mha.out

If you skip "-rebuild", web-archive will run in incremental mode, i.e. what is used in the cron job. Rebuilding archives can be time-consuming and noisy (depending on how many years you have to import). Best to redirect the output as shown above, or whatever.

Then you can check, e.g., http://lists.nongnu.org/archive/html/$listname.

Private mailing lists

Mailman, for archives of private mailing lists, behaves differently. The command ~mailman/bin/arch --wipe will regenerate list archives from a given mbox file (in the standard mailman location by default).