Wiki Administration

Savannah's WIKI is available at https://savannah.gnu.org/maintenance. Despite the name, it is more a static-website generator (like Jekyll) than a full-fledged wiki.

It is based on ikiwiki (hence the name). Updating is done solely throught Subversion, web-based editing is disabled. Savannah Administrators have write-access to the svn repository.

See RecentChanges for latest activity.

See http://svn.savannah.gnu.org/viewvc/trunk/sviki/?root=administration for the actual commits.

All commits are publically available at the 'savannah-cvs' mailing list https://lists.gnu.org/mailman/listinfo/savannah-cvs.

Format and style

The pages are written in IkiWiki markdown format, for details see https://ikiwiki.info/ikiwiki/formatting/ .

The files use the .mdwn extension.

Diagrams are welcomed, and should be included as PNG files. Please be sure to include an editable source for the diagram as well (e.g. GNU Dia files). Do not use non-free file formats.

One-off contributions

We welcome improvements to the wiki!

Use svn to anonymously checkout the repository:

svn co svn://svn.savannah.gnu.org/administration/trunk/sviki

Edit your changes, and create a diff file with:

svn diff > newstuff.patch

Then email your suggestion to savannah-hackers-public@gnu.org .

It is also possible to access Subversion repositories using Git through git-svn, which is an optional addition to the git suite of tools.

Anonymously clone the repository:

git svn clone --stdlayout svn://svn.savannah.gnu.org/administration/

Make your changes and create a diff using:

git diff > newstuff.patch

Then email your suggestion to savannah-hackers-public@gnu.org .

Write access (SVN)

Savannah administrators with write access to the savannah repositories can checkout the wiki pages using:

svn co svn+ssh://<USERNAME>@svn.savannah.gnu.org/administration/

To fetch the latest changes to an existing repository, run:

svn update

Make your changes, then commit and push with:

svn commit -m'log message'

The result should show up under http://savannah.gnu.org/maintenance/ in a few seconds, if all goes well. Check RecentChanges if you like.

Write access (GIT)

It is possible to access Subversion repositories using Git through git-svn, which is an optional addition to the git suite of tools.

Clone the repository:

git svn clone --stdlayout svn+ssh://<USERNAME>@svn.savannah.gnu.org/administration

To get the latest changes (of an existing cloned repository) and rebase your local repository, run:

git svn rebase

Make your changes, the commit with:

git commit -a -m "log message"

And push with:

git svn dcommit

Use git svn fetch to only fetch upstream changes but not rebase merge them.

If due to a race with another upstream svn committer there is a conflict then git svn rebase again to pull their new upstream changes. Then push again with git svn dcommit. (This is the same as the svn update and svn commit.)

Wiki Generation - NEW VMs

On VCS

The repository is in vcs0:/srv/svn/administration. A post-commit hook vcs0:/srv/svn/administration/hooks/post-commit.d/sv-ikiwiki-refresh-2 will be executed after a new SVN commit. The script simply triggers an update on frontend:

$ cat sv-ikiwiki-refresh-2
#!/bin/sh
curl https://savannah.gnu.org/update-sviki/update.php

On Frontend

Note: Savannah has a few servers named frontendN, and one of them runs Savannah web UI as of 2022-05-17, the working frontend is frontend2, that is, frontend0 below should in most cases read as frontend2.

The apache configuration file frontend0:/etc/apache2/sites-available/ikiwiki.inc contains the following:

# Directory with update trigger script.
Alias /update-sviki /var/www/sviki/php/

# Directory with static HTML files, auto-generated from wiki pages.
Alias /maintenance /var/www/sviki/www

The update PHP script frontend0:/var/www/sviki/php/update.php writes a message to the logs, and runs frontend0:/var/www/sviki/sviki-refresh.

The update script sviki-refresh updates the wiki with the following commands (the actual script is slightly longer, but conceptually the same):

cd /var/www/sviki/src/sviki
svn -q update
ikiwiki --setup /var/www/sviki/ikiwiki.setup --refresh

The checked-out SVN repository is in frontend0:/var/www/sviki/src. The generated HTML files are in frontend0:/var/www/sviki/www.

Troubleshooting

1. To force a rebuild, delete the checked-out SVN repository on frontend0 and re-run the update script as user www-data:

   cd /var/www/sviki
   rm -r src/sviki
   sudo -u www-data ./sviki-refresh
   

   This will run svn co again, and will run ikiwiki --rebuild --verbose to rebuild all files in /var/www/sviki/www.

2. To trigger an update, run:

   curl https://savannah.gnu.org/update-sviki/update.php
   

   There should be no output. To check for possible apache misconfiguration, check the server's HTTP output headers:

   curl -D /dev/stderr https://savannah.gnu.org/update-sviki/update.php
   

   Outdated: Apache configuration allowed updating only from frontend, frontend0, vcs0, vsc, mgt, mgt0. Other hosts were to get 'HTTP 403 forbidden'; as of 2022-05, it looks like all hosts get 'HTTP 200 OK'.

3. The update.php script writes log messages to apache's error/php log. Look for them in frontend1:/var/log/apache2/error.log:

   [Mon Oct 03 22:14:03.443969 2016] [:error] [pid 4562] [client 208.118.235.72:46847] sviki udpate (start)
   [Mon Oct 03 22:14:03.755440 2016] [:error] [pid 4562] [client 208.118.235.72:46847] sviki update (end) rc =
   

Wiki Initial Setup - New VMs

On VCS/VCS0: no setup is necessary.

On FrontEnd0:

Install IkiWiki + required packages:

apt-get install -y subversion ikiwiki
cpan -i XML/SAX
cpan -i XML::Simple

Create the sviki directories (as root):

cd /var/www
mkdir sviki ; cd sviki
mkdir src www php
mkdir .subversion    # this will be the configdir for svn

The sviki updates run as www-data, setup permissions:

chgrp www-data src www .subversion
chmod g+ws src www .subversion

Copied ikiwiki.setup from (the old) frontend, modified the following:

srcdir => '/var/www/sviki/src/sviki/',
destdir => '/var/www/sviki/www/',
rcs => '',

Created /var/www/sviki/sviki-refresh:

#!/bin/sh

SVNDIR=/var/www/sviki/.subversion/

cd /var/www/sviki/src
if ! test -d sviki ; then
    # first time check out
    svn --config-dir="$SVNDIR" co svn://svn.savannah.gnu.org/administration/trunk/sviki

    # force a rebuild for a new checkout
    method="--rebuild --verbose"
else
    # just an update
    ( cd sviki ; svn --config-dir="$SVNDIR" -q update )
    method="--refresh"
fi
ikiwiki --setup /var/www/sviki/ikiwiki.setup $method

The result is the following:

# ls -la
drwxrwsr-x   3 root www-data  4096 Oct  3 21:11 .subversion
-r--r--r--   1 root root     10047 Oct  3 21:24 ikiwiki.setup
drwxr-xr-x   2 root root      4096 Oct  3 22:10 php
drwxrwsr-x   3 root www-data  4096 Oct  3 21:26 src
-rwxr-xr-x   1 root root       866 Oct  3 22:13 sviki-refresh
drwxrwsr-x 161 root www-data 12288 Oct  3 21:26 www

Running this wiki locally

To edit and review changes locally (before commiting, or submitting a patch to the savannah-hackers-public@gnu.org mailing list), install ikiwiki and checkout the svn repository:

$ sudo apt-get install ikiwiki
$ svn co svn://svn.savannah.gnu.org/administration

The markdown files will be in ./administration/trunk/sviki:

$ ls ./administration/trunk/sviki/
AboutTranslations.mdwn         MigrationFromSourceForge.mdwn
AccessToCVSROOT.mdwn           Mirmon.mdwn
AdminSvnImport.mdwn            MirroringExternalRepository.mdwn
AnnouncementMailingList.mdwn   ModerationAutomation.mdwn
ApprovingSubmission.mdwn       ModifyingGroupType.mdwn
...

Manual wiki rendering

For a one-time (or occasional) wiki rendering, run ikiwiki manually:

$ mkdir /tmp/sviki/
$ ikiwiki --verbose --rebuild \
      ./administration/trunk/sviki \
      /tmp/sviki

The HTML files will be rendered and stored in /tmp/sviki.

--rebuild should be used if rebuilding the wiki in a new directory, to ensure caching of previous rendering are ignored.

The files can be viewed by simply visiting <file:///tmp/sviki/index.html> - however, navigation will be slightly problematic as IkiWiki requires a DirectoryIndex like option which automatically renders index.html as the default HTML file in a directory.

Apache configuration

If you have an apache server running locally, the following configuration mimics the GNU Savannah settings:

Alias /maintenance "/tmp/sviki"
<Directory "/tmp/sviki">
    Options Indexes
    AllowOverride None
    order deny,allow
    allow from all
    DirectoryIndex index.html
</Directory>

Quicker Updates

For quicker rendering, use ikiwiki without --rebuild - will only update the changed files. In this case, it is better to clean the cache once with --clean and --rebuild:

# Clean the cache (just once)
$ ikiwiki --verbose --clean ./administration/trunk/sviki /tmp/sviki

# Rebuild the entire wiki (just once)
$ ikiwiki --verbose --rebuild ./administration/trunk/sviki /tmp/sviki

After any further changes, just update the relevant files (without --rebuild, the default ikiwiki operation is --refresh):

$ ikiwiki --verbose ./administration/trunk/sviki /tmp/sviki

Wiki configuration - current VMs

In the file frontend0:~sviki/ikiwiki.setup. At present, we are using the default (quite minimal) configuration, with no extra plugins.

That file is in RCS, for simplest version control. Anyone in the staff group can edit it, which should be anyone with shell access to the machine. (Savannah administrators are welcome to have shell logins, but Savannah users do not, for obvious security reasons.)

The web server configuration bits are in frontend0:/etc/apache2/sites-available/sv.inc.

How it works - current VMs

Our ikiwiki setup on Savannah is unusual. ikiwiki expects the source repository and wiki to live on the same machine, and thus permissions are simply a matter of having the files be writable to the desired Unix users.

For Savannah, we very much wanted the source repository to be accessible in the standard Savannah ways, that is, through the administration project (http://savannah.gnu.org/projects/administration, which holds all the live Savannah stuff which is checked into version control). That way, access would be controlled just as to any other source repository. We used the Subversion repository in administration as the wiki source partly because it was otherwise unused and partly because Karl (who did all this) likes Subversion.

Well, using a Savannah project means that source repository lives on one host (vcs) while the web-accessible wiki lives on another (frontend). And frontend is where the actual refresh is run, using ~sviki/sviki-refresh. This necessitated the following:

1. A post-commit hook on vcs which ssh's back to frontend to do the refresh. This is in vcs0:/srv/svn/administration/hooks/post-commit.d/sv_ikiwiki_refresh.

2. A pseudo-user sviki on vcs, frontend, and mgt, with its own ssh key to allow communication (in /etc/ssh/authorized_keys/sviki).

3. An empty directory vcs0:/srv/svn/.ssh because ... it's a long story. ssh unconditionally tries to create the user's ~/.ssh directory, even when (as here) it will not be used for anything. On vcs, the post-commit is running as the user who did the commit, which is to say, a normal Savannah user who does not have a real home directory but exists only in the mysql database. The users' homes are arbitrarily set to /srv/svn instead, so that is where the .ssh needs to exist. Thankfully ssh does not initially require specific permissions or ownership on it; as long as it exists, ssh won't try to create it, and then it will never be accessed again (due to the options with which sv_ikiwiki_refresh invokes ssh).

Initial setup

For posterity, here is how the initial setup went.

The first problem was that the previous Zope-based zwiki was cracked in summer 2012, with many pages deleted or junked. We eventually got a restore from the FSF, but it never ran right, and then Zope would not restart after a reboot. We could find no way of extracting the actual text from the huge .fs files hanging around. Sigh.

So we manually saved the HTML pages from the Wayback Machine, at http://archive.org. Fortunately there had been essentially no edits since before the crack, and the pre-crack versions were available. Yay for Wayback.

So we had these HTML files. We used Pandoc http://johnmacfarlane.net/pandoc to do the basic conversion to Markdown, plus some custom Perl massaging to eliminate the Wayback boilerplate, etc. That script is in frontend0:~sviki/wayback-trans, along with the original HTML files. The Zope wiki pages that contained spaces in the names are probably broken links now, by the way.

We had previously decided to use ikiwiki, to avoid ever having to go through the agony of restoring a cracked wiki again. So next, on frontend, we just install the basic Debian package; we don't need the very latest.

apt-get install ikiwiki

Following steps on http://ikiwiki.info/setup/byhand ...

SRCDIR=/usr/local/share/sviki/liveco
DESTDIR=/var/www/sviki

In /etc/apache2/sites-available/sv.inc, comment out zope stuff (since deleted), add:

Alias /maintenance /var/www/sviki

And run:

service apache2 reload

As me, create directories, initial index.mdwn:

ikiwiki --verbose $SRCDIR $DESTDIR --url=http://savannah.gnu.org/maintenance/

Try visiting http://savannah.gnu.org/maintenance/. It works. Amazing.

Dump setup for easier editing:

!! --dumpsetup /usr/local/share/sviki/ikiwiki.setup

Edit setup file for wiki name, etc., then update:

ikiwiki --setup ikiwiki.setup

Skip CGI for now (forever?). People will just edit through their local vc checkout, not on the web.

Create administration's svn repository (can do this anyway):

svn co svn+ssh://karl@svn.savannah.gnu.org/administration
svn mkdir branches tags trunk
svn mkdir trunk/sviki
cd trunk/sviki
svn propset svn:ignore .ikiwiki .
echo hi there > index.mdwn
svn commit

(Full disclosure: the svn co command I actually ran was different from the above standard one, because I have a shell account on vcs and the other machines SavannahArchitecture. But that is an irrelevant detail here.)

Back on frontend, ikiwiki will need to do updates, so:

apt-get install subversion

Switch /usr/local/share/sviki/liveco/ to an svn checkout. Set rcs => 'svn' in ikiwiki.setup; cmdline seemingly has no effect. Update:

 ikiwiki --verbose --setup ikiwiki.setup

Looks like the changes are written in liveco/.ikiwiki; I see https://savannah.gnu.org/maintenance/recentchanges/ now, made from the svn log.

Try a commit from our checked-out repository, then on frontend0:

ikiwiki --setup ikiwiki.setup --refresh

Looks ok. Create user sviki, uid/gid 902, with generated id_rsa, and set up post-commit stuff on vcs, as described above.

Edit frontend0:/etc/default/iptables-rules to allow incoming ssh from vcs. (and from all other Savannah hosts while we're at it.)

Don't bother trying to start zope anymore:

update-rc.d zope2.10 remove

That got rid of these symlinks:

rc0.d/K02zope2.10 -> ../init.d/zope2.10
rc1.d/K02zope2.10 -> ../init.d/zope2.10
rc2.d/K02zope2.10 -> ../init.d/zope2.10
rc3.d/K02zope2.10 -> ../init.d/zope2.10
rc4.d/K02zope2.10 -> ../init.d/zope2.10
rc5.d/K02zope2.10 -> ../init.d/zope2.10
rc6.d/K02zope2.10 -> ../init.d/zope2.10

And we were back in business.