The following is the GNU All-permissive License as recommended in https://www.gnu.org/licenses/license-recommendations.en.html

Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright notice and this notice are preserved. This file is offered as-is, without any warranty.

Contributions are welcome. See https://savannah.gnu.org/maintenance/fsf/.

long lines break awstats

Description:

Cron <www-data@termite2p> /usr/local/bin/awstats-update |& grep -v "awstats lock is busy"

The fast solution is run the following command on log files, to remove overly long lines:

sed -i -e '/^.\{1024\}./d' /var/log/vhosts/209.51.188.FOO/apache-access.log

This error may be caused by long log lines that get split by syslog-ng before sending them over UDP to termite.fsf.org. syslog-ng should support setting longer lines as an option, but because we're sending them over UDP, we don't want them larger than a UDP packet. Otherwise they could get mixed up on the receiving host.

This issue did not obviously impact stats on the last two occurrences. If we see irregularities in the stats when this is a problem until logs are rotated, then we should update this page and make it a process to deal with this and to set up a workaround or solution.

The right thing to do here in the long term is to find a way to tell Awstats to ignore invalid log lines. "SkipFiles" might be the right directive, but the second record in a broken line can always start with arbitrary text, so a regex would have to match an absence of an ip address.

Check the awstats history for the affected host. if stats are missing or far less than usual over one or more days, then you'll have to re-play old logs with a backup of the awstats data file before it broke:

To do this, turn off the awstats cron job until you're done fixing the issue. Then fetch the awstats file and log files if they're too old to be still on termite. Then put the data file in the usual place, then change the path to the log files that you want to play. Cat the old logs into one log file. Edit the log files so they don't contain the corrupted data, then run the command to update the data file. Check the web page for the stat history.