Sun 02 Jan 2011 08:44:23 AM UTC, comment #11:
Tokenize module does not translate quoted pairs
("\" CHAR) when tokenizing. For now, it will stay that way,
so mhonarc::extract_email_name() modified to unquote chars in
quoted-string and comment token data, if used as the
email name.
Also, while examining tokenizer code, noticed bug in parsing
comments: quoted "\(" or "\)" not taken into account, which
could lead to improper tokenization. This has been fixed.
|
Sun 02 Jan 2011 12:07:30 AM UTC, comment #10:
I should also mention the Tamil problem went away by itself (I think it turned out to be a weird browser bug or something like that)
|
Sun 02 Jan 2011 12:04:35 AM UTC, comment #9:
I've added about 500 examples of From fields containing backslashes - all this data is within the last two weeks. Since by necessity this field contains email addresses, I recommend deleting the dataset when finished. It is very easy to regenerate.
|
Fri 31 Dec 2010 09:10:27 PM UTC, comment #8:
It's been too long since I last looked at this problem.
Jeff, do you mind providing a sample data set again since
the URL you previous provided is now 404. Thanks.
|
Mon 04 May 2009 03:30:03 AM UTC, comment #7:
I posted a response to your Tamil problem awhile back
to the user's list:
http://www.mhonarc.org/archive/cgi-bin/mesg.cgi?a=mhonarc-users&i=200903230354.n2N3sQD3008778%40gator.earlhood.com
Wrt to the Tamil problem, I could not verify the problem.
Please review my message and follow-up to the list to
either clarify what the problem is or confirm that there
is no problem.
|
Mon 04 May 2009 03:24:39 AM UTC, comment #6:
Note to self: check if this patch is a possible cause of the $SUBJECT$ $SUSBJECTNA$ weirdness with Tamil.
http://www.mail-archive.com/mhonarc-users@mhonarc.org/msg01387.html
|
Wed 10 Oct 2007 04:35:31 AM UTC, comment #5:
--- /var/tmp/mhutil.pl 2007-10-09 20:30:36.000000000 -0700
+++ /usr/share/mhonarc/mhutil.pl 2007-10-09 21:32:05.000000000 -0700
@@ -176,7 +176,8 @@
foreach $tok (@tokens) {
next if $skip;
if ($tok =~ /^"/) { # Quoted string
$tok =~ s/^"//; $tok =~ s/"$//;
+ $tok =~ s/\\(.)/$1/g; $tok =~ s/\\\"//g;
return $tok;
}
if ($tok =~ /^\(/) { # Comment
|
Wed 08 Aug 2007 05:32:36 AM UTC, comment #4:
I've placed a sample of raw messages at the following location. It is encrypted to the mhonarc signing key and is representative of production traffic. Maybe the size is a little bit of overkill for this particular problem, but the dataset might be useful for other
problem cases as well. Once the dataset is received, yell and I'll delete it. To find relevent messages in this dataset, use the grep command described in one of the other comments.
http://www.mail-archive.com/mail-sample.bz2.gpg
|
Sat 04 Aug 2007 06:51:55 PM UTC, comment #3:
Do you have any original messages that illustrate
the problem?
I want to make sure thay any work I do will address
the cases you are encountering.
Thanks.
|
Mon 30 Jul 2007 02:41:47 AM UTC, comment #2:
This is super useful, and really comes into play for us on $FROMNAME$.
Everything else can essentially stay the same. In particular, $SUBJECT$ will quite often have unescaped backslashes, for example, a message talking about Windows software might contain "C:\Progam Files\whatever"
So I admit a hack to fix \( \) \" on $FROMNAME$ isn't pretty, but it would be super useful and if I had a pointer on where to insert it, I'd be happy to attempt this patch myself.
|
Tue 26 Jun 2007 11:37:19 PM UTC, comment #1:
RFC 822 does support the use of '\' to escape characters.
I'll need to do some testing to see if the RFC-822 parser
has a bug or if it is mhonarc. Mhonarc historically has
tried to do some short-cuts in some cases to avoid full
rfc-822 parsing, but later versions starting leveraging
the 822 parser to be more robust (albeit with a hit on
performance).
|
Sun 10 Jun 2007 07:19:01 PM UTC, original submission:
It is not uncommon to have an escaped character in the From: field of an email header. I've noticed this mainly with parentheses and quotation marks, e.g. \( \) \". Out of the last 75000 mails, approximately one in 500 have this characteristic. Mhonarc leaves the backslash in and produces messages that look a little funnny. Especially when one person posts a lot to a list, and each time his or her name is littered with extra backslashes in the index pages.
I looked over RFC822 and had a tough time deciding whether this practive is legal or not, but I can say it is fairly common and I think it makes sense to strip out backslashes in the FROMNAME resource variable. Or maybe only do it for our friends \" \( and \)
I also looked at Subject: lines, and in those cases backslashes are used all over the place in a meaninful way, so probably should not be stripped. I'm happy to suppy additional data.
http://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg121886.html
gen7:/var/mail# cat archive2 | grep ^From: | wc -l
73667
gen7:/var/mail# cat archive2 | grep ^From: | grep '\\' | wc -l
164
|