bugMHonArc - Bugs: bug #20142, strip backslash in rfc822 From:...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #20142: strip backslash in rfc822 From: field

Submitter:  Jeff Breidenbach <jab>
Submitted:  Sun 10 Jun 2007 07:19:01 PM UTC
   
 
Category:  Resource Variables Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Behavior
Status:  Fixed Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Operating System:  Linux Perl Version:  v5.8.4
Component Version:  2.6.16 Fixed Release:  2.6.17

Jump to the original submission

Sun 02 Jan 2011 08:44:23 AM UTC, comment #11: 

Tokenize module does not translate quoted pairs
("\" CHAR) when tokenizing.  For now, it will stay that way,
so mhonarc::extract_email_name() modified to unquote chars in
quoted-string and comment token data, if used as the
email name.

Also, while examining tokenizer code, noticed bug in parsing
comments: quoted "\(" or "\)" not taken into account, which
could lead to improper tokenization.  This has been fixed.

Earl Hood <ehood>
Group administrator
Sun 02 Jan 2011 12:07:30 AM UTC, comment #10: 

I should also mention the Tamil problem went away by itself (I think it turned out to be a weird browser bug or something like that)

Jeff Breidenbach <jab>
Sun 02 Jan 2011 12:04:35 AM UTC, comment #9: 

I've added about 500 examples of From fields containing backslashes - all this data is within the last two weeks. Since by necessity this field contains email addresses, I recommend deleting the dataset when finished. It is very easy to regenerate.


Jeff Breidenbach <jab>
Fri 31 Dec 2010 09:10:27 PM UTC, comment #8: 

It's been too long since I last looked at this problem.
Jeff, do you mind providing a sample data set again since
the URL you previous provided is now 404.  Thanks.

Earl Hood <ehood>
Group administrator
Mon 04 May 2009 03:30:03 AM UTC, comment #7: 

I posted a response to your Tamil problem awhile back
to the user's list:
http://www.mhonarc.org/archive/cgi-bin/mesg.cgi?a=mhonarc-users&i=200903230354.n2N3sQD3008778%40gator.earlhood.com

Wrt to the Tamil problem, I could not verify the problem.
Please review my message and follow-up to the list to
either clarify what the problem is or confirm that there
is no problem.

Earl Hood <ehood>
Group administrator
Mon 04 May 2009 03:24:39 AM UTC, comment #6: 

Note to self: check if this patch is a possible cause of the $SUBJECT$ $SUSBJECTNA$ weirdness with Tamil.


http://www.mail-archive.com/mhonarc-users@mhonarc.org/msg01387.html

Jeff Breidenbach <jab>
Wed 10 Oct 2007 04:35:31 AM UTC, comment #5: 

--- /var/tmp/mhutil.pl  2007-10-09 20:30:36.000000000 -0700
+++ /usr/share/mhonarc/mhutil.pl        2007-10-09 21:32:05.000000000 -0700
@@ -176,7 +176,8 @@
     foreach $tok (@tokens) {
        next  if $skip;
        if ($tok =~ /^"/) {   # Quoted string
           $tok =~ s/^"//;  $tok =~ s/"$//;
+           $tok =~ s/\\(.)/$1/g; $tok =~ s/\\\"//g;
            return $tok;
        }
        if ($tok =~ /^\(/) {  # Comment

Jeff Breidenbach <jab>
Wed 08 Aug 2007 05:32:36 AM UTC, comment #4: 

I've placed a sample of raw messages at the following location. It is encrypted to the mhonarc signing key and is representative of production traffic. Maybe the size is a little bit of overkill for this particular problem, but the dataset might be useful for other
problem cases as well. Once the dataset is received, yell and I'll delete it. To find relevent messages in this dataset, use the grep command described in one of the other comments.

  http://www.mail-archive.com/mail-sample.bz2.gpg

Jeff Breidenbach <jab>
Sat 04 Aug 2007 06:51:55 PM UTC, comment #3: 

Do you have any original messages that illustrate
the problem?

I want to make sure thay any work I do will address
the cases you are encountering.

Thanks.

Earl Hood <ehood>
Group administrator
Mon 30 Jul 2007 02:41:47 AM UTC, comment #2: 

This is super useful, and really comes into play for us on $FROMNAME$.

Everything else can essentially stay the same. In particular, $SUBJECT$ will quite often have unescaped backslashes, for example, a message talking about Windows software might contain "C:\Progam Files\whatever"

So I admit a hack to fix \( \) \" on $FROMNAME$ isn't pretty, but it would be super useful and if I had a pointer on where to insert it, I'd be happy to attempt this patch myself.

Jeff Breidenbach <jab>
Tue 26 Jun 2007 11:37:19 PM UTC, comment #1: 

RFC 822 does support the use of '\' to escape characters.

I'll need to do some testing to see if the RFC-822 parser
has a bug or if it is mhonarc.  Mhonarc historically has
tried to do some short-cuts in some cases to avoid full
rfc-822 parsing, but later versions starting leveraging
the 822 parser to be more robust (albeit with a hit on
performance).

Earl Hood <ehood>
Group administrator
Sun 10 Jun 2007 07:19:01 PM UTC, original submission:  

It is not uncommon to have an escaped character in the From: field of an email header. I've noticed this mainly with parentheses and quotation marks, e.g. \( \) \". Out of the last 75000 mails, approximately one in 500 have this characteristic. Mhonarc leaves the backslash in and produces messages that look a little funnny. Especially when one person posts a lot to a list, and each time his or her name is littered with extra backslashes in the index pages.

I looked over RFC822 and had a tough time deciding whether this practive is legal or not, but I can say it is fairly common and I think it makes sense to strip out backslashes in the FROMNAME resource variable. Or maybe only do it for our friends \" \( and \)

I also looked at Subject: lines, and in those cases backslashes are used all over the place in a meaninful way, so probably should not be stripped. I'm happy to suppy additional data.
 
http://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg121886.html

gen7:/var/mail# cat archive2 | grep ^From: | wc -l
73667
gen7:/var/mail# cat archive2 | grep ^From: | grep '\\' | wc -l
164

Jeff Breidenbach <jab>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ehood (Posted a comment)
  • -email is unavailable- added by jab (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-01-09 ehood Open/ClosedOpen Closed
    2011-01-09 ehood StatusReady For Test Fixed
        Fixed Release 2.6.17
    2011-01-02 ehood StatusNeed Info Ready For Test
    2011-01-02 ehood Attached File#22335 Removed
    2011-01-02 jab Attached File- Added data.gz, #22335
    2010-12-31 ehood StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-aa77.
    Corresponding source code