Sat 01 Jan 2011 11:29:10 PM UTC, comment #10:
Yes, solution implemented is inline with comment #2. The decoding
phase is done after the header is parsed and the $fields
hash is populated. The decoding phase operates on the
values in the $fields hash.
|
Sat 01 Jan 2011 11:20:08 PM UTC, comment #9:
As far as I understand you, you implemented the solution that I recommended a long ago in comment #2 ? I agree that that's the right way to go, not my patch.
Looking forward to a new stable release, it's been soooo long :-)
|
Sat 01 Jan 2011 10:35:10 PM UTC, comment #8:
(mhtxthtml.pl, 2.39)
Text-encoding/decoding of non-ASCII encoded
header text is now a post-processing step after raw parsing
of header is done. This way, the decoding function works
against an unfolded string, so handling of white space between
two encoded atoms should be handled correctly.
|
Sat 01 Jan 2011 07:21:42 PM UTC, comment #7:
Thanks for resubmitting the patch. However, I'm going to
look at doing a different solution, where the decoding
is done as a post-process after the basic header parsing.
The header parsing does unfold lines, so it would
simplify things if MIME decoding is done against the $fields
hash after $fields is populated.
This should make things simpler, and avoid the use of
$' (and similiar vars) since the use of those can introduce
extra overhead in Perl's regex processing.
Also, it should avoid unneeded space dropping in the
extremely rare case where "=?" and "?=" may exist
in a message header, but are not part a MIME encoding
string.
|
Sat 01 Jan 2011 03:01:30 PM UTC, comment #6:
Sent you an e-mail, using gmail's web frontend as mua.
The subject is "mhonarc bug 20074 test fóobar1 fóobar2 fóobar3 fóobar4 fóobar5"
Gmail encodes it as
Subject: =?UTF-8?Q?mhonarc_bug_20074_test_f=C3=B3obar1_f=C3=B3obar2_f=C3=B3obar3_f?=
=?UTF-8?Q?=C3=B3obar4_f=C3=B3obar5?=
Mhonarc decodes this with a spurious space after "f" in the 4th foobar.
It's extremely easy to construct similar examples. Take basically any mail client, and type a subject line that is long enough, and contains a couple of non-ascii characters here and there.
|
Sat 01 Jan 2011 02:43:58 PM UTC, comment #5:
Oh, the patch was the other way around :) Then I mean to say we didn't see any missing space (no new bug), nor any spurious one (the old one is fixed) :)
|
Sat 01 Jan 2011 02:39:47 PM UTC, comment #4:
Oops, I've totally missed the e-mail notification about your last comment, a year and a half ago :(
Patch re-attached.
We've been using this patch ever since I created it (3.5y ago) on lists.uhulinux.hu, and haven't seen any spurious space inserted (no new bug), but it fixed a lot of missing spaces.
I can't have any example right now (and I no longer have access to those raw mailboxes, neither can I regenerate the archive without the patch), but I do remember that many messages suffered from the problem, something along the lines of 1 out of every 10 or 20 (given that the language of the mailing lists uses accents quite often)...
|
Sun 03 May 2009 09:52:23 PM UTC, comment #3:
First, you could PLEASE resubmit your latest patch.
I had a brain fart and accidently deleted it from this
bug item.
Second, would you provide example data that illustrates
the problem?
One thing is to modify behavior according to what is
seen in the "wild" versus what the standard state about
this. The core mail specs and the MIME specs come into
play about how to deal with line folding and extra spaces,
and I would have refresh my memory on how things should
work.
|
Tue 05 Jun 2007 11:26:13 AM UTC, comment #2:
I think the correct and elegant solution would be to first concatenate raw header fields that belong together, and then decode the result with MAILdecode_1522_str.
MAILdecode_1522_str already takes care of dropping the space or tab if it's surrounded by two =?...?= blocks, so it would drop the leading space or tab of a physical mailbox line if and only if it should really be dropped.
|
Tue 05 Jun 2007 11:09:05 AM UTC, comment #1:
I update a patch. Now it seems to me (looking at more mailbox files) that the space should be dropped only if the previous line ends with a rfc1522-encoded component and the new line starts with another rfc1522 block.
(file #12955)
|
Mon 04 Jun 2007 06:17:24 PM UTC, original submission:
I've just upgraded from an ancient sympa to 5.3 and this one runs mhonarc with different arguments. The relevant part is:
<TextEncode>
utf-8; MHonArc::UTF8::to_utf8; MHonArc/UTF8.pm
</TextEncode>
I've just noticed a bug that we haven't faced before: in plenty of subjects an extra space or tab character is inserted somewhere in the middle of a word. For example a mail with subject "Blah Bláh Foobar" that was correctly displayed so far might now be displayed as "Blah Bláh Foob ar".
This happens when mail clients encode the long accented subject using =?x?Q?...?= or =?x?B?...?= notation. In this case they can wrap into a new physical line everywhere, even in the middle of a word. For example, the raw mailbox file might look like this:
Subject: =?utf-8?Q?Blah_Bl=C3=A1h_Foob?=
=?utf-8?Q?ar?=
In this case mhonarc preserves the space or tab at the beginning of the new line as if it was part of the subject, though it should drop it. If there is actually a space then mail clients explicitely mention it within one of the =?.?.?...?= blocks.
Ugly but forks-for-me patch attached, I hope it doesn't have negative side effects. I haven't read and understood all the details of rfc 1522 and friends, this patch just happens to work for this situation. I bet you'll have a better solution :-)
|