MHonArc - Bugs: bug #35723, Chinese GB2312 character set is...
You are not allowed to post comments on this tracker with your current authentication level.
bug #35723: Chinese GB2312 character set is missing the newer GB18030 characters
Submitter: | Silas S. Brown <ssb22> | ||
Submitted: | Mon 05 Mar 2012 11:07:59 AM UTC | ||
Category: | Character Sets | Severity: | 3 - Normal |
Priority: | 5 - Normal | Item Group: | None |
Status: | None | Privacy: | Public |
Assigned to: | None | Open/Closed: | Open |
Operating System: | All | Perl Version: | 5.10.0 |
Component Version: | 1.3 | Fixed Release: |
Attached Files
Depends on the following items: None found
Items that depend on this one: None found
Carbon-Copy List
There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.
Follows 1 latest change.
Date | Changed by | Updated Field | Previous Value | => | Replaced by |
---|---|---|---|---|---|
2012-03-05 | ssb22 | Attached File | - | ![]() |
Added makeGB.py, #25266 |
Many Chinese systems send mail with a charset label of GB2312 but the actual charset is GBK or GB18030, which are both supersets of GB2312. The GB2312 label is designed to trick older Chinese software into displaying as many characters as it can and ignoring the rest, but newer Chinese software treats a "GB2312" label as GB18030. However Mhonarc's lib/perl5/site_perl/5.10.0/MHonArc/UTF8/GB2312.pm and lib/perl5/site_perl/5.10.0/MHonArc/CharEnt/GB2312.pm cover only the original GB2312 standard, not the full GB18030.
Attached is a Python script to generate the relevant files for GB18030 and call it GB2312. Additionally it would be nice if Mhonarc would recognise a "charset=GBK" or "charset=GB18030" header (search for gb2312 in Char.pm, CharEnt.pm and UTF8/MhaEncode.pm and add the other mappings).
It seems MHonArc sometimes bypasses these tables, so I also put
if ($from_enc eq 'gb2312') {
$from_enc = 'gb18030';
}
into MHonArc/Encode.pm's _encode_from_to and _unimap_from_to, and
if ($charset eq 'gb2312') {
$charset = 'gb18030';
}
into UTF8/Encode.pm.