bugMHonArc - Bugs: bug #35723, Chinese GB2312 character set is...


You are not allowed to post comments on this tracker with your current authentication level.

bug #35723: Chinese GB2312 character set is missing the newer GB18030 characters

Submitter:  Silas S. Brown <ssb22>
Submitted:  Mon 05 Mar 2012 11:07:59 AM UTC
Category:  Character Sets Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Operating System:  All Perl Version:  5.10.0
Component Version:  1.3 Fixed Release: 

Mon 05 Mar 2012 11:07:59 AM UTC, original submission:  

Many Chinese systems send mail with a charset label of GB2312 but the actual charset is GBK or GB18030, which are both supersets of GB2312.  The GB2312 label is designed to trick older Chinese software into displaying as many characters as it can and ignoring the rest, but newer Chinese software treats a "GB2312" label as GB18030.  However Mhonarc's lib/perl5/site_perl/5.10.0/MHonArc/UTF8/GB2312.pm and lib/perl5/site_perl/5.10.0/MHonArc/CharEnt/GB2312.pm cover only the original GB2312 standard, not the full GB18030.

Attached is a Python script to generate the relevant files for GB18030 and call it GB2312.  Additionally it would be nice if Mhonarc would recognise a "charset=GBK" or "charset=GB18030" header (search for gb2312 in Char.pm, CharEnt.pm and UTF8/MhaEncode.pm and add the other mappings).

It seems MHonArc sometimes bypasses these tables, so I also put

    if ($from_enc eq 'gb2312') {
    $from_enc = 'gb18030';

into MHonArc/Encode.pm's _encode_from_to and _unimap_from_to, and

    if ($charset eq 'gb2312') {
    $charset = 'gb18030';

into UTF8/Encode.pm.

Silas S. Brown <ssb22>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #25266:  makeGB.py added by ssb22 (2KiB - text/x-python)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by ssb22 (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-03-05 ssb22 Attached File- Added makeGB.py, #25266

    Back to the top

    Powered by Savane 3.13-6ae7.
    Corresponding source code