bugMHonArc - Bugs: bug #35723, Chinese GB2312 character set is...

 
 

You are not allowed to post comments on this tracker with your current authentification level.

bug #35723: Chinese GB2312 character set is missing the newer GB18030 characters

Submitted by:  Silas S. Brown <ssb22>
Submitted on:  Mon 05 Mar 2012 11:07:59 AM UTC  
 
Category: Character SetsSeverity: 3 - Normal
Priority: 5 - NormalItem Group: None
Status: NonePrivacy: Public
Assigned to: NoneOpen/Closed: Open
Operating System: AllPerl Version: 5.10.0
Component Version: 1.3Fixed Release: 

Mon 05 Mar 2012 11:07:59 AM UTC, original submission:

Many Chinese systems send mail with a charset label of GB2312 but the actual charset is GBK or GB18030, which are both supersets of GB2312. The GB2312 label is designed to trick older Chinese software into displaying as many characters as it can and ignoring the rest, but newer Chinese software treats a "GB2312" label as GB18030. However Mhonarc's lib/perl5/site_perl/5.10.0/MHonArc/UTF8/GB2312.pm and lib/perl5/site_perl/5.10.0/MHonArc/CharEnt/GB2312.pm cover only the original GB2312 standard, not the full GB18030.

Attached is a Python script to generate the relevant files for GB18030 and call it GB2312. Additionally it would be nice if Mhonarc would recognise a "charset=GBK" or "charset=GB18030" header (search for gb2312 in Char.pm, CharEnt.pm and UTF8/MhaEncode.pm and add the other mappings).

It seems MHonArc sometimes bypasses these tables, so I also put

if ($from_enc eq 'gb2312') {
$from_enc = 'gb18030';
}

into MHonArc/Encode.pm's _encode_from_to and _unimap_from_to, and

if ($charset eq 'gb2312') {
$charset = 'gb18030';
}

into UTF8/Encode.pm.

Silas S. Brown <ssb22>

 

Attached Files
file #25266:  makeGB.py added by ssb22 (2kB - text/x-python)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by ssb22 (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follows 1 latest change.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 05 Mar 2012 11:07:59 AM UTCssb22Attached File-=>Added makeGB.py, #25266

    Back to the top


    Powered by Savane 3.1-cleanup