bugWeeChat - Bugs: bug #38191, perl encoding hell

 
 

bug #38191: perl encoding hell

Submitter:  None
Submitted:  Tue 29 Jan 2013 10:23:24 AM UTC
   
 
Category:  script API plugins Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Originator Name:  Originator Email:  * -email is unavailable-
Open/Closed:  Open Release:  * 0.3.0
IRC nick:  Nei
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 29 Jan 2013 06:38:36 PM UTC, comment #2: 

oh, and here is a brute force patch which declares everything as utf8 (and therefore obviously wrong) http://anti.teamidiot.de/static/nei/*/Code/WeeChat/always_utf8.diff

Anonymous
Tue 29 Jan 2013 10:29:06 AM UTC, comment #1: 

from the api docs derives the following list of possibly non-utf8 data:

utf8* = iconv_to_internal charset, byte*string
byte* = iconv_from_internal charset, utf8*string
hook_process(_hashtable) callback (data, command, rc, byte*out, byte*err) [maybe the out and err are not utf8?]

maybe some of the following hook_signal s:
,irc_in_ [?]
,irc_raw_in_ [?]
,irc_out(1)_ [?]

following hook_modifier s:
Plugin |        Modifier |         Modifier data |         String |         Output
[byte*string,utf8*out[?]] charset | charset_decode | plugin.buffer_name | any string | string decoded from charset found for plugin/buffer to UTF-8
[utf8*string,byte*out[?]] charset | charset_encode | plugin.buffer_name | any string | string encoded from UTF-8 to charset found for plugin/buffer
[byte] irc | irc_in_xxx (1) | server name | content of message received from IRC server (before charset decoding) | new content of message
[should be fine] irc | irc_in2_xxx (1) | server name | content of message received from IRC server (after charset decoding) | new content of message
[?] irc | irc_out1_xxx (1)| server name | content of message about to be sent to IRC server (before automatic split to fit in 512 bytes) | new content of message
[should be bytes at this stage?] irc | irc_out_xxx (1) | server name | content of message about to be sent to IRC server (after automatic split to fit in 512 bytes) | new content of message

--> hook_modifier_exec is affected the same as hook_modifier

Anonymous
Tue 29 Jan 2013 10:23:24 AM UTC, original submission:  

it is annoying to use unicode from perl scripts because the weechat api presents everything as byte[]

here's a sample script demonstrating the problem:


weechat::register('utf8', 'Nei <anti.teamidiot.de>', '0.0', 'GPL3', 'demo utf8 bug in perl api (/utf8a and /utf8b)', '', '') || return;
weechat::hook_command('utf8a', 'bug visible when error prefix is set to unicode char', '', '', '', 'utf8a_cmd', '');
weechat::hook_command('utf8b', 'bug visible when argument is unicode char', '[text]', '', '', 'utf8b_cmd', '');

sub utf8a_cmd {
        weechat::print('', weechat::prefix('error')."bug when prefix_error is unicode: x{4f60}x{597d} (hello in chinese)");
        weechat::WEECHAT_RC_OK
}

sub utf8b_cmd {
        weechat::print('', "bug when argument is unicode: x{4f60}x{597d} (hello in chinese). your argument was:".$_[2]);
        weechat::WEECHAT_RC_OK
}


Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #27341:  utf8.pl added by None (686B - application/x-perl - so much for "verbatim". script without missing letters attached)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-01-29 None Attached File- Added utf8.pl, #27341

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code