Mon 13 Nov 2006 10:13:30 AM UTC, comment #14:
THIS ITEM WAS REASSIGNED TO PATCH #5547. Please, do not post any new comments to this item.
|
Mon 13 Nov 2006 05:54:31 AM UTC, comment #13:
The patch works fine for me, thanks.
|
Wed 08 Nov 2006 09:30:41 PM UTC, comment #12:
Updated patch for current CVS, please test and report back if it works for you
|
Sat 07 Oct 2006 03:40:57 PM UTC, comment #11:
ping
|
Thu 14 Sep 2006 05:42:10 PM UTC, comment #10:
Applied the language detection to CVS, for the display problems
I will leave this bug open.
|
Tue 12 Sep 2006 10:37:40 PM UTC, comment #9:
To fix conversion problems enclose the string variable, which
currently outputs bad chars, with Charset.to_utf8 ( )
|
Tue 12 Sep 2006 06:45:08 PM UTC, comment #8:
I updated two places of code with an UTF8-conv function
for the popups. Please test if this is enough or if
there are other places where incorrect chars are displayed.
Are search result filenames displayed well?
|
Tue 12 Sep 2006 09:01:49 AM UTC, comment #7:
The problem about tip is the filename are not converted to UTF-8 encoding properly when it displayed in the tip window.
For example, in the src/daemon/driver/driverCommands.ml:
(Http_server.html_real_escaped (Filename.basename impl.impl_shared_codedname))
...
(Filename.basename impl.impl_shared_codedname)
The codedname may not in UTF-8 if the system default encoding is not UTF-8, a conversion is needed before print out the filename. I found the "shorten" method did the conversion properly, so it will be a workaround by calling shorten for filename before write to buffer.
For example:
(shorten (Http_server.html_real_escaped (Filename.basename impl.impl_shared_codedname)) 999)
...
(shorten (Filename.basename impl.impl_shared_codedname) !!max_name_len)
|
Tue 12 Sep 2006 07:09:22 AM UTC, comment #6:
Hi
I verified that your fix can resolve the problem, thank you very much.
While I found another encoding issue in the tips window when mouse move over the filename. Shall I open a new bug for that one?
Thanks
|
Mon 11 Sep 2006 08:15:31 AM UTC, comment #5:
Please test the updated patch, it converts all string to uppercase before analyzing them.
|
Mon 11 Sep 2006 03:39:06 AM UTC, comment #4:
I have tested your patch but unfortunately it doesn't works.
The problem is caused by following line:
if String.sub s 0 2 = "ZH" && String.length s > 4 then
The "ZH" is used to match the locale but the locale string
starts with lowercase in most case. It will always fall into
default when the locale starts with language id in lowercase.
Since it's a convention that locale starts with lowercase,
I suggest to always use the lowercase for language id.
|
Thu 07 Sep 2006 11:38:06 AM UTC, comment #3:
"The normalize_language method should also be changed since it will trim the "_CC" in the locale string."
Example: for the German language these values exist:
de_DE (Germany)
de_AT (Austria)
de_CH (Switzerland)
de_LU (Luxemburg)
de_BE (Belgium - has German minority)
for all of them using "de" cut by the normalize function is perfectly ok, otherwise charset_list_from_language would be a
very long function.
In the case of "zh_CN" and "zh_TW" its a different story,
I changed some code so zh_CN and zh_TW are specially treated.
Please try the attached patch and report if it works for you.
|
Thu 07 Sep 2006 06:15:50 AM UTC, comment #2:
--Buildinfo--
Version: MLNet Multi-Network p2p client version 2.7.7
Networks: Global Shares Gnutella G2 FileTP BitTorrent Donkey (SUI)
Ocaml version: 3.09.2 - C compiler version: 4.1.2 - C++ compiler version: 4.1.2
Build on: Linux i686 2.6.17.11-p4 (little endian) with glibc 2.3.6
Configure args: /usr/local/share/config.site /usr/local/etc/config.site
Features: threads zlib-1.2.3 bzip2-1.0.3 gd(jpg/png-1.2.8) iconv no-magic no-check-bounds
--Runinfo--
User: admin (PW Protected) - uptime: 1d 37m 50s
Enabled nets: Donkey Overnet BitTorrent FileTP
Server usage: enabled
Geoip: enabled, GeoLite data created by MaxMind, available from http://maxmind.com/
IP blocking: no blocking list loaded
System info: Linux lily 2.6.17.11-c3 #1 Mon Sep 4 22:53:39 CST 2006 i686
language: ZH - locale: GB2312 - UTC offset: +0800
max_string_length: 16777211 - word_size: 32 - max_array_length: 4194303
max file descriptors: 1024 - max useable file size: 2^63-1 bits (do the maths ;-p)
NOTE:
This is built by myself from the debian patched source with the workaround.
|
Wed 06 Sep 2006 12:29:47 PM UTC, comment #1:
Please post output of "buildinfo" and "runinfo"
|
Wed 06 Sep 2006 03:42:00 AM UTC, original submission:
THIS ITEM WAS REASSIGNED TO PATCH #5547. Hi
I found a bug that the charset is incorrect in some environment. For example, when the locale is set to "zh_CN" or "zh_CN.GBK" for Simplified Chinese, the mldonkey should use charset GB2312 or GBK charset to convert filenames, but it indeed using the BIG5 charset to convert filenames. This will result in filenames with incorrect characters displayed on webpages.
I found the root cause of this problem is in src/utils/lib/charset.ml:
The charsets list are always set to "BIG5, ..., GBK" when the language is "zh"
| "SH"
| "SR" -> li := central_european :: cyrillic ::!li
| "ZH" -> li := chinese_traditional :: chinese_simplified :: !li
| "BE"
But please note the charset BIG5 for zh_TW is totally different from GBK for zh_CN, it's not a good idea to assume charsets for same language should always be compatible with rests.
To workaround this problem, we can remove the chinese_traditional from above code, but it's bad for poeple who using Traditional Chinese.
My suggestion is to match the locale in ll_CC format, while not only the language tag. For example:
"zh_CN" -> li := chinese_simplified :: !li
"zh_TW" -> li := chinese_traditional :: !li
The normalize_language method should also be changed since it will trim the "_CC" in the locale string.
A better solution for this issue may be that try to use the encoding for current locale provided by glibc, such as nl_langinfo(), it will returns the charset for current locale, same as result of the command: locale charmap
Thanks
|