bugMapOSMatic - Bugs: bug #28603, Non-latin characters are removed...


bug #28603: Non-latin characters are removed from generated file names

Submitted by:  David Mentré <dmentre>
Submitted on:  Thu 14 Jan 2010 08:17:29 AM UTC  
Category: i18nSeverity: 3 - Normal
Priority: 1 - LaterStatus: Confirmed
Privacy: PublicAssigned to: None
Open/Closed: OpenRelease: None

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Thu 02 Dec 2010 10:25:54 PM UTC, comment #1:

I don't really see Punycode as being a viable solution here. It does generate ASCII, but it doesn't add anything in terms of readable information to the filename:

>>> u'007864_2010-01-14_07-23_دمشق.png'.encode('punycode')


I would simply not strip non-ASCII characters. The filesystem supports it anyway. I'll give it a try on my local instance and see how it goes in terms of browser interaction, etc.

Maxime Petazzoni <mpetazzoni>
Project Administrator
Thu 14 Jan 2010 08:17:29 AM UTC, original submission:

From Bassem Jarkas:

If we use non-latin characters for naming the city, the name will be missing in the result files, and only the id and date will appears!

For example:
1- City: دمشق
URL: http://www.maposmatic.org/jobs/7864
File name: 007864_2010-01-14_07-23_.png
The correct name should be: 007864_2010-01-14_07-23_دمشق.png

2- City: Микрорайоны
URL: http://www.maposmatic.org/jobs/7863
File name: 007863_2010-01-14_06-48_.png
The correct name should be: 007863_2010-01-14_06-48_Микрорайоны.png

Things to take account when designing a solution: security issues, at both browser level (the name is used in the provided URL) and file system level (impact of non-latin encoding?).

Relevant dev@ discussion (with ideas):

David Mentré <dmentre>
Project Administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by mpetazzoni (Updated the item)
  • -unavailable- added by dmentre (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 30 Oct 2010 12:16:30 AM UTCmpetazzoniSeverity4 - Important=>3 - Normal
      Priority5 - Normal=>1 - Later
    Tue 11 May 2010 03:05:09 PM UTCmpetazzoniCategoryocitysmap=>i18n

    Back to the top

    Powered by Savane 3.1-cleanup1