bugMapOSMatic - Bugs: bug #36647, Add Portuguese (pt-PT) translation...

 
 

bug #36647: Add Portuguese (pt-PT) translation and localization for portuguese speaking countries

Submitter:  RPome <rpome>
Submitted:  Wed 13 Jun 2012 02:23:00 PM UTC
   
 
Category:  i18n Severity:  3 - Normal
Priority:  * 5 - Normal Status:  None
Privacy:  Public Assigned to:  mpetazzoni
Open/Closed:  Open Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 09 Jun 2015 07:37:17 PM UTC, comment #4: 

Something gone wrong. Here it is the second file:


http://git.savannah.gnu.org/cgit/maposmatic/ocitysmap.git/tree/ocitysmap/i18n.py




# -*- coding: utf-8; mode: Python -*-

# ocitysmap, city map and street index generator from OpenStreetMap data
# Copyright (C) 2010  David Decotigny
# Copyright (C) 2010  Frédéric Lehobey
# Copyright (C) 2010  Pierre Mauduit
# Copyright (C) 2010  David Mentré
# Copyright (C) 2010  Maxime Petazzoni
# Copyright (C) 2010  Thomas Petazzoni
# Copyright (C) 2010  Gaël Utard

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Affero General Public License for more details.

# You should have received a copy of the GNU Affero General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import re
import gettext

def _install_language(language, locale_path):
    t = gettext.translation(domain='ocitysmap',
                            localedir=locale_path,
                            languages=[language],
                            fallback=True)
    t.install(unicode=True)

class i18n:
    """Functions needed to be implemented for a new language.
       See i18n_fr_FR_UTF8 below for an example. """
    def language_code(self):
        pass

    def user_readable_street(self, name):
        pass

    def first_letter_equal(self, a, b):
        pass

    def isrtl(self):
        return False

    def upper_unaccent_string(self, s):
        return s.upper()

class i18n_template_code_CODE(i18n):
    def __init__(self, language, locale_path):
        """Install the _() function for the chosen locale other
           object initialisation"""

        # It's important to convert to str() here because the map_language
        # value coming from the database is Unicode, but setlocale() needs a
        # non-unicode string as the locale name, otherwise it thinks it's a
        # locale tuple.
        self.language = str(language)
        _install_language(language, locale_path)

    def language_code(self):
        """returns the language code of the specific language
           supported, e.g. fr_FR.UTF-8"""
        return self.language

    def user_readable_street(self, name):
        """ transforms a street name into a suitable form for
            the map index, e.g. Paris (Rue de) for French"""
        return name

    def first_letter_equal(self, a, b):
        """returns True if the letters a and b are equal in the map index,
           e.g. É and E are equals in French map index"""
        return a == b

    def isrtl(self):
        return False


class i18n_fr_generic(i18n):
    APPELLATIONS = [ u"Accès", u"Allée", u"Allées", u"Autoroute", u"Avenue",
                     u"Avenues", u"Barrage",
                     u"Boulevard", u"Carrefour", u"Chaussée", u"Chemin",
                     u"Chemin rural",
                     u"Cheminement", u"Cale", u"Cales", u"Cavée", u"Cité",
                     u"Clos", u"Coin", u"Côte", u"Cour", u"Cours", u"Descente",
                     u"Degré", u"Escalier",
                     u"Escaliers", u"Esplanade", u"Funiculaire",
                     u"Giratoire", u"Hameau", u"Impasse", u"Jardin",
                     u"Jardins", u"Liaison", u"Lotissement", u"Mail",
                     u"Montée", u"Môle",
                     u"Parc", u"Passage", u"Passerelle", u"Passerelles",
                     u"Place", u"Placette", u"Pont", u"Promenade",
                     u"Petite Avenue", u"Petite Rue", u"Quai",
                     u"Rampe", u"Rang", u"Résidence", u"Rond-Point",
                     u"Route forestière", u"Route", u"Rue", u"Ruelle",
                     u"Square", u"Sente", u"Sentier", u"Sentiers", u"Terre-Plein",
                     u"Télécabine", u"Traboule", u"Traverse", u"Tunnel",
                     u"Venelle", u"Villa", u"Virage"
                   ]
    DETERMINANTS = [ u" des", u" du", u" de la", u" de l'",
                     u" de", u" d'", u" aux", u""
                   ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäãæ]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõœ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    Y_ACCENT = re.compile(ur"[ÿ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.Y_ACCENT.sub("y", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_it_generic(i18n):
    APPELLATIONS = [ u"Via", u"Viale", u"Piazza", u"Scali", u"Strada", u"Largo",
                     u"Corso", u"Viale", u"Calle", u"Sottoportico",
                     u"Sottoportego", u"Vicolo", u"Piazzetta" ]
    DETERMINANTS = [ u" delle", u" dell'", u" dei", u" degli",
                     u" della", u" del", u" di", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_es_generic(i18n):
    APPELLATIONS = [ u"Avenida", u"Avinguda", u"Calle", u"Callejón",
            u"Calzada", u"Camino", u"Camí", u"Carrer", u"Carretera",
            u"Glorieta", u"Parque", u"Pasaje", u"Pasarela", u"Paseo", u"Plaza",
            u"Plaça", u"Privada", u"Puente", u"Ronda", u"Salida", u"Travesia" ]
    DETERMINANTS = [ u" de la", u" de los", u" de las",
                     u" dels", u" del", u" d'", u" de l'",
                     u" de", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"[ñ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.N_ACCENT.sub("n", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_ca_generic(i18n):

    APPELLATIONS = [ # Catalan
                     u"Autopista", u"Autovia", u"Avinguda",
                     u"Baixada", u"Barranc", u"Barri", u"Barriada",
                     u"Biblioteca", u"Carrer", u"Carreró", u"Carretera",
                     u"Cantonada", u"Església", u"Estació", u"Hospital",
                     u"Monestir", u"Monument", u"Museu", u"Passatge",
                     u"Passeig", u"Plaça", u"Planta", u"Polígon",
                     u"Pujada", u"Rambla", u"Ronda", u"Travessera",
                     u"Travessia", u"Urbanització", u"Via",
                     u"Avenida", u"Calle", u"Camino", u"Plaza",

                     # Spanish (being distinct from Catalan)
                     u"Acceso", u"Acequia", u"Alameda", u"Alquería",
                     u"Andador", u"Angosta", u"Apartamentos", u"Apeadero",
                     u"Arboleda", u"Arrabal", u"Arroyo", u"Autovía",
                     u"Avenida", u"Bajada", u"Balneario", u"Banda",
                     u"Barranco", u"Barranquil", u"Barrio", u"Bloque",
                     u"Brazal", u"Bulevar", u"Calle", u"Calleja",
                     u"Callejón", u"Callejuela", u"Callizo", u"Calzada",
                     u"Camino", u"Camping", u"Cantera", u"Cantina",
                     u"Cantón", u"Carrera", u"Carrero", u"Carreterín",
                     u"Carretil", u"Carril", u"Caserío", u"Chalet",
                     u"Cinturón", u"Circunvalación", u"Cobertizo",
                     u"Colonia", u"Complejo", u"Conjunto", u"Convento",
                     u"Cooperativa", u"Corral", u"Corralillo", u"Corredor",
                     u"Cortijo", u"Costanilla", u"Costera", u"Cuadra",
                     u"Cuesta", u"Dehesa", u"Demarcación", u"Diagonal",
                     u"Diseminado", u"Edificio", u"Empresa", u"Entrada",
                     u"Escalera", u"Escalinata", u"Espalda", u"Estación",
                     u"Estrada", u"Explanada", u"Extramuros", u"Extrarradio",
                     u"Fábrica", u"Galería", u"Glorieta", u"Gran Vía",
                     u"Granja", u"Hipódromo", u"Jardín", u"Ladera",
                     u"Llanura", u"Malecón", u"Mercado", u"Mirador",
                     u"Monasterio", u"Muelle", u"Núcleo", u"Palacio",
                     u"Pantano", u"Paraje", u"Parque", u"Particular",
                     u"Partida", u"Pasadizo", u"Pasaje", u"Paseo",
                     u"Paseo marítimo", u"Pasillo", u"Plaza", u"Plazoleta",
                     u"Plazuela", u"Poblado", u"Polígono", u"Polígono industrial",
                     u"Portal", u"Pórtico", u"Portillo", u"Prazuela",
                     u"Prolongación", u"Pueblo", u"Puente", u"Puerta",
                     u"Puerto", u"Punto kilométrico", u"Rampla",
                     u"Residencial", u"Ribera", u"Rincón", u"Rinconada",
                     u"Sanatorio", u"Santuario", u"Sector", u"Sendera",
                     u"Sendero", u"Subida", u"Torrente", u"Tránsito",
                     u"Transversal", u"Trasera", u"Travesía", u"Urbanización",
                     u"Vecindario", u"Vereda", u"Viaducto", u"Viviendas",

                     # French (being distinct from Catalan and Spanish)
                     u"Accès", u"Allée", u"Allées", u"Autoroute", u"Avenue", u"Barrage",
                     u"Boulevard", u"Carrefour", u"Chaussée", u"Chemin",
                     u"Cheminement", u"Cale", u"Cales", u"Cavée", u"Cité",
                     u"Clos", u"Coin", u"Côte", u"Cour", u"Cours", u"Descente",
                     u"Degré", u"Escalier",
                     u"Escaliers", u"Esplanade", u"Funiculaire",
                     u"Giratoire", u"Hameau", u"Impasse", u"Jardin",
                     u"Jardins", u"Liaison", u"Mail", u"Montée", u"Môle",
                     u"Parc", u"Passage", u"Passerelle", u"Passerelles",
                     u"Place", u"Placette", u"Pont", u"Promenade",
                     u"Petite Avenue", u"Petite Rue", u"Quai",
                     u"Rampe", u"Rang", u"Résidence", u"Rond-Point",
                     u"Route forestière", u"Route", u"Rue", u"Ruelle",
                     u"Square", u"Sente", u"Sentier", u"Sentiers", u"Terre-Plein",
                     u"Télécabine", u"Traboule", u"Traverse", u"Tunnel",
                     u"Venelle", u"Villa", u"Virage"
                   ]

    DETERMINANTS = [ # Catalan
                     u" de", u" de la", u" del", u" dels", u" d'",
                     u" de l'", u" de sa", u" de son", u" de s'",
                     u" de ses", u" d'en", u" de na", u" de n'",

                     # Spanish (being distinct from Catalan)
                     u" de las",  u" de los",

                     # French (being distinct from Catalan and Spanish)
                     u" du",
                     u""]


    DETERMINANTS = [ u" de", u" de la", u" del", u" de las",
                     u" dels", u" de los", u" d'", u" de l'", u"de sa", u"de son", u"de s'",
                     u"de ses", u"d'en", u"de na", u"de n'", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"[ñ]", re.IGNORECASE | re.UNICODE)
    C_ACCENT = re.compile(ur"[ç]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.N_ACCENT.sub("n", s)
        s = self.C_ACCENT.sub("c", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_pt_generic(i18n):
    APPELLATIONS = [ # Brazilian Portuguese
                     u"Aeroporto", u"Aer.", u"Alameda", u"Al.", u"Apartamento", u"Ap.",
                     u"Área", u"Avenida", u"Av.", u"Beco", u"Bc.", u"Bloco", u"Bl.",
                     u"Caminho", u"Cam.", u"Campo", u"Chácara", u"Colônia",
                     u"Condomínio", u"Conjunto", u"Cj.", u"Distrito", u"Esplanada", u"Espl.",
                     u"Estação", u"Est.", u"Estrada", u"Estr.", u"Favela", u"Fazenda",
                     u"Feira", u"Jardim", u"Jd.", u"Ladeira", u"Lago",
                     u"Lagoa", u"Largo", u"Loteamento", u"Morro", u"Núcleo",
                     u"Parque", u"Pq.", u"Passarela", u"Pátio", u"Praça", u"Pç.", u"Quadra",
                     u"Recanto", u"Residencial", u"Resid.", u"Rotatória", u"Rua", u"R.",
                     u"Setor", u"Sítio", u"Travessa", u"Tv.", u"Trecho", u"Trevo",
                     u"Vale", u"Vereda", u"Via", u"V.", u"Viaduto", u"Viela",
                     u"Vila", u"Vl.",

                     # Portuguese (being distinct from Brazilian portuguese)
                     u"Avª", u"Av.ª", u"Azinhaga", u"Bco.", u"Cais", u"Calçada", u"Cc.",
                     u"Calçadinha", u"Ccnh", u"Esc.", u"Escadaria", u"Escada", u"Escadas",
                     u"Escnh.", u"Escadinhas", u"Jrd.", u"Lg.", u"Lug.", u"Lugar", u"Part.",
                     u"Particular", u"Passeio", u"Pr.", u"Pc.", u"Pct.", u"Pctª", u"Pct.ª",
                     u"Ponte", u"Praceta", u"Pto.", u"Prolongamento", u"Prolng.", u"Rampa",
                     u"Rot.", u"Rotunda", u"Terreiro", u"Transversal", u"Transv.", u"Trv.",
                     u"Trav.", u"Trilho", u"Túnel",

                     # Angola (being distinct from Brazilian portuguese and Portuguese)
                     u"Via Expressa"
                   ]


    DETERMINANTS = [ # Brazilian portuguese
                     u" do", u" da", u" dos", u" das",

                     # Portuguese (being distinct from Brazilian portuguese)
                     u" de",
                     u""]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    C_ACCENT = re.compile(ur"[ç]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.C_ACCENT.sub("c", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_ar_generic(i18n):
    APPELLATIONS = [ u"شارع", u"طريق", u"زقاق", u"نهج", u"جادة",
                     u"ممر", u"حارة",
                     u"كوبري", u"كوبرى", u"جسر", u"مطلع", u"منزل",
                     u"مفرق", u"ملف", u"تقاطع",
                     u"ساحل",
                     u"ميدان", u"ساحة", u"دوار" ]

    DETERMINANTS = [ u" ال", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    A_ACCENT = re.compile(ur"[اإآ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.A_ACCENT.sub("أ", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

    def isrtl(self):
        return True

class i18n_ru_generic(i18n):
    # Based on list from Streetmangler:
    # https://github.com/AMDmi3/streetmangler/blob/master/lib/locales/ru.cc
    STATUS_PARTS = [
        (u"улица", [u"ул"]),
        (u"площадь", [u"пл"]),
        (u"переулок", [u"пер", u"пер-к"]),
        (u"проезд", [u"пр-д"]),
        (u"шоссе", [u"ш"]),
        (u"бульвар", [u"бул", u"б-р"]),
        (u"тупик", [u"туп"]),
        (u"набережная", [u"наб"]),
        (u"проспект", [u"просп", u"пр-кт", u"пр-т"]),
        (u"линия", []),
        (u"аллея", []),
        (u"метромост", []),
        (u"мост", []),
        (u"просек", []),
        (u"просека", []),
        (u"путепровод", []),
        (u"тракт", [u"тр-т", u"тр"]),
        (u"тропа", []),
        (u"туннель", []),
        (u"тоннель", []),
        (u"эстакада", [u"эст"]),
        (u"дорога", [u"дор"]),
        (u"спуск", []),
        (u"подход", []),
        (u"подъезд", []),
        (u"съезд", []),
        (u"заезд", []),
        (u"разъезд", []),
        (u"слобода", []),
        (u"район", [u"р-н"]),
        (u"микрорайон", [u"мкр-н", u"мк-н", u"мкр", u"мкрн"]),
        (u"посёлок", [u"поселок", u"пос"]),
        (u"деревня", [u"дер", u"д"]),
        (u"квартал", [u"кв-л", u"кв"]),
    ]

    # matches one or more spaces
    SPACE_REDUCE = re.compile(r"\s+")
    # mapping from status abbreviations (w/o '.') to full status names
    STATUS_PARTS_ABBREV_MAPPING = dict((f, t) for t, ff in STATUS_PARTS for f in ff)
    # set of full (not abbreviated) status parts
    STATUS_PARTS_FULL = set((x[0] for x in STATUS_PARTS))
    # matches any abbreviated status part with optional '.'
    STATUS_ABBREV_REGEXP = re.compile(r"\b(%s)\.?(?=\W|$)" % u"|".join(
        f for t, ff in STATUS_PARTS for f in ff), re.IGNORECASE | re.UNICODE)
    # matches status prefixes at start of name used to move prefixes to the end
    PREFIX_REGEXP = re.compile(
        ur"^(?P<num_prefix>\d+-?(ы?й|я))?\s*(?P<prefix>(%s)\.?)?\s*(?P<name>.+)?" %
        (u"|".join(f for f,t in STATUS_PARTS)), re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        # usually, there are no accents in russian names, only "ё" sometimes, but
        # not as first letter
        return s.upper()

    def language_code(self):
        return self.language

    @staticmethod
    def _rewrite_street_parts(matches):
        if (matches.group('num_prefix') is None and
            matches.group('prefix') is not None and
            matches.group('name') in i18n_ru_generic.STATUS_PARTS_FULL):
            return matches.group(0)
        elif matches.group('num_prefix') is None and matches.group('prefix') is None:
            return matches.group(0)
        elif matches.group('name') is None:
            return matches.group(0)
        else:
            #print matches.group('num_prefix', 'prefix', 'name')
            return ", ".join((matches.group('name'),
                " ". join(s.lower()
                    for s in matches.group('prefix', 'num_prefix')
                    if s is not None)
                ))

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        # Normalize abbreviations
        name = self.STATUS_ABBREV_REGEXP.sub(lambda m:
                self.STATUS_PARTS_ABBREV_MAPPING.get(
                    m.group(0).replace('.', ''), m.group(0)),
            name)
        # Move prefixed status parts to the end for sorting
        name = self.PREFIX_REGEXP.sub(self._rewrite_street_parts, name)
        # TODO: move "малая", "большая" after name but before status
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_be_generic(i18n):
    # Based on code for Russian language:
    STATUS_PARTS = [
        (u"вуліца", [u"вул"]),
        (u"плошча", [u"пл"]),
        (u"завулак", [u"зав", u"зав-к"]),
        (u"праезд", [u"пр-д"]),
        (u"шаша", [u"ш"]),
        (u"бульвар", [u"бул", u"б-р"]),
        (u"тупік", [u"туп"]),
        (u"набярэжная", [u"наб"]),
        (u"праспект", [u"праспект", u"пр-кт", u"пр-т"]),
        (u"алея", []),
        (u"мост", []),
        (u"парк", []),
        (u"тракт", [u"тр-т", u"тр"]),
        (u"раён", [u"р-н"]),
        (u"мікрараён", [u"мкр-н", u"мк-н", u"мкр", u"мкрн"]),
        (u"пасёлак", [u"пас"]),
        (u"вёска", [ u"в"]),
        (u"квартал", [u"кв-л", u"кв"]),
    ]

    # matches one or more spaces
    SPACE_REDUCE = re.compile(r"\s+")
    # mapping from status abbreviations (w/o '.') to full status names
    STATUS_PARTS_ABBREV_MAPPING = dict((f, t) for t, ff in STATUS_PARTS for f in ff)
    # set of full (not abbreviated) status parts
    STATUS_PARTS_FULL = set((x[0] for x in STATUS_PARTS))
    # matches any abbreviated status part with optional '.'
    STATUS_ABBREV_REGEXP = re.compile(r"\b(%s)\.?(?=\W|$)" % u"|".join(
        f for t, ff in STATUS_PARTS for f in ff), re.IGNORECASE | re.UNICODE)
    # matches status prefixes at start of name used to move prefixes to the end
    PREFIX_REGEXP = re.compile(
        ur"^(?P<num_prefix>\d+-?(і|ы|я))?\s*(?P<prefix>(%s)\.?)?\s*(?P<name>.+)?" %
        (u"|".join(f for f,t in STATUS_PARTS)), re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        return s.upper()

    def language_code(self):
        return self.language

    @staticmethod
    def _rewrite_street_parts(matches):
        if (matches.group('num_prefix') is None and
            matches.group('prefix') is not None and
            matches.group('name') in i18n_be_generic.STATUS_PARTS_FULL):
            return matches.group(0)
        elif matches.group('num_prefix') is None and matches.group('prefix') is None:
            return matches.group(0)
        elif matches.group('name') is None:
            return matches.group(0)
        else:
            #print matches.group('num_prefix', 'prefix', 'name')
            return ", ".join((matches.group('name'),
                " ". join(s.lower()
                    for s in matches.group('num_prefix', 'prefix')
                    if s is not None)
                ))

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        # Normalize abbreviations
        name = self.STATUS_ABBREV_REGEXP.sub(lambda m:
                self.STATUS_PARTS_ABBREV_MAPPING.get(
                    m.group(0).replace('.', ''), m.group(0)),
            name)
        # Move prefixed status parts to the end for sorting
        name = self.PREFIX_REGEXP.sub(self._rewrite_street_parts, name)
        # TODO: move "малая", "большая" after name but before status
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_nl_generic(i18n):
    #
    # Dutch streets are often named after people and include a title.
    # The title will be captured as part of the <prefix>
    #
    APPELLATIONS = [ u"St.", u"Sint", u"Ptr.", u"Pater",
                     u"Prof.", u"Professor", u"Past.", u"Pastoor",
                     u"Pr.", u"Prins", u"Prinses", u"Gen.", u"Generaal",
                     u"Mgr.", u"Monseigneur", u"Mr.", u"Meester",
                     u"Burg.", u"Burgermeester", u"Dr.", u"Dokter",
                     u"Ir.", u"Ingenieur", u"Ds.", u"Dominee", u"Deken",
                     u"Drs.", u"Maj.", u"Majoor",
                     # counting words before street name,
                     # e.g. "1e Walstraat" => "Walstraat (1e)"
                     u"\d+e",
                     u"" ]
    #
    # Surnames in Dutch streets named after people tend to have the middle name
    # listed after the rest of the surname,
    # e.g. "Prins van Oranjestraat" => "Oranjestraat (Prins van)"
    # Likewise, articles are captured as part of the prefix,
    # e.g. "Den Urling" => "Urling (Den)"
    #
    DETERMINANTS = [ u"\s?van der", u"\s?van den", u"\s?van de", u"\s?van",
                     u"\s?Den", u"\s?D'n", u"\s?D'", u"\s?De", u"\s?'T", u"\s?Het",
                     u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)),
                                      re.IGNORECASE | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        matches = self.PREFIX_REGEXP.match(name)
        #
        # If no prefix was captured, that's okay. Don't substitute
        # the name however, "<name> ()" looks silly
        #
        if matches == None:
            return name

        if matches.group('prefix'):
            name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_hr_HR(i18n):
    # for upper_unaccent_string
    C_ACCENT = re.compile(ur"[ćč]", re.IGNORECASE | re.UNICODE)
    D_ACCENT = re.compile(ur"đ|dž", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"nj", re.IGNORECASE | re.UNICODE)
    L_ACCENT = re.compile(ur"lj", re.IGNORECASE | re.UNICODE)
    S_ACCENT = re.compile(ur"š", re.IGNORECASE | re.UNICODE)
    Z_ACCENT = re.compile(ur"ž", re.IGNORECASE | re.UNICODE)

    def upper_unaccent_string(self, s):
        s = self.C_ACCENT.sub("c", s)
        s = self.D_ACCENT.sub("d", s)
        s = self.N_ACCENT.sub("n", s)
        s = self.L_ACCENT.sub("l", s)
        s = self.S_ACCENT.sub("s", s)
        s = self.Z_ACCENT.sub("z", s)
        return s.upper()

    def __init__(self, language, locale_path):
        """Install the _() function for the chosen locale other
           object initialisation"""
        self.language = str(language) # FIXME: why do we have unicode here?
        _install_language(language, locale_path)

    def language_code(self):
        """returns the language code of the specific language
           supported, e.g. fr_FR.UTF-8"""
        return self.language

    def user_readable_street(self, name):
        """ transforms a street name into a suitable form for
            the map index, e.g. Paris (Rue de) for French"""
        return name

    ## FIXME: only first letter does not work for Croatian digraphs (dž, lj, nj)
    def first_letter_equal(self, a, b):
        """returns True if the letters a and b are equal in the map index,
           e.g. É and E are equals in French map index"""
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_pl_generic(i18n):

    APPELLATIONS = [ u"Dr.", u"Doktora", u"Ks.", u"Księdza",
                     u"Generała", u"Gen.",
                     u"Aleja", u"Plac", u"Pl.",
                     u"Rondo", u"rondo", u"Profesora",
                     u"Prof.",
                     u"" ]

    DETERMINANTS = [ u"\s?im.", u"\s?imienia", u"\s?pw.",
                     u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)),
                                      re.IGNORECASE | re.UNICODE)


    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        matches = self.PREFIX_REGEXP.match(name)
        #
        # If no prefix was captured, that's okay. Don't substitute
        # the name however, "<name> ()" looks silly
        #
        if matches == None:
            return name

        if matches.group('prefix'):
            name = self.PREFIX_REGEXP.sub(r"\g<name>, \g<prefix>", name)
        return name

    def first_letter_equal(self, a, b):
        return a == b

class i18n_tr_generic(i18n):
    APPELLATIONS = [ u"Sokak", u"Sokağı" ]
    DETERMINANTS = []
    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_de_generic(i18n):
    #
    # German streets are often named after people and include a title.
    # The title will be captured as part of the <prefix>
        # Covering airport names and "New"/"Old" as prefixes as well
    #
    APPELLATIONS = [ u"Alte", u"Alter", u"Doktor", u"Dr.",
                     u"Flughafen", u"Flugplatz", u"Gen.,", u"General",
                     u"Neue", u"Neuer", u"Platz",
                     u"Prinz", u"Prinzessin", u"Prof.",
                     u"Professor" ]
    #
    # Surnames in german streets named after people tend to have the middle name
    # listed after the rest of the surname,
    # e.g. "Platz der deutschen Einheit" => "deutschen Einheit (Platz der)"
    # Likewise, articles are captured as part of the prefix,
    # e.g. "An der Märchenwiese" => "Märchenwiese (An der)"
    #
    DETERMINANTS = [ u"\s?An den", u"\s?An der", u"\s?Am",
                     u"\s?Auf den" , u"\s?Auf der"
                     u" an", u" des", u" der", u" von", u" vor"]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_ast_generic(i18n):

    APPELLATIONS = [ # Asturian
                     u"Accesu", u"Autopista", u"Autovia", u"Avenida",
                     u"Baxada", u"Barrancu", u"Barriu", u"Barriada",
                     u"Biblioteca", u"Cai", u"Caleya",
                     u"Calzada", u"Camín", u"Carretera", u"Cuesta",
                     u"Estación", u"Hospital", u"Iglesia", u"Monasteriu",
                     u"Monumentu", u"Muelle", u"Muséu",
                     u"Palaciu", u"Parque", u"Pasadizu", u"Pasaxe",
                     u"Paséu", u"Planta", u"Plaza", u"Polígonu",
                     u"Ronda", u"Travesía", u"Urbanización", u"Via",
                     u"Xardín", u"Xardinos",

                     # Spanish (different from Asturian)
                     u"Acceso", u"Acequia", u"Alameda", u"Alquería",
                     u"Andador", u"Angosta", u"Apartamentos", u"Apeadero",
                     u"Arboleda", u"Arrabal", u"Arroyo", u"Autovía",
                     u"Bajada", u"Balneario", u"Banda",
                     u"Barranco", u"Barranquil", u"Barrio", u"Bloque",
                     u"Brazal", u"Bulevar", u"Calle", u"Calleja",
                     u"Callejón", u"Callejuela", u"Callizo",
                     u"Camino", u"Camping", u"Cantera", u"Cantina",
                     u"Cantón", u"Carrera", u"Carrero", u"Carreterín",
                     u"Carretil", u"Carril", u"Caserío", u"Chalet",
                     u"Cinturón", u"Circunvalación", u"Cobertizo",
                     u"Colonia", u"Complejo", u"Conjunto", u"Convento",
                     u"Cooperativa", u"Corral", u"Corralillo", u"Corredor",
                     u"Cortijo", u"Costanilla", u"Costera", u"Cuadra",
                     u"Dehesa", u"Demarcación", u"Diagonal",
                     u"Diseminado", u"Edificio", u"Empresa", u"Entrada",
                     u"Escalera", u"Escalinata", u"Espalda", u"Estación",
                     u"Estrada", u"Explanada", u"Extramuros", u"Extrarradio",
                     u"Fábrica", u"Galería", u"Glorieta", u"Gran Vía",
                     u"Granja", u"Hipódromo", u"Jardín", u"Ladera",
                     u"Llanura", u"Malecón", u"Mercado", u"Mirador",
                     u"Monasterio", u"Núcleo", u"Palacio",
                     u"Pantano", u"Paraje", u"Particular",
                     u"Partida", u"Pasadizo", u"Pasaje", u"Paseo",
                     u"Paseo marítimo", u"Pasillo", u"Plaza", u"Plazoleta",
                     u"Plazuela", u"Poblado", u"Polígono", u"Polígono industrial",
                     u"Portal", u"Pórtico", u"Portillo", u"Prazuela",
                     u"Prolongación", u"Pueblo", u"Puente", u"Puerta",
                     u"Puerto", u"Punto kilométrico", u"Rampla",
                     u"Residencial", u"Ribera", u"Rincón", u"Rinconada",
                     u"Sanatorio", u"Santuario", u"Sector", u"Sendera",
                     u"Sendero", u"Subida", u"Torrente", u"Tránsito",
                     u"Transversal", u"Trasera", u"Travesía", u"Urbanización",
                     u"Vecindario", u"Vereda", u"Viaducto", u"Viviendas",
                   ]

    DETERMINANTS = [ # Asturian
                     u" de", u" de la", u" del", u" de les", u" d'",
                     u" de los", u" de l'",

                     # Spanish (different from Asturian)
                     u" de las",
                     u""]


    DETERMINANTS = [ u" de", u" de la", u" del", u" de les",
                     u" de los", u" de las", u" d'", u" de l'", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"[ñ]", re.IGNORECASE | re.UNICODE)
    H_ACCENT = re.compile(ur"[ḥ]", re.IGNORECASE | re.UNICODE)
    L_ACCENT = re.compile(ur"[ḷ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.N_ACCENT.sub("n", s)
        s = self.H_ACCENT.sub("h", s)
        s = self.L_ACCENT.sub("l", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_generic(i18n):
    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        return name

    def first_letter_equal(self, a, b):
        return a == b

# When not listed in the following map, default language class will be
# i18n_generic
language_class_map = {
    'fr_BE.UTF-8': i18n_fr_generic,
    'fr_FR.UTF-8': i18n_fr_generic,
    'fr_CA.UTF-8': i18n_fr_generic,
    'fr_CH.UTF-8': i18n_fr_generic,
    'fr_LU.UTF-8': i18n_fr_generic,
    'en_AG': i18n_generic,
    'en_AU.UTF-8': i18n_generic,
    'en_BW.UTF-8': i18n_generic,
    'en_CA.UTF-8': i18n_generic,
    'en_DK.UTF-8': i18n_generic,
    'en_GB.UTF-8': i18n_generic,
    'en_HK.UTF-8': i18n_generic,
    'en_IE.UTF-8': i18n_generic,
    'en_IN': i18n_generic,
    'en_NG': i18n_generic,
    'en_NZ.UTF-8': i18n_generic,
    'en_PH.UTF-8': i18n_generic,
    'en_SG.UTF-8': i18n_generic,
    'en_US.UTF-8': i18n_generic,
    'en_ZA.UTF-8': i18n_generic,
    'en_ZW.UTF-8': i18n_generic,
    'nl_BE.UTF-8': i18n_nl_generic,
    'nl_NL.UTF-8': i18n_nl_generic,
    'it_IT.UTF-8': i18n_it_generic,
    'it_CH.UTF-8': i18n_it_generic,
    'de_AT.UTF-8': i18n_de_generic,
    'de_BE.UTF-8': i18n_de_generic,
    'de_DE.UTF-8': i18n_de_generic,
    'de_LU.UTF-8': i18n_de_generic,
    'de_CH.UTF-8': i18n_de_generic,
    'es_ES.UTF-8': i18n_es_generic,
    'es_AR.UTF-8': i18n_es_generic,
    'es_BO.UTF-8': i18n_es_generic,
    'es_CL.UTF-8': i18n_es_generic,
    'es_CR.UTF-8': i18n_es_generic,
    'es_DO.UTF-8': i18n_es_generic,
    'es_EC.UTF-8': i18n_es_generic,
    'es_SV.UTF-8': i18n_es_generic,
    'es_GT.UTF-8': i18n_es_generic,
    'es_HN.UTF-8': i18n_es_generic,
    'es_MX.UTF-8': i18n_es_generic,
    'es_NI.UTF-8': i18n_es_generic,
    'es_PA.UTF-8': i18n_es_generic,
    'es_PY.UTF-8': i18n_es_generic,
    'es_PE.UTF-8': i18n_es_generic,
    'es_PR.UTF-8': i18n_es_generic,
    'es_US.UTF-8': i18n_es_generic,
    'es_UY.UTF-8': i18n_es_generic,
    'es_VE.UTF-8': i18n_es_generic,
    'ca_ES.UTF-8': i18n_ca_generic,
    'ca_AD.UTF-8': i18n_ca_generic,
    'ca_FR.UTF-8': i18n_ca_generic,
    'pt_AO.UTF-8': i18n_pt_generic,
    'pt_BR.UTF-8': i18n_pt_generic,
    'pt_CV.UTF-8': i18n_pt_generic,
    'pt_GW.UTF-8': i18n_pt_generic,
    'pt_MZ.UTF-8': i18n_pt_generic,
    'pt_PT.UTF-8': i18n_pt_generic,
    'pt_ST.UTF-8': i18n_pt_generic,
    'pt_TL.UTF-8': i18n_pt_generic,
    'da_DK.UTF-8': i18n_generic,
    'ar_AE.UTF-8': i18n_ar_generic,
    'ar_BH.UTF-8': i18n_ar_generic,
    'ar_DZ.UTF-8': i18n_ar_generic,
    'ar_EG.UTF-8': i18n_ar_generic,
    'ar_IN': i18n_ar_generic,
    'ar_IQ.UTF-8': i18n_ar_generic,
    'ar_JO.UTF-8': i18n_ar_generic,
    'ar_KW.UTF-8': i18n_ar_generic,
    'ar_LB.UTF-8': i18n_ar_generic,
    'ar_LY.UTF-8': i18n_ar_generic,
    'ar_MA.UTF-8': i18n_ar_generic,
    'ar_OM.UTF-8': i18n_ar_generic,
    'ar_QA.UTF-8': i18n_ar_generic,
    'ar_SA.UTF-8': i18n_ar_generic,
    'ar_SD.UTF-8': i18n_ar_generic,
    'ar_SY.UTF-8': i18n_ar_generic,
    'ar_TN.UTF-8': i18n_ar_generic,
    'ar_YE.UTF-8': i18n_ar_generic,
    'hr_HR.UTF-8': i18n_hr_HR,
    'ru_RU.UTF-8': i18n_ru_generic,
    'pl_PL.UTF-8': i18n_pl_generic,
    'nb_NO.UTF-8': i18n_generic,
    'nn_NO.UTF-8': i18n_generic,
    'tr_TR.UTF-8': i18n_tr_generic,
    'ast_ES.UTF-8': i18n_ast_generic,
    'sk_SK.UTF-8': i18n_generic,
    'be_BY.UTF-8': i18n_be_generic,
}

def install_translation(locale_name, locale_path):
    """Return a new i18n class instance, depending on the specified
    locale name (eg. "fr_FR.UTF-8"). See output of "locale -a" for a
    list of system-supported locale names. When none matching, default
    class is i18n_generic"""
    language_class = language_class_map.get(locale_name, i18n_generic)
    return language_class(locale_name, locale_path)


Rui <zenter>
Tue 09 Jun 2015 07:33:53 PM UTC, comment #3: 

All strings in Transifex are now translated. Now it's only missing changes in language specific code in ocitysmap and maposmatic. Can someone change the following 2 files in Git? Thanks.



http://git.savannah.gnu.org/cgit/maposmatic.git/tree/www/settings.py



# coding: utf-8

# maposmatic, the web front-end of the MapOSMatic city map generation system
# Copyright (C) 2009  David Decotigny
# Copyright (C) 2009  Frédéric Lehobey
# Copyright (C) 2009  David Mentré
# Copyright (C) 2009  Maxime Petazzoni
# Copyright (C) 2009  Thomas Petazzoni
# Copyright (C) 2009  Gaël Utard

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Affero General Public License for more details.

# You should have received a copy of the GNU Affero General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

# Django settings for www project.

import logging
import os.path

from django.conf.global_settings import TEMPLATE_CONTEXT_PROCESSORS
from django.utils.translation import ugettext_lazy as _

from settings_local import *
import logconfig

PROJECT_PATH = os.path.dirname(os.path.abspath(__file__))

TEMPLATE_DEBUG = DEBUG

MANAGERS = ADMINS

# Local time zone for this installation. Choices can be found here:
# http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
# although not all choices may be available on all operating systems.
# If running in a Windows environment this must be set to the same as your
# system time zone.
TIME_ZONE = 'Europe/Paris'

# Language code for this installation. All choices can be found here:
# http://www.i18nguy.com/unicode/language-identifiers.html
LANGUAGE_CODE = 'en-us'

SITE_ID = 1

# If you set this to False, Django will make some optimizations so as not
# to load the internationalization machinery.
USE_I18N = True

# Absolute path to the directory that holds media.
# Example: "/home/media/media.lawrence.com/"
MEDIA_ROOT = ''

# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash if there is a path component (optional in other cases).
# Examples: "http://media.lawrence.com", "http://example.com/media/"
MEDIA_URL = ''

# URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a
# trailing slash.
# Examples: "http://foo.com/media/", "/media/".
ADMIN_MEDIA_PREFIX = '/media/'

# Make this unique, and don't share it with anybody.
SECRET_KEY = 'tm+wb)lp5q%br=p0d2toz&km_-w)cmcelv!7inons&^v9(q!d2'

# List of callables that know how to import templates from various sources.
TEMPLATE_LOADERS = (
    'django.template.loaders.filesystem.Loader',
    'django.template.loaders.app_directories.Loader',
#     'django.template.loaders.eggs.load_template_source',
)


TEMPLATE_CONTEXT_PROCESSORS += (
     'django.core.context_processors.request',
     'www.maposmatic.context_processors.all',
)

MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.locale.LocaleMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
)

ROOT_URLCONF = 'www.urls'

TEMPLATE_DIRS = (
    os.path.join(PROJECT_PATH, 'templates'),
)

LOCAL_MEDIA_PATH = os.path.join(PROJECT_PATH, 'static')

INSTALLED_APPS = (
    'django.contrib.sessions',
    'www.maposmatic',
)

# Available website translations. Note that the language codes must be
# specified in Django i18n location (all lowercase, with the language and
# locale separated by a dash instead of an underscore: pt_BR -> pt-br)
LANGUAGES = {
    "fr":    u"Français",
    "en":    u"English",
    "de":    u"Deutsch",
    "it":    u"Italiano",
    "ca":    u"Català",
    "ru":    u"Русский",
    "ar":    u"العربية",
    "pt":    u"Português",
    "pt-br": u"Português do Brasil",
    "nb":    u"Norwegian Bokmål",
    "nl":    u"Nederlands",
    "hr":    u"Hrvatski",
    "pl":    u"Polski",
    "es":    u"Español",
    "id":    u"Bahasa Indonesia",
    "tr":    u"Türkçe",
    "ja":    u"日本人",
    "el":    u"ελληνικά",
    "be":    u"беларуская",
    "uk":    u"українська",
}

# Associate a Django language code with:
#  the language code used to select the Paypal button
#  the country code that allows to get the proper translation of the
#  PayPal payment page
# When no association is found, we automatically default to english
PAYPAL_LANGUAGES = {
    "fr": ("fr_FR", "FR"),
    "de": ("de_DE", "DE"),
    "it": ("it_IT", "IT"),
    "pt": ("pt_PT", "PT"),
    "pt-br": ("pt_BR", "BR"),
    "nl": ("nl_NL", "NL"),
    "pl": ("pl_PL", "PL"),
    "es": ("es_ES", "ES"),
    "el": ("el_GR", "GR"),
}

# Languages must be ordered by country (in xx_YY, YY is the country
# code), and then ordered with the most widely used language in the
# country first. For example, in France, we will want "French" to be
# the first, and catalan to be in the second place). The reason for
# this is that the order in the below list will be the order with
# which languages will be presented by the MapOSMatic website (after
# filtering the language list based on the country of the city that is
# being rendered).
MAP_LANGUAGES = {
    "ca_AD.UTF-8": u"Andorra (CA)",
    "ar_AE.UTF-8": u"دولة الإمارات العربية المتحدة (AR)",
    "en_AG":       u"Antigua and Barbuda (EN)",
    "pt_AO.UTF-8": u"Angola (PT)",
    "es_AR.UTF-8": u"Argentina (ES)",
    "de_AT.UTF-8": u"Österreich (DE)",
    "en_AU.UTF-8": u"Australia (EN)",
    "nl_BE.UTF-8": u"Koninkrijk België (NL)",
    "fr_BE.UTF-8": u"Royaume de Belgique (FR)",
    "de_BE.UTF-8": u"Königreich Belgien (DE)",
    "ar_BH.UTF-8": u"البحرين (AR)",
    "be_BY.UTF-8": u"Белару́сь (BY)",
    "es_BO.UTF-8": u"Bolivia (ES)",
    "pt_BR.UTF-8": u"Brasil (PT)",
    "en_BW.UTF-8": u"Botswana (EN)",
    "en_CA.UTF-8": u"Canada (EN)",
    "fr_CA.UTF-8": u"Canada (FR)",
    "de_CH.UTF-8": u"Schweiz (DE)",
    "fr_CH.UTF-8": u"Suisse (FR)",
    "it_CH.UTF-8": u"Svizzera (IT)",
    "el_GR.UTF-8": u"Ελλάδα (GR)",
    "es_CL.UTF-8": u"Chile (ES)",
    "es_CR.UTF-8": u"Costa Rica (ES)",
    "pt_CV.UTF-8": u"Cabo Verde (PT)",
    "de_DE.UTF-8": u"Deutschland (DE)",
    "da_DK.UTF-8": u"Danmark (DA)",
    "en_DK.UTF-8": u"Denmark (EN)",
    "es_DO.UTF-8": u"República Dominicana (ES)",
    "ar_DZ.UTF-8": u"الجزائر (AR)",
    "es_EC.UTF-8": u"Ecuador (ES)",
    "ar_EG.UTF-8": u"مصر (AR)",
    "es_ES.UTF-8": u"España (ES)",
    "ca_ES.UTF-8": u"Espanya (CA)",
    "ast_ES.UTF-8": u"España (AST)",
    "fr_FR.UTF-8": u"France (FR)",
    "ca_FR.UTF-8": u"França (CA)",
    "en_GB.UTF-8": u"United Kingdom (EN)",
    "es_GT.UTF-8": u"Guatemala (ES)",
    "pt_GW.UTF-8": u"Guiné Bissau (PT)",
    "en_HK.UTF-8": u"Hong Kong (EN)",
    "es_HN.UTF-8": u"Honduras (ES)",
    "hr_HR.UTF-8": u"Republika Hrvatska",
    "id_ID.UTF-8": u"Bahasa Indonesia (ID)",
    "en_IE.UTF-8": u"Ireland (EN)",
    "en_IN":       u"India (EN)",
    "ar_IQ.UTF-8": u"العراق (AR)",
    "it_IT.UTF-8": u"Italia (IT)",
    "ar_JO.UTF-8": u"الأردنّ‎ (AR)",
    "ar_KW.UTF-8": u"الكويت (AR)",
    "ar_LB.UTF-8": u"لبنان (AR)",
    "ja_JP.UTF-8": u"日本人 (JA)",
    "fr_LU.UTF-8": u"Luxembourg (FR)",
    "de_LU.UTF-8": u"Luxemburg (DE)",
    "ar_LY.UTF-8": u"ليبيا (AR)",
    "ar_MA.UTF-8": u"المملكة المغربية (AR)",
    "es_MX.UTF-8": u"México (ES)",
    "pt_MZ.UTF-8": u"Moçambique (PT)",
    "en_NG":       u"Nigeria (EN)",
    "es_NI.UTF-8": u"Nicaragua (ES)",
    "nl_NL.UTF-8": u"Nederland (NL)",
    "nb_NO.UTF-8": u"Norwegian Bokmål (NO)",
    "nn_NO.UTF-8": u"Norwegian Nynorsk (NO)",
    "en_NZ.UTF-8": u"New Zealand (EN)",
    "ar_OM.UTF-8": u"سلطنة عمان (AR)",
    "es_PA.UTF-8": u"Panamá (ES)",
    "es_PE.UTF-8": u"Perú (ES)",
    "en_PH.UTF-8": u"Philippines (EN)",
    "pl_PL.UTF-8": u"Rzeczpospolita Polska",
    "es_PR.UTF-8": u"Puerto Rico (ES)",
    "pt_PT.UTF-8": u"Portugal (PT)",
    "es_PY.UTF-8": u"Paraguay (ES)",
    "ar_QA.UTF-8": u"دولة قطر (AR)",
    "ru_RU.UTF-8": u"Русский",
    "ar_SA.UTF-8": u"المملكة العربية السعودية (AR)",
    "ar_SD.UTF-8": u"السودان (AR)",
    "en_SG.UTF-8": u"Singapore (EN)",
    "pt_ST.UTF-8": u"São Tomé e Principe (PT)",
    "es_SV.UTF-8": u"El Salvador (ES)",
    "ar_SY.UTF-8": u"سوريا (AR)",
    "pt_TL.UTF-8": u"Timor-Leste (PT)",
    "ar_TN.UTF-8": u"تونس (AR)",
    "en_US.UTF-8": u"United States (EN)",
    "es_US.UTF-8": u"Estados Unidos de América (ES)",
    "uk_UA.UTF-8": u"Україна (UK)",
    "es_UY.UTF-8": u"Uruguay (ES)",
    "es_VE.UTF-8": u"Venezuela (ES)",
    "ar_YE.UTF-8": u"اليَمَن (AR)",
    "en_ZA.UTF-8": u"South Africa (EN)",
    "en_ZW.UTF-8": u"Zimbabwe (EN)",
    "tr_TR.UTF-8": u"Türkçe (TR)",
    "sk_SK.UTF-8": u"Slovakien (SK)",
}

MAP_LANGUAGES_LIST = MAP_LANGUAGES.items()
MAP_LANGUAGES_LIST.sort(lambda x, y: cmp(x[1], y[1]))
# "C" must be the last entry
MAP_LANGUAGES_LIST.append(("C", _(u"No localization")))

# GIS database (read settings from OCitySMap's configuration). The
# default port to connect to the database is 5432, which is the
# default PostgreSQL port.
import ConfigParser
gis_config = ConfigParser.SafeConfigParser({'port': '5432'})

if OCITYSMAP_CFG_PATH is None:
    OCITYSMAP_CFG_PATH = os.path.expanduser('~/.ocitysmap.conf')
with open(OCITYSMAP_CFG_PATH) as fp:
    gis_config.readfp(fp)
GIS_DATABASE_HOST = gis_config.get('datasource', 'host')
GIS_DATABASE_USER = gis_config.get('datasource', 'user')
GIS_DATABASE_PASSWORD = gis_config.get('datasource', 'password')
GIS_DATABASE_NAME = gis_config.get('datasource', 'dbname')
GIS_DATABASE_PORT = gis_config.get('datasource', 'port')

def has_gis_database():
    return GIS_DATABASE_NAME and GIS_DATABASE_NAME != ''

# Job page refresh frequency, in seconds, for when the job is waiting in queue
# and when the job is currently being rendered.
REFRESH_JOB_WAITING = 30
REFRESH_JOB_RENDERING = 15

def is_daemon_running():
    return os.path.exists(MAPOSMATIC_PID_FILE)

# Logging
logconfig.setup_maposmatic_logging(
        int(os.environ.get("MAPOSMATIC_LOG_LEVEL",
                           DEFAULT_MAPOSMATIC_LOG_LEVEL)),
        os.environ.get('MAPOSMATIC_LOG_FILE', DEFAULT_MAPOSMATIC_LOG_FILE),
        os.environ.get("MAPOSMATIC_LOG_FORMAT", DEFAULT_MAPOSMATIC_LOG_FORMAT))
LOG = logging.getLogger('maposmatic')









http://git.savannah.gnu.org/cgit/maposmatic/ocitysmap.git/tree/ocitysmap/i18n.py




# -*- coding: utf-8; mode: Python -*-

# ocitysmap, city map and street index generator from OpenStreetMap data
# Copyright (C) 2010  David Decotigny
# Copyright (C) 2010  Frédéric Lehobey
# Copyright (C) 2010  Pierre Mauduit
# Copyright (C) 2010  David Mentré
# Copyright (C) 2010  Maxime Petazzoni
# Copyright (C) 2010  Thomas Petazzoni
# Copyright (C) 2010  Gaël Utard

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Affero General Public License for more details.

# You should have received a copy of the GNU Affero General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

import re
import gettext

def _install_language(language, locale_path):
    t = gettext.translation(domain='ocitysmap',
                            localedir=locale_path,
                            languages=[language],
                            fallback=True)
    t.install(unicode=True)

class i18n:
    """Functions needed to be implemented for a new language.
       See i18n_fr_FR_UTF8 below for an example. """
    def language_code(self):
        pass

    def user_readable_street(self, name):
        pass

    def first_letter_equal(self, a, b):
        pass

    def isrtl(self):
        return False

    def upper_unaccent_string(self, s):
        return s.upper()

class i18n_template_code_CODE(i18n):
    def __init__(self, language, locale_path):
        """Install the _() function for the chosen locale other
           object initialisation"""

        # It's important to convert to str() here because the map_language
        # value coming from the database is Unicode, but setlocale() needs a
        # non-unicode string as the locale name, otherwise it thinks it's a
        # locale tuple.
        self.language = str(language)
        _install_language(language, locale_path)

    def language_code(self):
        """returns the language code of the specific language
           supported, e.g. fr_FR.UTF-8"""
        return self.language

    def user_readable_street(self, name):
        """ transforms a street name into a suitable form for
            the map index, e.g. Paris (Rue de) for French"""
        return name

    def first_letter_equal(self, a, b):
        """returns True if the letters a and b are equal in the map index,
           e.g. É and E are equals in French map index"""
        return a == b

    def isrtl(self):
        return False


class i18n_fr_generic(i18n):
    APPELLATIONS = [ u"Accès", u"Allée", u"Allées", u"Autoroute", u"Avenue",
                     u"Avenues", u"Barrage",
                     u"Boulevard", u"Carrefour", u"Chaussée", u"Chemin",
                     u"Chemin rural",
                     u"Cheminement", u"Cale", u"Cales", u"Cavée", u"Cité",
                     u"Clos", u"Coin", u"Côte", u"Cour", u"Cours", u"Descente",
                     u"Degré", u"Escalier",
                     u"Escaliers", u"Esplanade", u"Funiculaire",
                     u"Giratoire", u"Hameau", u"Impasse", u"Jardin",
                     u"Jardins", u"Liaison", u"Lotissement", u"Mail",
                     u"Montée", u"Môle",
                     u"Parc", u"Passage", u"Passerelle", u"Passerelles",
                     u"Place", u"Placette", u"Pont", u"Promenade",
                     u"Petite Avenue", u"Petite Rue", u"Quai",
                     u"Rampe", u"Rang", u"Résidence", u"Rond-Point",
                     u"Route forestière", u"Route", u"Rue", u"Ruelle",
                     u"Square", u"Sente", u"Sentier", u"Sentiers", u"Terre-Plein",
                     u"Télécabine", u"Traboule", u"Traverse", u"Tunnel",
                     u"Venelle", u"Villa", u"Virage"
                   ]
    DETERMINANTS = [ u" des", u" du", u" de la", u" de l'",
                     u" de", u" d'", u" aux", u""
                   ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäãæ]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõœ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    Y_ACCENT = re.compile(ur"[ÿ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.Y_ACCENT.sub("y", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_it_generic(i18n):
    APPELLATIONS = [ u"Via", u"Viale", u"Piazza", u"Scali", u"Strada", u"Largo",
                     u"Corso", u"Viale", u"Calle", u"Sottoportico",
                     u"Sottoportego", u"Vicolo", u"Piazzetta" ]
    DETERMINANTS = [ u" delle", u" dell'", u" dei", u" degli",
                     u" della", u" del", u" di", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_es_generic(i18n):
    APPELLATIONS = [ u"Avenida", u"Avinguda", u"Calle", u"Callejón",
            u"Calzada", u"Camino", u"Camí", u"Carrer", u"Carretera",
            u"Glorieta", u"Parque", u"Pasaje", u"Pasarela", u"Paseo", u"Plaza",
            u"Plaça", u"Privada", u"Puente", u"Ronda", u"Salida", u"Travesia" ]
    DETERMINANTS = [ u" de la", u" de los", u" de las",
                     u" dels", u" del", u" d'", u" de l'",
                     u" de", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"[ñ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.N_ACCENT.sub("n", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_ca_generic(i18n):

    APPELLATIONS = [ # Catalan
                     u"Autopista", u"Autovia", u"Avinguda",
                     u"Baixada", u"Barranc", u"Barri", u"Barriada",
                     u"Biblioteca", u"Carrer", u"Carreró", u"Carretera",
                     u"Cantonada", u"Església", u"Estació", u"Hospital",
                     u"Monestir", u"Monument", u"Museu", u"Passatge",
                     u"Passeig", u"Plaça", u"Planta", u"Polígon",
                     u"Pujada", u"Rambla", u"Ronda", u"Travessera",
                     u"Travessia", u"Urbanització", u"Via",
                     u"Avenida", u"Calle", u"Camino", u"Plaza",

                     # Spanish (being distinct from Catalan)
                     u"Acceso", u"Acequia", u"Alameda", u"Alquería",
                     u"Andador", u"Angosta", u"Apartamentos", u"Apeadero",
                     u"Arboleda", u"Arrabal", u"Arroyo", u"Autovía",
                     u"Avenida", u"Bajada", u"Balneario", u"Banda",
                     u"Barranco", u"Barranquil", u"Barrio", u"Bloque",
                     u"Brazal", u"Bulevar", u"Calle", u"Calleja",
                     u"Callejón", u"Callejuela", u"Callizo", u"Calzada",
                     u"Camino", u"Camping", u"Cantera", u"Cantina",
                     u"Cantón", u"Carrera", u"Carrero", u"Carreterín",
                     u"Carretil", u"Carril", u"Caserío", u"Chalet",
                     u"Cinturón", u"Circunvalación", u"Cobertizo",
                     u"Colonia", u"Complejo", u"Conjunto", u"Convento",
                     u"Cooperativa", u"Corral", u"Corralillo", u"Corredor",
                     u"Cortijo", u"Costanilla", u"Costera", u"Cuadra",
                     u"Cuesta", u"Dehesa", u"Demarcación", u"Diagonal",
                     u"Diseminado", u"Edificio", u"Empresa", u"Entrada",
                     u"Escalera", u"Escalinata", u"Espalda", u"Estación",
                     u"Estrada", u"Explanada", u"Extramuros", u"Extrarradio",
                     u"Fábrica", u"Galería", u"Glorieta", u"Gran Vía",
                     u"Granja", u"Hipódromo", u"Jardín", u"Ladera",
                     u"Llanura", u"Malecón", u"Mercado", u"Mirador",
                     u"Monasterio", u"Muelle", u"Núcleo", u"Palacio",
                     u"Pantano", u"Paraje", u"Parque", u"Particular",
                     u"Partida", u"Pasadizo", u"Pasaje", u"Paseo",
                     u"Paseo marítimo", u"Pasillo", u"Plaza", u"Plazoleta",
                     u"Plazuela", u"Poblado", u"Polígono", u"Polígono industrial",
                     u"Portal", u"Pórtico", u"Portillo", u"Prazuela",
                     u"Prolongación", u"Pueblo", u"Puente", u"Puerta",
                     u"Puerto", u"Punto kilométrico", u"Rampla",
                     u"Residencial", u"Ribera", u"Rincón", u"Rinconada",
                     u"Sanatorio", u"Santuario", u"Sector", u"Sendera",
                     u"Sendero", u"Subida", u"Torrente", u"Tránsito",
                     u"Transversal", u"Trasera", u"Travesía", u"Urbanización",
                     u"Vecindario", u"Vereda", u"Viaducto", u"Viviendas",

                     # French (being distinct from Catalan and Spanish)
                     u"Accès", u"Allée", u"Allées", u"Autoroute", u"Avenue", u"Barrage",
                     u"Boulevard", u"Carrefour", u"Chaussée", u"Chemin",
                     u"Cheminement", u"Cale", u"Cales", u"Cavée", u"Cité",
                     u"Clos", u"Coin", u"Côte", u"Cour", u"Cours", u"Descente",
                     u"Degré", u"Escalier",
                     u"Escaliers", u"Esplanade", u"Funiculaire",
                     u"Giratoire", u"Hameau", u"Impasse", u"Jardin",
                     u"Jardins", u"Liaison", u"Mail", u"Montée", u"Môle",
                     u"Parc", u"Passage", u"Passerelle", u"Passerelles",
                     u"Place", u"Placette", u"Pont", u"Promenade",
                     u"Petite Avenue", u"Petite Rue", u"Quai",
                     u"Rampe", u"Rang", u"Résidence", u"Rond-Point",
                     u"Route forestière", u"Route", u"Rue", u"Ruelle",
                     u"Square", u"Sente", u"Sentier", u"Sentiers", u"Terre-Plein",
                     u"Télécabine", u"Traboule", u"Traverse", u"Tunnel",
                     u"Venelle", u"Villa", u"Virage"
                   ]

    DETERMINANTS = [ # Catalan
                     u" de", u" de la", u" del", u" dels", u" d'",
                     u" de l'", u" de sa", u" de son", u" de s'",
                     u" de ses", u" d'en", u" de na", u" de n'",

                     # Spanish (being distinct from Catalan)
                     u" de las",  u" de los",

                     # French (being distinct from Catalan and Spanish)
                     u" du",
                     u""]


    DETERMINANTS = [ u" de", u" de la", u" del", u" de las",
                     u" dels", u" de los", u" d'", u" de l'", u"de sa", u"de son", u"de s'",
                     u"de ses", u"d'en", u"de na", u"de n'", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"[ñ]", re.IGNORECASE | re.UNICODE)
    C_ACCENT = re.compile(ur"[ç]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.N_ACCENT.sub("n", s)
        s = self.C_ACCENT.sub("c", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_pt_generic(i18n):
    APPELLATIONS = [ # Brazilian Portuguese
                     u"Aeroporto", u"Aer.", u"Alameda", u"Al.", u"Apartamento", u"Ap.",
                     u"Área", u"Avenida", u"Av.", u"Beco", u"Bc.", u"Bloco", u"Bl.",
                     u"Caminho", u"Cam.", u"Campo", u"Chácara", u"Colônia",
                     u"Condomínio", u"Conjunto", u"Cj.", u"Distrito", u"Esplanada", u"Espl.",
                     u"Estação", u"Est.", u"Estrada", u"Estr.", u"Favela", u"Fazenda",
                     u"Feira", u"Jardim", u"Jd.", u"Ladeira", u"Lago",
                     u"Lagoa", u"Largo", u"Loteamento", u"Morro", u"Núcleo",
                     u"Parque", u"Pq.", u"Passarela", u"Pátio", u"Praça", u"Pç.", u"Quadra",
                     u"Recanto", u"Residencial", u"Resid.", u"Rotatória", u"Rua", u"R.",
                     u"Setor", u"Sítio", u"Travessa", u"Tv.", u"Trecho", u"Trevo",
                     u"Vale", u"Vereda", u"Via", u"V.", u"Viaduto", u"Viela",
                     u"Vila", u"Vl.",

                     # Portuguese (being distinct from Brazilian portuguese)
                     u"Avª", u"Av.ª", u"Azinhaga", u"Bco.", u"Cais", u"Calçada", u"Cc.",
                     u"Calçadinha", u"Ccnh", u"Esc.", u"Escadaria", u"Escada", u"Escadas",
                     u"Escnh.", u"Escadinhas", u"Jrd.", u"Lg.", u"Lug.", u"Lugar", u"Part.",
                     u"Particular", u"Passeio", u"Pr.", u"Pc.", u"Pct.", u"Pctª", u"Pct.ª",
                     u"Ponte", u"Praceta", u"Pto.", u"Prolongamento", u"Prolng.", u"Rampa",
                     u"Rot.", u"Rotunda", u"Terreiro", u"Transversal", u"Transv.", u"Trv.",
                     u"Trav.", u"Trilho", u"Túnel",

                     # Angola (being distinct from Brazilian portuguese and Portuguese)
                     u"Via Expressa"
                   ]


    DETERMINANTS = [ # Brazilian portuguese
                     u" do", u" da", u" dos", u" das",

                     # Portuguese (being distinct from Brazilian portuguese)
                     u" de",
                     u""]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)
    C_ACCENT = re.compile(ur"[ç]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        s = self.C_ACCENT.sub("c", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_ar_generic(i18n):
    APPELLATIONS = [ u"شارع", u"طريق", u"زقاق", u"نهج", u"جادة",
                     u"ممر", u"حارة",
                     u"كوبري", u"كوبرى", u"جسر", u"مطلع", u"منزل",
                     u"مفرق", u"ملف", u"تقاطع",
                     u"ساحل",
                     u"ميدان", u"ساحة", u"دوار" ]

    DETERMINANTS = [ u" ال", u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    A_ACCENT = re.compile(ur"[اإآ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.A_ACCENT.sub("أ", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

    def isrtl(self):
        return True

class i18n_ru_generic(i18n):
    # Based on list from Streetmangler:
    # https://github.com/AMDmi3/streetmangler/blob/master/lib/locales/ru.cc
    STATUS_PARTS = [
        (u"улица", [u"ул"]),
        (u"площадь", [u"пл"]),
        (u"переулок", [u"пер", u"пер-к"]),
        (u"проезд", [u"пр-д"]),
        (u"шоссе", [u"ш"]),
        (u"бульвар", [u"бул", u"б-р"]),
        (u"тупик", [u"туп"]),
        (u"набережная", [u"наб"]),
        (u"проспект", [u"просп", u"пр-кт", u"пр-т"]),
        (u"линия", []),
        (u"аллея", []),
        (u"метромост", []),
        (u"мост", []),
        (u"просек", []),
        (u"просека", []),
        (u"путепровод", []),
        (u"тракт", [u"тр-т", u"тр"]),
        (u"тропа", []),
        (u"туннель", []),
        (u"тоннель", []),
        (u"эстакада", [u"эст"]),
        (u"дорога", [u"дор"]),
        (u"спуск", []),
        (u"подход", []),
        (u"подъезд", []),
        (u"съезд", []),
        (u"заезд", []),
        (u"разъезд", []),
        (u"слобода", []),
        (u"район", [u"р-н"]),
        (u"микрорайон", [u"мкр-н", u"мк-н", u"мкр", u"мкрн"]),
        (u"посёлок", [u"поселок", u"пос"]),
        (u"деревня", [u"дер", u"д"]),
        (u"квартал", [u"кв-л", u"кв"]),
    ]

    # matches one or more spaces
    SPACE_REDUCE = re.compile(r"\s+")
    # mapping from status abbreviations (w/o '.') to full status names
    STATUS_PARTS_ABBREV_MAPPING = dict((f, t) for t, ff in STATUS_PARTS for f in ff)
    # set of full (not abbreviated) status parts
    STATUS_PARTS_FULL = set((x[0] for x in STATUS_PARTS))
    # matches any abbreviated status part with optional '.'
    STATUS_ABBREV_REGEXP = re.compile(r"\b(%s)\.?(?=\W|$)" % u"|".join(
        f for t, ff in STATUS_PARTS for f in ff), re.IGNORECASE | re.UNICODE)
    # matches status prefixes at start of name used to move prefixes to the end
    PREFIX_REGEXP = re.compile(
        ur"^(?P<num_prefix>\d+-?(ы?й|я))?\s*(?P<prefix>(%s)\.?)?\s*(?P<name>.+)?" %
        (u"|".join(f for f,t in STATUS_PARTS)), re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        # usually, there are no accents in russian names, only "ё" sometimes, but
        # not as first letter
        return s.upper()

    def language_code(self):
        return self.language

    @staticmethod
    def _rewrite_street_parts(matches):
        if (matches.group('num_prefix') is None and
            matches.group('prefix') is not None and
            matches.group('name') in i18n_ru_generic.STATUS_PARTS_FULL):
            return matches.group(0)
        elif matches.group('num_prefix') is None and matches.group('prefix') is None:
            return matches.group(0)
        elif matches.group('name') is None:
            return matches.group(0)
        else:
            #print matches.group('num_prefix', 'prefix', 'name')
            return ", ".join((matches.group('name'),
                " ". join(s.lower()
                    for s in matches.group('prefix', 'num_prefix')
                    if s is not None)
                ))

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        # Normalize abbreviations
        name = self.STATUS_ABBREV_REGEXP.sub(lambda m:
                self.STATUS_PARTS_ABBREV_MAPPING.get(
                    m.group(0).replace('.', ''), m.group(0)),
            name)
        # Move prefixed status parts to the end for sorting
        name = self.PREFIX_REGEXP.sub(self._rewrite_street_parts, name)
        # TODO: move "малая", "большая" after name but before status
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_be_generic(i18n):
    # Based on code for Russian language:
    STATUS_PARTS = [
        (u"вуліца", [u"вул"]),
        (u"плошча", [u"пл"]),
        (u"завулак", [u"зав", u"зав-к"]),
        (u"праезд", [u"пр-д"]),
        (u"шаша", [u"ш"]),
        (u"бульвар", [u"бул", u"б-р"]),
        (u"тупік", [u"туп"]),
        (u"набярэжная", [u"наб"]),
        (u"праспект", [u"праспект", u"пр-кт", u"пр-т"]),
        (u"алея", []),
        (u"мост", []),
        (u"парк", []),
        (u"тракт", [u"тр-т", u"тр"]),
        (u"раён", [u"р-н"]),
        (u"мікрараён", [u"мкр-н", u"мк-н", u"мкр", u"мкрн"]),
        (u"пасёлак", [u"пас"]),
        (u"вёска", [ u"в"]),
        (u"квартал", [u"кв-л", u"кв"]),
    ]

    # matches one or more spaces
    SPACE_REDUCE = re.compile(r"\s+")
    # mapping from status abbreviations (w/o '.') to full status names
    STATUS_PARTS_ABBREV_MAPPING = dict((f, t) for t, ff in STATUS_PARTS for f in ff)
    # set of full (not abbreviated) status parts
    STATUS_PARTS_FULL = set((x[0] for x in STATUS_PARTS))
    # matches any abbreviated status part with optional '.'
    STATUS_ABBREV_REGEXP = re.compile(r"\b(%s)\.?(?=\W|$)" % u"|".join(
        f for t, ff in STATUS_PARTS for f in ff), re.IGNORECASE | re.UNICODE)
    # matches status prefixes at start of name used to move prefixes to the end
    PREFIX_REGEXP = re.compile(
        ur"^(?P<num_prefix>\d+-?(і|ы|я))?\s*(?P<prefix>(%s)\.?)?\s*(?P<name>.+)?" %
        (u"|".join(f for f,t in STATUS_PARTS)), re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        return s.upper()

    def language_code(self):
        return self.language

    @staticmethod
    def _rewrite_street_parts(matches):
        if (matches.group('num_prefix') is None and
            matches.group('prefix') is not None and
            matches.group('name') in i18n_be_generic.STATUS_PARTS_FULL):
            return matches.group(0)
        elif matches.group('num_prefix') is None and matches.group('prefix') is None:
            return matches.group(0)
        elif matches.group('name') is None:
            return matches.group(0)
        else:
            #print matches.group('num_prefix', 'prefix', 'name')
            return ", ".join((matches.group('name'),
                " ". join(s.lower()
                    for s in matches.group('num_prefix', 'prefix')
                    if s is not None)
                ))

    def user_readable_street(self, name):
        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        # Normalize abbreviations
        name = self.STATUS_ABBREV_REGEXP.sub(lambda m:
                self.STATUS_PARTS_ABBREV_MAPPING.get(
                    m.group(0).replace('.', ''), m.group(0)),
            name)
        # Move prefixed status parts to the end for sorting
        name = self.PREFIX_REGEXP.sub(self._rewrite_street_parts, name)
        # TODO: move "малая", "большая" after name but before status
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_nl_generic(i18n):
    #
    # Dutch streets are often named after people and include a title.
    # The title will be captured as part of the <prefix>
    #
    APPELLATIONS = [ u"St.", u"Sint", u"Ptr.", u"Pater",
                     u"Prof.", u"Professor", u"Past.", u"Pastoor",
                     u"Pr.", u"Prins", u"Prinses", u"Gen.", u"Generaal",
                     u"Mgr.", u"Monseigneur", u"Mr.", u"Meester",
                     u"Burg.", u"Burgermeester", u"Dr.", u"Dokter",
                     u"Ir.", u"Ingenieur", u"Ds.", u"Dominee", u"Deken",
                     u"Drs.", u"Maj.", u"Majoor",
                     # counting words before street name,
                     # e.g. "1e Walstraat" => "Walstraat (1e)"
                     u"\d+e",
                     u"" ]
    #
    # Surnames in Dutch streets named after people tend to have the middle name
    # listed after the rest of the surname,
    # e.g. "Prins van Oranjestraat" => "Oranjestraat (Prins van)"
    # Likewise, articles are captured as part of the prefix,
    # e.g. "Den Urling" => "Urling (Den)"
    #
    DETERMINANTS = [ u"\s?van der", u"\s?van den", u"\s?van de", u"\s?van",
                     u"\s?Den", u"\s?D'n", u"\s?D'", u"\s?De", u"\s?'T", u"\s?Het",
                     u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)),
                                      re.IGNORECASE | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        matches = self.PREFIX_REGEXP.match(name)
        #
        # If no prefix was captured, that's okay. Don't substitute
        # the name however, "<name> ()" looks silly
        #
        if matches == None:
            return name

        if matches.group('prefix'):
            name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_hr_HR(i18n):
    # for upper_unaccent_string
    C_ACCENT = re.compile(ur"[ćč]", re.IGNORECASE | re.UNICODE)
    D_ACCENT = re.compile(ur"đ|dž", re.IGNORECASE | re.UNICODE)
    N_ACCENT = re.compile(ur"nj", re.IGNORECASE | re.UNICODE)
    L_ACCENT = re.compile(ur"lj", re.IGNORECASE | re.UNICODE)
    S_ACCENT = re.compile(ur"š", re.IGNORECASE | re.UNICODE)
    Z_ACCENT = re.compile(ur"ž", re.IGNORECASE | re.UNICODE)

    def upper_unaccent_string(self, s):
        s = self.C_ACCENT.sub("c", s)
        s = self.D_ACCENT.sub("d", s)
        s = self.N_ACCENT.sub("n", s)
        s = self.L_ACCENT.sub("l", s)
        s = self.S_ACCENT.sub("s", s)
        s = self.Z_ACCENT.sub("z", s)
        return s.upper()

    def __init__(self, language, locale_path):
        """Install the _() function for the chosen locale other
           object initialisation"""
        self.language = str(language) # FIXME: why do we have unicode here?
        _install_language(language, locale_path)

    def language_code(self):
        """returns the language code of the specific language
           supported, e.g. fr_FR.UTF-8"""
        return self.language

    def user_readable_street(self, name):
        """ transforms a street name into a suitable form for
            the map index, e.g. Paris (Rue de) for French"""
        return name

    ## FIXME: only first letter does not work for Croatian digraphs (dž, lj, nj)
    def first_letter_equal(self, a, b):
        """returns True if the letters a and b are equal in the map index,
           e.g. É and E are equals in French map index"""
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_pl_generic(i18n):

    APPELLATIONS = [ u"Dr.", u"Doktora", u"Ks.", u"Księdza",
                     u"Generała", u"Gen.",
                     u"Aleja", u"Plac", u"Pl.",
                     u"Rondo", u"rondo", u"Profesora",
                     u"Prof.",
                     u"" ]

    DETERMINANTS = [ u"\s?im.", u"\s?imienia", u"\s?pw.",
                     u"" ]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)),
                                      re.IGNORECASE | re.UNICODE)


    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        matches = self.PREFIX_REGEXP.match(name)
        #
        # If no prefix was captured, that's okay. Don't substitute
        # the name however, "<name> ()" looks silly
        #
        if matches == None:
            return name

        if matches.group('prefix'):
            name = self.PREFIX_REGEXP.sub(r"\g<name>, \g<prefix>", name)
        return name

    def first_letter_equal(self, a, b):
        return a == b

class i18n_tr_generic(i18n):
    APPELLATIONS = [ u"Sokak", u"Sokağı" ]
    DETERMINANTS = []
    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_de_generic(i18n):
    #
    # German streets are often named after people and include a title.
    # The title will be captured as part of the <prefix>
        # Covering airport names and "New"/"Old" as prefixes as well
    #
    APPELLATIONS = [ u"Alte", u"Alter", u"Doktor", u"Dr.",
                     u"Flughafen", u"Flugplatz", u"Gen.,", u"General",
                     u"Neue", u"Neuer", u"Platz",
                     u"Prinz", u"Prinzessin", u"Prof.",
                     u"Professor" ]
    #
    # Surnames in german streets named after people tend to have the middle name
    # listed after the rest of the surname,
    # e.g. "Platz der deutschen Einheit" => "deutschen Einheit (Platz der)"
    # Likewise, articles are captured as part of the prefix,
    # e.g. "An der Märchenwiese" => "Märchenwiese (An der)"
    #
    DETERMINANTS = [ u"\s?An den", u"\s?An der", u"\s?Am",
                     u"\s?Auf den" , u"\s?Auf der"
                     u" an", u" des", u" der", u" von", u" vor"]

    SPACE_REDUCE = re.compile(r"\s+")
    PREFIX_REGEXP = re.compile(r"^(?P<prefix>(%s)(%s)?)\s?\b(?P<name>.+)" %
                                    ("|".join(APPELLATIONS),
                                     "|".join(DETERMINANTS)), re.IGNORECASE
                                                                 | re.UNICODE)

    # for IndexPageGenerator.upper_unaccent_string
    E_ACCENT = re.compile(ur"[éèêëẽ]", re.IGNORECASE | re.UNICODE)
    I_ACCENT = re.compile(ur"[íìîïĩ]", re.IGNORECASE | re.UNICODE)
    A_ACCENT = re.compile(ur"[áàâäã]", re.IGNORECASE | re.UNICODE)
    O_ACCENT = re.compile(ur"[óòôöõ]", re.IGNORECASE | re.UNICODE)
    U_ACCENT = re.compile(ur"[úùûüũ]", re.IGNORECASE | re.UNICODE)

    def __init__(self, language, locale_path):
        self.language = str(language)
        _install_language(language, locale_path)

    def upper_unaccent_string(self, s):
        s = self.E_ACCENT.sub("e", s)
        s = self.I_ACCENT.sub("i", s)
        s = self.A_ACCENT.sub("a", s)
        s = self.O_ACCENT.sub("o", s)
        s = self.U_ACCENT.sub("u", s)
        return s.upper()

    def language_code(self):
        return self.language

    def user_readable_street(self, name):
        #
        # Make sure name actually contains something,
        # the PREFIX_REGEXP.match fails on zero-length strings
        #
        if len(name) == 0:
            return name

        name = name.strip()
        name = self.SPACE_REDUCE.sub(" ", name)
        name = self.PREFIX_REGEXP.sub(r"\g<name> (\g<prefix>)", name)
        return name

    def first_letter_equal(self, a, b):
        return self.upper_unaccent_string(a) == self.upper_unaccent_string(b)

class i18n_ast_generic(i18n):

    APPELLATIONS = [ # Asturian
                     u"Accesu", u"Autopista", u"Autovia", u"Avenida",
                     u"Baxada", u"Barrancu", u"Barriu", u"Barriada",
                     u"Biblioteca", u"Cai", u"Caleya",
                     u"Calzada", u"Camín", u"Carretera", u"Cuesta",
                     u"Estación", u"Hospital", u"Iglesia", u"Monasteriu",
                     u"Monumentu", u"Muelle", u"Muséu",
                     u"Palaciu", u"Parque", u"Pasadizu", u"Pasaxe",
                     u"Paséu", u"Planta", u"Plaza", u"Pol

Rui <zenter>
Thu 10 Jan 2013 08:11:33 PM UTC, comment #2: 

Translation now happens on Transifex https://www.transifex.com/projects/p/maposmatic

I've cloned the pt-BR translation and updated it with your translation files, but you'd want to go through it since I think there might have been some changes since you translated.

Guttorm Flatabø <dittaeva>
Wed 13 Jun 2012 10:25:24 PM UTC, comment #1: 

I forgot to tell about sorting order (I think it has something to do with those python files). At this moment if I render a map in MapOSMatic with Brazilian portuguese language selected I get the following index sort:

  • D. Francisco Manuel de Melo (Rua)
  • Diario de Noticias (Rua do)
  • D. João de Castro (Rua)


but it should be:

  • D. Francisco Manuel de Melo (Rua)
  • D. João de Castro (Rua)
  • Diario de Noticias (Rua do)




although the following sort order is correct (extracted from an actual MapOSMatic map index):
Dr. Alfredo Bensaúde (Avenida)
Dr Alfredo da Costa (Rua)
Dr. António Granjo (R.)

this one in the middle (Dr Alfredo da Costa) has a typing error since period (.) is missing and should be fixed in OpenStreetMap, but this happens a lot so would be better to keep this sorting order in this last example.

If this is a lot of work, just ignore. All those abbreviations should be cleaned in OpenStreetMap (Dr. > Doutor; D. > Dom; etc.)

RPome <rpome>
Wed 13 Jun 2012 02:23:00 PM UTC, original submission:  

Here it is the portuguese translation and localization for portuguese speaking countries.

I don't know if Macao should be included in MAP_LANGUAGES since Portuguese and Chinese are both official languages unless we use "Macau (PT)" and "澳門 (广州话 / 廣州話)" or something like that (I don't know cantonese, those symbols were extracted from en.wikipedia). Since Macao is not a country, but a special administrative region of China, and only a few people understand portuguese in Macao, maybe shouldn't be included. Although we can see in OpenStreetMap Macao has portuguese names in some streets. Equatorial Guinea is another story, I don't know if they have portuguese street names and if portuguese is now really a official language.

Note that all countries listed here should use portuguese interface (pt / pt-PT / standard portuguese / international portuguese) except Brazil (pt-BR)

I don't have experience in Python and this is only the second time I translate .po files. I only have experience translating some software (wikipedia and openstreetmap related software/interfaces) via web based translation tools/resources. So please check if the code is correct.



In http://git.savannah.gnu.org/cgit/maposmatic.git/tree/www/settings.py#n111 add the following code:


LANGUAGES =    ("pt", u"Português"),

PAYPAL_LANGUAGES = {
    "pt": ("pt_BR", "PT"),

MAP_LANGUAGES = [
    ("pt_PT.UTF-8", u"Portugal (PT)"),
    ("pt_PT.UTF-8", u"Cabo Verde (PT)"),
    ("pt_PT.UTF-8", u"Angola (PT)"),
    ("pt_PT.UTF-8", u"Moçambique (PT)"),
    ("pt_PT.UTF-8", u"São Tomé e Príncipe (PT)"),
    ("pt_PT.UTF-8", u"Guiné-Bissau (PT)"),
    ("pt_PT.UTF-8", u"Timor-Leste (PT)"),






http://git.savannah.gnu.org/cgit/maposmatic/ocitysmap.git/tree/ocitysmap/i18n.py

I didn't changed words in APPELLATIONS (Brazilian portuguese) although I don't agree on some words like the first one "Aeroporto" (Airport), it shouldn't be included. I could suggest another class for portuguese, but brazilian portuguese is portuguese with only a few differences (ISO code is the same: "pt") so I followed the example "class i18n_ca_generic(i18n)" in http://git.savannah.gnu.org/cgit/maposmatic/ocitysmap.git/tree/ocitysmap/i18n.py and annexed portuguese to brazilian portuguese

In http://git.savannah.gnu.org/cgit/maposmatic/ocitysmap.git/tree/ocitysmap/i18n.py add the following code ("Brazilian portuguese" is already in that page)


class i18n_pt_generic(i18n):        # (changed "i18n_pt_br_generic" to "i18n_pt_generic", ISO code: "PT")

    APPELLATIONS = [ # Brazilian portuguese
                     u"Aeroporto", u"Aer.", u"Alameda", u"Al.", u"Apartamento", u"Ap.",
                     u"Área", u"Avenida", u"Av.", u"Beco", u"Bc.", u"Bloco", u"Bl.",
                     u"Caminho", u"Cam.", u"Campo", u"Chácara", u"Colônia",
                     u"Condomínio", u"Conjunto", u"Cj.", u"Distrito", u"Esplanada", u"Espl.",
                     u"Estação", u"Est.", u"Estrada", u"Estr.", u"Favela", u"Fazenda",
                     u"Feira", u"Jardim", u"Jd.", u"Ladeira", u"Lago",
                     u"Lagoa", u"Largo", u"Loteamento", u"Morro", u"Núcleo",
                     u"Parque", u"Pq.", u"Passarela", u"Pátio", u"Praça", u"Pç.", u"Quadra",
                     u"Recanto", u"Residencial", u"Resid.", u"Rua", u"R.",
                     u"Setor", u"Sítio", u"Travessa", u"Tv.", u"Trecho", u"Trevo",
                     u"Vale", u"Vereda", u"Via", u"V.", u"Viaduto", u"Viela",
                     u"Vila", u"Vl.",

                     # Portuguese (being distinct from Brazilian portuguese)
                     u"Avª",  u"Av.ª", u"Calçada", u"Esc.", u"Escadaria", u"Escadas",
                     u"Escnh.", u"Escadinhas", u"Jrd.", u"Lg.", u"Part.",
                     u"Particular", u"Pr.", u"Pct.", u"Pctª", u"Praceta", u"Rot.", u"Rotunda",
                     u"Trv.", u"Trav.",

                     # "Angola" (being distinct from Brazilian portuguese and Portuguese)
                     u"Via Expressa"
                   ]


    DETERMINANTS = [ # Brazilian portuguese
                     u" do", u" da", u" dos", u" das",

                     # Portuguese (being distinct from Brazilian portuguese)
                     u" de", u""]



    # for IndexPageGenerator.upper_unaccent_string
    C_ACCENT = re.compile(ur"[ç]", re.IGNORECASE | re.UNICODE)


    def upper_unaccent_string(self, s):
        s = self.C_ACCENT.sub("c", s)


LANGUAGES =
             ("pt", u"Português"),


language_class_map = {
    'pt_BR.UTF-8': i18n_pt_generic,                     # (changed "i18n_pt_br_generic" to "i18n_pt_generic", ISO code is "PT")
    'pt_PT.UTF-8': i18n_pt_generic,
    'pt_AO.UTF-8': i18n_pt_generic,
    'pt_GW.UTF-8': i18n_pt_generic,
    'pt_ST.UTF-8': i18n_pt_generic,
    'pt_MZ.UTF-8': i18n_pt_generic,
    'pt_CV.UTF-8': i18n_pt_generic,
    'pt_TL.UTF-8': i18n_pt_generic,



See also the attached files with translations. Note that I used brazilian portuguese translation (since only a few things should be modified) although I checked the English version. I found a few errors (and a few sentences not translated) in Brazilian portuguese translation so I made modifications to that file too.

P.S. it would be great to use launchpad.net or translatewiki.net (even better) to create / update translations. Anyway thanks for this great tool.

RPome <rpome>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #26016:  django(brasilian_portuguese_pt-BR).po added by rpome (44KiB - application/octet-stream)
file #26015:  django(portuguese_pt).po added by rpome (44KiB - application/octet-stream)
file #26014:  ocitysmap(portuguese_pt).po added by rpome (4KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by zenter (Posted a comment)
  • -email is unavailable- added by dittaeva (Posted a comment)
  • -email is unavailable- added by dmentre (Updated the item)
  • -email is unavailable- added by rpome (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-09-05 dmentre Assigned toNone mpetazzoni
    2012-06-13 rpome Attached File- Added ocitysmap(portuguese_pt).po, #26014
        Attached File- Added django(portuguese_pt).po, #26015
        Attached File- Added django(brasilian_portuguese_pt-BR).po, #26016

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code