bugVarnamproject - Bugs: bug #41902, [libvarnam] Normalization of words...

 
 

bug #41902: [libvarnam] Normalization of words while learning

Submitted by:  Navaneeth <navaneethkn>
Submitted on:  Wed 19 Mar 2014 04:50:08 AM UTC  
 
Category: libvarnamSeverity: 3 - Normal
Item Group: BugStatus: None
Privacy: PublicAssigned to: None
Open/Closed: Open

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 28 Mar 2014 01:57:13 PM UTC, comment #2:

Kiran,

Thanks for looking into this. But the patch you have submitted is not what it needs. This is much larger and needs to be configured at the scheme file level. Basically the scheme file will have a rule which specify how normalization should happen for the particular language and libvarnam applies it during learning.

Navaneeth <navaneethkn>
Project Administrator
Thu 27 Mar 2014 10:47:38 AM UTC, comment #1:

added function to remove soft-hyphen

https://gitorious.org/varnamproject/kiru-libvarnam/commit/3d1ac24bc05e044b840a6874fdf3e1f3baac70bb

kiran <pskirann>
Wed 19 Mar 2014 04:50:08 AM UTC, original submission:

In Unicode, there are characters which look same but have different code points. The atomic chills is Malayalam is an example. Also Unicode text can contain metadata characters like "SOFT HYPHEN (xad)". This has to be removed and normalized to a standard form while learning a word.

Scheme file will define the word normalization rules and varnam will apply them while learning.

Navaneeth <navaneethkn>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by pskirann (Posted a comment)
  • -unavailable- added by navaneethkn (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup