taskSwathanthra Malayalam Computing - Tasks: task #9305, Convert English-Malayalam...


You are not allowed to post comments on this tracker with your current authentication level.

task #9305: Convert English-Malayalam dictionary to TEI XML format

Submitter:  Rajeesh K Nambiar <rajeeshknambiar>
Submitted:  Mon 06 Apr 2009 05:40:39 PM UTC
Category:  dictionaries Should Start On:  Tue 07 Apr 2009 12:00:00 AM UTC
Should be Finished on:  Sun 27 Feb 2011 06:30:00 PM UTC Priority:  5 - Normal
Status:  In Progress Privacy:  Public
Assigned to:  iamsunilk Percent Complete:  50%
Open/Closed:  Open Effort:  0.00

Mon 21 Feb 2011 03:05:39 PM UTC, comment #4: 

Dear Santhosh,

Only the teiheader part requires some modification such as proper copyright notice and maintainer etc.
I put some info there. But I dont know whether I missed anything

Other wise data part I thing it is complete. Tei to dict conversion is also working fine. But the following test I not able to do "xmllint --noout --valid eng-mal.tei"

I am not sure any other files are required.

Sunil K <iamsunilk>
Group Member
Mon 21 Feb 2011 11:31:40 AM UTC, comment #3: 

Did we submit this to upstream? What is pending ?

Santhosh Thottingal <santhosh>
Group administrator
Tue 25 Jan 2011 10:14:37 PM UTC, comment #2: 

Dear Santhosh,

Thanks for the feedback. Thanks to that I have a better version of tei file (Header info need lot of modification )

1. I am not able to validate using http://sourceforge.net/apps/mediawiki/freedict/index.php?title=FreeDict_HOWTO_-_12
I am using debian (TEI DTD are not available from rep)

But I validated using http://wiki.tei-c.org/index.php/TEI-to-DICT_howto

xmllint --noout your-dictionary.tei

Other test I am not able do due to some errors
xmllint --noout --valid eng-mal.tei
eng-mal.tei:2: validity error : Validation failed: no
DTD found !
<TEI xmlns="http://www.tei-c.org/ns/1.0">

also the conversion given in the link is also working fine

Transform the TEI file into something appropriate to be fed into dictfmt by entering

xsltproc -o dictionary.c5 -novalid --stringparam current-date
$(date) ../CVS/tools/xsl/tei2c5.xsl dictionary.tei

Transform the intermediate file into the database and index files by entering one (or a combination) of the following:

dictfmt -t --utf8 my_dictionary < dictionary.c5

dictfmt -t --headword-separator %%% --utf8 my_dictionary <

dictfmt -t -s <short_descriptive_name> --utf8 my_dictionary <


And the resulting dictionary file is working fine in goldendict

2. Header file is not modified ---now it just a copy from eng-fra  will do modify soon but I dont know the contents to be put.

3. whitespaces removed

one method described in http://wiki.tei-c.org/index.php/TEI-to-DICT_howto is working (given above) for converting from tei to dict
 Note: there are  few split lines in the dict, I removed quite a few of them but this may sometimes lead to missing part of the info(malayalam meaning). I will look into it once again 

I didn;t use dict2tei.py because initially it didn;t worked for me. even now it is not giving output that can be validated

output of dict2tei.py is

<def>       1. കന്യകാമഠം</def>
<def>       2. മഠത്തോട് ബന്ധപ്പെട്ട ദേവാലയം</def>
<def>       3. സന്യാസിമഠം</def>
<def>       1. മഠാധിപതി</def>

but the output in the format given in eng-fra and eng-hin dicts is of form
            <sense n="1">
               <cit type="trans">
               <cit type="trans">
            <sense n="2">
               <cit type="trans">

So the script because of that (I know little bit perl but no python even to debug)

to run the script
 perl tei2dict file.dict file.tei  (file should not have comments)

git I have to learn. I will do it tomorrow

Thanks & Best regards


Sunil K <iamsunilk>
Group Member
Tue 25 Jan 2011 04:30:20 PM UTC, comment #1: 

Thanks Sunil, good work!
Some points:
1. Did you validate the TEI xml file as documented here http://sourceforge.net/apps/mediawiki/freedict/index.php?title=FreeDict_HOWTO_-_12
2. In the XML file, I see wrong information in publicationStmt node. Similarly sourceDesc node. You may refer http://wiki.smc.org.in/Dictionary and dont forget to add your name in credits :)
3. In orth and quote node content, the white spaces before and after the word is intentional or by mistake?

Is there any too to convert the xml to dict format again?
Did you wrote the perl script yourself for this purpose? I see http://sourceforge.net/apps/mediawiki/freedict/index.php?title=FreeDict_HOWTO_-_7 talking about dict2tei.py - a python script.

And please add the xml and perl script to our git.

Santhosh Thottingal <santhosh>
Group administrator
Mon 06 Apr 2009 05:40:39 PM UTC, original submission:  

The existing data used to create English-Malayalam dictionary is to be formatted to Text Encoding Initiative(TEI)'s XML format before submitting to freedict.org for accepting.

See http://apps.sourceforge.net/mediawiki/freedict/index.php?title=FreeDict_HOWTO_-_TOC for further details.

Rajeesh K Nambiar <rajeeshknambiar>
Group administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #22514:  eng-mal1.tei.tar.gz added by iamsunilk (445KiB - application/x-gzip)
file #22515:  dict2tei.pl added by iamsunilk (654B - application/octet-stream)
file #22499:  eng-mal.tei.tar.gz added by iamsunilk (445KiB - application/x-gzip - Uploading eng-mal.tei based on eng-hin and eng-fra )
file #22500:  dict2tei.pl added by iamsunilk (639B - application/octet-stream - Uploading eng-mal.tei based on eng-hin and eng-fra )


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by santhosh (Posted a comment)
  • -email is unavailable- added by iamsunilk (Updated the item)
  • -email is unavailable- added by rajeeshknambiar (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-01-25 iamsunilk Attached File- Added eng-mal1.tei.tar.gz, #22514
        Attached File- Added dict2tei.pl, #22515
    2011-01-25 santhosh Should be Finished on2009-06-06 2011-02-27
        StatusNone In Progress
        Percent Complete0% 50%
    2011-01-23 iamsunilk Assigned toNone iamsunilk
    2011-01-23 iamsunilk Attached File- Added eng-mal.tei.tar.gz, #22499
        Attached File- Added dict2tei.pl, #22500

    Back to the top

    Powered by Savane 3.13-3cd0.
    Corresponding source code