bugWWWJDic - Bugs: bug #45558, Better Parsing

 
 

bug #45558: Better Parsing

Submitted by:  Tha Leang <vleango>
Submitted on:  Thu 16 Jul 2015 01:58:51 AM UTC  
 
Category: Improvement requestSeverity: 3 - Normal
Status: FixedPrivacy: Public
Assigned to: Marco Bresciani <marcobresciani>Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 28 Jul 2015 12:59:55 PM UTC, comment #7:

I love the new improvements from 9.2.0
You can close this and I want to make a "Better Parsing II" thread.

Tha Leang <vleango>
Wed 22 Jul 2015 07:43:20 PM UTC, comment #6:

See 9.2.0 release.

Marco Bresciani <marcobresciani>
Project AdministratorIn charge of this item.
Tue 21 Jul 2015 08:16:38 PM UTC, comment #5:

In version 9.1.0 there is a new "lines" element in the JSON result of the translation. It contains the split lines of the raw text translation field.

Marco Bresciani <marcobresciani>
Project AdministratorIn charge of this item.
Tue 21 Jul 2015 08:04:02 PM UTC, comment #4:

Better a "lines" element...

Marco Bresciani <marcobresciani>
Project AdministratorIn charge of this item.
Tue 21 Jul 2015 07:54:46 PM UTC, comment #3:

First step for 9.1.0 version: I'm going to add a "translations" array with simple split of the text lines.

Marco Bresciani <marcobresciani>
Project AdministratorIn charge of this item.
Thu 16 Jul 2015 12:21:45 PM UTC, comment #2:

Using this result http://www.edrdg.org/cgi-bin/wwwjdic/wwwjdic?1ZDEglasses as sample reference example, I'd add a "translations" array having these fields:

"translations": [
{
"word": "アイケアグラス;アイケア・グラス",
"reading": "",
"tags": [ "n" ],
"translation": ["safety glasses (esp. more fashionable models used in public for protection against pollen, UV rays, etc.) (wasei: eye-care glass)"]
},

{
"word": "杯盤狼藉",
"reading": "はいばんろうぜき",
"tags": [ "adj-t,adv-to", "arch", "yoji" ]
"translation": [ "glasses and plates being scattered about after a party or banquet", "articles lying about in a jumble" ]

},

{
"word": "杯盤",
"reading": "はいばん",
"tags": [ "n" ]
"translation": ["glasses and plates"]
}
]

and so on... there are interesting and different possibilities and issues that may raise from the parsing. Consider the line below:

蔓 [つる] /(n) (1) (uk) vine/bine/tendril/runner/(2) bow/temple arm/sidepiece/lug arm/earpiece/part of a pair of glasses that goes over the ear/(3) connections/contacts/influence/financial supporter/

If I split on "/", the (1) definition will be split in four different translations while it's actually the same...

取る [とる] /(v5r,vt) (1) to take/to pick up/to harvest/to earn/to win/to choose/(2) (See 盗る) to steal/(3) (See 摂る) to eat/to have (a meal)/(4) (also written as 脱る) to remove (one's glasses, etc.)/(5) to compete (in sumo, cards, etc.)/to play/(P)/

For this, I have to consider "tags" for each part (1)...(5) of the translation since the "See" would be considered like an information tag.

メガネっ娘;眼鏡っ娘;メガネっ子;眼鏡っ子;めがねっ娘;眼鏡娘 [めがねっこ(眼鏡っ娘,眼鏡っ子,めがねっ娘,眼鏡娘);メガネっこ(メガネっ娘,メガネっ子)] /(n) (m-sl) girl (usu. attractive) with glasses/glasses-wearing girl/

What about multiple translations or even multiple readings spread in the translation? How should I care about ";" separators? Some readings are also in [] other in () ...

凹眼鏡 [おうがんきょう] /(n) concave glasses/

This is easy... eh eh eh. ;)

老眼鏡 [ろうがんきょう] /(n) reading (farsighted) glasses/spectacles for the aged/

Gosh, here the (farsighted) comment would be considered like a tag? It depends...

ALSO TO BE CONFIRMED: actual output and example from Japanese to another language!

What about a first rough step in which I cut the lines in a "translations" array?

Marco Bresciani <marcobresciani>
Project AdministratorIn charge of this item.
Thu 16 Jul 2015 11:56:02 AM UTC, comment #1:

Good idea for an improvement. Could you please write me a full JSON output example, highlighting your idea about how to improve the output?

According to your example:

  • I'd split on the first white space to create the "word" field with the first element;
  • Use the bracket to highlight the reading;
  • split on the slash to compute an array of translations with the "type" from inside round parentheses(n for name, adj for adjective or so, probably) and the text that is contained between the slashes. Probably the language is not necessarily needed since the input language is known... is it?

Is it correct?

Do you prefer to keep this as a separate feature (a complete new entry in the resulting JSON) or the translations array, for example, should be in parallel with currently existing fields?

This:

------------------------------------------
word: ...
uri: ...
content: {
word: ...
reading: ...
translations: [...]
}
------------------------------------------

or this one?

------------------------------------------
word: ...
uri: ...
reading: ...
translations : [...]
------------------------------------------

Marco Bresciani <marcobresciani>
Project AdministratorIn charge of this item.
Thu 16 Jul 2015 01:58:51 AM UTC, original submission:

Feature request:

Have the json return more specific information.

Example:
学校医 [がっこうい] /(n) school physician/ another translation/

I think it would be great to get something like this:

{
"word": "学校医"
"reading": "がっこうい"
"translations" : [
{
"text" : "school physician",
"pos" : "n",
"lng" : "E" },
{
"text" : "another translation"
}
]
}

Of course, there are a lot more fields and data on each entry. I'm doing some research right now on all of the fields.

Tha Leang <vleango>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by vleango (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 29 Jul 2015 01:21:00 PM UTCmarcobrescianiStatusWorks For Me=>Fixed
      Open/ClosedOpen=>Closed
    Wed 22 Jul 2015 07:43:20 PM UTCmarcobrescianiStatusReady For Test=>Works For Me
    Tue 21 Jul 2015 08:16:38 PM UTCmarcobrescianiStatusIn Progress=>Ready For Test
    Tue 21 Jul 2015 06:49:02 PM UTCmarcobrescianiStatusNeed Info=>In Progress
    Thu 16 Jul 2015 11:56:02 AM UTCmarcobrescianiCategoryNone=>Improvement request
      StatusNone=>Need Info
      Assigned toNone=>marcobresciani

    Back to the top


    Powered by Savane 3.1-cleanup1