bugGrammatica - Bugs: bug #12767, Tokenizer does not do '/* **/'...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #12767: Tokenizer does not do '/* **/' comment correctly

Submitted by:  None
Submitted on:  Thu 21 Apr 2005 01:53:45 PM UTC  
 
Severity:  5 - Major Item Group:  Software
Status:  Fixed Assigned to:  Per Cederberg <cederberg>
Open/Closed:  Closed

Sat 14 Mar 2009 09:34:13 AM UTC, comment #4:

Version 1.5 was released on 2009-03-07. Closing this issue.

Per Cederberg <cederberg>
Project AdministratorIn charge of this item.
Sat 27 Aug 2005 10:53:23 AM UTC, comment #3:

Oops. That is correct. I think the correct fix here is to simplify the whole regexp by using a reluctant quantifier '*?' instead of the current greedy '*' quantifier. That change should fix the whole issue while also simplifying the regexp. Here is the new version:

/\*(.|\n)*?\*/

I've uploaded version 2.5.alpha2 with this fix (and further changes to the .NET API).

Per Cederberg <cederberg>
Project AdministratorIn charge of this item.
Wed 24 Aug 2005 10:53:16 PM UTC, comment #2:

The regexp in 1.5.alpha1 is imho still wrong. A small test shows that something like this fragment

/** c1 **/ code /** c2 **/

in a grammar file will not be tokenized correctly. c1 and c2 will become part of the same comment, and code will be dropped.

A more correct regexp would be <</\*([^*]|\*+[^*/])*\*+/>>.

Anonymous
Thu 21 Apr 2005 06:53:13 PM UTC, comment #1:

Ah, yes. This is indeed a bug. I'd go for the following solution, though:

<</\*([^*]|\*[^/])*\*?\*/>>

I haven't tested it yet, but by adding \*? to the regexp, we should cover up for the **/ case that isn't properly handled in the current version.

Thanks for finding and reporting this issue! (And for an excellent bug report!)

Per Cederberg <cederberg>
Project AdministratorIn charge of this item.
Thu 21 Apr 2005 01:53:45 PM UTC, original submission:

Considering the way multiline comments are defined in the Grammatica grammar (that is: MULTI_LINE_COMMENT = <</\*([^*]|\*[^/])*\*/>> %ignore%) it can tokenize the following comment (which is legal):

/* comment */

But it cannot tokenize this one (which should be legal):

/** comment **/

A workaround for this is to use the following multiline comment AS WELL: (MULTI_LINE_COMMENT = <</\*([^*]|\*[^/])*\*\*/>> %ignore%) which basically adds another * to the closing term, such that it can tokenize an even number of *'s before the closing /.

Anonymous

 

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by cederberg (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2009-03-14 cederberg Open/ClosedOpen => Closed
        Discussion LockUnlocked => Locked
    2005-05-07 cederberg StatusNone => Fixed
    2005-04-21 cederberg Assigned toNone => cederberg

    Back to the top


    Powered by Savane 3.4