bugGrammatica - Bugs: bug #12767, Tokenizer does not do '/* **/'...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #12767: Tokenizer does not do '/* **/' comment correctly

Submitter:  None
Submitted:  Thu 21 Apr 2005 01:53:45 PM UTC
   
 
Severity:  5 - Major Item Group:  Software
Status:  Fixed Assigned to:  cederberg
Open/Closed:  Closed

Discussion locked!

Sat 14 Mar 2009 09:34:13 AM UTC, comment #4: 

Version 1.5 was released on 2009-03-07. Closing this issue.

 Per Cederberg <cederberg>
Group administrator
Sat 27 Aug 2005 10:53:23 AM UTC, comment #3: 

Oops. That is correct. I think the correct fix here is to simplify the whole regexp by using a reluctant quantifier '*?' instead of the current greedy '*' quantifier. That change should fix the whole issue while also simplifying the regexp. Here is the new version:

/\*(.|\n)*?\*/

I've uploaded version 2.5.alpha2 with this fix (and further changes to the .NET API).

 Per Cederberg <cederberg>
Group administrator
Wed 24 Aug 2005 10:53:16 PM UTC, comment #2: 

The regexp in 1.5.alpha1 is imho still wrong. A small test shows that something like this fragment

  /** c1 */ code /* c2 **/

in a grammar file will not be tokenized correctly. c1 and c2 will become part of the same comment, and code will be dropped.

A more correct regexp would be <</\*([^*]|\*+[^*/])*\*+/>>.

Anonymous
Thu 21 Apr 2005 06:53:13 PM UTC, comment #1: 

Ah, yes. This is indeed a bug. I'd go for the following solution, though:

<</\*([^*]|\*[^/])*\*?\*/>>

I haven't tested it yet, but by adding \*? to the regexp, we should cover up for the **/ case that isn't properly handled in the current version.

Thanks for finding and reporting this issue! (And for an excellent bug report!)

 Per Cederberg <cederberg>
Group administrator
Thu 21 Apr 2005 01:53:45 PM UTC, original submission:  

Considering the way multiline comments are defined in the Grammatica grammar (that is: MULTI_LINE_COMMENT           = <</\*([^*]|\*[^/])*\*/>> %ignore%) it can tokenize the following comment (which is legal):

/* comment */

But it cannot tokenize this one (which should be legal):

/** comment **/

A workaround for this is to use the following multiline comment AS WELL: (MULTI_LINE_COMMENT           = <</\*([^*]|\*[^/])*\*\*/>> %ignore%) which basically adds another to the closing term, such that it can tokenize an even number of 's before the closing /.

 Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by cederberg (Posted a comment)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2009-03-14 cederberg Open/ClosedOpen Closed
        Discussion LockNone Locked
    2005-05-07 cederberg StatusNone Fixed
    2005-04-21 cederberg Assigned toNone cederberg

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-d582.
    Corresponding source code