bugGrammatica - Bugs: bug #13009, Tokenizer: String tokens are...


You are not allowed to post comments on this tracker with your current authentication level.

bug #13009: Tokenizer: String tokens are always considered first

Submitter:  Per Cederberg <cederberg>
Submitted:  Sat 07 May 2005 09:49:48 AM UTC
Severity:  1 - Minor Item Group:  Software
Status:  Fixed Assigned to:  cederberg
Open/Closed:  Closed

Discussion locked!

Sat 14 Mar 2009 09:33:50 AM UTC, comment #2: 

Version 1.5 was released on 2009-03-07. Closing this issue.

Per Cederberg <cederberg>
Group administrator
Sat 14 Feb 2009 12:38:59 PM UTC, comment #1: 

Fixed in r130.

Per Cederberg <cederberg>
Group administrator
Sat 07 May 2005 09:49:48 AM UTC, original submission:  

When tokenizing an input string, the string token patterns are always checked and considered first. This means that if a regular expression and a string token pattern matches the same input sequence, the string token will always be returned. This is wrong, as the token pattern order should determine which token is returned in cases where multiple patterns match.

Per Cederberg <cederberg>
Group administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by cederberg (Posted a comment)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2009-03-14 cederberg Open/ClosedOpen Closed
        Discussion LockNone Locked
    2009-02-14 cederberg StatusNone Fixed
        Assigned toNone cederberg

    Back to the top

    Powered by Savane 3.13-b921.
    Corresponding source code