Re: [Caml-list] ocamllex problem

[Berke Durak <obdurak@free.fr>]

On Thu, Aug 04, 2005 at 11:15:48PM -0700, james woodyatt wrote:
> One thing: the pattern [':'((letter|' ')* as s)] is interesting.   
> You're definitely right that something non-trivial is happening  
> inside the DFA.  My [Cf_dfa] module does not keep a stack of  
> backtracking sequences because I did something else to resolve the  
> problem.  Look at the ( $@ ) operators, which allow you to use a  
> parser monad on the recognized input sequence to obtain the result of  
> a lexical rule.  Using this, you can implement something like the  
> feature you're interested in by defining a nested hierarchy of parsers.

You may also wish to have a look at :

  Thomas Reps, "Maximal-Munch" Tokenization in Linear Time, ACM Trans.
  Program. Lang. Syst., vol. 20, num. 2, 1998, pp.259-273
  http://www.cs.wisc.edu/wpis/papers/toplas98b.pdf

-- 
Berke Durak