Re: [Caml-list] Dynamically switching lexer

["Nicolas Pouillard" <nicolas.pouillard@gmail.com>]

Excerpts from till.varoquaux's message of Wed Jan 09 23:18:57 +0100 2008:
> I'm currently doing a toy DSEL,
> Camlp4 pops to mind as a very nice way to embed it unfortunately the
> standard lexer it comes with is not all that great for my language.
> Although I do know how to use another lexer it comes with a catch:
> antiquotation need to be properly lexed to string to be passed to the
> standard ocaml parser as strings (I use
> let expr_of_string = Gram.parse_string Syntax.expr_eoi
> this seems simpler than what is shown in tutorials. Am I missing
> something?) off course the easy way to delimit such strings
> (respecting nested comments, escaped strings...) would be to use the
> former lexer, It would need to be switched between both lexers, is
> this all possibe?.
> Cheers,
> Till
> 

You cannot easily change the camlp4 lexer dynamically, but when you get a quotation you get a string that you can lex with a different lexer.

Their is only two restrictions:

- ">>" is forbidden since it will end the quotation
- if you start a new quotation inside, you must close it: '<' (':' ident)? ('@' locname)? '<' ... ">>"

Here is the intersting part of the lexer

---------------------------
and quotation c = parse
  | '<' (':' ident)? ('@' locname)? '<'    {                          store c ;
                                                    with_curr_loc quotation c ;
                                                            parse quotation c }
  | ">>"                                                            { store c }
  | eof                                  { err Unterminated_quotation (loc c) }
  | newline                                     { update_loc c None 1 false 0 ;
                                                      store_parse quotation c }
  | _                                               { store_parse quotation c }
---------------------------


-- 
Nicolas Pouillard aka Ertai