Re: [Caml-list] sedlex = ulex without camlp4

[Alain Frisch <alain.frisch@lexifi.com>]

I have to admit that I don't know much about Unicode and surrogates 
(moreover, support for utf-16 was contributed by someone else).  I'll 
happily update the documentation if someone looks at the source code and 
tells me that the property you mention indeed holds.

-- Alain


On 01/18/2013 04:32 PM, Daniel Bünzli wrote:
> Hello Alain,
>
> I rapidly went through your documentation.
>
> If your UTF-8 and UTF-16 decoders are conformant, your module, on output, doesn't generate Unicode code points, but Unicode scalar values (code points minus the UTF-16 surrogates [1]). If that is the case it would be nice to state this invariant explicitely in the documentation.
>
> This allows to directly pass the data generated by sedlex to modules like Uunf without further checks as those values belong to the Uunf.uchar type [2].
>
> Best,
>
> Daniel
>
>
> [1] http://www.unicode.org/glossary/#unicode_scalar_value
> [2] http://erratique.ch/software/uunf/doc/Uunf#TYPEuchar
>
>
>