Re: [Caml-list] 246 constructors?

Re: [Caml-list] 246 constructors?

[Emmanuel Onzon]

> On Sun, 2007-05-06 at 20:50 +0100, Jon Harrop wrote:
>> On Sunday 06 May 2007 20:49, skaller wrote:
>> > I just got this message:
>> >
>> > File "dyp_parse.ml", line 492, characters 5-19461:
>> > Too many non-constant constructors -- maximum is 246 non-constant
>> > constructors
>> >
>> > Really???
>>
>> Yeah, I was hardcoding a bunch of enums for my Mathematica 
>> interpreter and got
>> that. Easiest solution is to use polymorphic variants instead. Then you have
>> clashes to worry about instead...
>
> Unfortunately (or not .. ) the file is generated by
> the dypgen parser generator..

I have just added an option -symbols-with-variants which makes dypgen use
variants instead of constructors. (see the file CHANGES for other changes
in this version)

Emmanuel

Re: [Caml-list] 246 constructors?

[skaller]

On Mon, 2007-05-07 at 10:21 +0200, Emmanuel Onzon wrote:

> I have just added an option -symbols-with-variants which makes dypgen use
> variants instead of constructors. (see the file CHANGES for other changes
> in this version)

Hi! You're the author right? I there a repository somewhere?
I have a couple of patches. To the lexer I added:

and ocaml_type = parse
  | "->" 
      { ocaml_code_buffer := (!ocaml_code_buffer) ^ "->";
        ocaml_type lexbuf
      }  
  
otherwise function types can't be properly lexed.
Probably be cool if the lexer was made a bit more
sophisticated .. for example I had

%token<...>...

which failed due to a lack of a space between token and <.

I also added:

  | "%type" blank { look_for_type:=true; KWD_TYPE }

in a vain attempt to get it to ignore my %type commands.

It should be possible to make it work with an ocamlyacc
spec with a bit of fiddling shouldn't it?


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

246 constructors?

[skaller]

I just got this message:

File "dyp_parse.ml", line 492, characters 5-19461:
Too many non-constant constructors -- maximum is 246 non-constant
constructors

Really???

(The file is generated by dypgen, it has a constructor for
every terminal and non-terminal in the Felix grammar)

At least on a 64 bit machine this is a sad restriction:
any chance of a two byte code here?

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] 246 constructors?

[Jon Harrop]

On Sunday 06 May 2007 20:49, skaller wrote:
> I just got this message:
>
> File "dyp_parse.ml", line 492, characters 5-19461:
> Too many non-constant constructors -- maximum is 246 non-constant
> constructors
>
> Really???

Yeah, I was hardcoding a bunch of enums for my Mathematica interpreter and got 
that. Easiest solution is to use polymorphic variants instead. Then you have 
clashes to worry about instead...

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
The F#.NET Journal
http://www.ffconsultancy.com/products/fsharp_journal/?e

Re: [Caml-list] 246 constructors?

[skaller]

On Sun, 2007-05-06 at 20:50 +0100, Jon Harrop wrote:
> On Sunday 06 May 2007 20:49, skaller wrote:
> > I just got this message:
> >
> > File "dyp_parse.ml", line 492, characters 5-19461:
> > Too many non-constant constructors -- maximum is 246 non-constant
> > constructors
> >
> > Really???
> 
> Yeah, I was hardcoding a bunch of enums for my Mathematica interpreter and got 
> that. Easiest solution is to use polymorphic variants instead. Then you have 
> clashes to worry about instead...

Unfortunately (or not .. ) the file is generated by
the dypgen parser generator..

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] 246 constructors?

[Till Varoquaux]

I'm guessing a patch against dypgen could prove usefull then.

Till

On 5/6/07, skaller <skaller@users.sourceforge.net> wrote:
> On Sun, 2007-05-06 at 20:50 +0100, Jon Harrop wrote:
> > On Sunday 06 May 2007 20:49, skaller wrote:
> > > I just got this message:
> > >
> > > File "dyp_parse.ml", line 492, characters 5-19461:
> > > Too many non-constant constructors -- maximum is 246 non-constant
> > > constructors
> > >
> > > Really???
> >
> > Yeah, I was hardcoding a bunch of enums for my Mathematica interpreter and got
> > that. Easiest solution is to use polymorphic variants instead. Then you have
> > clashes to worry about instead...
>
> Unfortunately (or not .. ) the file is generated by
> the dypgen parser generator..
>
> --
> John Skaller <skaller at users dot sf dot net>
> Felix, successor to C++: http://felix.sf.net
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] 246 constructors?

[skaller]

On Sun, 2007-05-06 at 22:31 +0200, Till Varoquaux wrote:
> I'm guessing a patch against dypgen could prove usefull then.


I have 221 tokens, and 233 non-terminals .. both just barely
within the 246 limit. 

Looking at dypgen code it is using integer sequence numbers:
I can't tell at this time whether it uses magic. I guess
that Ocamlyacc uses magic. Not sure about Menhir.

* Magic which assumes sequential tags of constructors 
isn't compatible with polymorphic variants.

* It looks like Ocamlyacc is close to blowing up too

The dypgen grammar representation is built of symbols of the variant
type with a case for each terminal and non-terminal AND 
parametrised by all the non-terminals: so there are around
500 cases with 233 type parameters.

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] 246 constructors?

[Brian Hurt]

On Mon, 7 May 2007, skaller wrote:

> I just got this message:
>
> File "dyp_parse.ml", line 492, characters 5-19461:
> Too many non-constant constructors -- maximum is 246 non-constant
> constructors
>
> Really???
>
> (The file is generated by dypgen, it has a constructor for
> every terminal and non-terminal in the Felix grammar)
>
> At least on a 64 bit machine this is a sad restriction:
> any chance of a two byte code here?

The problem is that this is comming out of the tag word.  The current 
structure is:
 	8 bits for the tag information
 	2 bits for the GC color information
 	n-10 bits for the size of the block
where n is the number of bits in a word.

The 246 limitation comes from the 256 possible tag values minus 10 
"special" tag values for various objects that need to be handled special 
by the GC (objects that don't need to be scanned for pointers, like floats 
and strings, closures, objects, lazy thunks, etc.).

Unfortunately, this bumps right into the other favorite thread: the 2M 
limitation on arrays.  The maximum size of an array is the maximum size of 
a block- and the more bits that are given over to tag bits, the fewer bits 
there are for block size, so the smaller arrays can be.

I think a better solution, although I have no idea how difficult it'd be 
to implement, would be "large tag blocks".  For blocks with large tag 
values (say, >245), the tag value in the tag word would be the special 
"large tag" tag (say, value 246), and the last word of the block would be 
the "large" tag value.  I say last, as this means you can still get the 
nth word of a block without needing to special case large blocks.  This 
would allow a full word to be used for tag values (giving way more than 
enough tag values), and wouldn't change either memory usage or code for 
blocks with small tag values.

Brian

Re: [Caml-list] 246 constructors?

[skaller]

On Sun, 2007-05-06 at 17:11 -0400, Brian Hurt wrote:

> I think a better solution, although I have no idea how difficult it'd be 
> to implement, would be "large tag blocks".  
[]

That sounds like a good plan!

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] 246 constructors?

[Tom]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

I think a better and simpler (and probably faster as well) solution would be
to simply use an extra "tag word" for all constructors of the "large-tag
type". It could even probably be implemented using camlp4 extenstion
(turning all "large-tag types" into records with an implicit first word
indicating the tag, and then transforming the pattern matching of these
values to pattern matching of records. To simplify things, you could even
use a special annotation for such blocks (for the camlp4 version only, of
course).

- Tom

On 06/05/07, Brian Hurt <bhurt@spnz.org> wrote:
>
>
> I think a better solution, although I have no idea how difficult it'd be
> to implement, would be "large tag blocks".  For blocks with large tag
> values (say, >245), the tag value in the tag word would be the special
> "large tag" tag (say, value 246), and the last word of the block would be
> the "large" tag value.  I say last, as this means you can still get the
> nth word of a block without needing to special case large blocks.  This
> would allow a full word to be used for tag values (giving way more than
> enough tag values), and wouldn't change either memory usage or code for
> blocks with small tag values.
>

[-- Attachment #2: Type: text/html, Size: 1594 bytes --]