GtkSourceView2.0 syntax highlighting

GtkSourceView2.0 syntax highlighting

[Edgar Friendly]

[-- Attachment #1: Type: text/plain, Size: 2596 bytes --]

The upcoming version of GtkSourceView, the library used by many Gnome
text editors for syntax highlighting, supports new and more powerful
language parsing.  I volunteer to update the language definition for
Ocaml, and would like some feedback from the community regarding useful
things to highlight.

The following are currently matched:
====================================

* Comments (* *), and within them, email addresses, net addresses and
TODO/FIXME/XXX

* Decimal, Octal, Hex, Binary and Floating point literals

* Labeled and Optional function arguments

* Polymorphic variants and normal Variant constructors

* Module paths (as a prefix to anything)

* Strings and Character Literals, with all escape codes allowed within
them sub-matched


The reasonably large list of keywords has been broken into four
sections.  (I encourage comments on this division.)

1) booleans
	true	false

2) flow control & common keywords
	and	assert	begin	do	done	downto	else
	end	for	fun	function	if	in
	let	match	rec	then	to	try	val
	when	while	with

3) types, objects & modules
	as	class	constraint	exception	external
	functor	include	inherit	initializer	method	module
	mutable	new	object	of	open	private	struct
	sig	type	virtual

4) function-like keywords
	asr	land	lazy	lor	lsl	lsr	lxor
	mod	or


Things not matched currently
============================

* Line number directives  (probably never seen in actual code)

* Record constructors - { record with label:value; label:value }

* Object duplication - {< var = value; var = value >}

* List literals - [ elem1; elem2; elem3 ]

* Array literals - [| elem1; elem2; elem3 |]

* Tuples - elem1, elem2, elem3  (hard to parse - no parentheses needed,
only commas)

* Array access and modification - arr.(i), arr.(i) <- 5

* String access and modification - str.[i], str[i] <- 'w'

* Coercion - ( expr :> type ), (expr : type1 :> type2)

* Method calls - obj#method args

* There's a ton of character-sequence keywords, are there any that
should be handled as a keyword, or should they be handled only in the
above cases where they're used?
    !=    #     &     &&    '     (     )     *     +     ,     -
    -.    ->    .     ..    :     ::    :=    :>    ;     ;;    <
    <-    =     >     >]    >}    ?     ??    [     [<    [>    [|
    ]     _     `     {     {<    |     |]    }     ~

* What about camlp4 keywords?  New keywords in the new camlp4?
    parser    <<    <:    >>    $     $$    $:


Thanks for your comments.  Attached is a compressed version of my
current language definition in the XML format required by gtksourceview.

E.

[-- Attachment #2: ocaml2.lang.gz --]
[-- Type: application/x-gzip, Size: 1411 bytes --]

Re: [Caml-list] GtkSourceView2.0 syntax highlighting

[Richard Jones]

On Tue, May 29, 2007 at 02:04:52PM -0500, Edgar Friendly wrote:
> * Comments (* *), and within them, email addresses, net addresses and
> TODO/FIXME/XXX

Are (** ... *) (ocamldoc) comments shown differently?  If that's easy
to do, it might be worthwhile.

> * List literals - [ elem1; elem2; elem3 ]
> 
> * Array literals - [| elem1; elem2; elem3 |]

These would be nice, if possible.

I'm not really sure what else to say, except that it's great that you
are adding OCaml support to a common tool like this.

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] GtkSourceView2.0 syntax highlighting

[Edgar Friendly]

Richard Jones wrote:
> On Tue, May 29, 2007 at 02:04:52PM -0500, Edgar Friendly wrote:
>> * Comments (* *), and within them, email addresses, net addresses and
>> TODO/FIXME/XXX
> 
> Are (** ... *) (ocamldoc) comments shown differently?  If that's easy
> to do, it might be worthwhile.
> 
Yes, it's easy.

Inside ocamldoc comments, there's various @-tags.  Are there any that
should have special handling (specially formatting what follows them?),
or just highlight them, or highlight the lines they're in?

There also seem to be stop comments.  I might as well highlight those,
but I don't think it's possible (or if possible, worth the complexity)
to try enabling and disabling ocamldoc highlighting in the right places
so that comments that won't appear in the documentation get a different
highlighting.  If someone is really interested, they're more than
welcome to give it a try.

>> * List literals - [ elem1; elem2; elem3 ]
>>
>> * Array literals - [| elem1; elem2; elem3 |]
> 
> These would be nice, if possible.
>
How do you want the contents to look, same as they would outside the
literal, or the same style throughout (like strings)?

E.

Re: [Caml-list] GtkSourceView2.0 syntax highlighting

[Robert Roessler]

Edgar Friendly wrote:
> The upcoming version of GtkSourceView, the library used by many Gnome
> text editors for syntax highlighting, supports new and more powerful
> language parsing.  I volunteer to update the language definition for
> Ocaml, and would like some feedback from the community regarding useful
> things to highlight.

A "gotcha" to be aware of (if you aren't already handling this): don't 
forget that OCaml comments nest. ;)

Also, from my OCaml lexer in the Scintilla distribution:

# suggested; more could come from, say, Pervasives
keywordoptional1.caml= \
option Some None ignore ref lnot succ pred

# suggested; more could come from, say, user-defined types
keywordoptional2.caml= \
array bool char float int list string unit

The full set (including default coloring values) may be found in the 
caml.properties file in a Scintilla source tree.

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] GtkSourceView2.0 syntax highlighting

[skaller]

On Tue, 2007-05-29 at 14:04 -0500, Edgar Friendly wrote:

> The reasonably large list of keywords has been broken into four
> sections.  (I encourage comments on this division.)
> 
> 1) booleans
> 	true	false
> 
> 2) flow control & common keywords
> 	and	assert	begin	do	done	downto	else
> 	end	for	fun	function	if	in
> 	let	match	rec	then	to	try	val
> 	when	while	with
> 
> 3) types, objects & modules
> 	as	class	constraint	exception	external
> 	functor	include	inherit	initializer	method	module
> 	mutable	new	object	of	open	private	struct
> 	sig	type	virtual
> 
> 4) function-like keywords
> 	asr	land	lazy	lor	lsl	lsr	lxor
> 	mod	or
> 


I would do this slightly differently. The 'most important'
keywords are used in binders which often denote something
part of an interface: These are (roughly):

type, let, module, class, and, in, rec, val, exception
external functor

and these should be in a heavy font. Flow control is an implementation
detail and should be lighter weight. inherit, include, and open are
"directives", perhaps a weak color (boring houskeeping ..)



-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] GtkSourceView2.0 syntax highlighting

[Edgar Friendly]

Robert Roessler wrote:
> Edgar Friendly wrote:
>> The upcoming version of GtkSourceView, the library used by many Gnome
>> text editors for syntax highlighting, supports new and more powerful
>> language parsing.  I volunteer to update the language definition for
>> Ocaml, and would like some feedback from the community regarding useful
>> things to highlight.
> 
> A "gotcha" to be aware of (if you aren't already handling this): don't
> forget that OCaml comments nest. ;)
> 
Yup, that's being handled.  Another fun gotcha I found is that comments
aren't comments inside string literals.

> Also, from my OCaml lexer in the Scintilla distribution:
> 
> # suggested; more could come from, say, Pervasives
> keywordoptional1.caml= \
> option Some None ignore ref lnot succ pred
> 
> # suggested; more could come from, say, user-defined types
> keywordoptional2.caml= \
> array bool char float int list string unit
> 
wow, this is going to be a lot of keyword highlighting.  people might
run out of colors on their screen.

> The full set (including default coloring values) may be found in the
> caml.properties file in a Scintilla source tree.
> 
> Robert Roessler
> roessler@rftp.com
> http://www.rftp.com
> 

Thanks for the feedback,
E.