node.ocaml

node.ocaml

[Jeffrey Barber]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

I've been working with libevent2 and OCaml for the past couple of weeks to
build node.ocaml. It is far from done, but it is interesting enough to
share.

node.ocaml as of now contains a web server and a terminal server that
provides asynchronous programming to OCaml to enable some of my research.
The first example server is a key value pair server that brings OCaml's
Hashtbl to http and terminal IO:

example code:
http://github.com/mathgladiator/node.ocaml/blob/master/test/kvp.ml

This is the first test program, and it works fairly well in a single
threaded environment. I was inspired by node.js to build an evented io
system, and so I begged the question "how does 'OCaml's FFI' compare with
v8's in node.js". In my virtual machine environment (ubuntu 10.04), I got
the following results doing 10,000 requests 50 at a time:

node.js with the "Hello World" script:
2100 requests/second

node.ocaml with kvp.ml:
5300 requests/second

This seems to me to be a very positive first benchmark considering I haven't
optimized anything yet nor have I hacked caml_copy_string yet.

The code is licensed under BSD and available
http://github.com/mathgladiator/node.ocaml .

Any thoughts/questions are appreciated; thank you for your time.

-J

[-- Attachment #2: Type: text/html, Size: 1528 bytes --]

Re: [Caml-list] node.ocaml

[Jake Donham]

On Sat, Aug 21, 2010 at 8:27 PM, Jeffrey Barber <jeff@mathgladiator.com> wrote:
> example code:
> http://github.com/mathgladiator/node.ocaml/blob/master/test/kvp.ml

Have you considered using Lwt (http://ocsigen.org/lwt) as a layer on
top of raw continuation-passing style? It has a lot of nice functions
for writing asynchronous code, as well as a syntax extension to make
it look more like direct-style code. It provides a select loop
already, but you don't need to use it, or you could integrate
libevent2 with Lwt's event loop.

Jake

Re: [Caml-list] node.ocaml

[Jeffrey Barber]

[-- Attachment #1: Type: text/plain, Size: 2088 bytes --]

Hello,

No I haven't, thanks for the link. In looking through the docs, it seems
similiar at some level to what I am doing. I'm trying to get away from the
language of threading (In my code, I use chain and io_pumps for context
storage) since I'm thinking of node.ocaml as an event based io library. This
is probably just a marketing decision, :/, rather than a technical decision.

More context on node.ocaml:

I was recently inspired by unix, memcache, and redis and realized that all
you need as a packet format is a single line, so I'm taking that as the
packet format and am going to try to build some neat things with it for both
my own research/education.

For my research, I want to build a fault tolerant JSON data store where I
can get the performance benefits of Redis/Memcache but also have
transactional transformations over indexed sets. But not only that, I want
it to be easy to build them so it can be extended down the road. I've
noticed that from using Redis, there are a lot of design patterns emerging
that would be better migrated from the application server to a redis-like
server.

For my education, I really want to do something with Paxos that's useful.
I'm not sure what, but I've yet to master it and it sounds fun. I'm tempted
to build a chubby clone for distributed locking, but who knows.

Oh, I may also cause the universe to explode since I may enable OCaml to
call JavaScript via v8... In case it does, I'm sorry now.

On Mon, Aug 23, 2010 at 4:31 PM, Jake Donham <jake@donham.org> wrote:

> On Sat, Aug 21, 2010 at 8:27 PM, Jeffrey Barber <jeff@mathgladiator.com>
> wrote:
> > example code:
> > http://github.com/mathgladiator/node.ocaml/blob/master/test/kvp.ml
>
> Have you considered using Lwt (http://ocsigen.org/lwt) as a layer on
> top of raw continuation-passing style? It has a lot of nice functions
> for writing asynchronous code, as well as a syntax extension to make
> it look more like direct-style code. It provides a select loop
> already, but you don't need to use it, or you could integrate
> libevent2 with Lwt's event loop.
>
> Jake
>

[-- Attachment #2: Type: text/html, Size: 2728 bytes --]

Re: [Caml-list] node.ocaml

[Gerd Stolpmann]

You may also look at ocamlnet
(http://projects.camlcity.org/projects/ocamlnet.html), which provides a
stack of abstractions for event-driven programming:

- The Netsys library defines an object type for sets of file
  descriptors one can poll:

http://projects.camlcity.org/projects/dl/ocamlnet-3.0test5/doc/html-main/Netsys_pollset.pollset.html

  There is right now only an implementation for poll(), but
  no fancy things like what libevent provides (but this is 
  just lack of time).

- Above that, there is the equeue library with its main
  interface Unixqueue:

http://projects.camlcity.org/projects/dl/ocamlnet-3.0test5/doc/html-main/Unixqueue.html

  This provides a queue of events, and a mechanism to consume
  events.

- Above that, there is the engines API:

http://projects.camlcity.org/projects/dl/ocamlnet-3.0test5/doc/html-main/Uq_engines.html

  This is an abstraction allowing easy composition of code. An
  engine represents a suspended I/O operation to which one can
  attach continuations.

There are even implementations of high-level protocols using these APIs
(most hooking in at the Unixqueue level), and this includes HTTP,
Telnet, FTP (partly), and SunRPC. Companies are using this already in
products, so this is really a mature implementation. 

It is interesting that in your code the callbacks are implemented on the
C side. I think this is generally more complicated than doing the same
in Ocaml which is way more a "callback-friendly" language.

Gerd

Am Samstag, den 21.08.2010, 22:27 -0500 schrieb Jeffrey Barber:
> I've been working with libevent2 and OCaml for the past couple of
> weeks to build node.ocaml. It is far from done, but it is interesting
> enough to share.
> 
> node.ocaml as of now contains a web server and a terminal server that
> provides asynchronous programming to OCaml to enable some of my
> research. The first example server is a key value pair server that
> brings OCaml's Hashtbl to http and terminal IO:
> 
> example code:
> http://github.com/mathgladiator/node.ocaml/blob/master/test/kvp.ml
> 
> This is the first test program, and it works fairly well in a single
> threaded environment. I was inspired by node.js to build an evented io
> system, and so I begged the question "how does 'OCaml's FFI' compare
> with v8's in node.js". In my virtual machine environment (ubuntu
> 10.04), I got the following results doing 10,000 requests 50 at a
> time:
> 
> node.js with the "Hello World" script:
> 2100 requests/second
> 
> node.ocaml with kvp.ml:
> 5300 requests/second
> 
> This seems to me to be a very positive first benchmark considering I
> haven't optimized anything yet nor have I hacked caml_copy_string yet.
> 
> The code is licensed under BSD and available
> http://github.com/mathgladiator/node.ocaml .
> 
> Any thoughts/questions are appreciated; thank you for your time.
> 
> -J
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs


-- 
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------