[Caml-list] Functorising over Cohttp's backends

[Caml-list] Functorising over Cohttp's backends

[Dario Teixeira]

Hi,

Suppose I have a function "make_callback" which creates a callback
ready to be given to Cohttp's Server.make. Here's its signature:

  type callback =
    Cohttp_lwt_unix.Server.IO.conn * Cohttp.Connection.t ->
    Cohttp.Request.t ->
    Cohttp_lwt_body.t ->
    (Cohttp.Response.t * Cohttp_lwt_body.t) Cohttp_lwt_unix.Server.IO.t

  val make_callback: unit -> callback

Note that it's tied to the Lwt+Unix instantiation of Cohttp. Now,
I wish to make it more generic, so it can be used with the Async
backend, for instance.

There's more to it than just using a functor for abstracting over the
IO monad, unfortunately. In particular, the connection type and the
body are also backend-dependent. Anyway, Cohttp's API is notoriously
hard to navigate. Is the generic signature I'm looking for already
defined somewhere, or are the backends so distinct that I might
as well forget about functorising the interface and thus force the
Lwt+Unix backend on my API? (It's the only backend I use myself...)

Thanks in advance!
Best regards,
Dario Teixeira

Re: [Caml-list] Functorising over Cohttp's backends

[Rudi Grinberg]

The Cohttp API at its present state doesn't really offer a good abstraction for
creating backend independent HTTP clients and servers.

Luckily, the request and response types are backend independent. And so is the
http body type if you aren't streaming. This means that you can fairly easily
create a light application specific abstraction for yourself.

This means is that if you'd like to provide a backend independent server,
you should just worry about providing 2 functions of the type:

open Cohttp

val lwt : Request.t -> Body.t -> (Response.t * Body.t) Lwt.t
val async : Request.t -> Body.t -> (Response.t * Body.t) Deferred.t

and perhaps provide some light wrappers for turning these into full blown
cohttp servers in a backend dependent way.

With this technique, Cohttp's API will only help you as far as manipulating
request, response, and body values. This can often be enough for simple servers.
But in cases where it's not, such as when you need access to the client FD for
example Going back to implementing those 2 functions, you will need to cook up
your own abstractions on top.

Now going back to implementing the 2 functions mentioned above. For any non
trivial server, you will need to find a way to share code between your Async and
Lwt implementation. This is where it's common to use standard techniques such as
functors. But that's a topic you know about just as much as I do :P

To summarize, it's unfortunate that Cohttp doesn't provide a real server (or
client) abstraction. But it's kind of hard to do without making a wrapper that
tries to abstract away the differences between different backends. Hiding away
the details of Async and Lwt was always an anti-goal for Cohttp anyway.

Cheers,

Rudi.


On 08/29, Dario Teixeira wrote:
>Hi,
>
>Suppose I have a function "make_callback" which creates a callback
>ready to be given to Cohttp's Server.make. Here's its signature:
>
> type callback =
>   Cohttp_lwt_unix.Server.IO.conn * Cohttp.Connection.t ->
>   Cohttp.Request.t ->
>   Cohttp_lwt_body.t ->
>   (Cohttp.Response.t * Cohttp_lwt_body.t) Cohttp_lwt_unix.Server.IO.t
>
> val make_callback: unit -> callback
>
>Note that it's tied to the Lwt+Unix instantiation of Cohttp. Now,
>I wish to make it more generic, so it can be used with the Async
>backend, for instance.
>
>There's more to it than just using a functor for abstracting over the
>IO monad, unfortunately. In particular, the connection type and the
>body are also backend-dependent. Anyway, Cohttp's API is notoriously
>hard to navigate. Is the generic signature I'm looking for already
>defined somewhere, or are the backends so distinct that I might
>as well forget about functorising the interface and thus force the
>Lwt+Unix backend on my API? (It's the only backend I use myself...)
>
>Thanks in advance!
>Best regards,
>Dario Teixeira
>
>
>-- 
>Caml-list mailing list.  Subscription management and archives:
>https://sympa.inria.fr/sympa/arc/caml-list
>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Functorising over Cohttp's backends

[Dario Teixeira]

Hi,

And thanks for the comprehensive response, Rudi!


> The Cohttp API at its present state doesn't really offer a good
> abstraction for creating backend independent HTTP clients and
> servers.
> 
> Luckily, the request and response types are backend independent. And
> so is the http body type if you aren't streaming. This means that
> you can fairly easily create a light application specific abstraction
> for yourself.

Yes, I'm leaning towards just functorising over "IO" and nothing
else.  I thought about functorising also over "Body", but I reckon
the API will be much simpler if I dictate that the body must be of
type Cohttp.S.Body.t, and place on the user the burden of converting
to/from this type if the backend body implementation extends over
Cohttp.S.Body (as is the case with Cohttp_lwt_body).

I reckon a truly universal solution would also need to functorise
over "Request" and "Response", but a) that way lies madness, and b)
as you pointed out, luckily these are presently backend-independent.


> To summarize, it's unfortunate that Cohttp doesn't provide a real
> server (or client) abstraction. But it's kind of hard to do without
> making a wrapper that tries to abstract away the differences between
> different backends. Hiding away the details of Async and Lwt was
> always an anti-goal for Cohttp anyway.

Cohttp is perhaps the poster child for how the Lwt/Async schism
makes many APIs way more complex than they should be!  (Not that
there's an easy solution to this situation...)

Best regards,
Dario Teixeira

Re: [Caml-list] Functorising over Cohttp's backends

[Daniel Bünzli]

On Monday 29 August 2016 at 20:42, Dario Teixeira wrote:
> Cohttp is perhaps the poster child for how the Lwt/Async schism
> makes many APIs way more complex than they should be! (Not that
> there's an easy solution to this situation...)

I'd rather say it is the poster child on how not to cope with the lwt/async schism... Don't functorize over IO, provide non-blocking abstractions that can be used with either.

D

Re: [Caml-list] Functorising over Cohttp's backends

[Malcolm Matalka]

Daniel Bünzli <daniel.buenzli@erratique.ch> writes:

> On Monday 29 August 2016 at 20:42, Dario Teixeira wrote:
>> Cohttp is perhaps the poster child for how the Lwt/Async schism
>> makes many APIs way more complex than they should be! (Not that
>> there's an easy solution to this situation...)
>
> I'd rather say it is the poster child on how not to cope with the
> lwt/async schism... Don't functorize over IO, provide non-blocking
> abstractions that can be used with either.

Is there any HTTP library for Ocaml that separates I/O from the state
machine?  I have looked around a little bit for something I'm working
on, but with no luck.  I'm told HTTP is especially hard to separate
these things but I'm a bit ignorant as to why.

>
> D