Thanks, Gerd!

FWIW I could not reproduce the crash by using ocaml-ssl's blocking operations directly.

https://gist.github.com/4152047#file_ssl_threads.ml

This works fine- so, perhaps something nasty arises from using nonblocking I/O on ssl sockets from multiple threads. I'm sure if there is any other critical difference with how netclient/equeue-ssl and my example use ocaml-ssl.

I also don't have time to pursue this much further, so I will try to put all of my http operations on one thread as your example suggests.

Best,
Mike


On Tue, Nov 27, 2012 at 5:03 AM, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:
Hi Mike,

I can confirm the problem, but I'm not able to track it down.

First of all, I'm quite sure that you cannot share SSL contexts between threads. But even if I fix this, I get crashes - usually early during SSL_connect.

I'm wondering whether ocaml-ssl initializes openssl correctly for multi-threading. In my version of openssl the prototype is

int CRYPTO_THREADID_set_callback(void (*threadid_func)(CRYPTO_THREADID *));

but in ocaml-ssl the CRYPTO_THREADID pointer is ignored (it assumes void here). So, maybe this is the reason. I currently don't have time to follow this idea.

If everything fails, there is a way to use Netclient in threaded applications so that only one thread is doing the networking stuff, see this example:

https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/examples/netclient/simple/http_mt.ml

As Netclient is event-driven, the requests are processed in parallel (with one exception, though: DNS lookups are synchronous, but this can be worked around, as the DNS lookup function is settable in Netclient).

Gerd

Am 27.11.2012 03:54:16 schrieb(en) Mike Lin:

Hello Gerd, all,

I'm trying to find the right incantations to make parallel https requests
from multiple threads using Http_client. (I know the library is capable of
multiple requests from one thread -- I have other strong reasons to use
multithreading.)

https://gist.github.com/4152047

This program generally segfaults for me at some random point during the 10
requests.  Compiled as in the comment in line 2 (ubuntu quantal,
libocamlnet-ocaml-dev 3.5.1, ocaml 3.12.1).

Permutations attempted:

1) Change https to http in google URL -- works as expected
2) Serialize requests by uncommenting line 35 -- works as expected
3) Move invocations of Ssl.init, Ssl.create_context,
HTTPS.https_transport_channel_type in and out of fresh_pipeline (which runs
in the worker threads) -- no help

Any suggestions? Thanks!

Mike Lin




--
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------