From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from weis@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id TAA03786 for caml-redistribution@pauillac.inria.fr; Wed, 8 Mar 2000 19:27:39 +0100 (MET) Resent-Message-Id: <200003081827.TAA03786@pauillac.inria.fr> Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id OAA18200 for ; Tue, 7 Mar 2000 14:19:53 +0100 (MET) Received: from suburbia.net (suburbia.net [203.4.184.1]) by nez-perce.inria.fr (8.8.7/8.8.7) with ESMTP id OAA28929; Tue, 7 Mar 2000 14:19:33 +0100 (MET) Received: by suburbia.net (Postfix, from userid 110) id 842D86C6F8; Wed, 8 Mar 2000 00:19:28 +1100 (EST) Sender: proff@suburbia.net To: Xavier Leroy Cc: William Chesters , caml-list@inria.fr Subject: Re: Interpreter vs hardware threads References: <14518.34203.741447.637489@gargle.gargle.HOWL> <38BC93FC.65E3ED43@in.ot.com.au> <38bd773a@tequila.cs.yale.edu> <200003021818.SAA13646@toy.william.bogus> <20000306183546.18171@pauillac.inria.fr> Cc: proff@iq.org From: Julian Assange Date: 08 Mar 2000 00:19:28 +1100 In-Reply-To: Xavier Leroy's message of "Mon, 6 Mar 2000 18:35:46 +0100" Message-ID: User-Agent: Gnus/5.0802 (Gnus v5.8.2) XEmacs/21.1 (Big Bend) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Resent-From: weis@pauillac.inria.fr Resent-Date: Wed, 8 Mar 2000 19:27:39 +0100 Resent-To: caml-redistribution@pauillac.inria.fr Xavier Leroy writes: > > IIRC, Linux native pthreads, for one, aren't at the moment meant to > > be used in huge numbers---the flood-test performance of those Linux > > Java VMs which map Java threads onto them supposedly suffers compared > > with those that don't. But Xavier would be able to tell us more about > > that :). > > My pleasure :-) It is true that threads in LinuxThreads are not as > lightweight as some would like, mostly because they map 1-to-1 to > kernel processes, and the Linux kernel limits the number of processes > to 512 (for the super-user) or 256 (for normal users). Also, kernel > scheduling takes time linear in the number of processes (just like the > OCaml bytecode-thread scheduler, which was strongly inspired by that > of the Linux kernel), so context switching times increase as more > threads are run. I recently worked on a project in C that used a FSM to simulate several thousand concurrent IO threads, as part of a concurrent version of dig (able to recursively zone transfer all of *.au in under three minutes). Due to the complicated, but inherently forward moving nature of the operations involved (i.e do this. if there is no error or timeout, do the next step, and so on for about 30 steps) together with inter-state resources (i.e various queues, buffers and so on) this ended up as an extremely large and unintuitive piece of code. A lot of the added complexity was a result of having to explicity deallocate data and maintain reference counts, and manually follow allocation dependencies, which in the circumstances of multiple interdependent event queues and states became extremely tedious. What is the source of the linearity of thread context switches in ocaml? Are there ways to eliminate it? If so I'd be happy to have a look at doing so. Cheers, Julian. -- Stefan Kahrs in [Kah96] discusses the notion of completeness--programs which never go wrong can be type-checked--which complements Milner's notion of soundness--type-checked programs never go wrong [Mil78].