From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 229AF7FF9F for ; Mon, 7 Mar 2016 16:17:01 +0100 (CET) IronPort-PHdr: 9a23:HqUaLhe1kCDwIl3UxiYpV8mJlGMj4u6mDksu8pMizoh2WeGdxc6zYx7h7PlgxGXEQZ/co6odzbGG7OawACdesd6oizMrTt9lb1c9k8IYnggtUoauKHbQC7rUVRE8B9lIT1R//nu2YgB/Ecf6YEDO8DXptWZBUiv2OQc9HOnpAIma153xjLDtvc2KKFsYzBOGIppMbzyO5T3LsccXhYYwYo0Q8TDu5kVyRuJN2GlzLkiSlRuvru25/Zpk7jgC86l5r50IeezAcq85Vb1VCig9eyBwvZWz9EqLcQza13wAW2BeuBNSBQ/UpEXrWYv4tyHzrOx6yQGVOMT3SfY/XjH0vIlxTxq9qiocLzMjuEXQl81rxItdrB+7vBF5i9rWbZqNOeA4eqTAfMhcTGxNU9xKWippDYa1bo9JBO0Ea7UL57LhrkcD+EPtTTKnA/nin3oR3if7 Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=jesper.louis.andersen@gmail.com; spf=Pass smtp.mailfrom=jesper.louis.andersen@gmail.com; spf=None smtp.helo=postmaster@mail-wm0-f46.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of jesper.louis.andersen@gmail.com) identity=pra; client-ip=74.125.82.46; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="jesper.louis.andersen@gmail.com"; x-sender="jesper.louis.andersen@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of jesper.louis.andersen@gmail.com designates 74.125.82.46 as permitted sender) identity=mailfrom; client-ip=74.125.82.46; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="jesper.louis.andersen@gmail.com"; x-sender="jesper.louis.andersen@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-wm0-f46.google.com) identity=helo; client-ip=74.125.82.46; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="jesper.louis.andersen@gmail.com"; x-sender="postmaster@mail-wm0-f46.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DOAgAWmt1Wji5SfUpdg1iBIQaoPZILgWmGDwKBIgc6EgEBAQEBAQEBEAEBAQEHCwsJHzGCLYIVAQEDARIRHQEbHQEDAQsGBQs3AgIhAQERAQUBHAYTIodqAQMKCKI8gTE+MYs2gWqCV4UdChknDVGDVgEBAQEBBQEBAQEBARQBBQoEikaCPYR9gToFlyqLeIF1jnqHDIYLER6BDycGgisegVE7Lok9AQEB X-IPAS-Result: A0DOAgAWmt1Wji5SfUpdg1iBIQaoPZILgWmGDwKBIgc6EgEBAQEBAQEBEAEBAQEHCwsJHzGCLYIVAQEDARIRHQEbHQEDAQsGBQs3AgIhAQERAQUBHAYTIodqAQMKCKI8gTE+MYs2gWqCV4UdChknDVGDVgEBAQEBBQEBAQEBARQBBQoEikaCPYR9gToFlyqLeIF1jnqHDIYLER6BDycGgisegVE7Lok9AQEB X-IronPort-AV: E=Sophos;i="5.22,551,1449529200"; d="scan'208,217";a="167465867" Received: from mail-wm0-f46.google.com ([74.125.82.46]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 07 Mar 2016 16:17:00 +0100 Received: by mail-wm0-f46.google.com with SMTP id n186so90783187wmn.1 for ; Mon, 07 Mar 2016 07:17:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1lsQS4FCOE/lCw+GPViWyK5MDfLNbJh2cGkh4EytpRM=; b=kQEQ+l1E/sgH0OsG9Xuo9MVXagJNiIBJ7BNzZdKFvRwG5vBBHk/4/SEaJW1MErM7Ws riq8MTpaP/hKyXIYxMfZ5VbnNrrLQvWaDDc7iwEMT93AJ815D9bGWk46gQwsQQDTThMs /kvEm4WYg1lksF2UsGamyNY9ABV6a/6TzJV6je0LoLTtTRy8xJYmAuyFOaXP+dK+c2uo 1WmGFYCLc3xPjU6Wex7KdKLw/CfyEELuujDxNWNn53jC4DR5JSRUMKcGH5ZlLFj4YQvP PnMZygDBAP2n6cZpMqXC2LiVisiZZCzbu98lB06mSVu0wPu84cAKaFD35sdpmlRzWIWw LdSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1lsQS4FCOE/lCw+GPViWyK5MDfLNbJh2cGkh4EytpRM=; b=RyT2+BTAp1jn2ErD8A85FARhdlGPhlRo7y1EqRS2k+wBCxcFZ1pAcv9+Y2X6b2UDub IGjSZ2pUE7+kRv/Tewkg/Y/73cuXQw624tDOktgL3dxJKBsLjDA8gXvySH6ICPEKwV2n 3LD+hJHi3PfhgD4oGj9UJJMLBacoTHJiQeTejbnVbtODQhxLWO4wYWu3xyO8TwrutWQs sN20Y1WBLcZe9CSbyQ+RqeZjkJaBwvQxadVOm5iL7NG3O8GwZ0EDfUlSRE+J1i8Mls8x Ccs6POFwGTRdCFDg//iqXm/vTTG34nRY2uKn/fflnN4Y3X2h9dFiore6qFUhN1pK/obm tYhQ== X-Gm-Message-State: AD7BkJJfwQQ1918pg7VlS8g398eIoO5yir6xe1MUK/C5t+YFB6uW5A5z5l5HMkHTayOvpbS5SpmW8DpUea63KA== X-Received: by 10.194.190.6 with SMTP id gm6mr23822352wjc.115.1457363819802; Mon, 07 Mar 2016 07:16:59 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.144.212 with HTTP; Mon, 7 Mar 2016 07:16:20 -0800 (PST) In-Reply-To: References: From: Jesper Louis Andersen Date: Mon, 7 Mar 2016 16:16:20 +0100 Message-ID: To: Yotam Barnoy Cc: Ocaml Mailing List Content-Type: multipart/alternative; boundary=047d7bdca7fec9b4a9052d76f4f0 Subject: Re: [Caml-list] Question about Lwt/Async --047d7bdca7fec9b4a9052d76f4f0 Content-Type: text/plain; charset=UTF-8 On Mon, Mar 7, 2016 at 2:38 AM, Yotam Barnoy wrote: > Also, what happens to general utility functions that aren't rewritten for > Async/Lwt -- as far as I can tell, being in non-monadic code, they will > always starve other threads, since they cannot yield to another Async/Lwt > thread. Is this perception correct? Yes. On one hand, your observation is negative in the sense that now your code has "color" in the sense that it is written for one library only. And you have to transform code to having the right color before it can be used. This is not the case if the concurrency model is at a lower level[0]. On the other hand, your observation is positive: cooperative scheduling makes the points in which the code can switch explicit. This gives the programmer far more control over when you are done with a task and start to process the next task. You can also avoid the preemption check in the code all the time. If your code manipulates lots of shared data, it also simplifies things since you don't usually have to protect data with a mutex in a single-threaded context as much[1]. Cooperative models, if carefully managed, can exploit structure in the problem domain, whereas a preemptive model needs to fit all. My personal opinion is that the preemptive model eventually wins over the cooperative model, much like it has in most (all popular) operating systems. It is simply more productive to take an up-front performance hit as a sacrifice for a system which is more robust against stray code misbehaving. If a cooperative system fails, it is fails catastrophically. If a preemptive system fails, it degrades in performance. But given I have more than 10 years of Erlang programming behind me by now, I'm obviously biased toward certain computational models :) [0] Erlang would be one such example, where the system is preemptively scheduling for you and you can use any code in any place without having to worry about blocking for latency. Go is quasi-preemptive because it checks on function calls, but in contrast to Erlang a loop is not forced to factor through a recursion, so it can in principle run indefinitely. Haskell (GHC) is quasi-preemptive as well, checking on memory allocation boundaries. So the thing to look out for in GHC is latency from processing large arrays with no allocation, say. [1] Erlang has two VM runtimes for this reason. One is single-threaded and can avoid lots of locks which is far faster for certain workloads, or on embedded devices with a single core only. -- J. --047d7bdca7fec9b4a9052d76f4f0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On Mon, Mar 7, 2016 at 2:38 AM, Yotam Barnoy <yotambarnoy@gmail.com> wrote:
Also, what happens t= o general utility functions that aren't rewritten for Async/Lwt -- as f= ar as I can tell, being in non-monadic code, they will always starve other = threads, since they cannot yield to another Async/Lwt thread. Is this perce= ption correct?

Yes.<= br>
On one hand, your observation is ne= gative in the sense that now your code has "color" in the sense t= hat it is written for one library only. And you have to transform code to h= aving the right color before it can be used. This is not the case if the co= ncurrency model is at a lower level[0].

On the other hand, your observation is positive: cooperative schedulin= g makes the points in which the code can switch explicit. This gives the pr= ogrammer far more control over when you are done with a task and start to p= rocess the next task. You can also avoid the preemption check in the code a= ll the time. If your code manipulates lots of shared data, it also simplifi= es things since you don't usually have to protect data with a mutex in = a single-threaded context as much[1]. Cooperative models, if carefully mana= ged, can exploit structure in the problem domain, whereas a preemptive mode= l needs to fit all.

My personal opi= nion is that the preemptive model eventually wins over the cooperative mode= l, much like it has in most (all popular) operating systems. It is simply m= ore productive to take an up-front performance hit as a sacrifice for a sys= tem which is more robust against stray code misbehaving. If a cooperative s= ystem fails, it is fails catastrophically. If a preemptive system fails, it= degrades in performance.

But given= I have more than 10 years of Erlang programming behind me by now, I'm = obviously biased toward certain computational models :)

[0] Erlang would be o= ne such example, where the system is preemptively scheduling for you and yo= u can use any code in any place without having to worry about blocking for = latency. Go is quasi-preemptive because it checks on function calls, but in= contrast to Erlang a loop is not forced to factor through a recursion, so = it can in principle run indefinitely. Haskell (GHC) is quasi-preemptive as = well, checking on memory allocation boundaries. So the thing to look out fo= r in GHC is latency from processing large arrays with no allocation, say.

[1] Erlang has two VM runtimes for this reason. One is single-th= readed and can avoid lots of locks which is far faster for certain workload= s, or on embedded devices with a single core only.

--
J.
--047d7bdca7fec9b4a9052d76f4f0--