* [Caml-list] The road to OCaml 5.0
@ 2021-10-07 9:48 Florian Angeletti
2021-10-08 1:42 ` Francois Berenger
2021-10-08 20:34 ` Christophe Raffalli
0 siblings, 2 replies; 11+ messages in thread
From: Florian Angeletti @ 2021-10-07 9:48 UTC (permalink / raw)
To: caml-list
[-- Attachment #1: Type: text/plain, Size: 3413 bytes --]
With the convergence between the multicore and standard runtime across
OCaml 4.10.0 to 4.13.0, the development of OCaml multicore has reached a point
where further integration into OCaml's main branch requires fully committing to
a switch to OCaml multicore.
The OCaml team has decided that the time has come for such a commitment.
The new major version, OCaml 5, will be a multicore version of OCaml.
Moreover, OCaml 4.14 will be the last minor release of the 4.x series of OCaml.
Multicore Minimum Viable Product (MVP)
------------------------------------------------------
The first version of OCaml multicore, code-named OCaml 5.0, will be
a Minimum Viable Product focused on:
- x86-64
- Linux, MacOS, Windows mingw-w64
- Parallelism through Domains [1]
- Concurrency through Effect Handlers [2] (without syntactic support and exposed as functions from the standard library)
Our plan is to integrate the multicore branch into the main branch during the
next 6 months. Hopefully, OCaml 5.0 will then be released between March and
April 2022.
Note that OCaml 5.0 focuses on minimal (solid) support for the multicore
runtime system, and will not provide stable user-facing concurrency and
parallelism libraries. There has been a lot of experimentation [3,4] in the last
few years, and some work remains to offer long-term, user-facing concurrent and
parallel programming abstractions. OCaml 5.0 will be a great time to start
adding concurrency and parallelism to your OCaml programs, but the libraries
will still be in flux.
Long term support for OCaml 4.14
----------------------------------------------
While OCaml 5 is stabilising, we plan to extend the support period for
OCaml 4.14 by publishing minor bugfix releases whenever needed. In particular,
OCaml 4.14 will be supported until all tier-1 architectures and operating systems
are available in OCaml 5, and OCaml 5 sequential performance is close enough to
that of OCaml 4.
The sequential glaciation
---------------------------------
To ensure that maintainers can concentrate on Multicore integration, and avoid
any rebase work for the Multicore developers, the trunk branch will be
feature-frozen starting from November 2021. All non-bugfix non-multicore
contributions will be delayed to after the Multicore integration.
We are calling this period the "sequential glaciation".
We understand that this may be frustrating for our contributors, and apologize
for the delay in getting your nice work reviewed and merged into the codebase.
We hope that the sequential glaciation will be a good opportunity to help with
the Multicore integration, review and testing, and/or focus on non-core-compiler
efforts and the rest of the OCaml ecosystem.
With this early feature-freeze, we also plan to release OCaml 4.14.0 in advance,
between January-February 2022, reducing the concurrency between the OCaml 5.0
and OCaml 4.14.0 releases.
References
---------------
[1] "Retrofitting Parallelism onto OCaml", ICFP 2020, https://arxiv.org/abs/2004.11663
[2] "Retrofitting Concurrency onto OCaml", PLDI 2021, https://arxiv.org/abs/2104.00250
[3] Domainslib -- Parallel Programming over Multicore OCaml, https://github.com/ocaml-multicore/domainslib
[4] eio -- Effects-based Parallel IO for OCaml, https://github.com/ocaml-multicore/eio
Happy hacking,
Florian Angeletti, for the OCaml team.
[-- Attachment #2: Type: text/html, Size: 3863 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] The road to OCaml 5.0
2021-10-07 9:48 [Caml-list] The road to OCaml 5.0 Florian Angeletti
@ 2021-10-08 1:42 ` Francois Berenger
2021-10-08 20:34 ` Christophe Raffalli
1 sibling, 0 replies; 11+ messages in thread
From: Francois Berenger @ 2021-10-08 1:42 UTC (permalink / raw)
To: Florian Angeletti; +Cc: caml-list
On 07/10/2021 18:48, Florian Angeletti wrote:
> With the convergence between the multicore and standard runtime across
> OCaml 4.10.0 to 4.13.0, the development of OCaml multicore has reached
> a point
> where further integration into OCaml's main branch requires fully
> committing to
> a switch to OCaml multicore.
>
> The OCaml team has decided that the time has come for such a
> commitment.
> The new major version, OCaml 5, will be a multicore version of OCaml.
> Moreover, OCaml 4.14 will be the last minor release of the 4.x series
> of OCaml.
>
> Multicore Minimum Viable Product (MVP)
> ------------------------------------------------------
>
> The first version of OCaml multicore, code-named OCaml 5.0, will be
> a Minimum Viable Product focused on:
>
> - x86-64
> - Linux, MacOS, Windows mingw-w64
> - Parallelism through Domains [1]
> - Concurrency through Effect Handlers [2] (without syntactic support
> and exposed as functions from the standard library)
>
> Our plan is to integrate the multicore branch into the main branch
> during the
> next 6 months. Hopefully, OCaml 5.0 will then be released between
> March and
> April 2022.
>
> Note that OCaml 5.0 focuses on minimal (solid) support for the
> multicore
> runtime system, and will not provide stable user-facing concurrency
> and
> parallelism libraries. There has been a lot of experimentation [3,4]
> in the last
> few years, and some work remains to offer long-term, user-facing
> concurrent and
> parallel programming abstractions. OCaml 5.0 will be a great time to
> start
> adding concurrency and parallelism to your OCaml programs, but the
> libraries
> will still be in flux.
The Parany library:
---
Generalized map reduce for multicore computers (unfold, map in parallel,
fold).
Parany can process in parallel an "infinite" stream of elements (too big
to fit in memory).
Any Parmap functionality can be reimplemented using parany.
---
already has a git branch which is relying on multicore-OCaml:
https://github.com/UnixJunkie/parany/tree/domains
When OCaml-5.0 ships, I will ship a parany version compatible with it.
I am quite sure Parmap could do the same, by the way (but I'm not the
Parmap maintainer).
Regards,
F.
> Long term support for OCaml 4.14
> ----------------------------------------------
>
> While OCaml 5 is stabilising, we plan to extend the support period for
> OCaml 4.14 by publishing minor bugfix releases whenever needed. In
> particular,
> OCaml 4.14 will be supported until all tier-1 architectures and
> operating systems
> are available in OCaml 5, and OCaml 5 sequential performance is close
> enough to
> that of OCaml 4.
>
> The sequential glaciation
> ---------------------------------
>
> To ensure that maintainers can concentrate on Multicore integration,
> and avoid
> any rebase work for the Multicore developers, the trunk branch will be
> feature-frozen starting from November 2021. All non-bugfix
> non-multicore
> contributions will be delayed to after the Multicore integration.
> We are calling this period the "sequential glaciation".
>
> We understand that this may be frustrating for our contributors, and
> apologize
> for the delay in getting your nice work reviewed and merged into the
> codebase.
> We hope that the sequential glaciation will be a good opportunity to
> help with
> the Multicore integration, review and testing, and/or focus on
> non-core-compiler
> efforts and the rest of the OCaml ecosystem.
>
> With this early feature-freeze, we also plan to release OCaml 4.14.0
> in advance,
> between January-February 2022, reducing the concurrency between the
> OCaml 5.0
> and OCaml 4.14.0 releases.
>
> References
> ---------------
> [1] "Retrofitting Parallelism onto OCaml", ICFP 2020,
> https://arxiv.org/abs/2004.11663
> [2] "Retrofitting Concurrency onto OCaml", PLDI 2021,
> https://arxiv.org/abs/2104.00250
> [3] Domainslib -- Parallel Programming over Multicore OCaml,
> https://github.com/ocaml-multicore/domainslib
> [4] eio -- Effects-based Parallel IO for OCaml,
> https://github.com/ocaml-multicore/eio
>
> Happy hacking,
> Florian Angeletti, for the OCaml team.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] The road to OCaml 5.0
2021-10-07 9:48 [Caml-list] The road to OCaml 5.0 Florian Angeletti
2021-10-08 1:42 ` Francois Berenger
@ 2021-10-08 20:34 ` Christophe Raffalli
2021-10-09 0:07 ` [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0) Christophe Raffalli
1 sibling, 1 reply; 11+ messages in thread
From: Christophe Raffalli @ 2021-10-08 20:34 UTC (permalink / raw)
To: Florian Angeletti, caml-list
[-- Attachment #1: Type: text/plain, Size: 125 bytes --]
Hello dear camelers,
What is the recommanded way to test multicore, passibly via opam, using
flambda ?
Chears,
Christophe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0)
2021-10-08 20:34 ` Christophe Raffalli
@ 2021-10-09 0:07 ` Christophe Raffalli
2021-10-09 0:58 ` Michael Bacarella
2021-10-09 17:14 ` ygrek
0 siblings, 2 replies; 11+ messages in thread
From: Christophe Raffalli @ 2021-10-09 0:07 UTC (permalink / raw)
To: Florian Angeletti, caml-list
[-- Attachment #1: Type: text/plain, Size: 683 bytes --]
Hello,
I managed to install ocaml 4.12.0 with multicore. I could not parallelise my
code in 5mn ;-) but I check just the sequential speed and got a bit
surprised. On the same examples, same options (flambda everywhere), etc ...
Ex 1 Ex 2 Ex 3
4.13.1 normal 45s 12s 49s
4.12.0 normal 36s 11s 45s
4.12.0 multicore 31s 10s 40s
These are not small differences and it is rather surprising that
4.13.1 is significantly slower than 4.12.0 (20 to 25%)
4.12.0 + multicore is faster on sequential code.
Other people observe the same ?
Any idea ? Should I report an issue for the speed degradation of 4.13.1 ?
Christophe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0)
2021-10-09 0:07 ` [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0) Christophe Raffalli
@ 2021-10-09 0:58 ` Michael Bacarella
2021-10-09 1:11 ` Michael Bacarella
2021-10-09 2:10 ` Christophe Raffalli
2021-10-09 17:14 ` ygrek
1 sibling, 2 replies; 11+ messages in thread
From: Michael Bacarella @ 2021-10-09 0:58 UTC (permalink / raw)
To: Christophe Raffalli; +Cc: Florian Angeletti, caml-list
[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]
My gut here says you're unwittingly comparing regular ocaml to flambda
ocaml.
https://ocaml.org/manual/flambda.html
Perhaps multicore only comes in flambda flavor now (I notice it's not
available as a switch).
On Fri, Oct 8, 2021 at 5:07 PM Christophe Raffalli <christophe@raffalli.eu>
wrote:
>
> Hello,
>
> I managed to install ocaml 4.12.0 with multicore. I could not parallelise
> my
> code in 5mn ;-) but I check just the sequential speed and got a bit
> surprised. On the same examples, same options (flambda everywhere), etc ...
>
> Ex 1 Ex 2 Ex 3
> 4.13.1 normal 45s 12s 49s
> 4.12.0 normal 36s 11s 45s
> 4.12.0 multicore 31s 10s 40s
>
> These are not small differences and it is rather surprising that
>
> 4.13.1 is significantly slower than 4.12.0 (20 to 25%)
>
> 4.12.0 + multicore is faster on sequential code.
>
> Other people observe the same ?
> Any idea ? Should I report an issue for the speed degradation of 4.13.1 ?
>
> Christophe
>
[-- Attachment #2: Type: text/html, Size: 1586 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0)
2021-10-09 0:58 ` Michael Bacarella
@ 2021-10-09 1:11 ` Michael Bacarella
2021-10-09 1:20 ` Michael Bacarella
2021-10-09 2:10 ` Christophe Raffalli
1 sibling, 1 reply; 11+ messages in thread
From: Michael Bacarella @ 2021-10-09 1:11 UTC (permalink / raw)
To: Christophe Raffalli; +Cc: Florian Angeletti, caml-list
[-- Attachment #1: Type: text/plain, Size: 1493 bytes --]
Sorry, you actually said as much "On the same examples, same options *(flambda
everywhere)*,"
Though, the expectation that was thwarted for me is that there's no
specific multicore flambda switch. Just multicore. And that 20-25% speedup
felt like a familiar we-switched-to-flambda speedup to me.
On Fri, Oct 8, 2021 at 5:58 PM Michael Bacarella <
michael.bacarella@gmail.com> wrote:
> My gut here says you're unwittingly comparing regular ocaml to flambda
> ocaml.
>
> https://ocaml.org/manual/flambda.html
>
> Perhaps multicore only comes in flambda flavor now (I notice it's not
> available as a switch).
>
> On Fri, Oct 8, 2021 at 5:07 PM Christophe Raffalli <christophe@raffalli.eu>
> wrote:
>
>>
>> Hello,
>>
>> I managed to install ocaml 4.12.0 with multicore. I could not parallelise
>> my
>> code in 5mn ;-) but I check just the sequential speed and got a bit
>> surprised. On the same examples, same options (flambda everywhere), etc
>> ...
>>
>> Ex 1 Ex 2 Ex 3
>> 4.13.1 normal 45s 12s 49s
>> 4.12.0 normal 36s 11s 45s
>> 4.12.0 multicore 31s 10s 40s
>>
>> These are not small differences and it is rather surprising that
>>
>> 4.13.1 is significantly slower than 4.12.0 (20 to 25%)
>>
>> 4.12.0 + multicore is faster on sequential code.
>>
>> Other people observe the same ?
>> Any idea ? Should I report an issue for the speed degradation of 4.13.1 ?
>>
>> Christophe
>>
>
[-- Attachment #2: Type: text/html, Size: 2349 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0)
2021-10-09 1:11 ` Michael Bacarella
@ 2021-10-09 1:20 ` Michael Bacarella
0 siblings, 0 replies; 11+ messages in thread
From: Michael Bacarella @ 2021-10-09 1:20 UTC (permalink / raw)
To: Christophe Raffalli; +Cc: Florian Angeletti, caml-list
[-- Attachment #1: Type: text/plain, Size: 1894 bytes --]
Oh. Or I'm just out of date. Apparently the way you choose compilers
options (like flambda) in a switch changed awhile ago:
https://discuss.ocaml.org/t/experimental-new-layout-for-the-ocaml-variants-packages-in-opam-repository/6779
Sorry about the noise!
On Fri, Oct 8, 2021 at 6:11 PM Michael Bacarella <
michael.bacarella@gmail.com> wrote:
> Sorry, you actually said as much "On the same examples, same options *(flambda
> everywhere)*,"
>
> Though, the expectation that was thwarted for me is that there's no
> specific multicore flambda switch. Just multicore. And that 20-25% speedup
> felt like a familiar we-switched-to-flambda speedup to me.
>
> On Fri, Oct 8, 2021 at 5:58 PM Michael Bacarella <
> michael.bacarella@gmail.com> wrote:
>
>> My gut here says you're unwittingly comparing regular ocaml to flambda
>> ocaml.
>>
>> https://ocaml.org/manual/flambda.html
>>
>> Perhaps multicore only comes in flambda flavor now (I notice it's not
>> available as a switch).
>>
>> On Fri, Oct 8, 2021 at 5:07 PM Christophe Raffalli <
>> christophe@raffalli.eu> wrote:
>>
>>>
>>> Hello,
>>>
>>> I managed to install ocaml 4.12.0 with multicore. I could not
>>> parallelise my
>>> code in 5mn ;-) but I check just the sequential speed and got a bit
>>> surprised. On the same examples, same options (flambda everywhere), etc
>>> ...
>>>
>>> Ex 1 Ex 2 Ex 3
>>> 4.13.1 normal 45s 12s 49s
>>> 4.12.0 normal 36s 11s 45s
>>> 4.12.0 multicore 31s 10s 40s
>>>
>>> These are not small differences and it is rather surprising that
>>>
>>> 4.13.1 is significantly slower than 4.12.0 (20 to 25%)
>>>
>>> 4.12.0 + multicore is faster on sequential code.
>>>
>>> Other people observe the same ?
>>> Any idea ? Should I report an issue for the speed degradation of 4.13.1 ?
>>>
>>> Christophe
>>>
>>
[-- Attachment #2: Type: text/html, Size: 3181 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0)
2021-10-09 0:58 ` Michael Bacarella
2021-10-09 1:11 ` Michael Bacarella
@ 2021-10-09 2:10 ` Christophe Raffalli
2021-10-09 17:20 ` Xavier Leroy
1 sibling, 1 reply; 11+ messages in thread
From: Christophe Raffalli @ 2021-10-09 2:10 UTC (permalink / raw)
To: Michael Bacarella; +Cc: Florian Angeletti, caml-list
[-- Attachment #1: Type: text/plain, Size: 310 bytes --]
On 21-10-08 17:58:00, Michael Bacarella wrote:
> My gut here says you're unwittingly comparing regular ocaml to flambda ocaml.
>
> https://ocaml.org/manual/flambda.html
>
> Perhaps multicore only comes in flambda flavor now (I notice it's not available
> as a switch).
No, I am using flambda on all versions.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0)
2021-10-09 0:07 ` [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0) Christophe Raffalli
2021-10-09 0:58 ` Michael Bacarella
@ 2021-10-09 17:14 ` ygrek
1 sibling, 0 replies; 11+ messages in thread
From: ygrek @ 2021-10-09 17:14 UTC (permalink / raw)
To: caml-list
4.13 default GC changed (to best-fit), that might explain the difference.
--
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-10-10 3:37 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-07 9:48 [Caml-list] The road to OCaml 5.0 Florian Angeletti
2021-10-08 1:42 ` Francois Berenger
2021-10-08 20:34 ` Christophe Raffalli
2021-10-09 0:07 ` [Caml-list] Sequential speed 4.12.0 vs 4.13.1 vs multicore (Was: The road to OCaml 5.0) Christophe Raffalli
2021-10-09 0:58 ` Michael Bacarella
2021-10-09 1:11 ` Michael Bacarella
2021-10-09 1:20 ` Michael Bacarella
2021-10-09 2:10 ` Christophe Raffalli
2021-10-09 17:20 ` Xavier Leroy
2021-10-10 3:37 ` Christophe Raffalli
2021-10-09 17:14 ` ygrek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox