From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id AAA05250; Wed, 27 Mar 2002 00:02:45 +0100 (MET) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id AAA06049 for ; Wed, 27 Mar 2002 00:02:44 +0100 (MET) Received: from wetware.wetware.com (wetware.wetware.com [199.108.16.1]) by concorde.inria.fr (8.11.1/8.11.1) with ESMTP id g2QN2gb09967 for ; Wed, 27 Mar 2002 00:02:42 +0100 (MET) Received: from kallisti.apple.com(wetware.wetware.com[199.108.16.1]) (13979 bytes) by wetware.wetware.com via sendmail with P:esmtp/R:bind_hosts/T:inet_zone_bind_smtp (sender: ) id for ; Tue, 26 Mar 2002 15:02:41 -0800 (PST) (Smail-3.2.0.114 2001-Aug-6 #1 built 2002-Jan-4) Date: Tue, 26 Mar 2002 15:02:03 -0800 Mime-Version: 1.0 (Apple Message framework v481) Content-Type: text/plain; charset=US-ASCII; format=flowed Subject: [Caml-list] announcing iox-1.00 (beta 1) From: james woodyatt To: The Trade Content-Transfer-Encoding: 7bit Message-Id: <7CA7B522-410D-11D6-8A0C-000502DB38F5@wetware.com> X-Mailer: Apple Mail (2.481) Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk everyone-- Okay, so I wasn't completely forthcoming. While I do make the rent as a practitioner, it is true that I indulge in a little amateur software research from time to time. This message announces the availability in source code of my "Iox" library, which is a framework for concurrent, single-threaded Internet application services. It's released under the 2-clause BSD-style "as-is" license, but I might consider waiving even those terms for peanuts, so you shouldn't have any worries. It's the first result of my encounter with Objective Caml, and I'm probably more pleased with it than I should be. You can download the distribution directly from either of these locators: http://www.wetware.com/jhw/src/iox-1.00b1.tar.gz http://www.wetware.com/jhw/src/iox-1.00b1.tar.bz2 Alternatively, you may browse the source directly: http://www.wetware.com/jhw/src/iox-1.00b1/ There is documentation generated with the [excellent] ocamldoc tool: http://www.wetware.com/jhw/src/iox-1.00b1/doc/index.html Sometime in May this year, I will post a version of this library with appropriate patches applied to fix bugs (assuming some are found, which is probably safe to assume). Some time later, I plan to release another version which has improvements in the feature set. Please direct constructive comments or helpful tips to me. If you find a bug, please help me out by also finding a patch that fixes it. The remainder of this message is a reprint of the README file from the distribution. ----- NAME Iox -- A Framework for Concurrent, Single-threaded Network Applications SYNOPSIS The Iox library is a foundation layer for concurrent, single-threaded network application servers. It comprises the following subsystems: + Modular event loop kernel. + Functional buffer-chained messages. + Flow-control mechanisms. + I/O reactor (for multiplexing network events). + Abstractions for reactive socket I/O. See the accompanying INSTALL file, for instructions on integrating. INTRODUCTION For most network applications, a serializing design (forked or multi- threaded) performs adequately and scales sufficiently well enough that the extra difficulty of implementing a concurrent, single-threaded design is not worth the effort. Some applications-- in particular, application services which manage a set of common resources on behalf of many simultaneous users-- can benefit substantially from a concurrent, single- threaded design. This library is intended to help in the implementation of such designs. The common element in concurrent, single-threaded network application services is a top-level event loop. At the start of the loop, the program waits for operating system notification of I/O operations that may proceed on one or more of its communication channels. When the program receives this notification, it performs all possible I/O operations that do not require the process to block, then it re-enters the start of the loop. The loop is terminated only when all possible I/O operations are performed, and no more operations are pending in the operating system. To support this programming paradigm, the Iox library offers a modular top-level event loop, allowing small reuseable components to be combined into larger complex applications. Also, a flow control mechanism (which borrows some fundamental concepts from the architecture of the STREAMS environment in Unix SVR3) is introduced and used to implement reactive programming interfaces to the BSD sockets transport I/O layer. EXAMPLES The following annotated source code implements a simple loopback test of the Iox library. It establishes a loopback TCP connection and transfers a large string of octets through the socket. It shows how the flow control mechanism is used with the modular event loop. open Iox_kernel (* the modular event loop *) open Iox_circuit (* reactive interfaces to BSD sockets *) (* the component defining the main event loop module *) module C : Component_sig = struct (* need to run with the Iox_circuit event loop module *) let depend = [ Iox_circuit.ring ] (* the core of the component *) type core_t = { kernel_: Loop.t; (* event loop *) teststring_: string; (* string to transfer *) listener_fd_: Iox_socket.TCP.t; (* TCP listener socket *) listener_src_: TCP.listener; (* reactor for TCP listen *) } (* the internet address of localhost *) let loopback_addr = Unix.inet_addr_of_string "127.0.0.1" (* the maximum receive segment size *) let rss = 999 (* the maximum transmit segment size *) let tss = 640 (* constants for constructing the message to transport *) let msgpattern = "all work and no play makes jack a dull boy. " let msgsize = 100000 (* a mutable flag indicating success *) let complete = ref false (* function to convert Unix.Unix_error exceptions into Failure *) let catch_exn_ x = try raise x with | Unix.Unix_error (code, fname, _) -> failwith begin Printf.sprintf "error in %s: %s" fname (Unix.error_message code) end (* the class defining the receiver object. initialized with a core and a TCP socket *) class receiver c fd = let k = c.kernel_ in let p = TCP.protocol ~k ~x:catch_exn_ ~rss fd in object(self) inherit [STREAM.io_t, STREAM.io_t] Iox_protocol.client ~k as super (* the message received so far... *) val mutable message_ = Iox_message.empty (* receive events from the TCP circuit agent. *) method private rx_put = function | `Data m -> message_ <- Iox_message.join message_ m; | `Release -> if (Iox_message.size message_) <> msgsize then failwith "receiver: bad message size!"; let s = Iox_message.contents message_ in if (String.length s) <> msgsize then failwith "message: contents not same size!"; if s <> c.teststring_ then failwith "message: contents not equal!"; super#tx_recycle (* transmit release events to the TCP circuit agent. *) method private tx_cycle = match super#tx_canput with | None -> failwith "receiver#tx_cycle can't put" | Some put -> put `Release; super#detach; Iox_socket.close fd (* init: cancel the listener, attach the TCP agent and unblock the receive flow from the TCP agent. *) initializer c.listener_src_#cancel; super#attach p; super#rx_unblock end (* the class defining the transmitter object. initialized with a core and a TCP socket *) class transmitter c fd = let k = c.kernel_ in let p = TCP.protocol ~k ~x:catch_exn_ ~rss fd in let implemented_ = false in object(self) inherit [STREAM.io_t, STREAM.io_t] Iox_protocol.client ~k as super (* the remainder of the message to transmit still... *) val mutable message_ = let b = Iox_message.buffer_of_string c.teststring_ in Iox_message.create [ b ] (* receive release events from the TCP circuit agent. *) method private rx_put = function | `Data _ -> failwith "transmitter#rx_put unexpected input" | `Release -> super#detach; Iox_socket.close fd; complete := true (* transmit events to the TCP circuit agent. *) method private tx_cycle = match super#tx_canput with | None -> self#tx_recycle | Some put -> if message_ == Iox_message.empty then put `Release else begin let m = if tss < Iox_message.size message_ then begin let m0, m1 = Iox_message.split message_ tss in message_ <- m1; m0 end else begin let m = message_ in message_ <- Iox_message.empty; m end in assert (m != Iox_message.empty); assert (Iox_message.size m > 0); put (`Data m); self#tx_cycle end (* init: attach the TCP circuit agent, unblock the receive flow, initiate the transmit flow. *) initializer super#attach p; super#rx_unblock; super#tx_recycle end (* initialize the core of the test *) let init k = (* create the test string. *) let s = let b = Buffer.create msgsize in while (Buffer.length b) < msgsize do Buffer.add_string b msgpattern done; String.sub (Buffer.contents b) 0 msgsize in (* create the TCP listener socket *) let lfd = Iox_socket.TCP.create () in (* bind the TCP listener socket to an ephemeral loopback port *) let laddr = Iox_socket.TCP.Provider.A.to_sockaddr (loopback_addr, 0) in Iox_socket.bind lfd laddr; (* get the actual address of the listener socket *) let laddr = Iox_socket.TCP.Provider.A.of_sockaddr (Iox_socket.getsockname lfd) in (* listen for one connection on the listener socket *) Iox_socket.listen lfd 1; (* create a new TCP listener object *) let lsrc = new TCP.listener ~k ~x:catch_exn_ lfd in (* construct the core record *) let c = { kernel_ = k; teststring_ = s; listener_fd_ = lfd; listener_src_ = lsrc; } in (* construct a sink for the listener that creates receiver objects with each accepted socket, and start the listener on the flow. *) let lflow = Iox_flow.create ~f:(fun fd -> ignore (new receiver c fd)) in lsrc#start lflow; (* create a new TCP initiator object that constructs transmitter objects with successful connections. *) let connect = function | `Connect fd -> ignore (new transmitter c fd) | `Error e -> catch_exn_ e in let _ = new TCP.initiator ~k ~f:connect laddr in (* return the core *) c (* the finalization function for the test. closes the listener socket and fails if the test was not successful. *) let fini c = Iox_socket.close c.listener_fd_; if not !complete then failwith "not complete at fini" end (* create the event loop module *) include Component(C) (* run the event loop *) Loop.run ring BUGS There is no programmer's guide. Though, there is documentation generated with Maxence Guesdon's excellent 'ocamldoc' tool. On multi-CPU hardware architectures, the event loop needs to be implemented by a pool of N threads, one for each CPU. Threads should be cheap enough that this library is never worth the hassle. LICENSE See the accompanying LICENSE file for the details. It's the 2-clause BSD-style open-source license. AUTHOR j h woodyatt ACKNOWLEDGEMENTS -- j h woodyatt "somebody has to do something, and it's just incredibly pathetic that it has to be us." --jerry garcia ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners