NetMQ and IO Completion Ports

One of the original goals of NetMQ was to use IO Completion ports (a.k.a IOCP) on Windows.
I’m happy to let you know that after two years and multiple attempts NetMQ is now using IOCP.

IO Completion Ports

IO Completion ports is windows answer to C10k (http://www.kegel.com/c10k.html,
Wikipedia) problem, C10K problem is the problem of optimizing sockets to handle large number (10K) of clients at the same time. Linux has epoll, FreeBSD has kqueue and Windows has IO Completion ports.

ZeroMQ

ZeroMQ doesn’t scale well on Windows, on Linux ZeroMQ is using epoll which can scale to thousands of sockets. On windows ZeroMQ is using select which is slow and doesn’t scale well. NetMQ was ported from ZeroMQ and up until now it was using select as well.

In the past multiple attempts were made to integrate IOCP to ZeroMQ but none of them succeeded.

Reactor vs Proactor

The main problem of porting network project from Linux (or any other operating system) to Windows is the different asynchronous network model, Linux is using a pattern called reactor and windows is using proactor.

Both reactor and procator patterns enable multiple asynchronous receive and send operations without blocking the thread.
Both are using an event loop, the different is with the meaning of the event.

With reactor pattern you get an event when the socket is ready for an operation. For example you can register a socket for receive readiness and get an event when the data is available for receiving data from the socket.

Linux has a native support for the reactor pattern with epoll (which ZeroMQ is using) that can scale to thousands of clients. Windows also has support for the reactor pattern with select, but as I mentioned select is slow and doesn’t scale well.

With the proactor pattern you first call the method and get an event when the operation is completed. .Net is using the proactor pattern heavily with Begin/End pattern, tasks and Async pattern (from .net4.5) and that is no surprise because Windows has a native support for the proactor pattern with IO Completion ports.

As you can understand it hard to make same code-base support for both reactor and proactor patterns. This the main reason all the attempts to use IO Completion ports in ZeroMQ failed.
ZeroMQ supports multiple implantation of reactor pattern including epoll on linux, kqueue on FreeBSD and of course select on Windows.

On his new project nanomsg, Martin Sustrik, original developer of ZeroMQ, succeeded in using IO Completion ports and epoll linux on the same code-base. Martin’s approach was to make the epoll behave like proactor. In a nutshell, the send/receive is called in a non-blocking way, if the call failed because the socket was not ready the method will be called again once the ready event is sent and only then the procator completion event is raised.

Mono framework is using same approach as nanomsg when running on Linux.

AsyncIO Library

So as I mentioned earlier, in the past I attempted to make NetMQ use IO Completion ports and failed, the main reason is that .Net support for IOCP is a bit annoying because you don’t have a control over which thread the completion event will be handled on.

Eventually I decided to develop my own library for IO Completion ports with control over the thread and using events instead of callbacks. On windows native IO Completion ports API are used (with pinvoke). When running on other platforms (or when forced) the project is using native .Net Async API (which on Linux with Mono using epoll).

You can find the project on Github and Nuget.

Summary

So to summarize, NetMQ master repository is now using IO Completion ports, which means you can use it with thousands of clients (I don’t have the numbers yet).

So if you only used NetMQ to communicate between your servers you can now use it for client-server communication with multiple clients.

Nuget current version of NetMQ (3.3.0.11) is not using IOCP, to get NetMQ with IOCP you need to compile it from the source code.

comments powered by Disqus