RejectedSoftware Forums

Sign up

Asynchronous IO

avatar
Eldar Insafutdinov
Posted Sun, 30 Sep 2012 23:30:40 +0200
Reply

One thing I never really understood is how you do asynchronous IO
with Go's goroutines or D's fibers. The callback model of Node.js
is pretty simple:

fs.readFile(fileName, function(data) { 

  use(data);

});

The function returns immediately after invocation and processing
goes back to the event loop, until at some point data is read and
clojure is being invoked. The downside of this approach is
obviously that we cannot right sequential code anymore and
anything more complicated quickly becomes a mess.

Now, my understanding of Vibe's implementation is that incoming
requests are handled in individual Fibers, each of them has it's
own stack and so on. So when in the request handler we invoke an
IO function, my understanding that it should call yield() which
will return it back to the event loop while maintaining the state
of that fiber. Once the operation is ready to continue it will
resume the fiber. This brings me to the two questions:

  1. I/O functions in Phobos are not Fiber-aware, so we will have
    to have our own set of I/O operations, is that correct?
  2. How exactly does it resume the fiber after yield()?

Cheers,
Eldar

Re: Asynchronous IO

avatar
Sönke Ludwig
Posted Mon, 01 Oct 2012 10:23:25 +0200 in reply to Eldar Insafutdinov
Reply
  1. I/O functions in Phobos are not Fiber-aware, so we will have

to have our own set of I/O operations, is that correct?

It would be possible to use at least non-blockig sockets in the same way, but in general, most of Phobos is indeed not suitable for asynchronous I/O. But that's less a problem of fibers, as most of the code does not really have to be aware if it runs inside of a fiber or not (there is not really a difference since each function can usually only access it's own stack frame an globals, so it doesn't matter if the stack frame is on a fiber stack or not). The main problem is that phobos does not provide an interface to the asynchronous OS APIs (epoll, overlapped I/O etc.).

  1. How exactly does it resume the fiber after yield()?

Most of the time, the underlying asynchronous call takes a callback and a user defined pointer. Basically that pointer is used to tell which fiber has to be resumed. You can look for resumeTask() and yieldForEvent() in on of the drivers in vibe.core.drivers, that part is actually quite simple. Exceptions have to be passed from the callback to the inside of the fiber in a special way, but that's about it already.

Sönke

Re: Asynchronous IO

avatar
Sönke Ludwig
Posted Mon, 01 Oct 2012 10:38:06 +0200 in reply to Eldar Insafutdinov
Reply

On Sun, 30 Sep 2012 23:30:40 +0200, Eldar Insafutdinov wrote:

One thing I never really understood is how you do asynchronous IO
with Go's goroutines or D's fibers. The callback model of Node.js
is pretty simple:

fs.readFile(fileName, function(data) { 

  use(data);

});

Btw. if you want to do simulate something like this, you could do it like this: 

void read(InputStream str, void delegate(in ubyte[] data) callback) )
{
    conn.release();
    runTask({
        conn.acquire();
        while( !str.empty ){
            ubyte[] buf = new ubyte[conn.leastSize];
            conn.read(buf);
            callback(buf);
        }
        conn.release();
    });
}

(I didn't test this so it probably contains some mistakes)