RejectedSoftware Forums

Sign up

Multithreading support

1) I would like to have a couple 'server' threads dealing with different types of requests, e.g. 1 thread for http+websockets, one for raw sockets, ++. How would I do that? Is it enough to call the individual setup-functions in each thread and then call 'start()'?

2) For a server thread waiting for incoming connections or data on existing threads, e.g. thread A. How would I asynchronously send data to thread A based on a request from another thread?

3) For threads where the main focus is not listening for requests, but e.g. sending some data to a mongodb server. Is it possible to just call the related DB functions and continue executing code, checking back on the request at a later time if it was fulfilled? Note that I don't want this thread to be running an event-loop.

Re: Multithreading support

On Thu, 07 Feb 2013 19:56:13 GMT, Øivind Loe wrote:

1) I would like to have a couple 'server' threads dealing with different types of requests, e.g. 1 thread for http+websockets, one for raw sockets, ++. How would I do that? Is it enough to call the individual setup-functions in each thread and then call 'start()'?

It should be enough to create a new core.thread.Thread and then call runEventLoop(), setup should happen automatically in the module constructor of vibe.core.core.

2) For a server thread waiting for incoming connections or data on existing threads, e.g. thread A. How would I asynchronously send data to thread A based on a request from another thread?

In theory this would work using the Signal and Mutex classes in vibe.core. However, as it stands, they are not really suitable for multi-threading currently. I have that on the table for a while, but have been distracted by lots of other stuff. There will also be a std.concurrency-like message passing system that would be ideal for this.

For now something that works would be to use a core.sync.mutex.Mutex and poll in the receiver thread for new messages in a receiver queue that is protected by the mutex (DList, deque or similar). Polling can be done using a loop calling vibe.core.core.sleep(1.msecs()) between the polls.

3) For threads where the main focus is not listening for requests, but e.g. sending some data to a mongodb server. Is it possible to just call the related DB functions and continue executing code, checking back on the request at a later time if it was fulfilled? Note that I don't want this thread to be running an event-loop.

If you have no event loop running, all operations will be blocking. For any background operations the event loop is always required (or a different thread). You could do something like this, though:

void quasiThreadFunc()
{
   auto task = runTask({ db.querySomthing(); });

   // now do lots of blocking stuff and heavy computation causing the event
   // loop to stall...
   // but call getEventDriver().processEvents() periodically to keep
   // the asynchronous I/O going

   // check if the query is still running using task.running or
   // wait for it using task.join();
}

void threadFunc()
{
    runTask(&quasiThreadFunc);
    runEventLoop();
}

P.S.: I take it that you really need separate threads for this because the threads are performing computations or use synchronous I/O. If that were not the case, I would just use runTask() to acheive concurrency and not deal with threads at all. Or, if heavy computation or synchronous I/O is indeed needed, as long as task permits it, I would recommend to try to find a design that performs all I/O in a single thread and uses threads purely for the computational tasks. The computation threads could then be signaled using standard condition variable and all communication between the I/O tasks would not need to worry about threading issues, which can greatly simplify the code in some cases.

Re: Multithreading support

Thanks for the extensive reply

2) For a server thread waiting for incoming connections or data on existing threads, e.g. thread A. How would I asynchronously send data to thread A based on a request from another thread?

In theory this would work using the Signal and Mutex classes in vibe.core. However, as it stands, they are not really suitable for multi-threading currently. I have that on the table for a while, but have been distracted by lots of other stuff. There will also be a std.concurrency-like message passing system that would be ideal for this.

For now something that works would be to use a core.sync.mutex.Mutex and poll in the receiver thread for new messages in a receiver queue that is protected by the mutex (DList, deque or similar). Polling can be done using a loop calling vibe.core.core.sleep(1.msecs()) between the polls.

But this thread also has to run the event loop to process incoming requests on sockets etc. Will vibe.core.core.sleep(...) run the event loop in the background while sleeping? If not, I would really have to run 'processEvents()', check my incoming queue (with data from other threads), sleep, then loop back and do processEvents(), etc... Is there a better way?

I was hoping to use lockless queues between my threads to speed up message passing.. I don't know effective this would be in terms of cpu usage, but I could probably do something like:

while (...) {
processEvents();
checkQueue();
yield(); //To not go 100% on one cpu polling for events, but still have reasonably fast response to IO/messages
}

what do you think?

3) For threads where the main focus is not listening for requests, but e.g. sending some data to a mongodb server. Is it possible to just call the related DB functions and continue executing code, checking back on the request at a later time if it was fulfilled? Note that I don't want this thread to be running an event-loop.

If you have no event loop running, all operations will be blocking. For any background operations the event loop is always required (or a different thread). You could do something like this, though:

void quasiThreadFunc()
{
   auto task = runTask({ db.querySomthing(); });

   // now do lots of blocking stuff and heavy computation causing the event
   // loop to stall...
   // but call getEventDriver().processEvents() periodically to keep
   // the asynchronous I/O going

   // check if the query is still running using task.running or
   // wait for it using task.join();
}

void threadFunc()
{
    runTask(&quasiThreadFunc);
    runEventLoop();
}

P.S.: I take it that you really need separate threads for this because the threads are performing computations or use synchronous I/O. If that were not the case, I would just use runTask() to acheive concurrency and not deal with threads at all. Or, if heavy computation or synchronous I/O is indeed needed, as long as task permits it, I would recommend to try to find a design that performs all I/O in a single thread and uses threads purely for the computational tasks. The computation threads could then be signaled using standard condition variable and all communication between the I/O tasks would not need to worry about threading issues, which can greatly simplify the code in some cases.

Yes, I have other threads performing heavy computations. I can solve 3) by sending data to my IO thread(s), so I don't think this will be a problem.. As long as I get a good solution on 2).

I am really impressed by what you do here btw. Keep up the good work.

Re: Multithreading support

But this thread also has to run the event loop to process incoming requests on sockets etc. Will
vibe.core.core.sleep(...) run the event loop in the background while sleeping? If not, I would
really have to run 'processEvents()', check my incoming queue (with data from other threads), sleep,
then loop back and do processEvents(), etc... Is there a better way?

Yes, basically all blocking functions in vibe.d will keep the event loop running in the background so that other tasks/fibers can run in the meantime. So no processEvents() is necessary in this case. Druntime's core.thread.Thread.sleep() would indeed block the event loop on the other hand.

I was hoping to use lockless queues between my threads to speed up message passing.. I don't know
effective this would be in terms of cpu usage, but I could probably do something like:

while (...) {
processEvents();
checkQueue();
yield(); //To not go 100% on one cpu polling for events, but still have reasonably fast response
to IO/messages
}

what do you think?

If yield is vibe.core.core.yield(), this would work, and even without the processEvents() call (yield already calls that internally). It would still take up 100% CPU time though if nothing else is running, the sleep call would fix this at the cost of slightly worse response time, but something like sleep(100.usecs()) should minimize that... but since polling shouldn't be necessary once the messaging system is there, this should only be a temporary issue.

P.S.: I take it that you really need separate threads for this because the threads are performing
computations or use synchronous I/O. If that were not the case, I would just use runTask() to
acheive concurrency and not deal with threads at all. Or, if heavy computation or synchronous I/O
is indeed needed, as long as task permits it, I would recommend to try to find a design that
performs all I/O in a single thread and uses threads purely for the computational tasks. The
computation threads could then be signaled using standard condition variable and all communication
between the I/O tasks would not need to worry about threading issues, which can greatly simplify
the code in some cases.

Yes, I have other threads performing heavy computations. I can solve 3) by sending data to my IO
thread(s), so I don't think this will be a problem.. As long as I get a good solution on 2).

I am really impressed by what you do here btw. Keep up the good work.

Thanks! I hope that everything will be a really well-rounded package, once all the multi-threading related structures are in place (v0.9.13 should have them).