RejectedSoftware Forums

Sign up

Pages: 1 2

HTTPServerOption.distribute and Cassandra database driver conflict

I want to make use of the Cassandra driver located at
https://github.com/rjmcguire/cassandra-d

I also want to use Vibe's threading. However, when I use both, my program exits with error code -11. I tried to use gdb on the application, but when I run gdb ./test2, I can't load any pages.

I uploaded a simple case to connect to a local cassandra server:
https://github.com/gronka/vibetest01

I'd be happy to dig into this, but I'm not sure how to get a stack trace or other places to start.

I've also been looking at DataStax's C++ driver for Cassandra. I would like to contribute to cassandra-d since it would be a good exercise in D for me, but the DataStax driver is probably higher quality than I would achieve for quite some time. Would you expect it to be difficult to use that driver in D, or to convert it to a library? I've read a bit about these processes, but I have no idea how hard it might actually be.

Their driver is here:
https://github.com/datastax/cpp-driver

Thanks,

Re: HTTPServerOption.distribute and Cassandra database driver conflict

On Thu, 16 Jul 2015 20:53:05 GMT, Taylor Gronka wrote:

I want to make use of the Cassandra driver located at
https://github.com/rjmcguire/cassandra-d

I also want to use Vibe's threading. However, when I use both, my program exits with error code -11. I tried to use gdb on the application, but when I run gdb ./test2, I can't load any pages.

I uploaded a simple case to connect to a local cassandra server:
https://github.com/gronka/vibetest01

Since the DB driver code is not thread-safe, you'll have to create one CassandraClient instance per thread. In that case everything should work fine in theory. The simplest approach would be to make the Cassandra variables static. However, you'll have to be careful to not introduce any race-conditions in your own code, too, and use a TaskMutex (or normal Mutex) where appropriate.

It's actually an open issue with vibe.d's HTTPServerOption.distribute that it circumvents the type system and doesn't enforce shared request handlers. Especially for the vibe.web.web interface this needs some thought how to solve it nicely without causing too much disturbance.

I'd be happy to dig into this, but I'm not sure how to get a stack trace or other places to start.

I've also been looking at DataStax's C++ driver for Cassandra. I would like to contribute to cassandra-d since it would be a good exercise in D for me, but the DataStax driver is probably higher quality than I would achieve for quite some time. Would you expect it to be difficult to use that driver in D, or to convert it to a library? I've read a bit about these processes, but I have no idea how hard it might actually be.

Their driver is here:
https://github.com/datastax/cpp-driver

Thanks,

It should basically be portable to D relatively straight forward (well, it's tedious work of course), but, ironically, the fact that it uses asynchronous I/O will make the job harder than it should be (it uses a lot of explicit callbacks/event handlers that are not needed for vibe.d).

Since the protocol is relatively simple, I'd recommend to extend the cassandra-d driver instead. However, the original author doesn't seem to be active anymore, so this might mean taking the maintainership. Maybe it's possible to contact him and move the driver to a GitHub organization.

Re: HTTPServerOption.distribute and Cassandra database driver conflict

On Tue, 21 Jul 2015 19:48:15 GMT, Sönke Ludwig wrote:

Since the DB driver code is not thread-safe, you'll have to create one CassandraClient instance per thread. In that case everything should work fine in theory. The simplest approach would be to make the Cassandra variables static. However, you'll have to be careful to not introduce any race-conditions in your own code, too, and use a TaskMutex (or normal Mutex) where appropriate.

I was wishing the ConnectionPool code you added would magically make it thread-safe, haha. (You don't have to explain it I'll look up exactly what it does.)

Since the protocol is relatively simple, I'd recommend to extend the cassandra-d driver instead. However, the original author doesn't seem to be active anymore, so this might mean taking the maintainership. Maybe it's possible to contact him and move the driver to a GitHub organization.

I'll do that then. I've liked working with Cassandra, and I've liked working with D and vibe.d. I lack background though so it will take me a bit, but I think I can mostly follow it.

Is the vibe.d newsgroup the best place to follow discussion about the framework (and learn more)?

At exactly what point are threads spawned at? It seems to me that it's at the point listenHTTP(settings, router);.

Re: HTTPServerOption.distribute and Cassandra database driver conflict

Since the DB driver code is not thread-safe, you'll have to create one CassandraClient instance per thread. In that case everything should work fine in theory. The simplest approach would be to make the Cassandra variables static. However, you'll have to be careful to not introduce any race-conditions in your own code, too, and use a TaskMutex (or normal Mutex) where appropriate.

I'm reviewing these posts:
http://forum.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/25949/
http://forum.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/8304/

For clarity, every variable I define globally in vibe.d is the same as thread-safe, correct? Basically, in regards to vibe.d, terms can mostly be used interchangeably? (With the exception of initializing with __gshared.)

(This seems to be true for all of D, not just vibe.d, according to
http://dlang.org/migrate-to-shared.html)

Re: HTTPServerOption.distribute and Cassandra database driver conflict

For clarity, every variable I define globally in vibe.d is the same as thread-safe, correct? Basically, in regards to vibe.d, terms can mostly be used interchangeably? (With the exception of initializing with __gshared.)

I'm not sure if I said that right. I suppose, for a variable to be global, it must be full of static and const anyways. So, for a variable to be properly global, it must also be thread-safe (although I'm sure there are exceptions/mistakes that can be made). The two goals align.

Re: HTTPServerOption.distribute and Cassandra database driver conflict

On Wed, 22 Jul 2015 07:09:38 GMT, Taylor Gronka wrote:

For clarity, every variable I define globally in vibe.d is the same as thread-safe, correct? Basically, in regards to vibe.d, terms can mostly be used interchangeably? (With the exception of initializing with __gshared.)

I'm not sure if I said that right. I suppose, for a variable to be global, it must be full of static and const anyways. So, for a variable to be properly global, it must also be thread-safe (although I'm sure there are exceptions/mistakes that can be made). The two goals align.

static is a TLS specifier (thread-local storage), so you have a different instance there on Thread 1, Thread 2...

This means you don't need it to be thread-safe.

Connection pool should be specified as non-thread-safe, you have to wrap it in a TaskMutex if you make it shared : __gshared ConnectionPool!Object gs_objPool;

Otherwise, you'll have a single pool per thread, which is also fine.

Personally, I don't like listening on multiple threads, makes things more complicated. I would suggest you send "jobs" to worker tasks instead, using vibe.core.concurrency, and let the main thread deal with request/response.

Re: HTTPServerOption.distribute and Cassandra database driver conflict

Am 22.07.2015 um 16:08 schrieb Etienne Cimon:

Personally, I don't like listening on multiple threads, makes things more complicated. I would suggest you send "jobs" to worker tasks instead, using vibe.core.concurrency, and let the main thread deal with request/response.

Ditto, as long as you have payloads that you can reasonably put into
worker tasks. It simplifies the architecture a lot and can also improve
the response latency in case of CPU intensive tasks.

If you have an architecture where the process doesn't hold permanent
state in-memory (but in a database such as Redis), it's also often a
good idea to start multiple single-threaded processes instead of a
single multi-threaded one because of the garbage collector. The GC is
currently very inefficient in a multi-threaded scenario, so it's very
important to minimize it's use for high-performance multi-threaded
applications (use -version=VibeManualMemoryManagement for vibe.d in that
case).

Re: HTTPServerOption.distribute and Cassandra database driver conflict

Am 22.07.2015 um 08:49 schrieb Taylor Gronka:

On Tue, 21 Jul 2015 19:48:15 GMT, Sönke Ludwig wrote:

Since the DB driver code is not thread-safe, you'll have to create one CassandraClient instance per thread. In that case everything should work fine in theory. The simplest approach would be to make the Cassandra variables static. However, you'll have to be careful to not introduce any race-conditions in your own code, too, and use a TaskMutex (or normal Mutex) where appropriate.

I was wishing the ConnectionPool code you added would magically make it thread-safe, haha. (You don't have to explain it I'll look up exactly what it does.)

The best that ConnectionPool could reasonably do as it stands would be
to internally create one pool per thread, because individual connections
cannot safely be moved between threads anyway (or at least there is no
static guarantee that a particular connection type is able to handle
that). So it wouldn't really gain anything effectively.

Since the protocol is relatively simple, I'd recommend to extend the cassandra-d driver instead. However, the original author doesn't seem to be active anymore, so this might mean taking the maintainership. Maybe it's possible to contact him and move the driver to a GitHub organization.

I'll do that then. I've liked working with Cassandra, and I've liked working with D and vibe.d. I lack background though so it will take me a bit, but I think I can mostly follow it.

Is the vibe.d newsgroup the best place to follow discussion about the framework (and learn more)?

The newsgroup and also the GitHub repository. A lot of discussions
around implementation or design issues happen there.

At exactly what point are threads spawned at? It seems to me that it's at the point listenHTTP(settings, router);.

Yes, this is the latest point at which they will be started. On a lower
level, it happens when the first worker task is started
(runWorkerTask/runWorkerTaskDist). By default, the number of threads
will equal the number of logical CPU cores and starting more worker
tasks or listening for more HTTP connections will not change the number
of worker threads.

Re: HTTPServerOption.distribute and Cassandra database driver conflict

Am 22.07.2015 um 16:08 schrieb Etienne Cimon:

On Wed, 22 Jul 2015 07:09:38 GMT, Taylor Gronka wrote:

For clarity, every variable I define globally in vibe.d is the same as thread-safe, correct? Basically, in regards to vibe.d, terms can mostly be used interchangeably? (With the exception of initializing with __gshared.)

I'm not sure if I said that right. I suppose, for a variable to be global, it must be full of static and const anyways. So, for a variable to be properly global, it must also be thread-safe (although I'm sure there are exceptions/mistakes that can be made). The two goals align.

static is a TLS specifier (thread-local storage), so you have a different instance there on Thread 1, Thread 2...

Should be noted that vibe.d, in addition to the default static
thread-local variables, also provides analogous task-local variables:
http://vibed.org/api/vibe.core.core/TaskLocal

Re: HTTPServerOption.distribute and Cassandra database driver conflict

On Wed, 22 Jul 2015 17:30:19 +0200, Sönke Ludwig wrote:

single multi-threaded one because of the garbage collector. The GC is
currently very inefficient in a multi-threaded scenario, so it's very
important to minimize it's use for high-performance multi-threaded
applications (use -version=VibeManualMemoryManagement for vibe.d in that

The GC will lock (for all threads) during allocation and during collection, this happens also for manual memory management as well (but the collection is spared). Even the single-threaded scenario will also flush cpu L1 cache at every allocation because of this synchronization, costing probably >100 cycles + contention :/

The only real solution is to use an experimental TLS GC that I added to a custom druntime/phobos here:

https://github.com/etcimon/druntime/tree/2.068-custom
https://github.com/etcimon/phobos/tree/2.068-custom

I wonder what the benchmark numbers would be in comparison! I use it because I don't construct an object in a thread and let it get collected in another, so I never got any problems with it with my vibe.d fork.

Pages: 1 2