RejectedSoftware Forums

Sign up

Question about multithreading vs fibers, vs some other option... possible bug?

Hi,

So I'm currently using Adam's MsSql driver to connect to an existing database. I've got a server set up with:

/* Initalize Router */
auto router = new URLRouter;
router.registerWebInterface(new WebInterface);
router.get("*", serveStaticFiles("./public/"));

/* Initalize Server */
auto settings = new HTTPServerSettings;
settings.port = 8080;
settings.sessionStore = new MemorySessionStore;

listenHTTP(settings, router);

And have implemented the WebInterface class. I'm working on implementing the sql connection to best match what I'm trying to do. Basically I just need to be able to pull some arbitrary number of SQL queries before each page to get relevant information for that page.

Right now I have a wrapper for queries that looks like this:

private auto query (string sql)
{
    auto db = new MsSql(connectionString);
    return db.query(sql);
}

This then gets used some number of times for each page. For instance the index page calls it 6 times. Since opening that connection is fairly expensive I just went ahead and imported std.parallelism and threw each query into something like:

auto usersTask = scopedTask(&query,"
    SELECT Users.DisplayName
    FROM dbo.Users
");
usersTask.executeInNewThread;

And then at the end of all these tasks I have the following:

auto users = usersTask.yieldForce;
...
render!("index.dt", users, ...);

When it loads correctly it takes ~190ms, on localhost, time to first byte. This is compared to about 800ms without the multi-threading, so a marked improvement.

That said, it only loads correct about half the time (probably a bit less). For some reason it returns a 200 OK message before the page finishes loading one the diet templates. I do not have this problem when I'm not using std.parallelism and I just run the queries serially.

Any thoughts as to what I'm missing, or a better approach to doing this in general?

Thanks,
Charles

Re: Question about multithreading vs fibers, vs some other option... possible bug?

On Wed, 22 Apr 2015 16:48:10 GMT, Charles wrote:

That said, it only loads correct about half the time (probably a bit less). For some reason it returns a 200 OK message before the page finishes loading one the diet templates. I do not have this problem when I'm not using std.parallelism and I just run the queries serially.

A bit more on what appears to be going on when it fails:

  1. I run through the first five queries using scopedTask as I've described.
  2. The last query I have running in the main thread.
  3. Page begins to load the diet template.
  4. It may or may not load every row from each query, seemingly non-deterministic.
  5. Sometimes execution stops, and the template ceases loading other blocks completely.

For instance, on the following code, it'll never load the blocks scripts:

block datatable
    table#dataTable.display(cellspacing="0", width="100%")
        thead
            tr
                th ID
                th Name
                th Creator
                th State
                th Stage
                th Actions
        tbody
            - foreach (row; data)
                - auto rowArr = row.toAA;
                tr(data-username='#{rowArr["username"]}', data-project='#{"project"}', data-state='#{rowArr["state"]}', data-workflow='#{rowArr["workflow_name"]}')
                    td.text-center #{rowArr["sim_id"]}
                    td #{rowArr["sim_name"]}
                    td.text-center #{rowArr["username"]}
                    td.text-center #{rowArr["state"]}
                    td.text-center #{rowArr["stage"]}
                    td.text-center Not Implemented Yet
block scripts
    script(src='/js/pages/datatable.js')

It does seem to consistently load an entire row each time it begins a row.

Re: Question about multithreading vs fibers, vs some other option... possible bug?

On Thu, 23 Apr 2015 12:55:09 GMT, Charles wrote:

On Wed, 22 Apr 2015 16:48:10 GMT, Charles wrote:

That said, it only loads correct about half the time (probably a bit less). For some reason it returns a 200 OK message before the page finishes loading one the diet templates. I do not have this problem when I'm not using std.parallelism and I just run the queries serially.

A bit more on what appears to be going on when it fails:

  1. I run through the first five queries using scopedTask as I've described.
  2. The last query I have running in the main thread.
  3. Page begins to load the diet template.
  4. It may or may not load every row from each query, seemingly non-deterministic.
  5. Sometimes execution stops, and the template ceases loading other blocks completely.

For instance, on the following code, it'll never load the blocks scripts:

block datatable
    table#dataTable.display(cellspacing="0", width="100%")
        thead
            tr
                th ID
                th Name
                th Creator
                th State
                th Stage
                th Actions
        tbody
            - foreach (row; data)
                - auto rowArr = row.toAA;
                tr(data-username='#{rowArr["username"]}', data-project='#{"project"}', data-state='#{rowArr["state"]}', data-workflow='#{rowArr["workflow_name"]}')
                    td.text-center #{rowArr["sim_id"]}
                    td #{rowArr["sim_name"]}
                    td.text-center #{rowArr["username"]}
                    td.text-center #{rowArr["state"]}
                    td.text-center #{rowArr["stage"]}
                    td.text-center Not Implemented Yet
block scripts
    script(src='/js/pages/datatable.js')

It does seem to consistently load an entire row each time it begins a row.

Are you waiting for all the threads to finish? You mention you run 5 queries, but I only see you waiting for one.
The order in which they all terminate is not going to be deterministic, you must wait for each of them to finish before you try to use it's data.
The fact that sometimes it works could be when this last one terminates after all the others.

Alternatively, it could also mean that the MsSQL driver is not thread safe, though I have no idea if that is the case or not, but I guess it's fairly unlikely.

Also, I think it's generally a bad idea to mix std.parallelism with vibe.d. Vibe's has it's own versions of most of std.parallelism, which will not block the event loop thread on contention, and will generally do the right thing.
The major difference is that for example in your case, when you call yieldForce(), vibe.d cannot just go and do something else, like handle another request. The OS will just switch away from your event loop thread altogether. This is very inefficient and goes against vibe'd async model.

Re: Question about multithreading vs fibers, vs some other option... possible bug?

On Thu, 23 Apr 2015 13:22:46 GMT, Márcio Martins wrote:

Are you waiting for all the threads to finish? You mention you run 5 queries, but I only see you waiting for one.

Yeah, I'm waiting for all of them to finish.

but I guess it's fairly unlikely.

That it is thread safe?

Also, I think it's generally a bad idea to mix std.parallelism with vibe.d. Vibe's has it's own versions of most of std.parallelism, which will not block the event loop thread on contention, and will generally do the right thing.

That's probably my issue.

The major difference is that for example in your case, when you call yieldForce(), vibe.d cannot just go and do something else, like handle another request. The OS will just switch away from your event loop thread altogether. This is very inefficient and goes against vibe'd async model.

I was having issues getting async to work. I'll try it again.

Re: Question about multithreading vs fibers, vs some other option... possible bug?

On Thu, 23 Apr 2015 14:05:44 GMT, Charles wrote:

On Thu, 23 Apr 2015 13:22:46 GMT, Márcio Martins wrote:

Are you waiting for all the threads to finish? You mention you run 5 queries, but I only see you waiting for one.

Yeah, I'm waiting for all of them to finish.

but I guess it's fairly unlikely.

That it is thread safe?

Also, I think it's generally a bad idea to mix std.parallelism with vibe.d. Vibe's has it's own versions of most of std.parallelism, which will not block the event loop thread on contention, and will generally do the right thing.

That's probably my issue.

The major difference is that for example in your case, when you call yieldForce(), vibe.d cannot just go and do something else, like handle another request. The OS will just switch away from your event loop thread altogether. This is very inefficient and goes against vibe'd async model.

I was having issues getting async to work. I'll try it again.

This is an old thread, where I was trying to do something similar:
http://forum.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/20279/

Re: Question about multithreading vs fibers, vs some other option... possible bug?

On Thu, 23 Apr 2015 15:58:45 GMT, Márcio Martins wrote:

This is an old thread, where I was trying to do something similar:
http://forum.rejectedsoftware.com/groups/rejectedsoftware.vibed/thread/20279/

Went ahead and tried that. Seems to yield the same type of buggy results. I'll give the MsSql driver another look to see if its not thread safe or something.