On Thu, 7 Sep 2017 09:50:08 +0200, Sönke Ludwig wrote:

Am 06.09.2017 um 17:03 schrieb Clinton D Skakun:

Hi I'd like to get more familiar with vibe.d's method of multitaking routes.

Coming from a PHP and Node background I understand how, in Nodejs, a blocking operation (like a large loop) can hold up the event loop making other requests take longer to process.

With PHP/FCGI/Apache there seems to be a number of workers waiting for requests.

Been told Vibe.d uses fibers.

Just wondering, where's the bottlekneck in Vibe.d regarding blocking operations? I found the vibe.d sleep function, which works fine but I'm not sure if this is causing all routes to sleep. Does using sleep cause the fiber to wait?

I've also noticed that doing very large db queries tends to make the entire vibe API slow (there's tones of free CPU and memory)

Operations that actually block the thread, such as lengthy computations
or I/O operations from libraries that are not written with the event
loop in mind, indeed will block all tasks/fibers within the thread.
vibe.d's own sleep function (like all blocking functions of the API) is
written with the event loop in mind and lets other tasks process while
the sleep is in progress, though.

Database queries in general, as long as the database server can process
multiple queries at the same time, should not block the thread either,
because the requesting task will just sit idle waiting for the database
response, and other tasks of the same thread will be processed instead.
As soon as the database is at its limit, of course all database queries
will start to get delayed or slowed down, but parts of the API that
don't depend on the DB should stay as fast as usual.

There is one major factor for hidden blocking potential, and not just
for the thread, but for the whole process - high memory pressure on the
garbage collector. Sometimes when a lot of memory is allocated (large DB
queries executed at a high frequency could fall into this category), the
garbage collection cycles can take a rather long time, especially if the
overall amount of allocated memory is high.

Depending on the database used, there are ways to reduce or eliminate
the GC pressure, if you can post an example query, maybe I can give some
tips for improvement.

I delegate most long running processes to a worker outside of vibe.d
that runs on a queue. However, I'm unsure how vibe handles requests that
take around 1.5secs to complete. Sometimes, my app makes 5 requests at
nearly the same time from ajax and there seems to be a bit of a lag.

Generally, the timing behavior should be the same as for other async I/O
based frameworks, such as node.js. Only long CPU operations should ever
block the process, or libraries that use normal blocking functions
instead of event loop aware ones (I/O, mutexes etc.).

BTW, there is a built-in worker thread pool that can be used for lengthy
computations. Tasks can be started within this thread pool using
runWorkerTask. Everything else happens in the main thread by default,
concurrency only happens on a fiber level if no threads are used. The
small drawing at http://vibed.org/features#fibers tries to visualize
this - multiple fibers per thread, each fiber processes one tasks at a
time, and execution can shift between fibers as needed.

Thanks, this makes a lot of sense. The reason I asked is because I was having an issue with Vibe.d where it appeared that it wouldn't accept new connections, takes a long time, after a while and just returns a 502 instead of showing any error (not sure if this is Nginx timing out but I'm not getting a timeout error). Postgres is often under a heavy load, but that doesn't seem to be the main issue. After restarting the vibe process, it goes back to normal. If I don't restart it, it takes a bit of time, then goes back to accepting requests just fine and fairly quickly. I expect some lag because of postgres being pounded but I'm confused by the requests taking a very long time then coming back as 502.

I added connection pooling because I thought it may have been a latency issue with connecting to postgres on each request, which made it a bit faster, but still have this 502 problem every now and then.

I just noticed 0.8.1 was released and upgraded. This whole time I was using beta.5 of a past 0.8 release which may or may not have just been a beta bug.

Going to deploy and see how well this new version works. Reading your changelog, I see you added a ton of new fixes.