On Tue, 09 Jun 2015 20:01:41 GMT, Zdeněk wrote:

Hello everyone,

I've been having some performance problems in my REST interface and I've tracked it down to the performance of requestHTTP.

I've created a very simple application which demonstrates this problem: 

shared static this()
{
    import std.stdio;
    import vibe.http.client;
    import vibe.stream.operations;

    int cnt = 0;
    foreach(i; 0..1_000)
    {
        requestHTTP("an example URL I am working with",
            (scope req) {
                req.method = HTTPMethod.GET;
            },
            (scope res) {
                res.bodyReader.readAll();
                cnt++;
                if (i%50 == 0) i.writeln;
            }
        );
    }
}

This takes about 40 seconds, which is way too much. I am certain that it's not caused by the server, as that can do a few thousands requests per second.

What I've noticed is that the application probably spends a lot of time waiting, because it doesn't use a lot of processing power (it uses about 1% of my CPU). This is expected, as it has to wait for the network (even though I'm running the server on the same machine), but still seems too low.

The situation gets better when I run the request in a task (using runTask), but such an approach requires a lot of memory for all the tasks and is not applicable in my case, where my server has to send a request to another server when it itself receives a request.

Any ideas how to fix this? Thanks.

It takes 40s because your requests are being processed in a synchronous way.
Never, EVER, do heavy lifting in the module ctor (static this / shared static this). All it is for is listenHTTP / setTimer / runTask and friends.

Regarding your memory problem, are you sure it's hard memory usage, and not just virtual ? By default each task (fiber) allocate some space, most of which will be unused (each one will/should basically be a page).
If it's hard memory, my guess is that the delegates are allocated. Try to make them functions and see if that improves things.