RejectedSoftware Forums

Sign up

HTTP server gets progressively slower with every test run

I have a very simple web server:

import vibe.vibe;
import std.getopt;
import std.stdio;

ushort serverPort = 8080;

int main(string[] args)
{
	getopt(args, "port|p", &serverPort);
	
	auto settings = new HTTPServerSettings;
	settings.port = serverPort;

	listenHTTP(settings, &handleRequest);
	return runEventLoop();
}


void handleRequest(HTTPServerRequest req,
				   HTTPServerResponse res)
{
	res.writeBody("Hello World","text/plain");
}


When I run ab against it the first time, the results look great.
But every time I rerun it, it gets worse and worse until it slows down to a crawl.
Restarting the app resets it back to its initial speed and then it slows down again.

for example "ab -n 10000 -c 20" gives me:

a) run 1: 5000 TPS
b) run 2: 2800 TPS
c) run 3: 1900 TPS

etc.

I am on latest DMD, on Mint 64-bit.

Is this a GC problem or something else altogether?

Thanks
Jacek

Re: HTTP server gets progressively slower with every test run

Sorry, the markdown seemed to nuke my code, will try again

import vibe.vibe;
import std.parallelism;
import std.getopt;
import std.stdio;
ushort serverPort = 8080;
int main(string[] args)
{
	getopt(args, "port|p", &serverPort);
	auto settings = new HTTPServerSettings;
	settings.port = serverPort;
	listenHTTP(settings, &handleRequest);
	return runEventLoop();
}
void handleRequest(HTTPServerRequest req,
				   HTTPServerResponse res)
{
	res.writeBody("Hello World","text/plain");
}



Re: HTTP server gets progressively slower with every test run

The original results were with '~master. Once I changed the packages to

"vibe-d": ">=0.7.12"

the problem still exists, although less pronounced

1st run: 5000 TPS
2nd run: 4000 TPS
3rd run: 3000 TPS

Re: HTTP server gets progressively slower with every test run

Am 21.11.2013 16:55, schrieb Jacek Furmankiewicz:

The original results were with '~master. Once I changed the packages to

"vibe-d": ">=0.7.12"

the problem still exists, although less pronounced

1st run: 5000 TPS
2nd run: 4000 TPS
3rd run: 3000 TPS

I've observed the same behavior when I made some tests during the last
days, there seems to to be a sudden decline in performance after ~30k
requests. I'll take a closer look after 0.7.18 is out.

Re: HTTP server gets progressively slower with every test run

Would that not be a showstopper for any apps currently in production?

Re: HTTP server gets progressively slower with every test run

Am 21.11.2013 19:59, schrieb Jacek Furmankiewicz:

Would that not be a showstopper for any apps currently in production?

It doesn't seem to be progressive, but rather seems to be a single step,
so it's not that bad (I've tested up to 4M requests and it was more or
less steady except for the first 30k requests), plus it has probably
been there for quite some time.

Also note that vibe.d is not yet declared fully production ready
(although it is already used successfully for non-heavy-duty projects).
It's quite good now in most areas, but any new feature or any larger
change can break things and it's never possible to catch all regressions
prior to a release.

Having said that, in a not so distant future, the first stable branch
will emerge from master. And probably in a few weeks, a CI server will
not only check unit tests, but also monitor performance and memory
consumption for each commit. Things should get much more stable then.

Re: HTTP server gets progressively slower with every test run

Am 21.11.2013 20:47, schrieb Sönke Ludwig:

Am 21.11.2013 19:59, schrieb Jacek Furmankiewicz:

Would that not be a showstopper for any apps currently in production?

It doesn't seem to be progressive, but rather seems to be a single step,
so it's not that bad (I've tested up to 4M requests and it was more or
less steady except for the first 30k requests), plus it has probably
been there for quite some time.

Also note that vibe.d is not yet declared fully production ready
(although it is already used successfully for non-heavy-duty projects).
It's quite good now in most areas, but any new feature or any larger
change can break things and it's never possible to catch all regressions
prior to a release.

Having said that, in a not so distant future, the first stable branch
will emerge from master. And probably in a few weeks, a CI server will
not only check unit tests, but also monitor performance and memory
consumption for each commit. Things should get much more stable then.

Interestingly, it also gets faster over limited periods of time after
some more requests. But the important thing is that it's not
progressive, but can be considered a constant performance degradation.
I'll check with VTune in the coming days what causes this.

Re: HTTP server gets progressively slower with every test run

Hi, were you able to find the root causes of this degradation?

Thanks
Jacek

On Fri, 22 Nov 2013 09:53:04 +0100, Sönke Ludwig wrote:

Am 21.11.2013 20:47, schrieb Sönke Ludwig:

Am 21.11.2013 19:59, schrieb Jacek Furmankiewicz:

Would that not be a showstopper for any apps currently in production?

It doesn't seem to be progressive, but rather seems to be a single step,
so it's not that bad (I've tested up to 4M requests and it was more or
less steady except for the first 30k requests), plus it has probably
been there for quite some time.

Also note that vibe.d is not yet declared fully production ready
(although it is already used successfully for non-heavy-duty projects).
It's quite good now in most areas, but any new feature or any larger
change can break things and it's never possible to catch all regressions
prior to a release.

Having said that, in a not so distant future, the first stable branch
will emerge from master. And probably in a few weeks, a CI server will
not only check unit tests, but also monitor performance and memory
consumption for each commit. Things should get much more stable then.

Interestingly, it also gets faster over limited periods of time after
some more requests. But the important thing is that it's not
progressive, but can be considered a constant performance degradation.
I'll check with VTune in the coming days what causes this.

Re: HTTP server gets progressively slower with every test run

On 12/12/2013 12:27 AM, Jacek wrote:

Hi, were you able to find the root causes of this degradation?

Thanks
Jacek

Seems to be fixed with
https://github.com/rejectedsoftware/vibe.d/commit/1488d5c7426ff7acc1d0bceef5667066354c1191.

Sönke was this a case of the following bug?
https://d.puremagic.com/issues/show_bug.cgi?id=3523

Re: HTTP server gets progressively slower with every test run

Am 14.12.2013 00:52, schrieb Martin Nowak:

On 12/12/2013 12:27 AM, Jacek wrote:

Hi, were you able to find the root causes of this degradation?

Thanks
Jacek

Seems to be fixed with
https://github.com/rejectedsoftware/vibe.d/commit/1488d5c7426ff7acc1d0bceef5667066354c1191.

I can still see the drops on git master and also for that specific
commit. Invoking the "bench-http-request" example with "dub -- -c 100 -k
100" shows a drop for me after about the first 30k to 120k requests
(Windows, server and client on the same machine). Unfortunately I
haven't had the time to debug it, yet.

Sönke was this a case of the following bug?
https://d.puremagic.com/issues/show_bug.cgi?id=3523

It looks a lot like it. The vibe.d bug simply caused some Fiber
references to get lost instead of getting recycled, so they could have
been collected. And I'm pretty sure that there are no other references
in the code that would keep it alive.