Pages: 1 2
Hello. I got interested in the D language thanks to the D Programming Language book by Andrei Alexandrescu.
The relatively simple web benchmarks at http://www.techempower.com/benchmarks/ have been getting a lot of attention recently. Would it make sense to submit Vibe.d entries for the benchmarks?
On Thu, 09 May 2013 15:38:34 GMT, Mike wrote:
Hello. I got interested in the D language thanks to the D Programming Language book by Andrei Alexandrescu.
The relatively simple web benchmarks at http://www.techempower.com/benchmarks/ have been getting a lot of attention recently. Would it make sense to submit Vibe.d entries for the benchmarks?
It was recently discussed in main D newsgroup : http://forum.dlang.org/thread/urpqdftuofgwespkcdxg@forum.dlang.org
Initially I was eager to submit vibe.d there but after closer examination I am not sure anymore. Problem is that those benchmark actually focus on testing speed of user application modules, like JSON parsing/generation, while vibe.d main focus now is speed of HTTP server and event system core. Which will not be relevant as I/O and computational-heavy data generation will become a bottleneck earlier (I don't remember anyone optimizing json module in vibe.d so far!)
So it may not be a message we (community) want to be sent right now. I am still in doubts though.
Am 10.05.2013 10:03, schrieb Dicebot:
On Thu, 09 May 2013 15:38:34 GMT, Mike wrote:
Hello. I got interested in the D language thanks to the D Programming Language book by Andrei Alexandrescu.
The relatively simple web benchmarks at http://www.techempower.com/benchmarks/ have been getting a lot of attention recently. Would it make sense to submit Vibe.d entries for the benchmarks?
It was recently discussed in main D newsgroup : http://forum.dlang.org/thread/urpqdftuofgwespkcdxg@forum.dlang.org
Initially I was eager to submit vibe.d there but after closer examination I am not sure anymore. Problem is that those benchmark actually focus on testing speed of user application modules, like JSON parsing/generation, while vibe.d main focus now is speed of HTTP server and event system core. Which will not be relevant as I/O and computational-heavy data generation will become a bottleneck earlier (I don't remember anyone optimizing json module in vibe.d so far!)
So it may not be a message we (community) want to be sent right now. I am still in doubts though.
Agreed, I'd say we should start to prepare the benchmark cases and see
how they actually compare against the others. If/when (possibly after
some tuning) they are on par with the fastest ones, we can do an actual
pull request to the official techempower repository.
I don't know when I get enough time to do this properly, so if anyone
wants to start by creating some initial canonical benchmark case
implementations that can be used as a starting point for performance
tuning, that would help a lot.
On Fri, 10 May 2013 15:27:38 +0200, Sönke Ludwig wrote:
...
I was going to create one persistent suite (maybe with CI on top) on top of ones I have been doing last time, but only once I am able to devote some space and h/w to such "lab", and that is unlikely to happen until the late summer. Have been on the run a lot lately, not even sure in what country I'll live next month, so not in a good spot to setup networking suite :)
On Thu, 09 May 2013 15:38:34 GMT, Mike wrote:
Hello. I got interested in the D language thanks to the D Programming Language book by Andrei Alexandrescu.
The relatively simple web benchmarks at http://www.techempower.com/benchmarks/ have been getting a lot of attention recently. Would it make sense to submit Vibe.d entries for the benchmarks?
Okay, I've ported the GO version of the Techempower benchmark target to vibe.d - except for the DB statements. If someone with a running MySQL server (and mysql-native experience) could add those in, that would be great.
On Tue, 01 Oct 2013 07:02:57 GMT, Sönke Ludwig wrote:
On Thu, 09 May 2013 15:38:34 GMT, Mike wrote:
Hello. I got interested in the D language thanks to the D Programming Language book by Andrei Alexandrescu.
The relatively simple web benchmarks at http://www.techempower.com/benchmarks/ have been getting a lot of attention recently. Would it make sense to submit Vibe.d entries for the benchmarks?
Okay, I've ported the [GO version of the Techempower benchmark][1] target to vibe.d - except for the DB statements. If someone with a running MySQL server (and mysql-native experience) could add those in, that would be great.
https://github.com/s-ludwig/techempower-vibed
[1]: https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/go
Went a step further and integrated it into a fork of the benchmark suite: https://github.com/s-ludwig/FrameworkBenchmarks/tree/master/vibe.d
The setup.bat/setup.py scripts are completely untested right now, so those will probably need some work before a pull request can be made.
Hello,
I am an author of some more lightweight benchmark WebFrameworkBenchmark, it is much simpler but also more friendly benchmark than Techempower's Benchmark.
I wanted to ask you to have a look at my Vibe.d results, which are pretty disappointing, just below Ruby performance and far below C and Java frameworks. Vibe.d looks really nice, it has interesting features, but its performance really needs some love (aka optimizations).
On Tue, 24 Nov 2015 18:01:59 GMT, Adam Strzelecki wrote:
Hello,
I am an author of some more lightweight benchmark WebFrameworkBenchmark, it is much simpler but also more friendly benchmark than Techempower's Benchmark.
I wanted to ask you to have a look at my Vibe.d results, which are pretty disappointing, just below Ruby performance and far below C and Java frameworks. Vibe.d looks really nice, it has interesting features, but its performance really needs some love (aka optimizations).
Yeah I found the same. I really want to use D for web API development, but the benchmarks are currently not encouraging.
I wrote a simple function that loads 100 records from MySQL, populates some structs with said data, and then json encodes the lot as output. I then wrote the same in PHP and used Apache bench to benchmark 100,000 requests with a concurrency of 10. Essentially the D / Vibed solution was slightly slower than PHP. PHP does have a fast / efficient JSON encoder and I know D's JSON encoder isn't the best, but still...
Does anyone know if the performance is something that can/will be improved anytime soon?
Am 24.11.2015 um 19:01 schrieb Adam Strzelecki:
Hello,
I am an author of some more lightweight benchmark WebFrameworkBenchmark, it is much simpler but also more friendly benchmark than Techempower's Benchmark.
I wanted to ask you to have a look at my Vibe.d results, which are pretty disappointing, just below Ruby performance and far below C and Java frameworks. Vibe.d looks really nice, it has interesting features, but its performance really needs some love (aka optimizations).
I tested a bit and here are some results:
DMD debug, nogc, singlethread: 30kreq/s
DMD plain, nogc, singlethread: 32kreq/s
DMD release, nogc, singlethread: 61kreq/s
DMD release, nogc, singlethread, libasync: 62kreq/s
GDC release, nogc, singlethread: 63kreq/s
GDC release, nogc, multithread: 64kreq/s
GDC release, gc, singlethread: 58kreq/s
GDC release, gc, multithread: 59kreq/s
release: "dub -b release"
nogc: versions "VibeManualMemoryManagement"
multithread: HTTPServerOption.distribute
Using multiple cores doesn't improve performance by much, indicating
that some shared resource is the bottleneck, but this may simply because
the CPU on my Laptop is just a dual-core (plus hyperthreading) - I'll
have to retest on another box. But building in release mode had a big
impact. DMD interestingly did pretty well compared with GDC.
When looking at the profiler output, there is nothing obvious. A lot of
small things that could hopefully be eliminated, but nothing that looks
like a performance regression.
BTW, I think it would be good to add an additional, more complex test.
Nothing involving a database, but some typical operations, such as JSON
parsing/generation or dynamic page rendering and possibly some CPU
computation. That could easily dwarf the impact of pure request
processing. But as always, the benchmark setup of course depends a lot
on the targeted use case.
One thing that I found during my earlier benchmarks is that testing on
localhost vs. over a real network connection can change the outcome
considerably - also the relative outcome between the different server
implementations. So if possible I'd benchmark between two about equally
fast machines connected with a GbE connection (hopefully it won't saturate).
Am 25.11.2015 um 15:24 schrieb Andrew Chapman:
On Tue, 24 Nov 2015 18:01:59 GMT, Adam Strzelecki wrote:
Hello,
I am an author of some more lightweight benchmark WebFrameworkBenchmark, it is much simpler but also more friendly benchmark than Techempower's Benchmark.
I wanted to ask you to have a look at my Vibe.d results, which are pretty disappointing, just below Ruby performance and far below C and Java frameworks. Vibe.d looks really nice, it has interesting features, but its performance really needs some love (aka optimizations).
Yeah I found the same. I really want to use D for web API development, but the benchmarks are currently not encouraging.
I wrote a simple function that loads 100 records from MySQL, populates some structs with said data, and then json encodes the lot as output. I then wrote the same in PHP and used Apache bench to benchmark 100,000 requests with a concurrency of 10. Essentially the D / Vibed solution was slightly slower than PHP. PHP does have a fast / efficient JSON encoder and I know D's JSON encoder isn't the best, but still...
Does anyone know if the performance is something that can/will be improved anytime soon?
There are some main points outside of the HTTP server where performance
may get lost:
I don't know about the performance of the MySQL driver, but what did you
use for the other two steps? The conversion to JSON, if you use res.writeJsonBody(struct_data), should be pretty optimal speed-wise.
But the SQL query and converting the data representation might easily
make heavy use of the GC, which currently is the number one enemy of a
performant server.
Since it's a stop-the-world collector, it will destroy all concurrency
benefits of a multi-thread setup and its collection cycle can lead to
connection drops/timeouts under heavy load.
This doesn't mean that the GC must be avoided at all costs, but the
amount of allocations that happen per request should be reduced to a
minimum by using stack allocation, ranges, free lists and similar means.
Pages: 1 2
