RejectedSoftware Forums

Sign up

Support of HTTPS?

Hello. We are evaluating a platform for our gambling backend solution (mainly it's a REST API consumed by js client). Currently, we have written some code in .NET/C#, but we'd like a portable solution and probably a better performance. So we are considering D/GDC. I came across vibe.d. It looks really promising, but we couldn't find anything about HTTPS support. So does vibe.d supports SSL?

Thanks :)

Re: Support of HTTPS?

On Sat, 04 Apr 2015 23:21:39 GMT, Dato wrote:

Hello. We are evaluating a platform for our gambling backend solution (mainly it's a REST API consumed by js client). Currently, we have written some code in .NET/C#, but we'd like a portable solution and probably a better performance. So we are considering D/GDC. I came across vibe.d. It looks really promising, but we couldn't find anything about HTTPS support. So does vibe.d supports SSL?

Thanks :)

It certainly does!

Low level SSL/TLS stream support: http://vibed.org/api/vibe.stream.ssl/
HTTPS server support: http://vibed.org/api/vibe.http.server/HTTPServerSettings.sslContext

The HTTP and SMTP clients also support SSL/TLS connections.

For HTTPS server examples see also:
https://github.com/rejectedsoftware/vibe.d/tree/master/examples/https_server
https://github.com/rejectedsoftware/vibe.d/tree/master/examples/https_server_sni

Re: Support of HTTPS?

On Sun, 05 Apr 2015 14:44:20 GMT, Sönke Ludwig wrote:

On Sat, 04 Apr 2015 23:21:39 GMT, Dato wrote:

Hello. We are evaluating a platform for our gambling backend solution (mainly it's a REST API consumed by js client). Currently, we have written some code in .NET/C#, but we'd like a portable solution and probably a better performance. So we are considering D/GDC. I came across vibe.d. It looks really promising, but we couldn't find anything about HTTPS support. So does vibe.d supports SSL?

Thanks :)

It certainly does!

Low level SSL/TLS stream support: http://vibed.org/api/vibe.stream.ssl/
HTTPS server support: http://vibed.org/api/vibe.http.server/HTTPServerSettings.sslContext

The HTTP and SMTP clients also support SSL/TLS connections.

For HTTPS server examples see also:
https://github.com/rejectedsoftware/vibe.d/tree/master/examples/https_server
https://github.com/rejectedsoftware/vibe.d/tree/master/examples/https_server_sni

Thanks, nice to now :). Also before we start diving deep in vibe, is there any pitfall, drawbacks we should be aware of? Is vibe.d stable enough to run a web server 24/7? is there any production system built on it? how does it do within heavy loaded environment?

Please forgive me if my questions seems foolish, that's because we are very new to D language. We've been impressed with it, but we are just starting to scratch the surface. :)

Thanks again

Re: Support of HTTPS?

One more thing, we are using MongoDB as our DB engine and we are planning to use Redis as our cache layer. It would be great if you could share status of those drivers. are they stable enough? what about their performance compared to C#/C++ drivers?

Thanks again :)

Re: Support of HTTPS?

Am 05.04.2015 um 18:07 schrieb Dato:

(...)

Thanks, nice to now :). Also before we start diving deep in vibe, is there any pitfall, drawbacks we should be aware of? Is vibe.d stable enough to run a web server 24/7? is there any production system built on it? how does it do within heavy loaded environment?

There are several production systems running with vibe.d, so this is
certainly doable. However, one thing to be aware of is that it currently
still lacks certain kinds of DoS countermeasures, so if you want to use
it for a large-scale public service, there should be some kind of
proxy/load balancer system that provides those.

It scales well for high loads, but you have to be careful with using a
lot of garbage collected memory, because the garbage collector will stop
the whole process during collection runs. Generally it's a good idea to
use some form of reference counting for dynamic memory allocations (e.g.
vibe.utils.memory.FreeListRef or std.typecons.RefCounted).

Re: Support of HTTPS?

Am 05.04.2015 um 18:09 schrieb Dato:

One more thing, we are using MongoDB as our DB engine and we are planning to use Redis as our cache layer. It would be great if you could share status of those drivers. are they stable enough? what about their performance compared to C#/C++ drivers?

Thanks again :)

They both work pretty reliable and fast. The MongoDB driver may still
have issues with MongoDB 3.x.x as it hasn't really been tested with that
version and some details of the network protocol have changed compared
to 2.6.x, but older versions are fine.

I can't comment on the relative performance to the C++ counterparts
because there are no comparative benchmarks AFAIK, but they both are
pretty fast and there shouldn't be much of a difference.

Re: Support of HTTPS?

On Mon, 06 Apr 2015 14:43:52 +0200, Sönke Ludwig wrote:

Am 05.04.2015 um 18:07 schrieb Dato:

(...)

Thanks, nice to now :). Also before we start diving deep in vibe, is there any pitfall, drawbacks we should be aware of? Is vibe.d stable enough to run a web server 24/7? is there any production system built on it? how does it do within heavy loaded environment?

There are several production systems running with vibe.d, so this is
certainly doable. However, one thing to be aware of is that it currently
still lacks certain kinds of DoS countermeasures, so if you want to use
it for a large-scale public service, there should be some kind of
proxy/load balancer system that provides those.

It scales well for high loads, but you have to be careful with using a
lot of garbage collected memory, because the garbage collector will stop
the whole process during collection runs. Generally it's a good idea to
use some form of reference counting for dynamic memory allocations (e.g.
vibe.utils.memory.FreeListRef or std.typecons.RefCounted).

I see. I guess those classes are similar to smart pointers. Just to make sure, is D's GC so bad? I can use RefCounted and FreeListRef, but what about vibe's internal implementation? We are going to have several thousand small requests per second and GC should also be an issue for vibe's internal allocations/calls no? Currently, our C# implementation handles ~7k requests per second without caching on a single server. According to our (dirty work) benchmarks, the system should perform at least 35% better when ported to GDC, but we didn't check GC to be honest...

Thanks again :)

Re: Support of HTTPS?

On Mon, 06 Apr 2015 18:06:59 GMT, Dato wrote:

On Mon, 06 Apr 2015 14:43:52 +0200, Sönke Ludwig wrote:

Am 05.04.2015 um 18:07 schrieb Dato:

(...)

Thanks, nice to now :). Also before we start diving deep in vibe, is there any pitfall, drawbacks we should be aware of? Is vibe.d stable enough to run a web server 24/7? is there any production system built on it? how does it do within heavy loaded environment?

There are several production systems running with vibe.d, so this is
certainly doable. However, one thing to be aware of is that it currently
still lacks certain kinds of DoS countermeasures, so if you want to use
it for a large-scale public service, there should be some kind of
proxy/load balancer system that provides those.

It scales well for high loads, but you have to be careful with using a
lot of garbage collected memory, because the garbage collector will stop
the whole process during collection runs. Generally it's a good idea to
use some form of reference counting for dynamic memory allocations (e.g.
vibe.utils.memory.FreeListRef or std.typecons.RefCounted).

I see. I guess those classes are similar to smart pointers. Just to make sure, is D's GC so bad? I can use RefCounted and FreeListRef, but what about vibe's internal implementation? We are going to have several thousand small requests per second and GC should also be an issue for vibe's internal allocations/calls no? Currently, our C# implementation handles ~7k requests per second without caching on a single server. According to our (dirty work) benchmarks, the system should perform at least 35% better when ported to GDC, but we didn't check GC to be honest...

Thanks again :)

The issues with the GC start to become apparent when you get close to the maximum number of requests that the machine can handle. It's the pause times that occur during the collection that can result in delayed requests or dropped connections. This is something that basically happens for most GC implementations more or less and D's GC isn't that bad per se, but others, such as the Java one are still a lot better.

Vibe.d has a mode where the basic request processing doesn't perform any GC allocations (enabled by putting "versions": ["VibeManualMemoryManagement"] in the project's dub.json), so it's possible to create applications that don't use the GC during request processing at all. However, a number of DB operations do still perform GC allocations currently.

But I guess you'll just have to make a simple benchmark to see if the GC is really an issue at all. It depends a lot on the application, but a simple application that doesn't have a database as the bottleneck can easily handle more than 10k requests per second on mediocre hardware, even with the GC enabled.

In an old benchmark (simple HTTP reply with no dynamic computations) I got about 70k requests per second on an AMD Phenom2 quad core with manual memory management and using multi-core processing. I'd have to rerun that benchmark now to be sure that those numbers still hold, but that's about the maximum that you can currently expect for that kind of CPU.

Re: Support of HTTPS?

On Mon, 06 Apr 2015 19:00:02 GMT, Sönke Ludwig wrote:

On Mon, 06 Apr 2015 18:06:59 GMT, Dato wrote:

On Mon, 06 Apr 2015 14:43:52 +0200, Sönke Ludwig wrote:

Am 05.04.2015 um 18:07 schrieb Dato:

(...)

Thanks, nice to now :). Also before we start diving deep in vibe, is there any pitfall, drawbacks we should be aware of? Is vibe.d stable enough to run a web server 24/7? is there any production system built on it? how does it do within heavy loaded environment?

There are several production systems running with vibe.d, so this is
certainly doable. However, one thing to be aware of is that it currently
still lacks certain kinds of DoS countermeasures, so if you want to use
it for a large-scale public service, there should be some kind of
proxy/load balancer system that provides those.

It scales well for high loads, but you have to be careful with using a
lot of garbage collected memory, because the garbage collector will stop
the whole process during collection runs. Generally it's a good idea to
use some form of reference counting for dynamic memory allocations (e.g.
vibe.utils.memory.FreeListRef or std.typecons.RefCounted).

I see. I guess those classes are similar to smart pointers. Just to make sure, is D's GC so bad? I can use RefCounted and FreeListRef, but what about vibe's internal implementation? We are going to have several thousand small requests per second and GC should also be an issue for vibe's internal allocations/calls no? Currently, our C# implementation handles ~7k requests per second without caching on a single server. According to our (dirty work) benchmarks, the system should perform at least 35% better when ported to GDC, but we didn't check GC to be honest...

Thanks again :)

The issues with the GC start to become apparent when you get close to the maximum number of requests that the machine can handle. It's the pause times that occur during the collection that can result in delayed requests or dropped connections. This is something that basically happens for most GC implementations more or less and D's GC isn't that bad per se, but others, such as the Java one are still a lot better.

Vibe.d has a mode where the basic request processing doesn't perform any GC allocations (enabled by putting "versions": ["VibeManualMemoryManagement"] in the project's dub.json), so it's possible to create applications that don't use the GC during request processing at all. However, a number of DB operations do still perform GC allocations currently.

But I guess you'll just have to make a simple benchmark to see if the GC is really an issue at all. It depends a lot on the application, but a simple application that doesn't have a database as the bottleneck can easily handle more than 10k requests per second on mediocre hardware, even with the GC enabled.

In an old benchmark (simple HTTP reply with no dynamic computations) I got about 70k requests per second on an AMD Phenom2 quad core with manual memory management and using multi-core processing. I'd have to rerun that benchmark now to be sure that those numbers still hold, but that's about the maximum that you can currently expect for that kind of CPU.

Wonderful. Thank you for your informative reply. I guess, we will port one game to D and see how it works.

Thanks again :)

Re: Support of HTTPS?

On Mon, 06 Apr 2015 18:06:59 GMT, Dato wrote:

I see. I guess those classes are similar to smart pointers. Just to make sure, is D's GC so bad? I can use RefCounted and FreeListRef, but what about vibe's internal implementation? We are going to have several thousand small requests per second and GC should also be an issue for vibe's internal allocations/calls no? Currently, our C# implementation handles ~7k requests per second without caching on a single server. According to our (dirty work) benchmarks, the system should perform at least 35% better when ported to GDC, but we didn't check GC to be honest...

Thanks again :)

A small note on this topic: D is a language that evolve pretty fast. It doesn't mean breaking changes (those are quite rare nowadays), but a new version usually brings a lot of improvements / bugfixes, both for the compiler and the standard library.

GDC and LDC use DMD's frontend code (what 'parses' the language), but they have to port it every time a new version is released.
At the moment, latest LDC use the frontend of DMD 2.066.1, and GDC use the frontend of 2.065.0.

The problem is, the library evolve as fast as the frontend, and depends on it. So with a compiler that supports 2.065, you'll get the library as it was with 2.065.

The two mains issue I can see for performance critical systems is:

  • lack of the @nogc attribute (introduced in 2.066.0: http://dlang.org/changelog.html#nogc-attribute), because it also led to a lot of allocations being removed from Phobos.
  • A very annoying and longstanding bug where struct destructor allocated on the GC heap didn't had their destructor called was fixed in 2.067 ( http://dlang.org/changelog.html#heap-struct-destructors ).

So I'll suggest you to keep an eye on LDC, which produce code superior to DMD, but isn't as far behind in term of frontend as GDC is ATM (there's also the possibility that GDC is ported to 2.066.x / 2.067.x, but I'm not aware of any plan ATM).