RejectedSoftware Forums

Sign up

Pages: 1 2 3

Re: Automatic GC collection runs?

Am 11.09.2012 12:22, schrieb Eldar Insafutdinov:

On Monday, 10 September 2012 at 21:18:41 UTC, Sönke Ludwig wrote:

Am 10.09.2012 23:03, schrieb Nick Sabalausky:

On Mon, 10 Sep 2012 22:26:39 +0200
Sönke Ludwig sludwig@rejectedsoftware.com wrote:

My current plan is to default to safe GC memory, which is reasonably
fast, and to provide the option to use the faster but unsafe manual
version when speed really matters. (I just tested the GC version and
its memory consumption is stable and only slightly worse with
11MB/5,8MB)

Oh, so the memory leak is fixed in the GC version, too?

Jep, it just holds on a bit more memory than it should because of a pool
allocator that is filled with each request - but eventually the whole
pool will get collected. I'll test a bit more and switch back to the GC
version as the default tomorrow.

With the latest commit that enables GC allocator as a default one the
server crashes immediately after startup.

Should work again. The Windows GC seems to be lazy compared to the Linux
one and avoided the crash. There were some untracked GC allocations in
the libevent back end that are now back to manual.

Re: Automatic GC collection runs?

On Tuesday, 11 September 2012 at 11:30:36 UTC, Sönke Ludwig
wrote:

Am 11.09.2012 12:22, schrieb Eldar Insafutdinov:

On Monday, 10 September 2012 at 21:18:41 UTC, Sönke Ludwig
wrote:

Am 10.09.2012 23:03, schrieb Nick Sabalausky:

On Mon, 10 Sep 2012 22:26:39 +0200
Sönke Ludwig sludwig@rejectedsoftware.com wrote:

My current plan is to default to safe GC memory, which is
reasonably
fast, and to provide the option to use the faster but
unsafe manual
version when speed really matters. (I just tested the GC
version and
its memory consumption is stable and only slightly worse
with
11MB/5,8MB)

Oh, so the memory leak is fixed in the GC version, too?

Jep, it just holds on a bit more memory than it should
because of a pool
allocator that is filled with each request - but eventually
the whole
pool will get collected. I'll test a bit more and switch back
to the GC
version as the default tomorrow.

With the latest commit that enables GC allocator as a default
one the
server crashes immediately after startup.

Should work again. The Windows GC seems to be lazy compared to
the Linux
one and avoided the crash. There were some untracked GC
allocations in
the libevent back end that are now back to manual.

Thanks!

Re: Automatic GC collection runs?

I experience the same behavior with 0.7.17 (and 0.7.18) versions of vibe.d.

A 1 Gib of memory consumption for 1M requests. Calling GC.collect() doesn't help. Is this the same bug?

Re: Automatic GC collection runs?

On Tue, 19 Nov 2013 10:56:03 GMT, Sergei Nosov wrote:

I experience the same behavior with 0.7.17 (and 0.7.18) versions of vibe.d.

A 1 Gib of memory consumption for 1M requests. Calling GC.collect() doesn't help. Is this the same bug?

Is that for doing client requests or for handling server requests? There was a memory leak that has been fixed in the HTTP client recently, but the fix is in 0.7.18-beta.2 (if that is the 0.7.18 which you have tested). Just now, I've also fixed a possible fiber leak that may have been the cause for a memory leak in a HTTP server.

Using the bench-http-server and bench-http-request example projects, I get stable memory usage now. However, without "VibeManualMemoryManagement" in package.json (i.e. using pure GC allocations) there is still a moderate growth in memory consumption for the benchmark client (about 70MB after 1M requests). Not sure if this is due to false pointers or because of some actual left over references.

(using Windows for testing)

Re: Automatic GC collection runs?

On Tue, 19 Nov 2013 19:08:13 GMT, Sönke Ludwig wrote:

Using the bench-http-server and bench-http-request example projects, I get stable memory usage now. However, without "VibeManualMemoryManagement" in package.json (i.e. using pure GC allocations) there is still a moderate growth in memory consumption for the benchmark client (about 70MB after 1M requests).

After 3.5M requests, memory usage of the client is now still at 80MB, so it looks like it might be stabilizing there.

Re: Automatic GC collection runs?

On Tue, 19 Nov 2013 19:08:13 GMT, Sönke Ludwig wrote:

On Tue, 19 Nov 2013 10:56:03 GMT, Sergei Nosov wrote:

I experience the same behavior with 0.7.17 (and 0.7.18) versions of vibe.d.

A 1 Gib of memory consumption for 1M requests. Calling GC.collect() doesn't help. Is this the same bug?

Is that for doing client requests or for handling server requests? There was a memory leak that has been fixed in the HTTP client recently, but the fix is in 0.7.18-beta.2 (if that is the 0.7.18 which you have tested). Just now, I've also fixed a possible fiber leak that may have been the cause for a memory leak in a HTTP server.

Using the bench-http-server and bench-http-request example projects, I get stable memory usage now. However, without "VibeManualMemoryManagement" in package.json (i.e. using pure GC allocations) there is still a moderate growth in memory consumption for the benchmark client (about 70MB after 1M requests). Not sure if this is due to false pointers or because of some actual left over references.

(using Windows for testing)

I've been running bench-http-server and ab -n 1000000 -c 10 for testing. Using current master, there doesn't seam to be any leaks any more. The memory consumption is really low - smth about 3.5Mb for the whole time. So, I guess, that's fixed! Thx!

If you don't mind I have another question - I've added a readText call in diet file. It simply embeds another html into the page. I assumed that I would notice a memory usage growth, since it returns a string, but that didn't happen. Do you think it's due to the GC collects previously allocated string on each call, or smth else?

Re: Automatic GC collection runs?

Am 20.11.2013 09:14, schrieb Sergei Nosov:

On Tue, 19 Nov 2013 19:08:13 GMT, Sönke Ludwig wrote:

On Tue, 19 Nov 2013 10:56:03 GMT, Sergei Nosov wrote:

I experience the same behavior with 0.7.17 (and 0.7.18) versions of vibe.d.

A 1 Gib of memory consumption for 1M requests. Calling GC.collect() doesn't help. Is this the same bug?

Is that for doing client requests or for handling server requests? There was a memory leak that has been fixed in the HTTP client recently, but the fix is in 0.7.18-beta.2 (if that is the 0.7.18 which you have tested). Just now, I've also fixed a possible fiber leak that may have been the cause for a memory leak in a HTTP server.

Using the bench-http-server and bench-http-request example projects, I get stable memory usage now. However, without "VibeManualMemoryManagement" in package.json (i.e. using pure GC allocations) there is still a moderate growth in memory consumption for the benchmark client (about 70MB after 1M requests). Not sure if this is due to false pointers or because of some actual left over references.

(using Windows for testing)

I've been running bench-http-server and ab -n 1000000 -c 10 for testing. Using current master, there doesn't seam to be any leaks any more. The memory consumption is really low - smth about 3.5Mb for the whole time. So, I guess, that's fixed! Thx!

Good to hear. I've noticed that the performance is worse than it was the
last time I did some benchmarking, but that will also improve again
eventually.

If you don't mind I have another question - I've added a readText call in diet file. It simply embeds another html into the page. I assumed that I would notice a memory usage growth, since it returns a string, but that didn't happen. Do you think it's due to the GC collects previously allocated string on each call, or smth else?

It's very likely that the existing GC memory pool is just large enough
to hold the file in memory without letting the actual process memory
usage go up. The memory will then always be reclaimed for further use by
the GC just before the memory pool would have to be enlarged (the GC
always runs a collection in this case).

The only drawback is that now there will be a GC collection run every
few requests, but that will not matter unless the application has to
handle an extremely high number of requests.

Re: Automatic GC collection runs?

On 2013-11-19 19:23:41 +0000, Sönke Ludwig said:

On Tue, 19 Nov 2013 19:08:13 GMT, Sönke Ludwig wrote:

Using the bench-http-server and bench-http-request example projects,
I get stable memory usage now. However, without
"VibeManualMemoryManagement" in package.json (i.e. using pure GC
allocations) there is still a moderate growth in memory consumption for
the benchmark client (about 70MB after 1M requests).

After 3.5M requests, memory usage of the client is now still at 80MB,
so it looks like it might be stabilizing there.

Awesome! Vibe-D is awesome and so are you Sönke!

Thanks for your hard work.

-Shammah

Pages: 1 2 3