RejectedSoftware Forums

On Tue, 03 Sep 2013 19:18:42 +0200, Sönke Ludwig wrote:

Am 03.09.2013 18:19, schrieb Craig Dillabaugh:

Hello,
I have a small server set up to serve image tiles as raw binary files. The 'app' consists of a HTML canvas page which uses an XMLHttpRequest() to request tiles from the server, which serves the tiles as raw data, which is then displayed on the canvas.

The whole app (minus the canvas2.html file) can be found here:

http://dpaste.com/hold/1364821/

Initially, it was working and I was able to serve and display tiles on my Canvas. Now my application crashes whenever I attempt to access a tile. The output from DUB is as follows:

(...)

I have no idea what the cause of this error is. Any advice on how to track this down would be appreciated.

Craig

Try doing "dub build" followed by "gdb ./name-of-app", enter "r" to run
and then, after it has crashed, "bt" to get a stack trace. Hopefully
that gives enough information to find the cause.

I get the following stack trace. It seems like the problem is with passing the request data to the getTile() function.
I am not really sure how to interpret the stack trace, but it seems odd that the 'this' pointer (at #1) points to the
same address passed to req in #0, which isn't what I would expect, as I assume 'this' is the function pointer to
app.getTile().

Stack trace:
0x0000000000565e13 in app.getTile() (res=0x18, req=0x565e08 <app.getTile()>) at source/app.d:17
17 void getTile(HTTPServerRequest req, HTTPServerResponse res)
Missing separate debuginfos, use: zypper install glibc-debuginfo-2.15-22.17.1.x8664 libevent-20-5-debuginfo-2.0.10-9.1.2.x8664 libopenssl100-debuginfo-1.0.1e-2.8.1.x8664 zlib-debuginfo-1.2.7-2.1.2.x8664
(gdb) bt
#0 0x0000000000565e13 in app.getTile() (res=0x18, req=0x565e08 ) at source/app.d:17
#1 0x0000000000589500 in std.functional._T13DelegateFakerTPFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZvZ.DelegateFaker.doIt() (

this=0x565e08 <app.getTile()>, a1=0x999010, a0=0x9b93c0) at /usr/include/dmd/phobos/std/functional.d:705

#2 0x00000000005e54b9 in vibe.http.router.URLRouter.handleRequest() (this=0x7ffff7ecac00, res=0x999010, req=0x9b93c0) at ../../.dub/packages/vibe-d-master/source/vibe/http/router.d:159
#3 0x00000000005e2a7a in vibe.http.server.handleRequest() (keepalive=, settings=0x7ffff7ff8d98, listeninfo=..., peeraddress=..., httpstream=0x9985b8)

at ../../.dub/packages/vibe-d-master/source/vibe/http/server.d:1267

#4 0x00000000005e1373 in vibe.http.server.handleHTTPConnection() (listeninfo=..., connection=0x9985b8) at ../../.dub/packages/vibe-d-master/source/vibe/http/server.d:1060
#5 0x00000000005dda0c in vibe.http.server.listenHTTPPlain() (this=0x7ffff7ec84c0, conn=0x9985b8) at ../../.dub/packages/vibe-d-master/source/vibe/http/server.d:122
#6 0x0000000000619b36 in vibe.core.drivers.libevent2tcp.onConnect() (this=0x997e90) at ../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2tcp.d:444
#7 0x000000000061e46a in vibe.core.core.CoreTask.run() (this=0x7ffff7ec5d00) at ../../.dub/packages/vibe-d-master/source/vibe/core/core.d:588
#8 0x000000000068f786 in core.thread.Fiber.run() ()
#9 0x000000000068f691 in fiberentryPoint ()
#10 0x0000000000000000 in ?? ()

Craig

On Tue, 03 Sep 2013 18:40:21 GMT, Craig Dillabaugh wrote:

On Tue, 03 Sep 2013 19:18:42 +0200, Sönke Ludwig wrote:

Am 03.09.2013 18:19, schrieb Craig Dillabaugh:

Hello,
I have a small server set up to serve image tiles as raw binary files. The 'app' consists of a HTML canvas page which uses an XMLHttpRequest() to request tiles from the server, which serves the tiles as raw data, which is then displayed on the canvas.

The whole app (minus the canvas2.html file) can be found here:

http://dpaste.com/hold/1364821/

Initially, it was working and I was able to serve and display tiles on my Canvas. Now my application crashes whenever I attempt to access a tile. The output from DUB is as follows:

(...)

I have no idea what the cause of this error is. Any advice on how to track this down would be appreciated.

Craig

Try doing "dub build" followed by "gdb ./name-of-app", enter "r" to run
and then, after it has crashed, "bt" to get a stack trace. Hopefully
that gives enough information to find the cause.

I get the following stack trace. It seems like the problem is with passing the request data to the getTile() function.
I am not really sure how to interpret the stack trace, but it seems odd that the 'this' pointer (at #1) points to the
same address passed to req in #0, which isn't what I would expect, as I assume 'this' is the function pointer to
app.getTile().

Stack trace:

clips

Craig

So I have identified the line of code causing the error, but I am no less perplexed than before.

It is the line:

	ubyte[tile_size * tile_size] data;

If tile_size >= 246 (total 60516 bytes) then the program crashes with error code -11, otherwise it seems to run fine.
Since this is a static array, and D has a limit on static array sizes, I thought that might be the problem, but D allows static arrays of size up to 16Mb, and this is clearly much smaller. Also, it should have generated a compile time error.

Is it perhaps a fiber problem. Maybe fibers don't allow such large static data structures? Any ideas?

Craig

Am 04.09.2013 17:39, schrieb Craig Dillabaugh:

So I have identified the line of code causing the error, but I am no less perplexed than before.

It is the line:

	ubyte[tile_size * tile_size] data;

If tile_size >= 246 (total 60516 bytes) then the program crashes with error code -11, otherwise it seems to run fine.
Since this is a static array, and D has a limit on static array sizes, I thought that might be the problem, but D allows static arrays of size up to 16Mb, and this is clearly much smaller. Also, it should have generated a compile time error.

Is it perhaps a fiber problem. Maybe fibers don't allow such large static data structures? Any ideas?

I see, the problem is that the stack size of fibers is kept relatively
small to save on virtual address space on 32-bit systems. It can be
enlarged using vibe.core.core.setTaskStackSize.

However, I would strongly recommend to replace the section of the code with:

try {
	auto file = openFile(filename, FileMode.read);
	res.bodyWriter.write(file);
}

That will avoid having to buffer the whole file in memory and will avoid
potentially blocking the event loop for a long while (and thus blocking
any other parallel requests).

To improve it further, res.writeRawBody could be used (this will use
DMA to transfer directly from disk to network if possible), but in that
case care needs to be taken to set the right HTTP headers. Maybe it
actually makes sense to make vibe.http.fileserver.sendFile public and
use that...