RejectedSoftware Forums

Sign up

Pages: 1 2

Understanding crashes

avatar
Luís Marques
Posted Fri, 09 May 2014 18:02:59 GMT
Reply

Hello,

My site became offline. I logged in to the server and the vibe.d process was not running. I tailed the .log file and the last request was completely normal (a status monitor request). The file to where the stderr is piped was empty (I think; the run script was overriding it on startup, so I can't double check; I changed it to append now). The ulimit for the core dump was set to 0, so if it crashed no core dump was produced. I have now set the core dump limit to unlimited. I don't really know what happened... Hmm.

Luís

Re: Understanding crashes

avatar
Sönke Ludwig
Posted Fri, 09 May 2014 18:20:48 GMT in reply to Luís Marques
Reply

On Fri, 09 May 2014 18:02:59 GMT, Luís Marques wrote:

Hello,

My site became offline. I logged in to the server and the vibe.d process was not running. I tailed the .log file and the last request was completely normal (a status monitor request). The file to where the stderr is piped was empty (I think; the run script was overriding it on startup, so I can't double check; I changed it to append now). The ulimit for the core dump was set to 0, so if it crashed no core dump was produced. I have now set the core dump limit to unlimited. I don't really know what happened... Hmm.

Luís

I guess the core dump is the only way to find out (or directly running it inside GDB, of course). Another possibility might be to use something like backtrace-d (never tried it), to get a proper stack trace on Linux.

In any case, even if everything runs stable, for a single server process of this kind, I'd always set up a watchdog process that restarts the server if necessary. There is also a little tool for simple cases in the registry: forever-d

Re: Understanding crashes

avatar
Luís Marques
Posted Fri, 09 May 2014 20:00:22 GMT in reply to Sönke Ludwig
Reply

On Fri, 09 May 2014 18:20:48 GMT, Sönke Ludwig wrote:

There is also a little tool for simple cases in the registry: forever-d

There's something I don't understand about dub and forever-d. forever-d has a package.json, and the page at http://code.dlang.org/packages/forever-d says you can add forever-d as a dependency in your .json file. But forever-d is not a library (is it?). If you add it as a dependency nothing is built, so not only do you not get a lib but you also don't get the unix program built either. Even though nothing is built, dub complains that "Package forever-d contains no source files. Please add {"targetType": "none"} to it's package description to avoid building it." Furthermore, dub has no option to just say "fetch and build forever-d", as a general package manager, instead of a dependency manager -- does it? I guess it would not be too hard to extend dub to do that. That would be nicer than having to find the forever-d repo, making sure a stable version is checked-out, building it, installing it, etc.

Re: Understanding crashes

avatar
Sönke Ludwig
Posted Fri, 09 May 2014 20:26:02 GMT in reply to Luís Marques
Reply

On Fri, 09 May 2014 20:00:22 GMT, Luís Marques wrote:

On Fri, 09 May 2014 18:20:48 GMT, Sönke Ludwig wrote:

There is also a little tool for simple cases in the registry: forever-d

There's something I don't understand about dub and forever-d. forever-d has a package.json, and the page at http://code.dlang.org/packages/forever-d says you can add forever-d as a dependency in your .json file. But forever-d is not a library (is it?). If you add it as a dependency nothing is built, so not only do you not get a lib but you also don't get the unix program built either.

This is indeed sub-optimal. The registry should ideally detect if a package is an application or a library (or both) and adjust the instructions accordingly.

Even though nothing is built, dub complains that "Package forever-d contains no source files. Please add {"targetType": "none"} to it's package description to avoid building it."

DUB should most probably rather error out with something like "Package forever-d doesn't define a library configuration. Cannot build as a dependency.". I'd need to look at the package to see why this is not the case.

Furthermore, dub has no option to just say "fetch and build forever-d", as a general package manager, instead of a dependency manager -- does it? I guess it would not be too hard to extend dub to do that. That would be nicer than having to find the forever-d repo, making sure a stable version is checked-out, building it, installing it, etc.

That's a two step procedure currently: 

dub fetch forever-d
dub run forever-d     # or: dub build forever-d

Re: Understanding crashes

avatar
Luís Marques
Posted Fri, 09 May 2014 20:50:49 GMT in reply to Sönke Ludwig
Reply

On Fri, 09 May 2014 20:26:02 GMT, Sönke Ludwig wrote:

That's a two step procedure currently: 

dub fetch forever-d
dub run forever-d     # or: dub build forever-d

If you want to run forever-d /bin/ls then how do you do that with dub? dub run forever-d /bin/ls does not work.

Re: Understanding crashes

avatar
Sönke Ludwig
Posted Fri, 09 May 2014 23:07:41 +0200 in reply to Luís Marques
Reply

Am 09.05.2014 22:50, schrieb Luís Marques:

On Fri, 09 May 2014 20:26:02 GMT, Sönke Ludwig wrote:

That's a two step procedure currently: 

dub fetch forever-d
dub run forever-d     # or: dub build forever-d

If you want to run forever-d /bin/ls then how do you do that with dub? dub run forever-d /bin/ls does not work.

dub run forever-d -- /bin/ls

Re: Understanding crashes

avatar
Luís Marques
Posted Sun, 11 May 2014 17:06:59 GMT in reply to Sönke Ludwig
Reply

My site crashed again. I have a core dump now. Here's a backtrace: 

Program terminated with signal 11, Segmentation fault.
#0  __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:2744
2744	../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S: No such file or directory.
(gdb) bt
#0  __memcpy_ssse3_back () at ../sysdeps/x86_64/multiarch/memcpy-ssse3-back.S:2744
#1  0x00000000007fe72f in _d_arraycopy ()
#2  0x00000000007da9cf in vibe.utils.array.__T13AllocAppenderTAhZ.AllocAppender.put() (this=0x16f49a8, arr=...)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/utils/array.d:105
#3  0x00000000007daa81 in vibe.utils.array.__T13AllocAppenderTAhZ.AllocAppender.put() (this=0x16f49a8, arr=...)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/utils/array.d:111
#4  0x000000000079100a in vibe.http.common.ChunkedOutputStream.write() (this=0x16f4990, bytes_=...)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/common.d:392
#5  0x00000000006f1504 in vibe.core.stream.OutputStream.write() (this=0x16f49e8, bytes=...)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/core/stream.d:77
#6  0x00000000006a5f74 in vibe.templ.diet.__T21compileDietFileIndentVAyaa8_64617263682e6474Vi0S227_D5darch9darchHomeFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZv10sourceListC3ddb8postgres90__T11PGResultSetTS3std8typecons50__T5TupleTAyaVAyaa4_73686131TAyaVAyaa5_7469746c65Z5TupleZ11PGResultSetZ.compileDietFileIndent() (this=0x7f16f4dd80a0, __applyArg0=<error reading variable>)
    at darch.dt:15
#7  0x00000000006a710c in ddb.postgres.__T11PGResultSetTS3std8typecons50__T5TupleTAyaVAyaa4_73686131TAyaVAyaa5_7469746c65Z5TupleZ.PGResultSet.opApply() (this=0x7f16f4d0b800, dg=...) at ../.dub/packages/ddb-0.2.0/source/ddb/postgres.d:2221
#8  0x00000000006a5e5f in vibe.templ.diet.__T21compileDietFileIndentVAyaa8_64617263682e6474Vi0S227_D5darch9darchHomeFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZv10sourceListC3ddb8postgres90__T11PGResultSetTS3std8typecons50__T5TupleTAyaVAyaa4_73686131TAyaVAyaa5_7469746c65Z5TupleZ11PGResultSetZ.compileDietFileIndent() (this=0x7f16f4dd8170, stream__=0x16f49e8) at darch.dt:13
#9  0x00000000006a5dad in vibe.templ.diet.__T15compileDietFileVAyaa8_64617263682e6474S227_D5darch9darchHomeFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZv10sourceListC3ddb8postgres90__T11PGResultSetTS3std8typecons50__T5TupleTAyaVAyaa4_73686131TAyaVAyaa5_7469746c65Z5TupleZ11PGResultSetZ.compileDietFile() (this=0x7f16f4dd8170, stream__=0x16f49e8)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/templ/diet.d:50
#10 0x00000000006a5d8c in vibe.http.server.__T6renderVAyaa8_64617263682e6474S227_D5darch9darchHomeFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZv10sourceListC3ddb8postgres90__T11PGResultSetTS3std8typecons50__T5TupleTAyaVAyaa4_73686131TAyaVAyaa5_7469746c65Z5TupleZ11PGResultSetZ.render() (this=0x7f16f4dd8170, res=0x17847e0)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/server.d:238
#11 0x00000000006a4e8a in darch.darchHome() (res=0x17847e0, req=0x17a59c0) at source/darch.d:23
#12 0x0000000000756504 in std.functional.__T13DelegateFakerTPFC4vibe4http6server17HTTPServerRequestC4vibe4http6server18HTTPServerResponseZvZ.DelegateFaker.doIt() (this=0x6a4e18 <darch.darchHome()>, a1=0x17847e0, a0=0x17a59c0)
    at /usr/include/dmd/phobos/std/functional.d-mixin-694:705
#13 0x00000000006fa55d in vibe.http.router.URLRouter.handleRequest() (this=0x7f16f4dd83f8, values=..., ridx=3)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/router.d:161
#14 0x00000000006faba4 in vibe.http.router.__T9MatchTreeTS4vibe4http6router5RouteZ.MatchTree.match() (this=0x7f16f740f810, del=..., 
    text=...) at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/router.d:464
#15 0x00000000006fa320 in vibe.http.router.URLRouter.handleRequest() (this=0x7f16f740f800, res=0x17847e0, req=0x17a59c0)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/router.d:154
#16 0x00000000007049cc in vibe.http.server.handleRequest() (keep_alive=0x7f16f4dd8cf0, settings=0x7f16f4dd8ce8, listen_info=..., 
    tcp_connection=0x16f8dd8, http_stream=0x16f8dd8) at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/server.d:1425
#17 0x0000000000703297 in vibe.http.server.handleHTTPConnection() (listen_info=..., connection=0x16f8dd8)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/server.d:1214
#18 0x00000000006ff468 in vibe.http.server.listenHTTPPlain() (this=0x7f16f7406440, conn=0x16f8dd8)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/http/server.d:121
#19 0x00000000006dccf0 in vibe.core.drivers.libevent2_tcp.onConnect() (this=0x16cafa0)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/core/drivers/libevent2_tcp.d:495
#20 0x00000000006c9ddb in vibe.core.core.__T7runTaskZ.runTask() (fiber=0x7f16f7433200)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/core/core.d:197
#21 0x00000000006c6392 in vibe.core.core.CoreTask.run() (this=0x7f16f7433200)
    at ../.dub/packages/vibe-d-0.7.20-beta.1/source/vibe/core/core.d:879
#22 0x00000000008364d2 in core.thread.Fiber.run() ()
#23 0x00000000008363dd in fiber_entryPoint ()
#24 0x0000000000000000 in ?? ()

(gdb) info registers
rax            0x7f16f5734ffc	139736583983100
rbx            0x33	51
rcx            0x4	4
rdx            0x7f16f5ecce42	139736591945282
rsi            0x7f16f4d1dd80	139736573402496
rdi            0x7f16f5734ffc	139736583983100
rbp            0x7f16f4dd7e00	0x7f16f4dd7e00
rsp            0x7f16f4dd7d88	0x7f16f4dd7d88
r8             0x8f6080	9396352
r9             0x7f16f4d1dd80	139736573402496
r10            0x40	64
r11            0x7f16f5f08720	139736592189216
r12            0x0	0
r13            0x0	0
r14            0x0	0
r15            0x0	0
rip            0x7f16f5ecce47	0x7f16f5ecce47 <__memcpy_ssse3_back+10663>
eflags         0x10297	[ CF PF AF SF IF RF ]
cs             0xe033	57395
ss             0xe02b	57387
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0

Any idea what the problem is?

BTW, I had never though about it, but I am running a debug build in the server, since that's the default of dub build.

(PS: does anyone remember how to change the core dump file name template, so that the core dump is not overwritten for every crash?)

Re: Understanding crashes

avatar
Luís Marques
Posted Sun, 11 May 2014 17:15:14 GMT in reply to Luís Marques
Reply

On Sun, 11 May 2014 17:06:59 GMT, Luís Marques wrote: 

    at darch.dt:15

Assuming the lines of diet templates are correctly preserved in the backtrace, we have: 

13: - foreach(source; sourceList)
14:    li
15:        - auto url = "archeology/" ~ source.sha1;

The handler for that URL, which uses darch.dt is: 

void darchHome(HTTPServerRequest req, HTTPServerResponse res)
{
	auto cmd = new PGCommand(pgdb, "SELECT sha1, title from darch_tests");
	auto sourceList = cmd.executeQuery!(Tuple!(string, "sha1", string, "title"))();
    scope(exit) sourceList.close();
    res.render!("darch.dt", sourceList);
}

I don't see anything obviously wrong there. Would it help if I put some kind of assert regarding the sourceList fields?

Re: Understanding crashes

avatar
Etienne Cimon
Posted Mon, 12 May 2014 02:49:39 GMT in reply to Luís Marques
Reply

On Sun, 11 May 2014 17:15:14 GMT, Luís Marques wrote: 

``auto sourceList = cmd.executeQuery!(Tuple!(string, "sha1", string, "title"))();``

I think passing a Tuple would mean that you're trying to map the fields to a composite type according to this:

https://github.com/pszturmaj/ddb/blob/master/source/ddb/postgres.d#L64

Try using auto sourceList = cmd.executeQuery!(string, "sha1", string, "title"); according to this example:

https://github.com/pszturmaj/ddb/blob/master/source/ddb/postgres.d#L1747

Btw, I don't get errors when attempting to post without a full name nor account :/ - and posting anonymously failed without errors too?

Re: Understanding crashes

avatar
Etienne Cimon
Posted Mon, 12 May 2014 02:53:16 GMT in reply to Etienne Cimon
Reply

On Mon, 12 May 2014 02:49:39 GMT, Etienne Cimon wrote:

Btw, I don't get errors when attempting to post without a full name nor account :/ - and posting anonymously failed without errors too?

Also over the newsgroup I get:

"A News (NNTP) error occurred: Not allowed to post with a foreign email address, please use simendsjo@gmail.com" (added underscores for bots..)

Pages: 1 2