RejectedSoftware Forums

Sign up

Pages: 1 2

Re: listenTCP with threads

On Sat, 23 Nov 2013 22:39:18 GMT, Atila Neves wrote:

Can you prepare a little (or large) test case, so that I can debug
locally? I think exit code -11 indicates a segmentation fault.

https://github.com/atilaneves/mqtt

Add "enableWorkerThreads()" and TCPListenOptions.distribute to mqttd/app.d, "dub" to run it then "telnet localhost 1883" with any input reproduces the problem. Comment out "enableWorkerThreads" and the app complains about an invalid message instead (since telnet isn't sending anything meaningful) and carries on. Put it back in and the complaint is still there but it exits soon thereafter.

Atila

Okay, the issue is that in your main code (app.d) gServer needs to be shared or __gshared for its value to be visible in threads other than the main thread. It is a null-dereference in line 26 that is responsible for the crash.

BTW I've just noticed that in package.json, there is "importPaths": ["mqttd"], but to match the module declarations in the actual D files, it needs to be "importPaths": ["."] (which then also makes the test/ files visible, which in turn is the reason why I'd always recommend a separate source folder for D projects). Otherwise there will be compile errors when it's used as a library and compiled separately (e.g. when "dub generate visuald" is used).

Re: listenTCP with threads

Okay, the issue is that in your main code (app.d) gServer needs to be shared or __gshared for its value to be visible in threads other than the main thread. It is a null-dereference in line 26 that is responsible for the crash.

Oops. I feel silly now. Thanks!

BTW I've just noticed that in package.json, there is "importPaths": ["mqttd"], but to match the module declarations in the actual D files, it needs to be "importPaths": ["."] (which then also makes the test/ files visible, which in turn is the reason why I'd always recommend a separate source folder for D projects). Otherwise there will be compile errors when it's used as a library and compiled separately (e.g. when "dub generate visuald" is used).

I don't want the tests directory to be visible actually. I run them separately with my tool dtest from unit_threaded. All I want to go into the lib is in mqttd.

Re: listenTCP with threads

I fixed the server not being shared, but now the app dumps core when it exits, even if nobody connects.

I got two connections to work though. Then it still crashed on exit.

Re: listenTCP with threads

Am 25.11.2013 08:07, schrieb Atila Neves:

Okay, the issue is that in your main code (app.d) gServer needs to be shared or __gshared for its value to be visible in threads other than the main thread. It is a null-dereference in line 26 that is responsible for the crash.

Oops. I feel silly now. Thanks!

BTW I've just noticed that in package.json, there is "importPaths": ["mqttd"], but to match the module declarations in the actual D files, it needs to be "importPaths": ["."] (which then also makes the test/ files visible, which in turn is the reason why I'd always recommend a separate source folder for D projects). Otherwise there will be compile errors when it's used as a library and compiled separately (e.g. when "dub generate visuald" is used).

I don't want the tests directory to be visible actually. I run them separately with my tool dtest from unit_threaded. All I want to go into the lib is in mqttd.

They won't go into the lib (i.e. they won't get compiled), but they will
be importable using import tests.xxx; - nothing really that can be
done about it without either breaking separate builds or putting the
library sources into their own root directory.

Re: listenTCP with threads

Am 25.11.2013 08:24, schrieb Atila Neves:

I fixed the server not being shared, but now the app dumps core when it exits, even if nobody connects.

I got two connections to work though. Then it still crashed on exit.

Which OS/compiler do you use? It works OK for me on Windows/DMD2.064,
32-bit.

Re: listenTCP with threads

On Mon, 25 Nov 2013 09:42:13 +0100, Sönke Ludwig wrote:

Am 25.11.2013 08:24, schrieb Atila Neves:

I fixed the server not being shared, but now the app dumps core when it exits, even if nobody connects.

I got two connections to work though. Then it still crashed on exit.

Which OS/compiler do you use? It works OK for me on Windows/DMD2.064,
32-bit.

dmd 2.064.2, Arch Linux 64-bit. I got the program to work like it used to using threads, but it also crashes in the middle of execution sometimes. Always in the same place:

Program terminated with signal 11, Segmentation fault.
#0 0x0000000000626c14 in vibe.utils.hashmap.__T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.resize() (this=0x7fdef0e47ba8, k=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:205

205 mhasher = k => k.toHash();
(gdb) bt
#0 0x0000000000626c14 in vibe.utils.hashmap.
_T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.resize() (this=0x7fdef0e47ba8, k=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:205

#1 0x00000000006266c2 in vibe.utils.hashmap.__T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.findIndex() (this=0x7fdef0e47ba8, key=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:165

#2 0x00000000006ba34a in vibe.utils.hashmap.T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.T13opBinaryRightVAyaa2_696eZ.opBinaryRight() (this=0x7fdef0e47ba8, key=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:121

#3 0x0000000000622ee5 in vibe.core.drivers.libevent2.Libevent2ManualEvent.amOwner() (this=0x7fdef0e47b80) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:511
#4 0x0000000000622785 in vibe.core.drivers.libevent2.Libevent2ManualEvent.wait() (this=0x7fdef0e47b80, referenceemitcount=3) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:449
#5 0x0000000000622710 in vibe.core.drivers.libevent2.Libevent2ManualEvent.wait() (this=0x7fdef0e47b80) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:444
#6 0x0000000000616fa2 in vibe.core.core.exitEventLoop() (shutdownworkers=true) at ../../../.dub/packages/vibe-d-master/source/vibe/core/core.d:113
#7 0x0000000000619149 in onSignal (signal=2) at ../../../.dub/packages/vibe-d-master/source/vibe/core/core.d:101
#8
#9 0x00007fdeef8a9a73 in epoll
wait () from /usr/lib/libc.so.6
#10 0x00007fdef0931793 in ?? () from /usr/lib/libevent-2.0.so.5
#11 0x00007fdef091dae7 in eventbaseloop () from /usr/lib/libevent-2.0.so.5
#12 0x000000000061fbdb in vibe.core.drivers.libevent2.Libevent2Driver.runEventLoop() (this=0x7fdef0e47e00) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:141
#13 0x0000000000616dba in vibe.core.core.runEventLoop() () at ../../../.dub/packages/vibe-d-master/source/vibe/core/driver.d:26
#14 0x00000000006c167e in D main () at ../../../.dub/packages/vibe-d-master/source/vibe/appmain.d:54
#15 0x000000000074f3a8 in rt.dmain2.drunmain() ()
#16 0x000000000074f302 in rt.dmain2.
drunmain() ()
#17 0x000000000074f368 in rt.dmain2.drunmain() ()
#18 0x000000000074f302 in rt.dmain2.
drunmain() ()
#19 0x000000000074f283 in drun_main ()
#20 0x00000000006c16cd in main ()

Re: listenTCP with threads

On Mon, 25 Nov 2013 09:50:57 GMT, Atila Neves wrote:

On Mon, 25 Nov 2013 09:42:13 +0100, Sönke Ludwig wrote:

Am 25.11.2013 08:24, schrieb Atila Neves:

I fixed the server not being shared, but now the app dumps core when it exits, even if nobody connects.

I got two connections to work though. Then it still crashed on exit.

Which OS/compiler do you use? It works OK for me on Windows/DMD2.064,
32-bit.

dmd 2.064.2, Arch Linux 64-bit. I got the program to work like it used to using threads, but it also crashes in the middle of execution sometimes. Always in the same place:

Program terminated with signal 11, Segmentation fault.
#0 0x0000000000626c14 in vibe.utils.hashmap.__T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.resize() (this=0x7fdef0e47ba8, k=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:205

205 mhasher = k => k.toHash();
(gdb) bt
#0 0x0000000000626c14 in vibe.utils.hashmap.
_T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.resize() (this=0x7fdef0e47ba8, k=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:205

#1 0x00000000006266c2 in vibe.utils.hashmap.__T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.findIndex() (this=0x7fdef0e47ba8, key=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:165

#2 0x00000000006ba34a in vibe.utils.hashmap.T7HashMapTC4core6thread6ThreadTS4vibe4core7drivers9libevent210ThreadSlotZ.HashMap.T13opBinaryRightVAyaa2_696eZ.opBinaryRight() (this=0x7fdef0e47ba8, key=0x0)

at ../../../.dub/packages/vibe-d-master/source/vibe/utils/hashmap.d:121

#3 0x0000000000622ee5 in vibe.core.drivers.libevent2.Libevent2ManualEvent.amOwner() (this=0x7fdef0e47b80) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:511
#4 0x0000000000622785 in vibe.core.drivers.libevent2.Libevent2ManualEvent.wait() (this=0x7fdef0e47b80, referenceemitcount=3) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:449
#5 0x0000000000622710 in vibe.core.drivers.libevent2.Libevent2ManualEvent.wait() (this=0x7fdef0e47b80) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:444
#6 0x0000000000616fa2 in vibe.core.core.exitEventLoop() (shutdownworkers=true) at ../../../.dub/packages/vibe-d-master/source/vibe/core/core.d:113
#7 0x0000000000619149 in onSignal (signal=2) at ../../../.dub/packages/vibe-d-master/source/vibe/core/core.d:101
#8
#9 0x00007fdeef8a9a73 in epoll
wait () from /usr/lib/libc.so.6
#10 0x00007fdef0931793 in ?? () from /usr/lib/libevent-2.0.so.5
#11 0x00007fdef091dae7 in eventbaseloop () from /usr/lib/libevent-2.0.so.5
#12 0x000000000061fbdb in vibe.core.drivers.libevent2.Libevent2Driver.runEventLoop() (this=0x7fdef0e47e00) at ../../../.dub/packages/vibe-d-master/source/vibe/core/drivers/libevent2.d:141
#13 0x0000000000616dba in vibe.core.core.runEventLoop() () at ../../../.dub/packages/vibe-d-master/source/vibe/core/driver.d:26
#14 0x00000000006c167e in D main () at ../../../.dub/packages/vibe-d-master/source/vibe/appmain.d:54
#15 0x000000000074f3a8 in rt.dmain2.drunmain() ()
#16 0x000000000074f302 in rt.dmain2.
drunmain() ()
#17 0x000000000074f368 in rt.dmain2.drunmain() ()
#18 0x000000000074f302 in rt.dmain2.
drunmain() ()
#19 0x000000000074f283 in drun_main ()
#20 0x00000000006c16cd in main ()

The call stack looks a bit strange (HashMap.findIndex doesn't call resize), but I've at least found a place that could explain the crash on shutdown. This is fixed in 1facb41. I'll test on Linux/64 later.

Pages: 1 2