RejectedSoftware Forums

On Thu, 27 Apr 2017 12:49:02 GMT, Ľudovít Lučenič wrote:

My vibe.d-based web server application serves currently as an augmented proxy (or a mediator) between a set of application logic servers (listening on TCP sockets) and users' browsers (receiving data via websockets).

In particular, the use case goes as follows:

• after a user logs in in a browser,
• a TCP connection is established from vibe.d server to one of the end servers,
• that server verifies the credentials and
• the user is supplied an HTML page with session-based web socket establishment code,
• in turn, the browser creates that websocket connection to my vibe.d server,
• the websocket handling function notifies the other task, so that it can start resending data from the TCP connection,
• this websocket handling function then resends data to the browser,
• finally, the user sees online updated data (sent from the end server) in the browser.

As I need to manage the connection status of both tied connections (ws, tcp) in various places while processing data from either end, it would be beneficial to have these (both) connections served within a single thread by different fibers, so they can shared TLS data. As I do not see a way how to make vibe.d open a websocket connection in a specific thread (the one of the already established TCP connection to an end server), I would go with a passing of the websocket over to the end server connection handling task and setting up the handling of the websocket with runTask() from there to stay in the same thread.

What is a proper way how to pass a websocket connection to another task in a different thread?

Thank you very much in advance for hints of any kind.
Ľudovít

TCP connections generally can't be safely moved between threads. It might work for some of the back ends by accident, but it can't be relied upon, since they are embedded in a thread-specific data structure.

Since the web socket connection is an incoming one, the only way to control the accepting thread would be to open up one port for each thread. So when a page gets served from the first thread, it might generate code to connect to ws://server:81, while the second one connects to ws://server:82.

Generally, if possible, I'd always favor a multi-process approach instead of a multi-threaded one. So instead of using the distribute option, HTTPServerSettings.reusePort would be used, and then n processes could all listen on the same port. However, that would require to make the application completely state-less and to move any session related data to a separate database/cache, so that may be a performance issue.

If the multiple-port suggestion isn't feasible for some reason, it would be possible to build in a way to move incoming connections between threads before the internal management structures have been built (using an additional pre-connection callback). That would require a bit of work, but could be added to vibe-core/eventcore.

I do not really see the point in the mentioned multi-process approach. How would this ensure a websocket is bound to the particular TCP handling task's thread. Or have you just meant it to be an add-on comment on the port-per-thread solution? I need to ask as wll, why do you favor multi-process over multi-threaded? Having multiple processes sharing the same port seems to me equivalent to having multiple threads within a single process listening on the same port.

Yeah, this was more of a general remark, because it usually makes sense to design with that setup in mind. Some advantages:

• No global GC lock that can otherwise easily destroy the multi-threading advantage
• Easy to scale across multiple machines
• More robust against fatal application errors (n-1 out of n processes will continue to run)
• Avoids error prone thread synchronization and shared memory state, which is otherwise often tempting to do

I am definitively thankful for the last hint, Sönke. Could you please guide me a bit in the direction where should I hook that callback in? I am prepared to contribute the solution in a PR with a little help of yours then :-)

I think I missed that the decision to which thread an incoming WS request belongs needs to be made based on the session ID. The callback based approach would basically just work for a very early decision, for example based on the remote IP address. Everything else would already have created too many thread-specific data dependencies.

Since all other thread transfer solutions would really be very involved (especially if thread safety is supposed to be guaranteed), would it maybe be possible to approach this from the other side and establish the web socket connection first and then the TCP connection afterwards? That would probably mean that the credentials would also need to go over the web socket and that the TCP connection can't outlive the web socket, but maybe that's acceptable?

Spontaneously, I'm a bit out of ideas for a true single-thread solution. As another shot in the dark, would it make sense to try to keep the network code to a single thread, and to just offload computationally expensive tasks to worker threads?

As a final thought, if there is no practical solution on the application side, enabling moving of just the TCP connection between threads would be a lot easier than for web socket connections:

• Implementing the reverse of EventDriverSockets.adoptStream in eventcore
  • makes sure that there is only a single reference to the stream socket left
  • clears all data from the slot associated with the socket and removes it from the event loop
  • returns the socket FD without closing it
• Implement a special snapshot type within vibe-core
  • captures the socket FD, as well as the current state of a TCP connection (contents of TCPConnection.m_context)
  • is typed immutable or shared, so that it can be passed between threads
• A new API function would then copy the state and use the eventcore function above to create such a snapshot
• Another API function would use adoptStream and would reapply the state to create a new TCPConnection

Just naming these things well will be interesting...