OK, the main problem is, as you said, with yielding and the resulting possible fragmentation of the data stream. I see how it would be possible to add some kind of queue or a temporary lock that yields until all other fibers have finished writing, but there is still a large probability (for most applications) that high level code introduces race-conditions above the packet (or
send()) granularity level. The other reason is that multi-threading is actually supported and the
TCPConnection is typically only usable with one thread at a time (because the events only arrive in one thread).
Right, I can certainly understand this. Like I said, I had basically just ensured that packet sends were "atomic" in my system, and thus whether a broadcast is done by one fiber or another was not really that relevant... it could just loop over all active connections and send the data.
Yes it's possible that things could happen somewhat "out of order", but in this case there's not a really well-defined order anyways since most/all of the broadcast-level events are triggered by clients themselves, so they already "race" to send data, etc.
But a simpler alternative to a message queue is to use a
vibe.core.mutex.Mutex, which interacts gracefully with the event loop and then lock and aquire the connection only during a send:
So a vibe Mutex will yield the fiber when blocked? That's indeed an interesting solution.
Btw. do you need to guarantee the same order of packets for all clients? If so, I think a message queue or a kind of mutex is needed anyway. Of course, it's possible to handle out-of-order packets on the receiver side, just wondering.
Yeah as I mentioned I don't really have this constraint, but I'm starting to warm up to the notion of just storing a short packet queue for each client. If I use a "Signal" like in the broadcast example, I'm assuming I lose a lot of the niceness of blocking "deep" within TcpConnection routines (send/receive) though, right? i.e. because I need to be able to pop all the way back up to the top level and handle something unrelated (i.e. broadcast), and then find out how to go back down to where I was? Seems to negate a lot of the Fibering model if I'm forced to do it that way though. I think I'd rather it just work that if the fiber is blocked on "something else" (send/receive say) that it stays blocked, and only when it finishes all that stuff will it get back to the message queue.
I guess I'm not totally clear how Signal's interact with other events, including ones internal to Vibe. Is it basically a "wait on multiple objects" sort of scenario? If so, will the Vibe send/receive routines wake up, check that they still are blocked and yield again if the event being emitted is not relevant to what the Fiber is currently doing?
I might try the Mutex thing for now... seems like it should be a fairly good solution for me. A poorly-behaved TcpConnection could then of course block a lot of stuff though, so perhaps I need to rethink the design slightly.
I'm afraid I was too lazy to return some kind of handle from
listenTcp that would make this possible ;) Will do as soon as possible.
No issues, I can live without it in the short term as well :) Was just curious if I had missed some bit of API.
Thanks for all the information! Getting pretty close to switching the mainline over to the vibe version :)