RejectedSoftware Forums

On 2013-11-28 19:21:31 +0000, Etienne Cimon said:

Does libevent take care of the byte order in tcp requests?

No. Use network byte order. What are you trying to do?

On Thu, 28 Nov 2013 21:44:15 -0500, Etienne wrote:

On 2013-11-28 20:55, Shammah Chancellor wrote:

On 2013-11-28 19:21:31 +0000, Etienne Cimon said:

Does libevent take care of the byte order in tcp requests?

No. Use network byte order. What are you trying to do?

I'm using postgresql in my project and I chose to rewrite the ddb
postgresql native connector to use vibe.d's TCPConnection and be able to
yield. I need it to run LISTEN/NOTIFY commands on active websockets and
I also thought of a long term project to create a MemoryStore /
SessionStore around it because it has some good caching abilities too.

So I'm thinking of building on this:
https://github.com/pszturmaj/ddb/

I fixed most of the bugs but it uses a stream.read(ref out) function to
consume data. I might have to replace it's general use of
std.socketstream read/write functions with consume!T() and sendCmd(),
and rethink the structure around that. I don't know how similar MySQL is
to the frontend/backend api of pgsql. It sucks that I don't have a
similar read function to work with but it's good to know the hton() are
there and used in vibe's tcp library.

Does this look interesting to you?

Yes, htonX can be used without issues (std.bitmanip.bitEndianToNative might be more efficient, but that's a different topic). So looking at the code, I would go about implementing this by replacing class PGStream : SocketStream { with

class PGStream {
    private {
        version (Have_vibe_d) TCPConnection stream;
        else SocketStream stream;
    }

and then for each type instead of

override void write(int x)
{
    super.write(hton(x));
}

do

void write(int x)
{
    stream.write(nativeToBigEndian(x));
}

that should work for both types of stream, because nativeToBigEndian returns a byte array, and have the same effect. Then there is only PGConnection.getMessage which reads the message length, where int len; stream.read(len); len = ntoh(len) - 4; needs to be replaced by ubyte[4] lenbytes; stream.stream.read(lenbytes[]); auto len = bigEndianToNative!int(lenbytes) - 4;.

I think that should be about all.

On 2013-11-29 10:36 AM, Etienne Cimon wrote:

On 2013-11-29 2:20 AM, Sönke Ludwig wrote:

version (Havevibed) TCPConnection stream;

That's amazing, thanks ! Also, I couldn't find stream.stream.read so I
assumed I'd have to read as ubyte[] all the way. Should be very simple
there then!

On another subject, I was looking into ConnectionPool and it seems like
(for any reason) an unresponsive database (with maybe a bad timeout or
trapped in locking) would let connections accumulate until size_t.max
(one for every visitor). Wouldn't a 2^63 of these objects clog the RAM
and make the application crash? It would probably be a good idea to
limit this in an options associative array as to keep it at a consistent
value with the database's max connections. Something of the sorts:

struct PoolOptions {

 size_t m_maxConnections;

} m_poolOptions;

:32 this(Connection delegate() connection_factory, PoolOptions options =
new PoolOptions(100))

:39 sizet cidx = mpoolOptions.m_maxConnections;

:44 if( cidx <= mpoolOptions.mmaxConnections ){

I would image it to be wise to put lockConnection in a yielded loop
where a connection is eventually returned. What do you think?

Here's the Postgresql connector for vibe.d in my github fork case anyone
is interested, it's being tested to be pulled later into the main project.

https://github.com/etcimon/ddb

It's tested on windows dmd x86_64 with postgresql 9.3.1. but I didn't
add ConnectionPool support yet

e.g.

auto conn = new PGConnection;
conn.open([
	"host" : "localhost",
	"database" : "postgres",
	"user" : "postgres",
	"password" : ""
]);

scope(exit) conn.close;

auto cmd = new PGCommand(conn, "SELECT typname, typlen FROM pg_type");
auto result = cmd.executeQuery;

try
{
	foreach (row; result)
	{
		writeln(row["typname"], ", ", row[1]);
	}
}
finally
{
	result.close;
}

On Thu, 28 Nov 2013 21:44:15 -0500, Etienne wrote:

I'm using postgresql in my project and I chose to rewrite the ddb
postgresql native connector to use vibe.d's TCPConnection and be able to
yield. I need it to run LISTEN/NOTIFY commands on active websockets and
I also thought of a long term project to create a MemoryStore /
SessionStore around it because it has some good caching abilities too.

So I'm thinking of building on this:
https://github.com/pszturmaj/ddb/

I fixed most of the bugs but it uses a stream.read(ref out) function to
consume data. I might have to replace it's general use of
std.socketstream read/write functions with consume!T() and sendCmd(),
and rethink the structure around that. I don't know how similar MySQL is
to the frontend/backend api of pgsql. It sucks that I don't have a
similar read function to work with but it's good to know the hton() are
there and used in vibe's tcp library.

Does this look interesting to you?

Great!See also: https://github.com/rejectedsoftware/vibe.d/issues/282