RejectedSoftware Forums

Sign up

Log connection count in connectionpool

Hi I created an implementation of vibe's connectionpool with my postgtres lib. Now I'd like to log the output to show when a connection is opened, locked and when it's unlocked (which should be at the end of a route callback call.

Looking at the source, I don't see a clear way to create a callback on unlocking a connection and I'd like to make sure the connection unlocks if a http connection is cancelled or abandoned.

So, for example, I have this(just sample code, not actual D):

static this() {
PgPool pool = new PgPool(connParams);

... routes defined here
}

void getUser(req, res) {
auto db = pool.lock()

db.query(...)

req.writeBody...
}

What happens when getUser is called but the request is cancelled halfway through before the query finishes?

Is there ever a time when the lock will never be released?

I'd like to create log output to say "pg unlock: 4 locked connections, 10 open connections in pool"

Re: Log connection count in connectionpool

On 10/11/17 1:50 PM, Clinton D Skakun wrote:

void getUser(req, res) {
auto db = pool.lock()

db.query(...)

req.writeBody...
}

What happens when getUser is called but the request is cancelled halfway through before the query finishes?

Is there ever a time when the lock will never be released?

Unlikely. The lock is handled automatically via reference counting. So
the unlock will be done as the destructor for db is called exiting the
scope. This happens even if an exception is thrown.

-Steve

Re: Log connection count in connectionpool

On Wed, 11 Oct 2017 13:57:08 -0400, Steven Schveighoffer wrote:

On 10/11/17 1:50 PM, Clinton D Skakun wrote:

void getUser(req, res) {
auto db = pool.lock()

db.query(...)

req.writeBody...
}

What happens when getUser is called but the request is cancelled halfway through before the query finishes?

Is there ever a time when the lock will never be released?

Unlikely. The lock is handled automatically via reference counting. So
the unlock will be done as the destructor for db is called exiting the
scope. This happens even if an exception is thrown.

-Steve

Thanks Steve, do you know if there's a straightforward method to do a writeln once a connection is unlocked?

Re: Log connection count in connectionpool

On 10/12/17 12:41 PM, Clinton D Skakun wrote:

On Wed, 11 Oct 2017 13:57:08 -0400, Steven Schveighoffer wrote:

On 10/11/17 1:50 PM, Clinton D Skakun wrote:

void getUser(req, res) {

auto db = pool.lock()

db.query(...)

req.writeBody...

}

What happens when getUser is called but the request is cancelled halfway through before the query finishes?

Is there ever a time when the lock will never be released?

Unlikely. The lock is handled automatically via reference counting. So
the unlock will be done as the destructor for db is called exiting the
scope. This happens even if an exception is thrown.

Thanks Steve, do you know if there's a straightforward method to do a writeln once a connection is unlocked?

Not sure what you mean, maybe there's a typo here?

-Steve

Re: Log connection count in connectionpool

Am 12.10.2017 um 18:41 schrieb Clinton D Skakun:

On Wed, 11 Oct 2017 13:57:08 -0400, Steven Schveighoffer wrote:

On 10/11/17 1:50 PM, Clinton D Skakun wrote:

void getUser(req, res) {

auto db = pool.lock()

db.query(...)

req.writeBody...

}

What happens when getUser is called but the request is cancelled halfway through before the query finishes?

Is there ever a time when the lock will never be released?

Unlikely. The lock is handled automatically via reference counting. So
the unlock will be done as the destructor for db is called exiting the
scope. This happens even if an exception is thrown.

-Steve

Thanks Steve, do you know if there's a straightforward method to do a writeln once a connection is unlocked?

If it's just for debugging, the easiest approach would be to edit
core/vibe/core/connectionpool.d of the vibe.d package downloaded to
~/.dub/packages/ and put the log message at line 147 (there is already
a commented out logTrace call there).

Apart from this debugging issue, it may also make sense to remember this
as an enhancement request, so that the connection object can get
notified when the connection is locked/unlocked, as that may enable
certain optimizations.

Re: Log connection count in connectionpool

On Fri, 13 Oct 2017 21:58:19 +0200, Sönke Ludwig wrote:

Am 12.10.2017 um 18:41 schrieb Clinton D Skakun:

On Wed, 11 Oct 2017 13:57:08 -0400, Steven Schveighoffer wrote:

On 10/11/17 1:50 PM, Clinton D Skakun wrote:

void getUser(req, res) {

auto db = pool.lock()

db.query(...)

req.writeBody...

}

What happens when getUser is called but the request is cancelled halfway through before the query finishes?

Is there ever a time when the lock will never be released?

Unlikely. The lock is handled automatically via reference counting. So
the unlock will be done as the destructor for db is called exiting the
scope. This happens even if an exception is thrown.

-Steve

Thanks Steve, do you know if there's a straightforward method to do a writeln once a connection is unlocked?

If it's just for debugging, the easiest approach would be to edit
core/vibe/core/connectionpool.d of the vibe.d package downloaded to
~/.dub/packages/ and put the log message at line 147 (there is already
a commented out logTrace call there).

Apart from this debugging issue, it may also make sense to remember this
as an enhancement request, so that the connection object can get
notified when the connection is locked/unlocked, as that may enable
certain optimizations.

Yeah, I saw that in the code. Might uncomment it to try it out.

A deeper question I have that I just thought of was, what happens to the lock when the tcp connection is cancelled? As in, someone is loading a route, then hit's refresh in the browser or Ctrl-C in Curl. I see vibe.d throws a non-fatal error saying the TCP connection was cut off. But testing my db, any long running queries continue on. Wondering if the ConnectionPool has any insight on when the tcp connection is closed. If so, I'd like to create a callback that send a cancel query cmd to the Postgres connection being locked. I already know how to do that through libpq but there's a missing link between the TCP connection cutting off and the db knowing when to cancel.

Just thinking, some large queries, if refreshed many times (or in the event of a DDOS attack) would cause a race condition on the DB because of the queries are still being returned in the background.