RejectedSoftware Forums

Sign up

mongodb index support

I noticed there's no actual support for indexes in vibe's current mongo driver. Be it related error handling or 'ensureIndex'.

Since this is a feature I need, I'll either implement it or use a different mongo driver.

Is anyone working on this feature?
Is it planned for a later milestone?

Re: mongodb index support

On Wed, 28 Nov 2012 21:16:34 GMT, Ekyo wrote:

I noticed there's no actual support for indexes in vibe's current mongo driver. Be it related error handling or 'ensureIndex'.

Since this is a feature I need, I'll either implement it or use a different mongo driver.

Is anyone working on this feature?
Is it planned for a later milestone?

I definitely planned to do this, but haven't gotten around to reverse engineer an existing client to see what exactly needs to be sent.

Of course, it's possible to use the mongo shell to create an index (this is what I've done so far).

Re: mongodb index support

On Thu, 29 Nov 2012 07:45:54 GMT, Sönke Ludwig wrote:

On Wed, 28 Nov 2012 21:16:34 GMT, Ekyo wrote:

I noticed there's no actual support for indexes in vibe's current mongo driver. Be it related error handling or 'ensureIndex'.

Since this is a feature I need, I'll either implement it or use a different mongo driver.

Is anyone working on this feature?
Is it planned for a later milestone?

I definitely planned to do this, but haven't gotten around to reverse engineer an existing client to see what exactly needs to be sent.

Of course, it's possible to use the mongo shell to create an index (this is what I've done so far).

Indexes are a collection: system.indexes

Indexes are inserted as any other data in any other collection, for example in mongo:

db.system.indexes.insert({ "v" : 1, "key" : { "Username" : 1 }, "unique" : true, "ns" : "users.account", "name" : "Username_1" })

would add a unique index on the field "Username" for the "users.account" collection where "users" is the db name and "account" the collection

The index flags, after a look in other drivers, seems to be:

Unique = 1<<0,
DropDuplicates = 1<<2,
Background = 1<<3,
Sparse = 1<<4

Note that these flags are part of the bson to be inserted, it is a regular insert.

With vibe's current implementation, the db and collection are both in the same variable: m_collection in collection.d:52 (MongoCollection)

so we'd need to extract the db name (I suggest in the constructor)

_dbname = m_collection[0 .. indexOf('.')];

and insert a bson representing the index

...
MongoConnection.insert(_dbname ~ "system.indexes", flags, doc);


ensureIndex only queries the collection in the first place before trying to insert the index in it to avoid useless insertions.

For errors related to indexes I'll need to look further but my guess is that the errors would be in system.indexes and we only currently check for errors in the current collection.

Re: mongodb index support

Am 30.11.2012 01:15, schrieb Ekyo:

(...)
ensureIndex only queries the collection in the first place before trying to insert the index in it
to avoid useless insertions.

Thanks for the info, I've committed a simple implementation to master. It seems like the DB already
checks if the index exists and does not reindex, so an additional query might not be necessary... or
do you see a concrete reason for this?

Btw. is there some kind of authoritative source for this kind of information? I just found the low
level protocol information and high level client information, as well as some of those internal
things scattered around. But most of the stuff either had to be reverse engineered by looking at the
log output of the server or by looking at other client's source code.

For errors related to indexes I'll need to look further but my guess is that the errors would be in
system.indexes and we only currently check for errors in the current collection.

The error checking code was generally broken. I've fixed it and indexing errors are now reported as
an exception.

Re: mongodb index support

On Fri, 30 Nov 2012 12:15:57 +0100, Sönke Ludwig wrote:

Am 30.11.2012 01:15, schrieb Ekyo:

(...)
ensureIndex only queries the collection in the first place before trying to insert the index in it
to avoid useless insertions.

Thanks for the info, I've committed a simple implementation to master. It seems like the DB already
checks if the index exists and does not reindex, so an additional query might not be necessary... or
do you see a concrete reason for this?

I don't know, in all drivers I checked there is an 'ensureIndex' function that check first and add only if it doesn't exist.

Btw. is there some kind of authoritative source for this kind of information? I just found the low
level protocol information and high level client information, as well as some of those internal
things scattered around. But most of the stuff either had to be reverse engineered by looking at the
log output of the server or by looking at other client's source code.

yes there is: MongoDB - Writing Drivers
but I mostly looked in other drivers (js, C, python and Haskell).

Re: mongodb index support

On Fri, 30 Nov 2012 14:15:35 GMT, Ekyo wrote:

Btw. is there some kind of authoritative source for this kind of information? I just found the low
level protocol information and high level client information, as well as some of those internal
things scattered around. But most of the stuff either had to be reverse engineered by looking at the
log output of the server or by looking at other client's source code.

yes there is: MongoDB - Writing Drivers
but I mostly looked in other drivers (js, C, python and Haskell).

Okay, that's the stuff I have looked at. But it seems to be very sparse in some areas. But well, I guess looking at other implementations will be good enough.