RejectedSoftware Forums

On Sun, 16 Apr 2017 18:17:21 GMT, Sönke Ludwig wrote:

There seems to be a 16MB limit on the size of a BSONobj passed mongodb. For now I'd like to increase this limit and carry on. Is there a simple way to achieve this?

I'm pretty sure that this is unfortunately a fixed limit. See also https://docs.mongodb.com/manual/reference/limits/#bson-documents

Thanks for pointing that out the unfortunate source of that unexpected limitation. OK, so my document contains a dozens of very long strings (XML), so I decided to compress them. To that end I defined a compressedString type that automatically converts to and from string, and used that type instead of string in the struct used by the mongodb insert. Here's the essential idea.

struct compressedString
{
    import std.zlib;
    ubyte[] data;
    this( string s)
    {
        this = s;
    }
    string cnvrt() @property
    {
        if( data.length == 0) return "";
        return cast(string)uncompress(data);
    }
    void cnvrt(string value) @property
    {
        data = compress(value);
    }
    alias cnvrt this;
}

However, apparently vibe.d serialization to BSON engaged by the insert query detects the ability to convert struct fields of type compressedString to/from string, and uses that conversion to string as part of serialization, thus undoing the compression.

Easy enough to work around, by making conversion to string manual. But does vibe.d provide a way for this to be done with both conversions present?