Am 13.11.2014 16:10, schrieb Etienne Cimon:
On Thu, 13 Nov 2014 11:15:24 GMT, David Monagle wrote:
Apologies for the essay, I'm really interested in where this is going so as to know the best place to work and contribute.
I can't say much about the direction it's taking, but there's a little bit of room for improvement using buffers, mostly for BIG json documents (> 1MB).
There's currently no way of having access to part of the schema during the transfer, which is useful to stop transfer if the metadata is wrong (e.g. mixing in a session ID with Json data that contains a base64 image)
There's possibilities for a memory attack with Json documents that exceed a certain size. Currently, the entire document piles up in the RAM.
I have a "mix" of solutions that also enable other openings ie. in the database world. This involves using Json as a schema, with the values being light integers to file offsets. The Json object would query a disk file for the string data, and throw an error if the schema has too many keys. If you add a small log writer / reader for the BigJson modifications, you get a nice little embedded database.
Other than that, anything related to the standards seems nicely implemented.
There is the range based parser, which doesn't store anything other than
the current parser node, so it is possible to skip parts of a big
document. However, string values are always stored in-memory (could be a
slice to a memory mapped buffer), so large Base-64 strings for example
could still be an issue.