RejectedSoftware Forums

On Fri, 14 Oct 2016 11:49:21 GMT, Grigorii Smorkalov wrote:

I have a lazy input range that produce alot of data (e.g. 1GB). I want to send this data via http without creating temporary array. The most intuitive way would be overloaded method "writeBody" that takes any range of bytes or chars. But there is no such overload. I tried to use StreamOutputRange but it looks wierd:

(scope req) {
	auto writer = StreamOutputRange(req.bodyWriter);
	foreach (c; range) {
		writer.put(c);
	}
}

I am sure that it is not effective and requires internal buffer for whole data. Is it possible to move this foreach iteration into vibe.d? Something like

(scope req) {
	auto reader = InputRangeStream(range); // must be template to inline ranges iteration
	req.writeBody(reader);
},

In this case vibe.d may write data directly to output buffer after this delegate completed. No additional memory required (only small buffer for one tcp frame that already exists). I am not sure about Content-Length. In case of RandomAccessRange everything is ok, in case of InputRange the only way is chunked encoding.

Same improvement would be perfect for raw TCP.

The StreamOutputRange approach has the drawback of having to copy the data once, but other than that is efficient (it copies and writes the data to output stream in chunks). However, due to the nature of ranges, this copy cannot be avoided, except for the corner case of plain arrays. The reverse approach with InputRangeStream would have to do exactly the same, just with a different API.

If you know the length of the stream, you can set the Content-Length header before writing to bodyWriter, otherwise it will default to chunked transfers.

Generally, I think it makes sense to offer a writeBody overload that encapsulates this logic. An OutputStream.write overload taking an input range could also make sense.

I've opened a ticket #1594