On Tue, 11 Mar 2014 09:53:27 +0100, Sönke Ludwig wrote:

Am 11.03.2014 00:17, schrieb Stephan Dilly:

On Mon, 10 Mar 2014 21:21:51 GMT, Craig Dillabaugh wrote:

In an application I am developing I have a process, initiated by the client, that can run for a considerable length of time (perhaps 10 minutes or more). However rather than have my client application wait around for the server response, I want to simply send a 'OK, I've started the work" message, and let the client check in later to view any results.

My attempt to do this looks like this: 

void longProcess(HTTPServerRequest req, HTTPServerResponse res) {

     //Do a bit of work.

     res.bodyWriter.write( "{\"process\": \"started\"}" );
     res.bodyWriter.finalize();

     //Do a lot of work.
}

My initial idea was that the JSON message returned to the client would contain the info. it needed to follow up on progress completing the process.

However, it seems that the message written to res.bodyWriter is not returned to the client until my 'longProcess' function terminates.

Is there any way to achieve what I am trying to do? Alternately, I am happy to hear of any better approaches to dealing with this problem.

You can do "a lot of work" in a separate task started in longProcess and check if that task is finished when the client checks back again.

This. A simple example would be: 

alias JobID = ulong;
JobID generateJobID() { static JobID counter = 0; return counter++; }

Task[JobID] tasks;

void longProcess(HTTPServerRequest req, HTTPServerResponse res) {
	// Do a bit of work
	auto id = generateJobID();
	tasks[id] = runTask({
		/* Do a lot of work */
		tasks.remove(id);
	});
	res.writeJsonBody(["process": "started", "id": to!string(id)]);
}

void getProcessStatus(HTTPServerRequest req, HTTPServerResponse res) {
	auto pt = to!JobID(req.query["id"]);
	enforce(pt !is null);
	res.writeJsonBody(["finished": !pt.running]);
}

First, thank you Sonke, Etienne and Stephan for your suggestions. I've tried the code that Sonke has suggested, however my program still waits until the task is complete before returning a response to the client. I tried to reduce my problem by replacing my long process with a simple timer that paused for 5 seconds, but when I did that everything worked just fine of course (ie. response was returned immediately, then the task executed).

The specific code that executes the task now looks like: 

    logInfo("Starting classification: " ~ Clock.currTime.toISOExtString() );
    
    auto id = generateJobID();
    
    tasks[id] = runTask( 
        {
	    classifyTask( bands, 
                      "cluster", 
                      g_config.tmp_dir ~ "isoclus.out.json", 
                      image_dir,
                      image );
                  
	    tasks.remove(id);
	}
    );
    res.writeJsonBody(["process": "started", "time": Clock.currTime.toISOExtString(), "id": to!string(id)]);

My classifyTask() function also logs the time at which it terminates. And the time written to by `writeJsonBody()` is roughly the same as the time at which classifyTask terminates, rather than the time printed at "Starting classification".
Since, I figure someone may ask what classifyTask does here it is: 

 void classifyTask( ulong[] band_list, 
		   string classifier, 
		   string classifier_config, 
		   string image_dir,
		   ImageDB src_image )
{
    string[] classify_args;
    classify_args.length = 4 + band_list.length;    
    tileDesc tl = src_image.tiles;
    
    classify_args[0] = "../bin/classify";
    classify_args[1] = "cluster"; //Currently only supported classifier
    classify_args[2] = g_config.tmp_dir ~ "isoclus.out.json";
    
     foreach( int t_x; iota( 0, tl.xTileBaseCount()  ) ) {
	foreach( int t_y; iota( 0, tl.yTileBaseCount()  ) ) {
  
	   classify_args[3] =  g_config.tmp_dir ~ "class/" ~ 
			       rawTileName( src_image.nextId(), t_x, t_y);
	  
	   foreach(idx, bnd; band_list ) {
	      classify_args[4 + idx] = image_dir ~ rawTileName( to!int(bnd), to!int(t_x), to!int(t_y) );
	   }
	    
           debug {
	       foreach( str; classify_args) {
	           std.stdio.write(str, " ");
	       }
	       writeln();
           }
	    
	   auto cpid = std.process.spawnProcess(classify_args);
	   std.process.wait(cpid); //wait until its done to proceed.
	}
      }
      
      logInfo("Classification complete: " ~ Clock.currTime.toISOExtString() );
}

Is it possible that something I am doing in my classifyTask() function is blocking the server response?