Event-driven multi-thread scheme for Batch files
Posted: 12 Aug 2015 16:10
This is a method that allows a Batch file to start a series of simultaneous threads as auxiliary processes. The threads normally stay in a wait state, but anyone of they can be activated upon the receipt of a string (the "event") that cause that the intended thread ("event handler") process the event and return to the wait state. This mechanism is assembled via pipelines, that is, all event handlers form a chain of processes connected by pipes, so the output of the first handler pass as input to the second one and so on. When the main Batch file send an event to the chain, each event handler take the event, review it and process it if the event is intended for such handler; otherwise the handler just pass the event to the next handler in the chain. This method allows to have a large number of event handlers and yet be very efficient. Some details can aid to assemble a more efficient chain; for example, place the most frequently called or fastest handlers at beginning of the chain, and the less used or larger ones at end.
In the example below, JScript is used to write the event handlers because they must read the event from a pipe, and SET /P Batch command have problems with this input; if such problems can be compensated, the event handlers can be written in Batch. Each handler use its Stdout to send the event to next handler, so they must use Stderr in order to show any result on the screen. The same point apply to the main Batch file, but in this case the "2>&1 1>&3" trick is used in order to avoid multiple "> CON" redirections in all messages displayed in the screen. In this example the event handler is a simple generic routine, but in a real application each handler may be very different.
An useful application of this technique could be a Batch file with three auxiliary processes: Directory, File and Text file. The Batch file send a directory to the first handler that process the nested subdirectories by itself, but send files to the next handler. The File handler do a basic processing on all files, but send text files to the third handle for additional processing. You may even insert many additional handlers to process several different types of files because the addition of such handlers in the chain no waste additional resources. This point represent a very simple way of multi-thread a wide range of applications.
Antonio
In the example below, JScript is used to write the event handlers because they must read the event from a pipe, and SET /P Batch command have problems with this input; if such problems can be compensated, the event handlers can be written in Batch. Each handler use its Stdout to send the event to next handler, so they must use Stderr in order to show any result on the screen. The same point apply to the main Batch file, but in this case the "2>&1 1>&3" trick is used in order to avoid multiple "> CON" redirections in all messages displayed in the screen. In this example the event handler is a simple generic routine, but in a real application each handler may be very different.
Code: Select all
@if (@CodeSection == @Batch) @then
@echo off
setlocal
if "%~1" equ "Restart" goto Restart
set newThread=CScript //nologo //E:JScript "%~F0"
echo Before start the chain of event handlers
"%~F0" Restart 2>&1 1>&3 | %newThread% 2 | %newThread% 3 | %newThread% 4
echo After the chain of event handlers ended
goto :EOF
:Restart
echo/
echo Press a key to send event to handler 2
pause
echo 2:Event sent to handler TWO>&2
echo/
echo Press a key to send event to handler 4
pause
echo 4:Event sent to handler FOUR>&2
echo/
echo Press a key to send event to handler 5 (unhandled)
pause
echo 5:Event sent to handler FIVE>&2
echo/
echo Press a key to send event to handler 3
pause
echo 3:Event sent to handler THREE>&2
echo/
echo Press a key to terminate
pause
echo/
goto :EOF
@end
var myId = WScript.Arguments(0);
WScript.Stderr.WriteLine();
WScript.Stderr.WriteLine("Event handler #"+myId+" started");
// Wait for an event intended for this handler in order to process it
while ( ! WScript.Stdin.AtEndOfStream ) {
var event = WScript.Stdin.ReadLine();
if ( event.substr(0,1) == myId ) {
WScript.Stderr.WriteLine(' -> Handler '+myId+' responding to event: "'+event+'"');
} else {
// Pass event not intended for this handler to next handler in the chain
WScript.Stdout.WriteLine(event);
}
}
WScript.Stderr.WriteLine("Event handler #"+myId+" ending");
An useful application of this technique could be a Batch file with three auxiliary processes: Directory, File and Text file. The Batch file send a directory to the first handler that process the nested subdirectories by itself, but send files to the next handler. The File handler do a basic processing on all files, but send text files to the third handle for additional processing. You may even insert many additional handlers to process several different types of files because the addition of such handlers in the chain no waste additional resources. This point represent a very simple way of multi-thread a wide range of applications.
Antonio
That was some cool collective sleuthing 
But one must be aware that it is only good for multi-threaded jobs where each thread receives roughly the same amount of work. The last thread in the chain can only receive commands that have been passed through the earlier threads in the pipe chain. And that can only happen when the earlier threads are idle, waiting for input. If an early thread receives a long task, then the subsequent threads are stuck in an idle state, waiting for the earlier one to finish so it can pass the command down the chain.
The only way to avoid that is to serialize the writes, which reintroduces semaphores and defeats the whole purpose.
