How to use chained redirections?

[Aacini]

This works!!!

Code: Select all

@if (@CodeSection == @Batch) @then

@echo off
if "%1" equ "Restart" goto Restart
cd . >pipe.txt

"%~F0" Restart  3>&1 1>&2 4<pipe.txt  |  CScript //nologo //E:JScript "%~F0" >>pipe.txt
goto :EOF

:Restart
echo This appear on the screen
set /P "stdinput=Keyboard input: "
echo READ via keyboard: "%stdinput%"
echo ACTION: This is a command sent to the JScript section >&3
echo Next line was sent to Stderr
verify bad
call :sendReceive "ECHO: This line must be taken in the Batch file"
echo BATCH: Received "%ln%"
echo Last line on screen
REM del pipe.txt
goto :EOF


:sendReceive
echo BATCH: Sending %1
>&3 echo %~1

set "ln="
:getInput
set /p "ln=" <&4
if not defined ln goto :getInput
exit /b


@end


// JScript section

var line, command;
while ( !WScript.StdIn.AtEndOfStream ) {
   line = WScript.Stdin.ReadLine();
   WScript.Stderr.WriteLine('JScript received: "'+line+'"');
   command = line.split(" ")[0];
   if ( command == "ECHO:" ) {
      WScript.Stderr.WriteLine('JScript sending: "OK - '+line+'"');
      WScript.Stdout.WriteLine("OK - "+line);
   } else {
      // Any other action, no screen output
   }
}

Output:

Code: Select all

This appear on the screen
Keyboard input: keys+keys
READ via keyboard: "keys+keys"
JScript received: "ACTION: This is a command sent to the JScript section "
Next line was sent to Stderr
Parámetro
incorrecto en el comando.
BATCH: Sending "ECHO: This line must be taken in the Batch file"
JScript received: "ECHO: This line must be taken in the Batch file"
JScript sending: "OK - ECHO: This line must be taken in the Batch file"
BATCH: Received "OK - ECHO: This line must be taken in the Batch file"
Last line on screen

Antonio

[dbenham]

Yep - that works. It is basically a combination of my last two posts, except you use a non-standard stream for the input on the left, which is probably a good idea. But I thought you didn't want to use files

I'm pretty sure a file is the only working possibility for communicating from right to left.

The code should work fine, but it sets an unhealthy precedent. If you use the same redirection chain without a pipe, then it will permanently disable stdout upon completion of the command This is not a problem with the pipe because it occurs in a child process that terminates upon completion, so no harm. But without the pipe, this is what you get when only standard streams are defined at the start - Note how everything is sort of OK if you start a new CMD with stdout redirected to stderr:

Code: Select all

C:\test>break 3>&1 1>&2 4<test.txt
Access is denied.
Access is denied.
dir
There is not enough space on the disk.
Access is denied.
Access is denied.
cmd >&2
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\test>dir test.txt
 Volume in drive C has no label.
 Volume Serial Number is 5ED1-638E

 Directory of C:\test

08/23/2015  12:12 PM                97 test.txt
               1 File(s)             97 bytes
               0 Dir(s)  577,455,001,600 bytes free

C:\test>exit
Access is denied.
Access is denied.

At this point, the only thing you can do is kill your CMD session and start a new one.

The "proper" way to do the chain is to redirect in descending order, (again assuming only standard streams at start):

Code: Select all

C:\test>break 4<test.txt 3>&1 1>&2

C:\test>dir test.txt
 Volume in drive C has no label.
 Volume Serial Number is 5ED1-638E

 Directory of C:\test

08/23/2015  12:12 PM                97 test.txt
               1 File(s)             97 bytes
               0 Dir(s)  577,454,796,800 bytes free

C:\test>

But the above can still get screwy results if 4 is already defined at the start, but 3 is undefined. At the end, 4 will be permanently redirected to stdout (&1) instead of whatever it was redirected to before. The only bullet proof way to redirect multiple non-standard file handles is to stage it, in which case the order does not really matter.

Code: Select all

(break  3>nul 1>&2)4<test.txt

Here is a demonstration. It is designed to be run when only standard streams 0, 1, and 2 are defined.

Code: Select all

@echo off

call :test1 4>&1
echo The following error message is to be expected.
echo This should fail with an error message because 3 and 4 are properly undefined >&4

echo(
call :test2 4>&1
echo This mistakenly shows to the screen because 3 is already defined so 1^>^&4 redirects to itself >&4
exit /b


:test1  Correctly staged redirection chain
echo :test 1 entry - This works >&4
(break  3>nul 1>&2)4<test.txt
echo :test1 exit - This works >&4
exit /b


:test2  Incorrect redirection chain
echo :test2 entry - This works >&4
break 4<test.txt 3>nul 1>&2
echo :test2 exit - This silently fails because 3 is permanently defined as stdout and 4 is is redefined as nul through remainder of routine >&4
exit /b

--OUTPUT-- (only the first run)

Code: Select all

C:\test>test
:test 1 entry - This works
:test1 exit - This works
The following error message is to be expected.
The handle could not be duplicated
during redirection of handle 1.

:test2 entry - This works
This mistakenly shows to the screen because 3 is already defined so 1>&4 redirects to itself

C:\test>echo 3 is still defined, so this still shows on screen >&4
3 is still defined, so this still shows on screen

C:\test>

Subsequent runs will not look the same because 3 has been permanently defined as stdout. The only recourse to get normal behavior is to kill the CMD session and start anew.


Dave Benham

[einstein1969]

A little question.

This redirection X>&Y is the same of Y<&X ?

einstein1969

[penpen]

Depending on how you mean it the answer is "Yes." or "No".
You know that this happens:

Code: Select all

X>&Y:   Writes the output from handle X to the input of handle Y.
Y<&X:   Reads the input from handle X and writes it to the output of handle Y.

Internally the same operation is performed (java style) to forward the data from stream[X] to stream[Y]:
"stream[Y].write(stream[X].read());"
=> "Yes."

But the data flow direction is reversed:
The first is used to write output, while the second is used to read input.
==> "No."

Example:

Code: Select all

@echo off
setlocal enableExtensions enableDelayedExpansion

((
   set "input="
   >&3 <nul set /P "input=STDOUT: TEXT"
) 3>&4
) 4>"test.txt"

((
   set "input="
   <&4 set /P "input="
) 4<&3
) 3<"test.txt"
echo(#!input!#

endlocal
goto :eof

penpen

Edit: Added the batch example.

[einstein1969]

Thanks penpen.

I have explicited handles. Is correct?

Code: Select all

@echo off
setlocal enableExtensions enableDelayedExpansion

((
   set "input="
   1>&3 <nul set /P "input=STDOUT: TEXT"
) 3>&4
) 4>"test.txt"

((
   set "input="
   0<&4 set /P "input="
) 4<&3
) 3<"test.txt"
echo(#!input!#

endlocal
goto :eof

einstein1969

[penpen]

Yes, you have given the handles (by number X, Y) explicitely in both cases "X>&Y" and "Y<&X".
But the handle type is different (input versus output), so you have to specify it, too.
Because of that "Y<&X" cannot replaced by "X>&Y" (and vice versa).

penpen

[Aacini]

There are some interesting news on this topic!

The operation mechanism of how cmd.exe implements file redirection had been finally discovered. This was done with the valuable aid from SO user eryksun, who used Microsoft Console Debugger (one of the Debugging Tools for Windows) to analyse the internal operation of cmd.exe when a series of chained file/handle redirections are given. The mechanism is basically the same used in old MS-DOS command.com (that I previously described at this post), plus a new unexpected bug.

As we known, when a couple redirections like 1>NUL 3>out.txt are processed, cmd.exe perform these steps: 1- duplicate handle 1 giving 3, 2- close handle 1, 3- open device NUL in handle 1, 4- duplicate handle 3 giving 4, 5- close handle 3, and 6- open file out.txt in handle 3. At this point the redirected command is executed, so it inherits all these opened handles. When the command ends, the original state must be recovered resetting all previously saved handles in the opposite order they were saved; however (this is the bug), they are restored in the same left-to-rigth order of the original redirections! User eryksun called this an "hilariously wrong" error...

Below is the analysis using the new rules of how 3>&1 1>&2 4<test.txt chained redirections are performed by cmd.exe. In this description we use dup(N)->M to duplicate handle N giving number M, that is the next available handle number (equivalent to "DuplicateHandle" function 45H of INT 21H in old MS-DOS; and dup2(N,M) to duplicate handle N precisely into number M (equivalent to "ForceDuplicateHandle" function 46H of INT 21H).

Code: Select all

3>&1 1>&2 4<test.txt
Before redirections:                            0=stdin, 1=stdout, 2=stderr
Redirect 3>&1           dup2(1, 3)                       3=stdout
Redirect 1>&2           dup(1) -> 4                      4=stdout,          saved[1]=4
                        close(1)        1=nothing
                        dup2(2, 1)                                 1=stderr
Redirect 4<test.txt     dup(4) -> 5                      5=stdout           saved[4]=5
                        close(4)        4=nothing
                        open(test.txt) -> 4                                 4=test.txt

At this point, execute the command. When it ends, restore previous redirections IN LEFT TO RIGHT ORDER:

Restore 3>&1            close(3)        3=nothing
Restore 1>&2            dup2(4, 1)                                          1=test.txt
                        close(4)        4=nothing
Restore 4<test.txt      dup2(5, 4)                       4=stdout
                        close(5)        5=nothing

After previous command ends, we have these handles open: 0=stdin, 1=test.txt, 2=stderr, 4=stdout. The standard output is send to handle 1, as usual. However, this handle is connected to test.txt file that was opened in INPUT MODE (4<test.txt), so any attempt to write on it issue an error message.

If you want to review the original thread that was the source of this information, see the comments below this SO answer.

Antonio

[penpen]

The operation mechanism of how cmd.exe implements file redirection had been finally discovered.
(...)
plus a new unexpected bug.
(...)
When the command ends, the original state must be recovered resetting all previously saved handles in the opposite order they were saved; however (this is the bug), they are restored in the same left-to-rigth order of the original redirections! User eryksun called this an "hilariously wrong" error...

I don't want to diminish the work of eryksun - it is a great work.
But it was discovered before, so i could use this restore order earlier:
http://www.dostips.com/forum/viewtopic.php?p=40839#p40839.

I actually cannot remember who has found it (earlier/first) or where to find it; i'm not sure - i guess it was someone here on dostips (but i cannot find the post, so it might be somewhere else... for example hidden in the MS C++ documentation).


penpen

[lockedscope]

@penpen check this for the algorithm using duplicate handle and force duplicate handle.

Art of assembly book, chapter 19.1.5 Redirection of I/O for Child Processes

https://www.plantation-productions.com/ ... HEADING3-4

[dbenham]

I actually cannot remember who has found it (earlier/first) or where to find it; i'm not sure - i guess it was someone here on dostips (but i cannot find the post, so it might be somewhere else... for example hidden in the MS C++ documentation).

I originally discovered and documented the Microsoft error in 2012 at viewtopic.php?p=14612#p14612 (and the subsequent post)
Around the same time I documented the issue on StackOverflow at https://stackoverflow.com/a/9880156/1012053
My original description of the Microsoft bug was not the best. Then in 2014 I clarified the StackOverflow post by stating that the restoration process was implemented as a queue (FIFO - First In First Out), when it should have been implemented as a stack (LIFO - Last In First Out)

I suppose someone else may have discovered the problem even earlier, but I've never seen it.

Dave Benham