I assume that as each phase processes a token, it reads from the source buffer, does some manipulation, and writes to an output buffer. The manipulation may cause the token to expand beyond the limits of the output buffer.
Here are the obvious buffered operations I am aware of, each with the potential of having its own overflow rules:
- Phase 0 file read buffer
- Phase 2 percent expansion
- Command token buffer
- Command arguments buffer
- Other possible buffers: redirection, FOR and IF special tokens ...
- Phase 4 For variable expansion buffer
- Phase 5 Delayed expansion buffer
- Phase 6 CALL. I don't think a new buffer is needed, but if CALL is parsed from a larger command token, then the balance of the command token must be appended to the arguments token.
- Phase 7 Execution
- Parse command from command token, balance must be appended to arguments token
- FOR iteration: Each FOR argument gets its own token buffer
- ECHO output buffer
So I had planned to run a series of tests to investigate all of the above. But I didn't get very far because I am seeing surprising differences between two machines with different versions of Windows 10
Work computer running Windows 10 Enterprise 64 bit, build 16299
Code: Select all
INFO.BAT version 1.5
--------------------------------------------------------------------------------
Windows version : Microsoft Windows [Version 10.0.16299.1625]
Product name : Windows 10 Enterprise, 64 bit
Performance indicators : Processor Cores: 8 Visible RAM: 8283220 kilobytes
Date/Time format : (mm/dd/yy) Thu 01/30/2020 11:40:32.47
__APPDIR__ : C:\WINDOWS\system32\
ComSpec : C:\WINDOWS\system32\cmd.exe
PathExt : .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
Extensions : system: Enabled user: Enabled
Delayed expansion : system: Disabled user: Disabled
Locale name : en-US Code Pages: OEM 437 ANSI 1252
DIR format : 01/29/2020 03:48 AM 1,677,721,600 pagefile.sys
Permissions : Elevated Admin=No, Admin group=No
Missing from the tool collection: debug
Code: Select all
INFO.BAT version 1.5
--------------------------------------------------------------------------------
Windows version : Microsoft Windows [Version 10.0.17763.1039]
Product name : Windows 10 Pro, 64 bit
Performance indicators : Processor Cores: 4 Visible RAM: 4192432 kilobytes
Date/Time format : (mm/dd/yy) Thu 03/19/2020 21:53:56.39
__APPDIR__ : C:\WINDOWS\system32\
ComSpec : C:\WINDOWS\system32\cmd.exe
PathExt : .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
Extensions : system: Enabled user: Enabled
Delayed expansion : system: Disabled user: Disabled
Locale name : en-US Code Pages: OEM 437 ANSI 1252
DIR format : 03/16/2020 12:39 PM 6,947,000,320 pagefile.sys
Permissions : Elevated Admin=No, Admin group=Yes
Missing from the tool collection: debug
Test 1) Command token 16405 bytes long with stdout redirection at end, followed by concatenated ECHO( and ECHO OK
starts at position 1 with ECHO.
position 8192 starts with ZECHO.
position 16382 starts with 23456789012345678901234 8888888888888
Work result:
Reads first 8191 bytes into command token, buffer is full, and the command token is terminated.
The next 8191 bytes are read and the first Z character is stripped and the rest placed in the argument token.
Continues to read the "arguments token", but throws it away until it reaches redirection which is successfully carried out.
Phase 7 strips out the ECHO command and the remainder after the dot goes into a new arguments buffer. The input arguments token is appended to the arguments buffer until it is full, and the rest is discarded.
The ECHO is successful, but the output is 8191 bytes, so there is no room for the the terminating \r\n
The concatenated commands are parsed and executed just fine.
Home result:
Reads first 8191 bytes into command token, buffer is full, and the command parsing is finished without any arguments token
The echo command is extracted and the balance is placed in the arguments buffer. ECHO writes out the argument
The next 8191 bytes is read into the command token, buffer is again full and the command parsing is finished without any arguments token
This time the command is invalid, but the error message exceeds 8191 bytes, so it is lost.
The balance of the line is parsed as a command, with redirection
The 23456789012345678901234 command is invalid and the error message is successful because it is less than 8191 bytes long.
The concatenated commands are parsed and executed just fine.
Test 2) Same as 1 except line begins with XCHO. and there are no commands concatenated at the end.
Work result:
Mostly the same as test 1, except now the command is invalid, and the error message exceeds 8191 bytes, so it fails
Home result:
Mostly the same as test 1, except now the first command is also invalid with an error message that exceeds 8191 bytes, so it fails.
Only the 3rd invalid command successfully prints out an error message as before.
Test 3) stdout and stderr redirected at beginning. Invalid command token is 8101 bytes long. Line is 24987 bytes long
Work and Home give same result:
The redirection succeeds and the command token is parsed.
Everything after the command token is parsed into the arguments token until it is full, the rest is discarded.
The error message (including two sets of \r\n) is exactly 8191 bytes, and succeeds
Test 4) Same as test 3, except now the command token is 8102 bytes long.
Work and Home give same result:
The same as test 3, except now the error message with two sets of \r\n is 8192 bytes, which exceeds the 8191 byte limit, and the error message fails (no output)
Tests 5 and 6 I began to look at Phase 1 expansion, but I didn't bother analyzing once I discovered the difference between two machines.
=========
Thoughts anyone?
I'm curious what the behavior is with XP (or any other version).
I'm also curious how more recent builds of Windows 10 behave.
Is the difference I am seeing related to Enterprise vs. Pro? (I doubt it) or differences between the builds? (I'm assuming the latter)
Dave Benham
