DosTips.com

Posted: **12 Dec 2012 12:28**

Hello.
Maybe this will be mentioned before. But I never post something that I read before.

For /F fails, maybe this is a bug, when you expand delayed a variable inside a pipe command.
This permits a command code execution.

Code: Select all

@Echo Off
SetLocal EnableDelayedExpansion

Set "Text=My Text&Calc.exe"

Rem Works:
Echo !Text!
Pause

Rem Fails:
For /F "delims=" %%# in ('Echo !Text!') do Echo %%#
Pause

The batch run the next command:

Code: Select all

Echo !Text!

Outside the For /F it print:

Code: Select all

My Text&Calc.exe

But, inside the For /F using pipe mode, it print:

Code: Select all

My Text

and run Calc.exe.

Bug?

Posted: **12 Dec 2012 12:33**

A for command outputs most poison characters in the metavariable just fine, so that is expected.

The fact that a !var! doesn't encapsulate the characters could be an issue in some cases... another 'gotcha' to remember, but it is the same behaviour that you get when echoing %var%

Posted: **12 Dec 2012 12:41**

echoing %var% is not the same that echoing !var! inside for /f in pipe mode:

Code: Select all

Set "Text=My Text&Calc.exe"

Rem Print: My Text&Calc.exe
Echo !Text!

Rem Print: My Text and run Calc.exe
For /F "delims=" %%# in ('Echo !Text!') do Echo %%#

Rem Abort the batch file, because the & character is not escaped.
For /F "delims=" %%# in ('Echo %Text%') do Echo %%#

Posted: **12 Dec 2012 12:48**

You really should add delayedexpansion to your snippet but yes, I see the difference.

Posted: **12 Dec 2012 12:55**

It fails because the commands within IN('command') are run in a new CMD.EXE session with delayed expansion turned off, even if the parent script has delayed expansion on.

The Windows pipe and FOR command use similar techniques to execute the commands in a new CMD.EXE context.

See jeb's excellent answer to my StackOverflow Question: Why does delayed expansion fail when inside a piped block of code?. I recommend reading the entire thread, as the other answers help give context to jeb's answer.

Dave Benham

Posted: **12 Dec 2012 13:06**

Thanks Dave.
But this also fail:

Code: Select all

SetLocal EnableDelayedExpansion
Set "Text=My Text&Calc.exe"

Rem Fails, print: My Text and run calc.exe
For /F "delims=" %%# in (
'SetLocal EnableDelayedExpansion ^&Echo !Text!'
) do Echo %%#

But this works:

Code: Select all

SetLocal EnableDelayedExpansion
Set "Text=My Text&Calc.exe"
Rem Works:
For /F "delims=" %%# in (
'SetLocal EnableDelayedExpansion ^&Echo ^^!Text^^!'
) do Echo %%#

I have the explanation of this.

Posted: **12 Dec 2012 13:41**

But this next print the help of echo command, even using echo(

Code: Select all

SetLocal EnableDelayedExpansion
Set "Text=/?"

Rem Fails, print: My Text and run calc.exe
For /F "delims=" %%# in (
'SetLocal EnableDelayedExpansion ^&Echo(^^!Text^^!'
) do Echo %%#

Maybe, the use of pipe mode in for /f in some scenarios is unreliable.

Posted: **12 Dec 2012 17:09**

carlos wrote:But this next print the help of echo command, even using echo(
Code: Select all
SetLocal EnableDelayedExpansion
Set "Text=/?"

Rem Fails, print: My Text and run calc.exe
For /F "delims=" %%# in (
'SetLocal EnableDelayedExpansion ^&Echo(^^!Text^^!'
) do Echo %%#
Maybe, the use of pipe mode in for /f in some scenarios is unreliable.

The problem is 'Echo %%#' which results in 'Echo /?' which displays the help.

Code: Select all

@echo off &SetLocal EnableDelayedExpansion
Set "Text=/?"

echo
echo /?
echo./?
echo.test
For /F "delims=" %%# in (

   'echo.^^^!Text^^^!'

) do echo.%%#

pause
exit

The for obfuscates the fact that you used 'echo' without a dot, simply add a dot and your problem is solved.

Code: Select all

ECHO is off (uit).
Meldingen weergeven of de opdracht ECHO aan- of uitschakelen.

  ECHO [ON | OFF]
  ECHO [melding]

ECHO zonder parameters geeft de huidige instelling voor de opdracht ECHO weer.
/?
test
/?
Druk op een toets om door te gaan. . .

ed

Posted: **12 Dec 2012 18:45**

@Ed - Not quite the full story. :;

!Text! is not getting expanded within the IN() clause CMD.EXE session. It is getting expanded by the parent batch within the DO clause. Disable delayed expansion and you can see what I mean.

Code: Select all

@echo off
SetLocal EnableDelayedExpansion
Set "Text=/?"
For /F "delims=" %%# in (
  'SetLocal EnableDelayedExpansion ^&Echo(^^!Text^^!'
) do (
  setlocal disableDelayedExpansion
  Echo(%%#
  endlocal
)

result

Code: Select all

!Text!

The problem is that the IN() clause is executed under a command line context, not a batch file context. SETLOCAL does not work under a command line context.

To get delayed expansion within the IN() clause requires using CMD /V:ON. It is easiest if quotes are used. But it is also possible without quotes.

Code: Select all

@echo off
SetLocal EnableDelayedExpansion
Set "Text=/?"

For /F "delims=" %%# in (
  'cmd /v:on /c "echo(^!Text^!"'
) do (
  setlocal disableDelayedExpansion
  Echo(%%#
  endlocal
)
echo(

For /F "delims=" %%# in (
  'cmd /v:on /c echo(^^^!Text^^^!'
) do (
  setlocal disableDelayedExpansion
  Echo(%%#
  endlocal
)
exit /b

But, how to prove that the expansion is happening within the IN() clause CMD session, and not within the parent batch? Here I provide 2 proofs within one set of examples.

I'm going to use !ERRORLEVEL! instead of !Text!. I want to echo the ERRORLEVEL that results from a command that executed within the IN() clause. I'll use a command (HELP HELP) that generates an error, and print the ERRORLEVEL both before and after.

The other proof is the use of !CMDCMDLINE! to show the command line that was used to launch the currently executing CMD.EXE session. This gets very confusing because there are actually two CMD sessions in the example. One is created implicitly by the FOR /F command, and the second is created explicitly by our command. This also makes the escape rules horrificly complicated. :twisted:

My first test uses quotes around everything. I show both CMD contexts.

My 2nd and 3rd tests require more escaping, so I simplified by dropping the printout of the initial CMD context.

But my 4th example attempts to put it all together, showing both contexts without any quotes, and it gives really screwy results. The CMD /V:ON... command gets executed twice :!:

Edit - removed extra ^ from escaped (not quoted) delayed expansion: ^^^!var^^^! --> ^^!var^^!
Edit 2 - removed some additional extra ^: ^^^= --> ^= and ^^^^^^^& --> ^^^^^& and ^^^^^^^> --> ^^^^^>

Code: Select all

@echo off
setLocal enableDelayedExpansion
for /f "delims=" %%# in (
  '"echo 1st cmdLine: %%cmdcmdline%%&cmd /v:on /c echo 2nd cmdLine: ^!cmdcmdline^!^^&echo err=^!errorlevel^!^^&^^>nul help help^^&echo err=^!errorlevel^!"'
) do (
  setlocal disableDelayedExpansion
  echo test1: %%#
  endlocal
)
echo(

for /f "delims=" %%# in (
  'cmd /v:on /c "echo 2nd cmdLine: ^!cmdcmdline^!&echo err=^!errorlevel^!&>nul help help&echo err=^!errorlevel^!"'
) do (
  setlocal disableDelayedExpansion
  echo test2: %%#
  endlocal
)
echo(

for /f "delims=" %%# in (
  'cmd /v:on /c echo 2nd cmdLine: ^^!cmdcmdline^^!^^^^^&echo err^=^^!errorlevel^^!^^^^^&^^^^^>nul help help^^^^^&echo err^=^^!errorlevel^^!'
) do (
  setlocal disableDelayedExpansion
  echo test3: %%#
  endlocal
)
echo(

for /f "delims=" %%# in (
  'echo 1st cmdLine: %%cmdcmdline%%^&cmd /v:on /c echo 2nd cmdLine: ^^!cmdcmdline^^!^^^^^&echo err^=^^!errorlevel^^!^^^^^&^^^^^>nul help help^^^^^&echo err^=^^!errorlevel^^!'
) do (
  setlocal disableDelayedExpansion
  echo test4: %%#
  endlocal
)

results

Code: Select all

test1: 1st cmdLine: C:\Windows\system32\cmd.exe /c "echo 1st cmdLine: %cmdcmdline%&cmd /v:on /c echo 2nd cmdLine: !cmdcmdline!^&echo err=!errorlevel!^&^>nul help help^&echo err=!errorlevel!"
test1: 2nd cmdLine: cmd  /v:on /c echo 2nd cmdLine: !cmdcmdline!&echo err=!errorlevel!&>nul help help&echo err=!errorlevel!
test1: err=0
test1: err=1

test2: 2nd cmdLine: cmd  /v:on /c "echo 2nd cmdLine: !cmdcmdline!&echo err=!errorlevel!&>nul help help&echo err=!errorlevel!"
test2: err=0
test2: err=1

test3: 2nd cmdLine: cmd  /v:on /c echo 2nd cmdLine: !cmdcmdline!&echo err=!errorlevel!&>nul help help&echo err=!errorlevel!
test3: err=0
test3: err=1

test4: 1st cmdLine: C:\Windows\system32\cmd.exe /c echo 1st cmdLine: %cmdcmdline%
test4: 2nd cmdLine: cmd  /v:on /c echo 2nd cmdLine: !cmdcmdline!&echo err=!errorlevel!&>nul help help&echo err=!errorlevel!
test4: err=0
test4: err=1
test4: 2nd cmdLine: cmd  /v:on /c echo 2nd cmdLine: !cmdcmdline!&echo err=!errorlevel!&>nul help help&echo err=!errorlevel!
test4: err=0
test4: err=1

Dave Benham

Posted: **12 Dec 2012 19:18**

'
O I just responded to his last post, where he make an obvious mistake.

That last test4 is quite weird indeed, don't understand the logic which would require such constructs, couldn't explain it,
don't care and leave it fully up to you :mrgreen:

Posted: **12 Dec 2012 20:04**

dbenham wrote:@Ed - Not quite the full story. :;

!Text! is not getting expanded within the IN() clause CMD.EXE session. It is getting expanded by the parent batch within the DO clause. Disable delayed expansion and you can see what I mean.
Code: Select all
@echo off
SetLocal EnableDelayedExpansion
Set "Text=/?"
For /F "delims=" %%# in (
  'SetLocal EnableDelayedExpansion ^&Echo(^^!Text^^!'
) do (
  setlocal disableDelayedExpansion
  Echo(%%#
  endlocal
)
result
Code: Select all
!Text!
The problem is that the IN() clause is executed under a command line context, not a batch file context. SETLOCAL does not work under a command line context.

@Ed Dyreen: Thanks for the fix.
@Dave: Thanks for the example. Now, the problem is more clear.

Also I found that run Setlocal EnableDelayedExpansion have none effect if you run it inside the pipe command of for /f.
But also, I found that If you do the delayed expansion inside of the pipe command of a variable with LF characters, it will not be splitted with for /F.
Check this example of code:

Code: Select all

@Echo Off
SetLocal EnableDelayedExpansion
(Set \n=^

)

Set "text=text1!\n!text2!\n!text3"

Set /a cnt=0

For /F %%# in (
'Echo(^^!text^^!'
) Do (
Set /a cnt+=1
Echo Loop !cnt!:%%#
)

It print:

Code: Select all

Loop 1:text1
text2
text3

(For /F don't split the variable in lines. The echo command do it. Bug?)

Posted: **12 Dec 2012 20:45**

carlos wrote:(For /F don't split the variable in lines. The echo command do it. Bug?)

http://www.dostips.com/forum/viewtopic.php?p=11528#p11528

Code: Select all

@echo off
setlocal enableDelayedExpansion
set LF=^


set "pLF=%%LF%%"

set ^"@macro=(^
   echo text1 %pLF%^
   echo text2 %pLF%^
   echo text3 %pLF%^
)"

set @macro

setlocal enableExtensions enableDelayedExpansion
echo --- Output is ---
for /f %%1 in ( '!@macro!' ) do (
   Set /a cnt+=1
   Echo Loop !cnt!:%%1
)

pause
exit

Code: Select all

@macro=(   echo text1 %LF%   echo text2 %LF%   echo text3 %LF%)
--- Output is ---
Loop 1:text1
Loop 2:text2
Loop 3:text3
Druk op een toets om door te gaan. . .

like that

Code: Select all

@echo off &setlocal enableDelayedExpansion &set $lf=^


::
set "$=text1!$lf!text2!$lf!text3!$lf!"
set "$"
echo --- Output is ---
for /f delims^=^ eol^= %%r in ("!$!") do set /a §+= 1 &echo.loop !§!:%%r

pause
exit

Code: Select all

$=text1
text2
text3

$lf=

--- Output is ---
loop 1:text1
loop 2:text2
loop 3:text3
Druk op een toets om door te gaan. . .

Posted: **12 Dec 2012 22:01**

@Carlos - You have made the same mistake. Your !text! is getting expanded by the parent batch in the DO() clause. You want it to be expanded by the child process in the IN() clause, so you need to enable delayed expansion with CMD /V:ON /C

Code: Select all

@Echo Off
SetLocal EnableDelayedExpansion
(Set \n=^

)

Set "text=text1!\n!text2!\n!text3"

Set /a cnt=0

For /F %%# in ('cmd /v:on /c Echo(^^!text^^!') Do (
  Set /a cnt+=1
  Echo Loop !cnt!:%%#
)

result

Code: Select all

Loop 1:text1
Loop 2:text2
Loop 3:text3

dbenham wrote:But my 4th example attempts to put it all together, showing both contexts without any quotes, and it gives really screwy results. The CMD /V:ON... command gets executed twice

Achh - it is so obvious :oops:

%cmdcmdline% is not quoted and the contents are not escaped, so it functions like a macro. The first unescaped, unquoted & terminates the ECHO and then the CMD /V:ON /C command is executed. I simply surround the variable in quotes and all is well.

Code: Select all

@echo off
setLocal enableDelayedExpansion
for /f "delims=" %%# in (
  'echo 1st cmdLine: "%%cmdcmdline%%"^&cmd /v:on /c echo 2nd cmdLine: ^^!cmdcmdline^^!^^^^^&echo err^=^^!errorlevel^^!^^^^^&^^^^^>nul help help^^^^^&echo err^=^^!errorlevel^^!'
) do (
  setlocal disableDelayedExpansion
  echo test4: %%#
  endlocal
)

results

Code: Select all

test4: 1st cmdLine: "C:\Windows\system32\cmd.exe /c echo 1st cmdLine: "%cmdcmdline%"&cmd /v:on /c echo 2nd cmdLine: !cmdcmdline!^&echo err=!errorlevel!^&^>nul help help^&echo err=!errorlevel!"
test4: 2nd cmdLine: cmd  /v:on /c echo 2nd cmdLine: !cmdcmdline!&echo err=!errorlevel!&>nul help help&echo err=!errorlevel!
test4: err=0
test4: err=1

Dave Benham

Posted: **12 Dec 2012 22:14**

'
Now I look stupid :roll:

Posted: **12 Dec 2012 22:23**

No MicroSoft FOR bug involved within this thread. But the escape rules are crazy :twisted:

I think one can make an argument that there is a design flaw in how pipes and FOR /F ... IN('command') are executed. It shouldn't have to be this difficult.

Dave Benham

DosTips.com

DelayedExpansion fails inside For /F pipe mode ( bug ?)

DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)

Re: DelayedExpansion fails inside For /F pipe mode ( bug ?)