Problem with delayed expansion of FOR /F options

[dbenham]

I want to paramatize the FOR /F EOL and DELIMS options, but the code is already in a FOR block so I need to use delayed expansion. I know from prior posts that there is special FOR parsing in phase 2 and it is preventing code like the following from working. My real code is more complex, but this code illustrates the core problem:

Code: Select all

@echo off
setlocal enableDelayedExpansion
(
  set "str= "
  for /f "EOL=!str! tokens=* delims=!str!" %%a in ("    Hello world") do echo %%a
)

I really want to avoid a CALL statement, and the SET and FOR statements must be in the same code block.

Does anyone (jeb ) have a solution to this problem I'm not very optimistic, but you have surprised me in the past jeb.

Dave Benham

[aGerman]

I remember a post of jeb (only some weeks ago) where he told you this construct will always fail. Unfortunately I can't find it.
I tried that, but it results in a syntax error:

Code: Select all

@echo off
setlocal enableDelayedExpansion
(
  set "str= "
  for /f "delims=" %%a in ("!str!") do (
    for /f "EOL=%%a tokens=* delims=%%a" %%b in ("    Hello world") do echo %%b
  )
)

I bet you have to escape something to get it work

Regards
aGerman

[dbenham]

Yes, I looked at that post - it dealt with IF, but also commented that the same applied for FOR as well. That is why I am not optimistic

I tried putting the options in a FOR variable as well, and upon reflecting on the failure, realized that I should have predicted it wouldn't work. How could the parser differentiate between the %%a and the %%b, especially when the quotes are not a requirement for the options.

I've also tried putting the entire line in a delayed expansion variable - again no luck

Dave

[trebor68]

In the posting I can see that the parameter "eol" and "delims" will have the same character " " (space). With the parameter "delims" separated the tokens. The first signs are all delimeter, so can not match the first letter the parameter "eol".

Please check the following code. There are some differences between the various command line. See the parameter "tokens" and "delims".

Here the Test.bat.

Code: Select all

@echo off
:: setlocal enableDelayedExpansion
set "str= "
set "str2=#"
set FORCMD=for /f "eol=%str2% tokens=* delims=%str%" %%a in (TestText.txt) do echo _%%a_
echo %forcmd%
%forcmd%
echo.
echo ##########
echo.
set FORCMD=for /f "eol=%str2% tokens=1-3 delims=%str%" %%a in (TestText.txt) do echo _%%a_%%b_%%c_
echo %forcmd%
%forcmd%
echo.
echo ##########
echo.
set FORCMD=for /f "eol=%str2% tokens=1-3 delims=" %%a in (TestText.txt) do echo _%%a_%%b_%%c_
echo %forcmd%
%forcmd%

And here the file TestText.txt.

Code: Select all

# actors from the serie "Andromeda"
# character Dylan Hunt
Kevin Sorbo
#
# character Beka Valentine
Lisa Ryder
#
# character Trance Gemini
Laura Bertram
#
# character Andromeda Ascendant and Rommie
  Lexa Doig  

The last row will have two spaces in front and at the end of the name.

[dbenham]

In the posting I can see that the parameter "eol" and "delims" will have the same character " " (space). With the parameter "delims" separated the tokens. The first signs are all delimeter, so can not match the first letter the parameter "eol".

My code was structured that way on purpose. It is a mechanism I discovered to disable the EOL option such that FOR will not skip any lines no matter what the 1st character is. The only other way to disable it is to set EOL to a line feed character via special syntax that jeb discovered. For more info, read through this thread, starting with the 3rd post.

But this discussion has nothing to do with the purpose of my original post.

Dave Benham

[jeb]

Hi Dave,

I know from prior posts that there is special FOR parsing in phase 2 and it is preventing code like the following from working.

Does anyone (jeb ) have a solution to this problem I'm not very optimistic, but you have surprised me in the past jeb.

Sorry , this time I didn't see any way other than using a CALL or nested FOR, as you said the main problem is the parsing phase of the FOR-parameters.
Even with a %%var you can't bypass the problem, as the FOR-parameters are fixed before execution.

Code: Select all

setlocal enableDelayedExpansion
(
  set "str= "
  FOR %%D in ("!str!") DO (
    for /f "EOL=%%~D tokens=* delims=%%~D" %%a in ("   Hello world") do echo %%a
  )
)

The nested FOR would work, but I suppose it's speed is less or equal to CALL, and it's not so flexible like a CALL.
The CALL-variant can access and change the normal variables and also access the other FOR-variables, both fails with the nested-FOR variant.

Code: Select all

@echo off   
setlocal enableDelayedExpansion
set "cmd=for /f "EOL=%%str%% tokens=* delims=%%str%%" %%a in ("   Hello world") do @echo %%a"
set cmd
(
  set "str= "
  for /F "delims=" %%x in ('!cmd!') do echo %%x
)

As I didn't know any way to "reparse" a FOR or IF in a block, I suppose there aren't better ways to solve it.

jeb

[dbenham]

Thanks jeb.

You did manage to surprise me - I hadn't thought of your nested FOR solution

I think it would have worked for me even with its limitations, But... like you predicted, the nested FOR is significantly slower than a simple CALL (factor of 5 at work)

Dave Benham