DosTips.com • HOW TO: FOR /F Disabling EOF or using a quote as delim

Page 1 of 1

HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 27 Oct 2011 16:22

by jeb

Hi,

I repeat here some description from other posts, as it's a bit hard to find them anymore, and the corresponding threads are quite long.

I will try to explain some advanced FOR /F tricks/usages

1. Disabling of the EOL character
If the first character of a line is the EOL-character, the line will be ignored.
Sometimes you didn't want any EOL-character, but it's not obvious how to disable it.

As this only sets the EOL-Standard semicolon to a quote character

Code: Select all

FOR /F "EOL=" %%A in ("test") do echo %%A

In my opinion the EOL option reads always the next character and doesn't regards any special characters.

But dbenham found a good trick how to disable it, if you use one ore more delim-characters,
if the EOL character is also a delim character the EOL is disabled at all.

Code: Select all

FOR /F "EOL=x delims=xyz"

But how to disable it, if the delims list is empty?
You could use a line feed character, as this is always the line termination character, the effect is like the EOL is disabled.
But it's not so obvious how to set the EOL to the line feed character, but it's possible.

Code: Select all

FOR /F ^"tokens^=1^,2^ EOL^=^

^" %%A in ...

This isn't a typo, the empty line is necessary, also the carets, as the key is the multiline caret at the end of line1,
but this can only work without quoting, therefore the first quote is escaped too.
But as the quote is escaped all batch line delimiters (not the FOR/F delims-option) are comma, semicolon, tab,space and the equal sign have to be escaped now with a caret.

And then there is the old question: "Is it possible to use the quote itself as a delim-character?"
Yes, but it's much more worse than the EOL-line feed trick.

FOR /F tokens^=1^,2^,3^ delims^=^" %%A in ("one"two"three") do echo %%A--%%B--%%C

This trick is found by pieh-ejdsch at adminstrator.de: FOR loop double quotes as Delimiter

jeb

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 27 Oct 2011 18:56

by dbenham

jeb wrote:And then there is the old question: "Is it possible to use the quote itself as a delim-character?"
Yes, but it's much more worse than the EOL-line feed trick.

FOR /F tokens^=1^,2^,3^ delims^=^" %%A in ("one"two"three") do echo %%A--%%B--%%C

Thanks for finding and sharing that one jeb. I was wondering how to do that.

Dave Benham

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 28 Oct 2011 22:20

by dbenham

jeb wrote:But it's not so obvious how to set the EOL to the line feed character, but it's possible.
Code: Select all
FOR /F ^"tokens^=1^,2^ EOL^=^

^" %%A in ...
This isn't a typo, the empty line is necessary, also the carets, as the key is the multiline caret at the end of line1, but this can only work without quoting, therefore the first quote is escaped too. But as the quote is escaped all batch line delimiters (not the FOR/F delims-option) are comma, semicolon, tab,space and the equal sign have to be escaped now with a caret.

Since this only works without quoting, the quotes aren't needed at all :wink:

:wink:

The EOL=<LF> can be written similarly to the DELIMS=" solution

Code: Select all

for /f tokens^=1^,2^ eol^=^

  %%a in ...

It's even possible to set both EOL=" and DELIMS="

Code: Select all

for /f tokens^=1^,2^ eol^=^"^ delims^=^" %%a in ...

Dave Benham

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 07 Aug 2012 11:19

by Ed Dyreen

'
I compile a help file

Code: Select all

:: -------------------------------------------------------------------------------------------------------------
:: - The delimiter
:: -------------------------------------------------------------------------------------------------------------
::
:: http://www.dostips.com/forum/viewtopic.php?f=3&t=2385
::
:: (
   @echo off &setlocal DisableDelayedExpansion

   echo.
   echo.sets " as delimiter
   for /f tokens^=1-3^ delims^=^" %%a in ("one"two"three") do echo %%a--%%b--%%c

   echo.
   echo.sets delims to null
   for /f "tokens=1-3 delims=" %%a in ("one"two"three") do echo %%a--%%b--%%c

   echo.
   echo.disables eol because it matches the delim
   for /f "tokens=1-3 eol=: delims=:" %%a in (":one"two"three") do echo %%a--%%b--%%c

   echo.
   echo.sets eol to LF and delims to null
   for /f tokens^=1-3^ eol^=^

   delims^= %%a in (";Please <preserve=^"this line"!>") do echo %%a--%%b--%%c

   for /f ^"tokens^=1-3^ eol^=^

   delims^=^" %%a in (";Please <preserve=^"this line"!>") do echo %%a--%%b--%%c

   pause
   exit

   -- output --
   sets " as delimiter
   one--two--three

   sets delims to null
   one"two"three----

   disables eol because it matches the delim
   one"two"three----

   sets eol to LF and delims to null
   ;Please <preserve=^"this line"!>----
   ;Please <preserve=^"this line"!>----
   -- output --
:: )
:: -------------------------------------------------------------------------------------------------------------

Tags; quote as delim, disable delims, disable eol, eol to linefeed, preserve special characters.

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 25 Oct 2012 08:08

by dbenham

SS64 forum user carlos published a much simpler method to disable both EOL and DELIMS :!:

:!:

Code: Select all

    for /f delims^=^ eol^= %%A in ...

We are all so conditioned to put DELIMS at the end so that we can include space as a delimiter that we never considered putting EOL at the end when DELIMS is disabled. But it is so logical if you think about it. :roll:

:roll:

Dave Benham

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 25 Oct 2012 23:13

by jeb

Hmm, but this isn't really new, I suppose, I saw this posted before on Dostips.

I thought that Aacini, Ed or you posted this :!:

:!:

dbenham wrote:We are all so conditioned to put DELIMS at the end so that we can include space as a delimiter that we never considered putting EOL at the end when DELIMS is disabled. But it is so logical if you think about it.

It supprised me, that delims isn't set to space in this case, but apperntly it sets it really to nothing.

jeb

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 26 Oct 2012 05:15

by dbenham

jeb wrote:Hmm, but this isn't really new, I suppose, I saw this posted before on Dostips.

I thought that Aacini, Ed or you posted this

dbenham wrote:We are all so conditioned to put DELIMS at the end so that we can include space as a delimiter that we never considered putting EOL at the end when DELIMS is disabled. But it is so logical if you think about it.

It supprised me, that delims isn't set to space in this case, but apperntly it sets it really to nothing.

jeb

It doesn't surprise me at all. EOL will always take the next character in the option string. But there is no next character because the space is not quoted or escaped.

This works - EOL is disabled (undefined or set to nothing)

Code: Select all

for /f delims^=^ eol^= %a in (" hello world") do @echo %a

None of these work - EOL is set to space or quote

Code: Select all

for /f delims^=^ eol^=^  %a in (" hello world") do @echo %a
for /f "delims= eol= " %a in (" hello world") do @echo %a
for /f "delims= eol=" %a in (""hello world"") do @echo %a

Perhaps someone posted this before, but if so, then I certainly missed it.

Either way, all I know is I am switching to this "new" syntax to disable EOL

Dave Benham

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 26 Oct 2012 06:36

by Ed Dyreen

It doesn't saves any bytes but it surely is less error prone than ^%LF%%LF%.

dbenham wrote:Either way, all I know is I am switching to this "new" syntax to disable EOL

I only had to change one location in a single file :twisted:

:twisted:

:lol:

Code: Select all

set  ^"forLineR_=for /f delims^^=^^ eol^^= %%r in"
set ^"mForLineR_=for /f delims^^^^=^^^^ eol^^^^= %%r in"

Thanks for sharing

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 13 Apr 2018 02:28

by sst

Hi Everyone
This is my first post here.

Sorry for bringing up this 6 years old thread but I would like to mention that there is no need to unquote and escape the entire FOR /F parameters.
It is sufficient to put just the eol out of the quote with escaped syntax.

Code: Select all

FOR /F "skip=n tokens=1,2,3 delims=xyz"eol^= %%a in ...

needless to say that the eol must come immediately after closing quote with no spaces between.
This will disable EOL and also avoids the ugly and confusing escaped syntax

Test Script:

Code: Select all

@echo off

setlocal EnableExtensions DisableDelayedExpansion

set "TextBeginningWithSemicolon=;abc xyz"
set "TextBeginningWithSpace= ;abc xyz"

echo,
echo TextBeginningWithSemicolon=%TextBeginningWithSemicolon%
echo Assigning 'TextBeginningWithSemicolon' value to MyVar via FOR /F...

echo,
echo Testing with default EOL...
for /F "tokens=1 delims=" %%a in ("%TextBeginningWithSemicolon%") do set "MyVar=%%a"
if not defined MyVar (
   echo MyVar not defined. This was expected.
) else (
   echo MyVar defined! WTF
)

echo,
echo Testing with supposedly disabled EOL...
for /F "tokens=1 delims="eol^= %%a in ("%TextBeginningWithSemicolon%") do set "MyVar=%%a"
if defined MyVar (
    echo It works! EOL was disabled.
    echo MyVar=%MyVar%

    echo,
    echo I'm Paranoid. Let's make sure EOL was not set to SPACE...
    for /F "tokens=1 delims="eol^= %%a in ("%TextBeginningWithSpace%") do set "MyVar=%%a"
    if defined MyVar (
        echo Yes It really works.
        call echo MyVar=%%MyVar%%
    ) else (
        echo Something is wrong.
    )
    
) else (
    echo It doesnt work :(
)

Output:

Code: Select all


TextBeginningWithSemicolon=;abc xyz
Assigning 'TextBeginningWithSemicolon' value to MyVar via FOR /F...

Testing with default EOL...
MyVar not defined. This was expected.

Testing with supposedly disabled EOL...
It works! EOL was disabled.
MyVar=;abc xyz

I'm Paranoid. Let's make sure EOL was not set to SPACE...
Yes It really works.
MyVar= ;abc xyz

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 13 Apr 2018 04:56

by dbenham

Very nice

And perfectly logical

Although I still think that looks pretty ugly, and is confusing upon a casual glance. One has to understand how both quotes and escapes work with FOR /F, whereas the other form only requires understanding of escapes.

Perhaps the biggest advantage of this new syntax is that it is easier to type (for me anyway)

Thanks for sharing

Dave Benham

Re: HOW TO: FOR /F Disabling EOF or using a quote as delim

Posted: 13 Apr 2018 05:35

by dbenham

So here is a test case for the new discovery that gives the expected behavior

Code: Select all

for /f  "tokens=1,2 delims=xyz"eol^= %a in (";abcxyz"^"123^") do @echo [%a] [%b]

-- OUTPUT --

Code: Select all

[;abc] ["123]

Putting an extra character before the eol gives the expected error

Code: Select all

for /f  "tokens=1,2 delims=xyz".eol^= %a in (";abcxyz"^"123^") do @echo [%a] [%b]

-- OUTPUT --

Code: Select all

.eol= was unexpected at this time.

But inserting an escaped quote avoids any error, yet the EOL remains defined as the default semicolon

Code: Select all

for /f "tokens=1,2 delims=xyz"^"eol^= %a in (";abcxyz"123^") do @echo [%a] [%b]

NO OUTPUT

In fact, you get the same result with an escaped quote tacked on the end - EOL does not treat the trailing quote as a value

Code: Select all

for /f "tokens=1,2 delims=xyz"^"eol^=^" %a in (";abcxyz"123^") do @echo [%a] [%b]

NO OUTPUT

In fact, the 2nd quoted section need not use any escapes. The quoted EOL assignment is simply ignored without error

Code: Select all

for /f "tokens=1,2 delims=xyz""eol=a" %a in ("abcxyz"123^") do @echo [%a] [%b]

--OUTPUT--

Code: Select all

[abc] ["123]

But revert to the escaped form and remove the first escaped quote, and EOL is set to a quote

Code: Select all

for /f "tokens=1,2 delims=xyz"eol^=^" %a in (""abcxyz"123") do @echo [%a] [%b]

NO OUTPUT

Dave Benham