[SOLVED] FOR /F with 15 tokens not working

[pendragon]

Hi,

I need to reorder a set of fields before writing them into a file; for this, I'm using the following FOR /F loop:

Code: Select all

for /f "tokens=1,7,4,5,6,9,11,10,15,8,26,34,52,28,27 delims=," %%A in (%input%) do (
  echo.%%A,%%B,%%C,%%D,%%E,%%F,%%G,%%H,%%I,%%J,%%K,%%L,%%M,%%N,%%O>>%report%
)

But, for a reason I can't seem to be able to grasp, the output looks like this:

12004,0,WIN_S_SMH_AF18,daily_incr,frrmsmh-fp02,frrmsmh-fp02,1427043600,0000000881,1427044481,16141036,frrmsmh-fp02,2,2,%N,%O

I made sure there's no empty token in input file; I tried to reverse the ordering logic in the following way:

Code: Select all

for /f "tokens=1,4-11,15,26-28,34,52 delims=," %%A in (%input%) do (
  echo.%%A,%%E,%%B,,%%C,%%D,%%G,%%I,%%H,%%J,%%F,%%K,%%N,%%O,%%M,%%L>>%report%
)

But all I get is a %N,%O in between other fields.

Is there a limitation I missed, forbidding me to have more than 13 tokens? Am I missing something? How can I make it work?

Many thanks for your help!

[Squashman]

Can't say that I have ever tried declaring the tokens out of order with the TOKENS option.

Have you tried putting them in numerical order and then outputting the TOKENS in the order they need to be in.

[dbenham]

It has nothing to do with the order of the TOKENS= specification (though the letters are always assigned in increasing token position order, regardless what order they are listed.)

The problem is you are attempting to access tokens 34 and 52, but FOR /F cannot parse more than 32 tokens.


Dave Benham

[Squashman]

It has nothing to do with the order of the TOKENS= specification (though the letters are always assigned in increasing token position order, regardless what order they are listed.)

The problem is you are attempting to access tokens 34 and 52, but FOR /F cannot parse more than 32 tokens.


Dave Benham

Hmm, I knew about the 31 tokens but thought that was based on the output. So regardless of how many tokens you are keeping it is based on the actual input and not what you are trying to keep. So in theory you cannot go higher than the number 31.

[dbenham]

Almost correct - you cannot parse past token 32 within a single FOR /F loop. You can get past the limit by using * followed by additional loop(s). Pendragon's problem can be solved by the following:

Code: Select all

for /f "tokens=1,4-11,15,26-28,30* delims=," %%A in (%input%) do  for /f "tokens=4,22 delims=," %%P in ("%%O") do (
  echo.%%A,%%E,%%B,,%%C,%%D,%%G,%%I,%%H,%%J,%%F,%%K,%%P,%%Q,%%M,%%L>>%report%
)

I chose to use 30* instead of 31* just to make the math easier. Simply add 30 to the token position in the 2nd loop to get the actual token position in the original line.


Dave Benham

[pendragon]

Hi all,

Can't say that I have ever tried declaring the tokens out of order with the TOKENS option.

Have you tried putting them in numerical order and then outputting the TOKENS in the order they need to be in.

I had thought of it too, refer to my second code snippet
Thanks anyway for looking into it.

Almost correct - you cannot parse past token 32 within a single FOR /F loop. You can get past the limit by using * followed by additional loop(s). Pendragon's problem can be solved by the following:

Code: Select all

for /f "tokens=1,4-11,15,26-28,30* delims=," %%A in (%input%) do  for /f "tokens=4,22 delims=," %%P in ("%%O") do (
  echo.%%A,%%E,%%B,,%%C,%%D,%%G,%%I,%%H,%%J,%%F,%%K,%%P,%%Q,%%M,%%L>>%report%
)

I chose to use 30* instead of 31* just to make the math easier. Simply add 30 to the token position in the 2nd loop to get the actual token position in the original line.


Dave Benham

You, Sir, are brilliant! This solution works perfectly well, many thanks for your inputs.

By the way and for the sake of curiosity, where did you find reference to this 32 tokens limit?

Thanks again for your help!

[Squashman]

The 31 token reference is here. Which contradicts what is actually in the FOR help file. The help file says 26.
http://ss64.com/nt/for_f.html