loop without for to read lines of a file

[lockedscope]

I use following to read from a file but it does not continue reading after first line.

I do not want to use a for-loop and want to experiment with set/p.

So, how could i achieve reading from a file in a loop(without for-loop)? I tried staged redirection but failed.

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "ndx=0"
(  
  :loop
  echo loop index: !ndx!
  
  set val=
  set /a ndx=ndx+1
  set /p val=
  if "!val!"=="" (goto :exitLoop)
  set a!ndx!=!val!
  echo !val!
  goto :loop
) <variables.txt

:exitLoop

echo after loop
echo a1: !a1!
echo a2: !a2!

endlocal

I tried with call but it hits stack limit after 338 lines of text.

Code: Select all

setlocal EnableDelayedExpansion

set "ndx=0"
(  
  :loop
  echo loop index: !ndx!
  
  set val=
  set /a ndx=ndx+1
  set /p val=
  if "!val!"=="" (goto :exitLoop)
  set a!ndx!=!val!
  echo !val!
  call :loop
  exit /b
) <variables.txt

:exitLoop

echo after loop
echo a1: !a1!
echo a2: !a2!

endlocal

[lockedscope]

I could achieve it with following but it exits current command line without any error(may be the weird redirection is culprit) so i call it with cmd/c bat.cmd.
So why does it exit? Do you have any idea?

bat.cmd

Code: Select all

@echo off

setlocal EnableDelayedExpansion

set "ndx=0"

(@echo off<&4 4<&3) 3<variables.txt


:loop
  echo loop index: !ndx!
  
  set val=
  set /a ndx=ndx+1
  
  (set /p val=) <&3
  rem It works without any explicit redirection too
  rem set /p val=
  
  if "!val!"=="" (goto :endLoop)
  set a_!ndx!=!val!
  echo !val!
  
  goto :loop

:endLoop

echo after loop
set a_
echo some values
echo a_1: !a_1!
echo a_2: !a_2!

endlocal

[Squashman]

Pretty sure I stole this from someone on this forum. Can't remember who.

Code: Select all

@echo off

call :Read <"file.txt"
pause
GOTO :EOF

:Read
set "line="
set /p "line="
if defined line (
  echo %line%
) else (
  GOTO :EOF
)
goto :Read

[lockedscope]

That's good. I have the batch file calling itself version.
But I am still curios why go-to does not work and why the second one works but exits.

[penpen]

Ad "goto does not work":
The goto leaves the scope of the compbound command (no matter if you want to jump right into it again), so it was fully executed and therefore all redirections are restored.

Ad "second one works, but exits":
Multiple redirections within one atomic command are restored in the order they are assigned.
So the compbound command '(@echo off<&4 4<&3) 3<variables.txt"' should affect the handles as follows:

Code: Select all

handle mapping   handle    initial   assign                    restore
                                     3<file  <&4     4<&3      0       4       3
                      0    STDIN     STDIN   STDIN   STDIN     file    file    file
                      1    STDOUT    STDOUT  STDOUT  STDOUT    STDOUT  STDOUT  STDOUT
                      2    STDERR    STDERR  STDERR  STDERR    STDERR  STDERR  STDERR
                      3    unused    file    file    file      file    file    file
                      4    unused    unused  STDIN   file      unused  STDIN   STDIN
                      5    unused    unused  unused  STDIN     STDIN   unused  unused
                     6+    unused    unused  unused  unused    unused  unused  unused
restore handle             -         3       3, 0    3, 0, 4   3, 4    3       -
        with               -         3       3, 4    3, 4, 5   3, 5    3       -

That means that the command "(set /p val=) <&3" reads from file. When the batch ends, the command line tries to read from device 0, which is the file that is closed (because you reached the end of it), so the command line loop is terminated, which exits the cmd.exe instance.


penpen

[T3RRY]

The following modification to Squahmans answer is a bit more robust:

Code: Select all

@Echo Off & CD "%~dp0"
Setlocal EnableDelayedExpansion
Set LB=0
Set L#=0
call :Read <"%~F0"
pause
GOTO :EOF
:Read
set "line[%L#%]="
Set /p "line[%L#%]="
if not "!line[%L#%]!"=="" (
  Set LB=0
) else (
  Set /A LB+=1
  If !LB! GTR 2 (GOTO :EOF)
)
Echo/!line[%L#%]!
Set /A L#+=1
goto :Read

Squashmans posted script may quit prematurely due to the If else construct using the Definition of Line to terminate the loop when the first empty line is found, which may often not be the last line of a file. The modified version Uses a count of the number of consecutive empty lines to quit execution, an Arbitrary number in the above example of three consecutive lines.

[lockedscope]

hi penpen,
thanks for the comprehensive answer.
I tried following and some other variations to reset stdin at the end but it did not help.
How could we restore handles at the end to terminate smoothly without exiting cmd.exe instance?

Code: Select all

(echo off) 5<variables.txt
endlocal

I realized that it works with following and still command terminates when it finishes so it might be easier to restore handles but still i do not know how?

Code: Select all

@echo off <variables.txt 3<&0
or
@echo off <&3 3<variables.txt

[penpen]

You could add the following command at the end of your batch program to restore the default input stream of cmd.exe:

Code: Select all

@echo off <&3 3<&4

But note that the handle mapping still remains messed up, so you might get unexpected results (for example when you start your batch a second time):

Code: Select all

0    STDIN
1    STDOUT
2    STDERR
3    file
4    STDIN
5+   unused

The unused mappings can't be restored using batch, because you can't redirect them.


penpen

[Aacini]

Hi. I wrote this working example based on your original code:

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "ndx=0"
call :ReadFile < variables.txt
echo After read:
for /L %%i in (1,1,%ndx%) do echo !a[%%i]!
goto :EOF

:ReadFile
   set "val="
   set /P "val="
   if "%val%" == "" goto exitLoop
   set /A ndx+=1
   set "a[%ndx%]=%val%"
goto ReadFile
:exitLoop
exit /B

Output:

Code: Select all

After read:
First line
Second line
Line three

I suggest you to read this thread about using several standard handles to read several input files.

Antonio

[lockedscope]

I learned mechanics of redirections from dbenham's explanations at viewtopic.php?p=14612#p14612
So, i come up with following it works fine but handles seem to be messed up as penpen told and it could not read stdin when it's launched again.

Code: Select all

@echo off

setlocal EnableDelayedExpansion
set "ndx=0"

rem @echo off 3<variables.txt <&4  4<&3
rem @echo off <&3 3<variables.txt 

 @echo off <variables.txt 3<&0 
 rem file redirected to 0 so 0 is hold in 3, 0 is redirected to 3 so 3 is hold in 4.
 rem file is in 0 so 0(stdin) pushed to 3; 0(file) is in 3 so 3 pushed to 4(stdin)
 rem 0 restored from 3(file), 3 restored from 4(stdin), 4 undefined.
 rem so 0 has file and 3 has stdin.

:loop
  echo loop index: !ndx!
  
  set val=
  set /a ndx=ndx+1
  
  set /p val=
  
  if "!val!"=="" (goto :endLoop)
  set a_!ndx!=!val!
  echo !val!
  
  goto :loop

:endLoop

echo after loop
set a_
echo some values
echo a_1: !a_1!
echo a_2: !a_2!

@echo off 0<&3 4<&0
rem 3(stdin) is in 0 so 0(file) pushed to 4; 0(stdin) is in 4 so 4(file) pushed to 5.
rem 0 restored from 4(stdin), 4 restored from 5(file)

rem Following ones work too
rem @echo off 0<&3 4<&0 4<&3
rem @echo off 0<&3 4<&3


endlocal

[lockedscope]

Thank you Aacini for the answer, i will check that article.

I suggest you to read this thread about using several standard handles to read several input files.

Antonio

[Aacini]

Simple question: what is the purpose of using several handles in your code? Messing up all?

Handles are used to read files. You need one handle for each file. Are your reading more than one file? If not, what is the purpose of using more than one handle?

Do you want to solve your reading file problem or to learn how this stuff works? If the answer is the second one, then I suggest you to complete several tests, and post the results!

Antonio

PS - Here it is an interesting application of this stuff:

Code: Select all

@echo off

(
   echo Standard output
   echo More standard output
   echo Error output >&2
   echo More error output >&2
) 1>&2 2>&3 | findstr /A:4E /N "^"

[lockedscope]

Hi Aacini,

Actually i want to lean how this stuff works and i am not sure i learned with well after trying your sample.

I tried it with following to understand what is happening. Error is redirected to standard input or output but not sure about inner mechanics.
So could you explain it please.

Code: Select all

@echo off
(
   echo Standard output
   echo More standard output
   echo Error output >&2
   echo More error output >&2
) 1>&2 2>&3 | (echo # Running findstr & findstr /A:4E /N "^")

I think with 1>&2; stdout is redirected to stream 2 and original stderror is saved to stream 3.
And with 2>&3; stream 2 is redirected to stream 3 and original stream 3 is saved to stream 4.
(Piping writes output streams from left to input streams of right sides. But i am not sure which streams are written.)
So I think it does not cause a chain redirection and redirections run separately. Otherwise stdout must be redirected to stream 3. But only >&2 lines are redirected to findstr.

[lockedscope]

So, i mixed up things in previous post. After reading viewtopic.php?t=6610&start=15#p43422, i could understand it better but still have some suspicions.

This one is fine.

Code: Select all

(
   echo Output1
   echo Output2 >&2
   echo Output3 >&3
   echo Output4 >&4
   echo Output5 >&5
) 1>&2 2>&3 | (echo # Running findstr & findstr /A:4E /N "^") 

### Result
Output1
Output4
Output5
# Running findstr
1:Output2
2:Output3

- 1>&2 
  - 2 need to be duplicated to 1 but 1 is not empty.
    - dup(1)->3(currently empty one), 3=stdout
    - close(1), 1=nothing
    - dup2(2,1), 1=stderr(2 contains stderr so its duplicated to 1)
- 2>&3
  - 3 need to be duplicated to 2 but 2 is not empty.
    - dup(2)->4(currently empty one), 4=stderr
    - close(2)
    - dup2(3, 2), 2=3(3 has stdout)

- Finally 1=stderr, 2=3=stdout, 4=stderr

But for this one, i could not find out how handle 5 is redirected?

Code: Select all

(
   echo Output1
   echo Output2 >&2
   echo Output3 >&3
   echo Output4 >&4
   echo Output5 >&5
) 1>&3 2>&3 | (echo # Running findstr & findstr /A:4E /N "^") 

### Result
Output4
# Running findstr
1:Output1
2:Output2
3:Output3
4:Output5

# Interpretation 1(3 is assumed to be undefined)
- 1>&3
  - 3 need to be duplicated to 1 but 1 is not empty.
    - dup(1)->3(currently empty one), 3=stdout
    - close(1), 1=nothing
    - dup2(3,1), 1=3(3 is 3)
- 2>&3
  - 3 need to be duplicated to 2 but 2 is not empty.
    - dup(2)->4(currently empty one), 4=stderr
    - close(2)
    - dup2(3, 2), 2=3(3 is 3)

- Finally 1=2=3=stdout, 4=stderr, what about 5?

# Interpretation 2(when 3 is assumed to be used or not available because it is used in this redirection);

- 1>&3
  - 3 need to be duplicated to 1 but 1 is not empty.
    - dup(1)->4(currently empty one), 4=stdout
    - close(1), 1=nothing
    - dup2(3,1), 1=3(3 is 3)
- 2>&3
  - 3 need to be duplicated to 2 but 2 is not empty.
    - dup(2)->5(currently empty one), 5=stderr
    - close(2)
    - dup2(3, 2), 2=3(3 is 3)

- Finally 1=2=3=stdout, 4=stdout, 5=stderr but 5 must be stdout and 4 must not be stdout!

2>&3 is another riddle Solving it will solve the previous one i think.

Code: Select all

(
   echo Output1
   echo Output2 >&2
   echo Output3 >&3
   echo Output4 >&4
   
) 2>&3 | (echo # Running findstr & findstr /A:4E /N "^") 

### Result
Output2
Output3
# Running findstr
1:Output1
2:Output4

# Interpretation 1
- 2>&3
  - 3 need to be duplicated to 2 but 2 is not empty.
    - dup(2)->4(currently empty one), 4=stderr
    - close(2), 2=nothing
    - dup2(3,2), 2=3(3 is 3)

- Finally, 1=stdout, 2=3=stdout, 4=stderr ??

# Interpretation 2
- 2>&3
  - 3 need to be duplicated to 2 but 2 is not empty.
    - dup(2)->3(currently empty one), 3=stderr
    - close(2), 2=nothing
    - dup2(3,2), 2=3(3 is stderr)

- Finally, 1=stdout, 2=3=stderr, 4=?

- According to Aacini (https://stackoverflow.com/a/9888382/8227155)

Handle 3 is connected to CON: device (keyboard for input, screen for output); handles 4-9 does not.

I also think, handle 3 is connected to stdin/stdout and it seems to be correct.

- But according to dbenham(https://stackoverflow.com/a/9880156/8227155) handle 3 is undefined

It also explains Aacini's observation that handle 3 appears to be connected to CON: (It isn't, it is actually undefined as per Windows documentation).

- So, while interpreting redirection algorithm should we assume handle 3 is undefined or stdin/stdout?
And in either case, we couldn't successfully interpret how above two samples work! Although for the first one assuming it as undefined worked fine.

[dbenham]

Handle 3 definitely starts off as undefined.

Explanation of your middle test:

Starting condition (call it START)
1 = stdout
2 = stderr
3 = undefined
4 = undefined
5 = undefined

Redirection of left side block:
1>&3, current definition of 1 is stored in first available (3), and then 1 is set to current &3, resulting in
1 = stdout
2 = stderr
3 = stdout
4 = undefined
5 = undefined

2>&3, current definition of 2 is stored in first available (4), and then 2 set to current &3, resulting in
1 = stdout
2 = stdout
3 = stdout
4 = stderr
5 = undefined

The last state above (call it BLOCK) applies to each of the commands within the block

Execution of commands within block:
Output 1 - self explanatory
Output 2 1>&2 - current definition of 1 stored in first available (5), and then 1 set to current &2, resulting in
1 = stdout
2 = stdout
3 = stdout
4 = stderr
5 = stdout
Output 2 appears on stdout (which gets piped), then state is reset back to BLOCK

Output 3 >&3 - current definition of 1 stored in first available (5), and then 1 set to current &3, resulting in
1 = stdout
2 = stdout
3 = stdout
4 = stderr
5 = stdout
Output 3 appears on stdout (which gets piped), then state is reset back to BLOCK

Output 4 >&4 - current definition of 1 stored in first available (5), and then 1 set to current &4, resulting in
1 = stderr
2 = stdout
3 = stdout
4 = stderr
5 = stdout
Output 4 appears on stderr (which does NOT get piped), then state is reset back to BLOCK

Output 5 >&5 - current definition of 1 stored in first available (5), and then 1 set to current &5, resulting in
1 = stdout
2 = stdout
3 = stdout
4 = stderr
5 = stdout
Output 5 appears on stdout (which gets piped), then state is reset back to BLOCK

Upon exiting the block we are in BLOCK state, and then everything is reset to START via:
first 1 restored to current &3, and 3 reset to undefined
lastly 2 restored to current &4, and 4 reset to undefined

All Results are accounted for.

I leave it up to you to trace the first and last tests


Dave Benham