infinite loop with break condition

[einstein1969]

It doesn't improve the performance in my tests.
Typical result:

Code: Select all

19:39:18,23
short code
19:39:22,16
long code
19:39:29,44
einstein code
19:39:31,02
einstein code 2
19:39:33,36
Drücken Sie eine beliebige Taste . . .

I understand why
you were using an old version

[aGerman]

Maybe. I just used the one that you called from within the test batch ¯\_(ツ)_/¯

Steffen

[Aacini]

I just read the posts from 2022/Oct/22 onwards. IMHO you have lost the point of this thread...

Of course, there are several factors that influence the speed of a Batch file that have loops. However, I think that the purpose of this thread was to speed up a loop that was assembled via a goto command that jumps to a previous label.

And the only/best way to achieve this speed up is changing the goto loop by a for /L %%i in () do (... command (perhaps with the top limit that dbenham suggested in order to avoid the cmd /C - exit mechanism: for /L %%i in (1,1,10000) do (...).

And the method to achieve this change is described at beginning of this thread and in The ultimate While loop one.

IMHO if you combine this method with any other factor that impact the performance you make confusing the result of this method, instead of improving it...

I think that beginners could be confused by your extensive description of other factors that affect the performance of a program. My advice is: if you want to speed up a slow Batch program that use loops assembled via goto commands, try to change they by for /L %%i in () do ( ... commands. Period. This change is totally independent of any other change you could do in order to improve the performance.

Just my two cents...

Antonio

[jeb]

Hi,

I'm a little late to the party, but my two cents ...

First, I agree with Aacini, some of the later posts are a bit off topic to the main problem.

Then I rememberer, that I thought about the problem before with a different approach using nested loops.
I see two drawbacks of the current sub-cmd-loop:
1. The command invocation is slow
2. The while macro itself is a bit complex and needs :strLen

My own solution doesn't need a sub cmd at all, therefore returning variables is not problem.
And it's much faster!

Code: Select all

setlocal EnableDelayedExpansion

set max=16
set nested=9
set "loop_base=for /L %%# in (1 1 %MAX%) do if not defined _break "
set loop_cnt=0
set "loop_cmd="
for /L %%# in (1 1 %nested%) do (
    set "loop_cmd=!loop_cmd!!loop_base!"
)
set "loop=set "_break=" & !loop_cmd!"

Or the complete source for the Aacini example of factorial, lcm and calculation of e

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set max=16
set nested=9
set "loop_base=for /L %%# in (1 1 %MAX%) do if not defined _break "
set loop_cnt=0
set "loop_cmd="
for /L %%# in (1 1 %nested%) do (
    set "loop_cmd=!loop_cmd!!loop_base!"
)
set "loop=set "_break=" & !loop_cmd!"

REM *** Tests begins ***
call :factorial 10 factor=
echo Factorial of 10: %factor%
echo/
echo/

call :lcm 3527 3784 var=
echo Least common multiple of 3527 and 3784: %var%
echo/
echo/

echo Calculation of e:
echo/
set /A digits=8, one=1
for /L %%i in (1,1,%digits%) do set one=!one!0
set /A num=0, fact=1, term=1, whileResult=0
echo #- #   term=1/#    summation
%loop% (
    set /A term=one/fact, whileResult+=term
    echo !num!- !fact!    !term!    !whileResult!
    set /A num+=1, fact*=num
    if !term! leq 0 set _break=1
)
echo/
echo Number e = !whileResult:~0,-%DIGITS%!.!whileResult:~-%DIGITS%!
echo/
echo/

goto :EOF

:factorial
setlocal

set num=1
set whileResult=1
%loop% (
    set /A whileResult*=num, num+=1
    if !num! gtr %1 set _break=1
)
endlocal & if "%2" neq "" (set %2=%whileResult%) else echo %whileResult%
exit /b

:lcm n1 n2 lcm=
setlocal
set /a "whileResult=%1, j=%2"

%loop% ( 
    set /a k=j, j=whileResult %% j, whileResult=k 
    if !j! equ 0 set _break=1
)
set /a j=%1*%2/whileResult
endlocal & if "%3" neq "" (set %3=%j%) else echo %j%
exit /B

How it works: MAGIC

It does nearly loop infinite, but in reality only 16^9 = 68719476736 times, but I suppose it's rarely a problem.

The trick is to nest multiple FOR-Loops, each with a low counting number, like

Code: Select all

FOR /L %%# in (1 1 16) do (
  FOR /L %%# in (1 1 16) do (
    FOR /L %%# in (1 1 16) do (
       FOR /L %%# in (1 1 16) do (
         ... the loop code is here
         REM Stop the code by
         goto :break
       )
    )
  )     
)
:break     

Obviously this iterates 16^4=65536 times, BUT it can be stopped by a goto :break in just 4*16 nop cycles instead of 65536.

But the goto :label technique can't be used in macros, so I improved it by using a break variable instead

Code: Select all

set "_break=" & FOR /L %%# in (1 1 16) do if not defined _break (
  FOR /L %%# in (1 1 16) do if not defined _break (
    FOR /L %%# in (1 1 16) do if not defined _break (
       FOR /L %%# in (1 1 16) do if not defined _break (
         ... the loop code is here
         REM Stop the code by
         set _break=1
       )
    )
  )     
)

The speed boost is enormous.
Tested on my PC with complete output:
Aacini: 800ms
jeb: 20ms

Obviously, the set break=1 can be used everywhere in the loop, so it's simple to build a while or a repeat until loop

[miskox]

Very good findings, Jeb.

I remembered that Aacini made a filepointer.exe.

Original Steffen's code:

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set "n=0" &echo %time% & set /a "n+=1"
:loop
if !n! leq 10000 ( set /a "n+=1" ) else goto :exit
goto :loop
:exit
echo %time% %n% &pause

With filepointer.exe

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set "n=0" &echo %time% & set /a "n+=1"
findstr /O /B /L /C:":loop" %0.cmd >where.tmp
for /f "tokens=1 delims=:" %%f in (where.tmp) do set where=%%f
del where.tmp
echo %where%
:loop
if !n! leq 10000 ( set /a "n+=1" ) else goto :exit
echo !n!
filepointer.exe !where!
set where=!errorlevel!
echo where was filepointer set?=!where!
:exit
echo %time% %n% &pause

Result:

Code: Select all

c:\>test1
20:39:30,81
191
2
where was filepointer set?=91
20:39:30,85 2
Press any key to continue . . .

You see that it does not go back.

The problem is that Aacini's FILEPOINTER.EXE doesn't work (on x64 - can't test this on x86 at the moment). According to viewtopic.php?p=34051#p34051 errorlevel returns the file offset where it's at.

This might be very fast solution - to replace the GOTO with the FILEPOINTER.EXE where it would set FP to the offset where the label is.

Ideas?

Saso

[Aacini]

Your idea is interesting! Use FilePointer.exe to move the file pointer of cmd.exe in order to perform a fast goto!

However, you missed a very important point from FilePointer.exe documentation:

Get or set the file pointer position OF A REDIRECTED FILE HANDLE.

FilePointer HANDLE [position [/C|/E]]

The file pointer that FilePointer.exe moves is NOT the one that cmd.exe uses to read the Batch file! How could a child program access a file handle of its parent process? Unless such a file handle be documented, but the Stdin, Stdout and Stderr (handles 0, 1 and 2) available for Batch files are just connected to the cmd.exe's console...

The only way that FilePointer.exe could move the file pointer of a cmd.exe's file handle is if it is executed in this form:

Code: Select all

cmd.exe < textFile.txt

In this way, cmd.exe read the commands it execute from the redirected textFile.txt. If one of these commands is a FilePointer.exe 0 position then the cmd.exe's Stdin IS moved to another position!

Some time (more than 8 years) ago I completed a series of tests on this idea and achieved to "execute" not just a GOTO, but also a CALL in the redirected text file. You can review these tests in Executing GOTO/CALL in a cmd.exe < NotBatch.txt file!.

Antonio

[miskox]

Of course. Handle. The moment I turned the computer off I thought of that.

Thanks.
Saso

[Sponge Belly]

Hi Jeb!

You wrote:

The trick is to nest multiple FOR-Loops, each with a low counting number, like

Code: Select all

FOR /L %%# in (1 1 16) do (
  FOR /L %%# in (1 1 16) do (
    FOR /L %%# in (1 1 16) do (
       FOR /L %%# in (1 1 16) do (
         ... the loop code is here
         REM Stop the code by
         goto :break
       )
    )
  )     
)
:break     

Obviously this iterates 16^4=65536 times, BUT it can be stopped by a goto :break in just 4*16 nop cycles instead of 65536.

I love the idea of nesting the loops. But I thought to myself why not use plain old for loops instead of buggy for /l loops?

Code: Select all

@echo off & setLocal enableExtensions disableDelayedExpansion

set "hex=0 1 2 3 4 5 6 7 8 9 A B C D E F"

for %%A in (%hex%) do (
    for %%B in (%hex%) do (
        rem echo(%%A%%B
        if 0x%%A%%B equ 26 goto break
    )
)
:break

endLocal & goto :EOF

When the test becomes true, the program jumps to the :break label and the for loops stop iterating immediately.

HNY!

- SB

[jeb]

Hi Sponge Belly,

it's a really nice idea and so simple.
And I like the combineable hex values to the total counter.

But you inspired me to think about another problem, I want to use the infinite loop in macros where goto can't be used.

It can be solved either with an if defined expression or even simpler with a simple minus sign.

Code: Select all

set "break="
if not defined break FOR /L %%1 in (1 1 16) do if not defined break FOR /L %%1 in (1 1 16) do if not defined break FOR /L %%2 in (1 1 16) do if not defined break (
  if "%%1.%%2" == "5.7" set "break=X"
)

Code: Select all

set "break="
FOR /L %%1 in (1 1 !break!16) do FOR /L %%1 in (1 1 !break!16) do FOR /L %%2 in (1 1 !break!16) do if not defined break (
  if "%%1.%%2" == "5.7" set "break=-"
)

But in this case, your solution with a simple FOR can't improve the speed.

[jfl]

I really liked this way of breaking out of a loop, and immediately started to experiment with it

My first idea was to define a %BREAK% macro like this:

Code: Select all

set BREAK=set "-break=-"

This works, and is very readable, with code looking like: if condition %BREAK%
But there's a catch: It should only be used on the last line of a code block.
Else, the following lines do get executed before breaking out of the loop. This might be OK in somes cases, but this is risky, as this %BREAK% macro does not behave like the break instructions in high level languages, which immediately break out.

This made me think that the right thing to do is actually to define WHILE or REPEAT macros, and get rid of this BREAK macro altogether.
And, thanks to the new ideas I borrowed above, this is actually simple and efficient.
Eventually, I ended up defining two sets of macros:

• %WHILE(% condition %)DO% ( block of code )
• %REPEAT% ( block of code ) %UNTIL(% condition %)ENDREP%

The condition is any valid test that can be used in an if statement.
Note that as the condition is evaluated once per loop, it must be used with delayed expansion enabled, so that the variables get updated every time.

The parenthesis in the %WHILE(% / %)DO% and %UNTIL(% / %)ENDREP% macro names are there only to make the macro language look more like C. If this proves to be a problem, we can just remove them.

Here's a test that demonstrates these two sets of macros:
(Note that it's using !TIME:~0,8! to quickly get the current time without the fractional part of a second. This is correct on my system, which uses the ISO time format HH:MM:SS.xx. But this may need to be adapted to your system depending on your locale.)

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set REP16X=for /l %%- in (0,1,15) do if not defined -
set REP1MX=%REP16X% %REP16X% %REP16X% %REP16X% %REP16X%

set WHILE(=set "-=" ^& %REP1MX% ( if
set )DO=(set "-=") else set "-=-") ^& if not defined -

set REPEAT=set "-=" ^& %REP1MX%
set UNTIL(=^& if
set )ENDREP=set "-=-"

:# Wait for the end of the current second
%WHILE(% "%TIME:~0,8%"=="!TIME:~0,8!" %)DO% rem

:# Initial test with a goto
set "N=0"
set "T0=%TIME:~0,8%"
:encore
if "%T0%"=="!TIME:~0,8!" (
  set /a "N+=1"
  goto :encore
)
echo goto loop looped !N! times in 1 second

:# Wait for the end of the current second
%WHILE(% "%TIME:~0,8%"=="!TIME:~0,8!" %)DO% rem

:# Test with a while loop
set "N=0"
set "T0=%TIME:~0,8%"
%WHILE(% "%T0%"=="!TIME:~0,8!" %)DO% (
  set /a "N+=1"
)
echo while loop looped !N! times in 1 second

:# Wait for the end of the current second
%WHILE(% "%TIME:~0,8%"=="!TIME:~0,8!" %)DO% rem

:# Test with a repeat loop
set "N=0"
set "T0=%TIME:~0,8%"
%REPEAT% (
  set /a "N+=1"
) %UNTIL(% not "%T0%"=="!TIME:~0,8!" %)ENDREP%

echo repeat loop looped !N! times in 1 second

endlocal & exit /b 0

The output of that code shows that the new while and repeat loops are considerably more efficient than loops using a goto:

Code: Select all

goto loop looped 93 times in 1 second
while loop looped 7098 times in 1 second
repeat loop looped 11700 times in 1 second

A bit disappointing though, the WHILE( macro is significantly slower than the REPEAT macro. We need to find a way to further simplify it to make it as fast as the other.

I've not tested them inside other macros, but I think this should work.
The only limitation so far is that we can't nest multiple such loops inside one another, because they all share the same - exit variable.

[jfl]

Update:
With 10000 loops per second, the 1 million loops limit is sufficient for just 100 seconds.
This is obviously too short for an infinite loop!
I repeated the test with 4 billion loops, enough for a week. And the good news is that the impact on performance is negligible.
Here are the updated macros:

Code: Select all

set REP16X=for /l %%- in (0,1,15) do if not defined -
set REP4GX=%REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X%

set WHILE(=set "-=" ^& %REP4GX% ( if
set )DO=(set "-=") else set "-=-") ^& if not defined -

set REPEAT=set "-=" ^& %REP4GX%
set UNTIL(=^& if
set )ENDREP=set "-=-"

[jfl]

OK, I resolved the two issues I had left

1) The slower performance of the %WHILE(% macro was due to a (set "-=") instruction that was used purely as a NOP.
(This NOP is necessary because I needed to negate the continuation condition, and I couldn't do it by ending the %WHILE(% macro with 'if not', because in case that condition were negative, this would generate invalid code like: if not not !N!==%NMAX% )
Anyway, after timing several possible alternatives, I found that (if 1==0 .) was considerably faster than (set "-="), and a better choice as a NOP anyway.
Using that, and despite its greater complexity, %WHILE(% is now just as fast as %REPEAT%.

2) The problem of having the same break variable in all macros is actually not that much of a problem for embedding several loops inside each other:
The trick is simply to clear that break variable when the macro ends.
This way, the break variable of an outer loop is returned to its initial state when exiting from an inner loop.
The only drawback is that this forces to introduce a %WEND% macro, making the meta language syntax a bit less pleasant.
Old timers will remember the WEND name from Visual Basic!

Here's the updated set of macros:
(By analogy with WEND, I renamed yesterday's ENDREP as REND.)

Code: Select all

set REP16X=for /l %%- in (0,1,15) do if not defined -
set REP4GX=%REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X%

set "NOP=(if 1==0 .)"
set WHILE(=(set "-=" ^& (%REP4GX% ( if
set )DO=(%NOP%) else set "-=-") ^& (if not defined -
set WEND=)) ^& set "-=")

set REPEAT=(set "-=" ^& (%REP4GX%
set UNTIL(=^& if
set )REND=set "-=-") ^& set "-=")

And with that, we can write code such as...

Code: Select all

echo :# Test while(repeat(while()))
set "N=0"
%WHILE(% not "!N!"=="2" %)DO% (
  set "M=0"
  %REPEAT% (
    set "L=0"
    %WHILE(% not "!L!"=="2" %)DO% (
      echo N=!N! M=!M! L=!L!
      set /a "L+=1"
    ) %WEND%
    set /a "M+=1"
  ) %UNTIL(% "!M!"=="2" %)REND%
  set /a "N+=1"
) %WEND%

... which outputs:

Code: Select all

:# Test while(repeat(while()))
N=0 M=0 L=0
N=0 M=0 L=1
N=0 M=1 L=0
N=0 M=1 L=1
N=1 M=0 L=0
N=1 M=0 L=1
N=1 M=1 L=0
N=1 M=1 L=1

[T3RRY]

(By analogy with WEND, I renamed yesterday's ENDREP as REND.)

Particularly in the situation of defining multiple Whiles into a macro, there's a simple alternative to WEND

Use substring substitution to replace the break variable for each additional while
IE \b1 for the first break, \b2 for the second and so on

[jfl]

Use substring substitution to replace the break variable for each additional while
IE \b1 for the first break, \b2 for the second and so on

I understand the advantage of generating unique break variable names for each loop.
But I don't understand which substring substitution would allow to do that.
Could you elaborate?

[T3RRY]

Use substring substitution to replace the break variable for each additional while
IE \b1 for the first break, \b2 for the second and so on

I understand the advantage of generating unique break variable names for each loop.
But I don't understand which substring substitution would allow to do that.
Could you elaborate?

It would have to be done at the time the component while end repeat etc Macros are defined into the macro end product, for each set of component variables.
Additionally, the "Base" variable must be a unique substring within the While and associated macros. I tend to favor the # character for such uses

IE

Code: Select all

%While:#=\b1%
%EndWhile:#=\b1%

%While:#=\b2%
%EndWhile:#=\b2%

Or as per your use example:

Code: Select all

@echo off

set REP16X=for /l %%- in (0,1,15) do if not defined #
set REP4GX=%REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X% %REP16X%

set "NOP=(if 1==0 .)"
set WHILE(=(set "#=" ^& (%REP4GX% ( if
set )DO=(%NOP%) else set "#=-") ^& (if not defined #
set WEND=)))

set REPEAT=(set "#=" ^& (%REP4GX%
set UNTIL(=^& if
set )REND=set "#=-") ^& set "#=")

Setlocal EnableDelayedExpansion

echo :# Test while(repeat(while()))
set "N=0"
%WHILE(:#=b1% not "!N!"=="2" %)DO:#=b1% (
  set "M=0"
  %REPEAT:#=b1% (
    set "L=0"
    %WHILE(:#=b2% not "!L!"=="2" %)DO:#=b2% (
      echo N=!N! M=!M! L=!L!
      set /a "L+=1"
    ) %WEND%
    set /a "M+=1"
  ) %UNTIL(:#=b1% "!M!"=="2" %)REND:#=b1%
  set /a "N+=1"
) %WEND%
Endlocal