CALL me, or better avoid call

[jeb]

Hi,

call or not to call was the question.
Sometimes there are tips to use call to reparse a line a second time, usefull for pseudo-array access or inside of loops/brackets.

Code: Select all

for /L %%n in (1,1,10) DO (
  set cnt=%%n
  echo %cnt% fails
  call echo %%cnt%% works
  set array[%%n]=Content%%n
  call echo %%array[%%n]%%
)

Obviously the same problem can be solved with delayed expansion,
but are there differences?

Yes, many.
-call echo can produce unexpected results

Code: Select all

call echo ;=^/,,^==^ ?hasta? la Vista

Outputs the help screen of the call command

- call can crash the cmd-window

Code: Select all

set "caret=^"
call echo Vista crash %%caret%%

- call is extremely slow

Code: Select all

setlocal EnableDelayedExpansion
set "x=1"
set start=%time%
for /L %%n in (1,1,1000) DO (
   rem
   CALL set var=!x!
)
set end=%time%

Masured on my system:
without call: 70ms
with one call: 3690ms
with two calls: 7330ms
with three calls: 11270ms

Perhaps it is slow, as it always creates a new variable context and has to copy all variables.
I tested this with a creation of a big bunch of variables (100*5000byte=500kB).
without call: 20ms
with one call: 5420ms
with one call, but yMAX=1: 3720ms

Code: Select all

setlocal EnableDelayedExpansion
set "y="
set yMAX=100
for /L %%n in (1,1,5000) do set "y=!y!#"
for /L %%n in (1,1,%yMAX%) do (
   set array%%n=!y!
)
set "x=1"
echo start
set start=%time%

for /L %%n in (1,1,1000) DO (
   rem
   call rem 
)
set end=%time%
echo end
call :bl.DateTime.timeDiff res "%start%" "%end%"
echo %res%

Result: Try to avoid call, better use the delayed expansion

jeb

EDIT: I append the real cause and the problematic side effects of call in another post in this thread
CALL me, or better avoid call#16

[Ed Dyreen]

Thanx Jeb, I appreciate your research

Why does everyone seems to be interested in speed lately, we used to be happy if we got it working

[allal]

nice time saving but i don't use delayedexpansion anymore

[Cleptography]

Thanx Jeb, I appreciate your research

Why does everyone seems to be interested in speed lately, we used to be happy if we got it working

Classic.

[orange_batch]

nice time saving but i don't use delayedexpansion anymore

Do you use CALL then, or...?

[allal]

nice time saving but i don't use delayedexpansion anymore

Do you use CALL then, or...?

yes i use call echo + pipe + find or findstr or grep

[dbenham]

I've been looking at this as well and drawn the same conclusion that CALL is to be avoided if possible.

Perhaps it is slow, as it always creates a new variable context and has to copy all variables.

There must be some other mechanism going on. I compared the speed of CALL REM to the speed of SETLOCAL&ENDLOCAL and the latter is nearly 5 times faster. I also tested speed of CALL to a function that is closely following the CALL as well as CALL to a function that is 50K after the CALL. I ran each of the tests with 1000 iterations.

I used my Timer macros that I've previously posted in New Function Template - return ANY string safely and easily:

Code: Select all

@echo off
if not defined macro\load.macro_TimerLib call macr_TimerLib
set n=%~1
if "%n%"=="" set n=1000
%macro_call% ("t1") %macro.getTime%
FOR /L %%N IN (1,1,%N%) DO CALL REM
%macro_call% ("t2") %macro.getTime%
FOR /L %%N IN (1,1,%N%) DO CALL :funcTop
%macro_call% ("t3") %macro.getTime%
FOR /L %%N IN (1,1,%N%) DO CALL :funcBottom
%macro_call% ("t4") %macro.getTime%
FOR /L %%N IN (1,1,%N%) DO SETLOCAL&ENDLOCAL
%macro_call% ("t5") %macro.getTime%
%macro_call% ("t1 t2 timeCallRem") %macro.diffTime%
%macro_call% ("t2 t3 timeCallFuncTop") %macro.diffTime%
%macro_call% ("t3 t4 timeCallFuncBottom") %macro.diffTime%
%macro_call% ("t4 t5 timeSetLocal") %macro.diffTime%
set time
exit /b

:funcTop
exit /b

::BEGIN 50 kilobytes of text
<not shown>
::END of not shown text

:funcBottom
exit /b

Results:

Code: Select all

timeCallFuncBottom=00:00:45.14
timeCallFuncTop=00:00:03.12
timeCallRem=00:00:02.29
timeSetLocal=00:00:00.45

What is CALL doing that is so slow

We can explain the label scan time, but that is not involved with CALL REM. The saving of the environment is only a small portion of the overhead. It must be something else

Addendum: To make the SETLOCAL vs CALL comparison more fair, I changed the SETLOCAL test to SETLOCAL&ENDLOCAL&REM. Same results.

Dave Benham

[jeb]

First I test the call a bit more, creating temporary files

Code: Select all

@echo off
set start=%time%
call :Label1
exit /b
:Label1
call :Label2
exit /b
:Label2
call :Label3
exit /b
:Label3
set end=%time%

And

Code: Select all

@echo off
set start=%time%
CALL REM
CALL REM
CALL REM
set end=%time%

With an iterator of 500 I measure
call :NextLabel 680ms
call REM 1970ms
But if I change the callCMD to call (REM) and a single call I got
call (REM) 130ms
call 120ms

I create this with

Code: Select all

@echo off
setlocal EnableDelayedExpansion
SET iterate=500
echo Creating callTemp.bat ...
(
   (echo @echo off)
   (echo set start=%%time%%)   
   for /L %%n in (1,1,%iterate%) do (
      (echo call :Label%%n)
      (echo exit /b)
      (echo :Label%%n)
   )
   (echo set end=%%time%%)
) > callTemp.bat
echo Ready

call callTemp.bat
rem set end=%time%
call :bl.DateTime.timeDiff res "%start%" "%end%"
echo %res%
set "callCMD=REM"
echo call %callCMD%
(
   (echo @echo off)
   (echo set start=%%time%%)   
   for /L %%n in (1,1,%iterate%) do (
      echo CALL !callCMD!
   )
   (echo set end=%%time%%)   
) > callTemp.bat
call callTemp.bat
call :bl.DateTime.timeDiff res "%start%" "%end%"
echo %res%
exit /b

::###############################
::### WARNING, enclose the time in quotes ", because it can contain comma seperators
:bl.DateTime.timeDiff <resultVar> <start_time> <end_time>
(
  SETLOCAL
  if "%~4" NEQ "" (
    echo ERROR in :bl.DateTime.timeDiff too much parameter
   exit /b 4
  )
  call :bl.DateTime.timeToMs ms_start "%~2"
  call :bl.DateTime.timeToMs ms_end "%~3"
)
set /a diff_ms=ms_end - ms_start
(
  ENDLOCAL
  set %~1=%diff_ms%
  goto :eof
)
::End of function

::###############################
:bl.DateTime.timeToMs <resultVar> <time>
::### WARNING, enclose the time in quotes ", because it can contain comma seperators
::### WARNING it does not convert time in am/pm format, because it's regressive
SETLOCAL
FOR /F "tokens=1,2,3,4 delims=:,.^ " %%a IN ("%~2") DO (
  set /a ms=^(^(^(30%%a%%100^)*60+7%%b^)*60+3%%c-42300^)*1000+^(1%%d0 %% 1000^)
)
(
  ENDLOCAL
  set %~1=%ms%
  goto :eof
)
::End of function

REM **** end of library BATCHLIB.DateTime

Currently I didn't understand it

But now I have found a totally new behaviour

Code: Select all

@echo off
setlocal EnableDelayedExpansion
(
   echo echo off
   echo echo This is "%%0" "%%~0" "%%~f0"
   echo echo    cmdcmdline=%%cmdcmdline%%
   echo echo    %%%%*="%%*" 
   echo echo(
) > 2.bat
copy 2.bat 1.bat > NUL
copy 2.bat 0.bat > NUL
copy 2.bat "+.bat" > NUL
copy 2.bat ",.bat" > NUL
set "amp=&"
set "pipe=|"
call ( echo aaa %%pipe%%%%pipe%% echo bbb0 )
call ( echo aaa %%amp%%%%amp%% echo bbb1 )
call ( echo aaa %%pipe%% echo bbb2 )
call ( echo aaa %%amp%% echo bbb/ )
echo(
call ( for %%%%x in (1,1,4) do echo aaa+   )
call ( IF a==a echo aaa,   )

Output:

Code: Select all

This is "0" "0" "C:\temp\0.bat"
        cmdcmdline="C:\Windows\system32\cmd.exe"
        %*=""

This is "1" "1" "C:\temp\1.bat"
        cmdcmdline="C:\Windows\system32\cmd.exe"
        %*=""

This is "2" "2" "C:\temp\2.bat"
        cmdcmdline="C:\Windows\system32\cmd.exe"
        %*=""

Der Befehl "/" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

This is "+" "+" "C:\temp\+.bat"
        cmdcmdline="C:\Windows\system32\cmd.exe"
        %*=""

This is "," "," "C:\temp\,.bat"
        cmdcmdline="C:\Windows\system32\cmd.exe"
        %*=""

The call command starts a file "0", "1", "2", "+" or "," depending of using a "||", "&&", "|", "FOR-Loop", "IF-statement".
A little bit curious

jeb

[allal]

please does anyone knows how to make findstr command faster and finds result in less time because last time i was not able to find all file containing a specific string after at least 10 minutes and it was horribly too slow while printing the result on screen.

i also want the same thing for the command dir if exist some possible tricks to make it faster this would very appreciated.

i don't have any problem for var expansion speed whether it's the call or delayedexpansion is used.

cheer

[dbenham]

With an iterator of 500 I measure
call :NextLabel 680ms
call REM 1970ms
But if I change the callCMD to call (REM) and a single call I got
call (REM) 130ms
call 120ms
...
Currently I didn't understand it

Exactly Why would a CALL requiring a scan for the label be faster then a CALL without a label

Remember that PM I sent you regarding variability of CALL timing? I ran my code from my prior post on my work machine and got the following:

Code: Select all

timeCallFuncBottom=00:00:02.22
timeCallFuncTop=00:00:02.19
timeCallRem=00:00:07.27
timeSetLocal=00:00:00.40

My work machine is generally faster than my home machine except the CALL REM is extremely slow at work. What is the CALL statement doing

Your CALL (REM) is fast because we know the CALL is short circuited and doesn't execute.
But...

The call command starts a file "0", "1", "2", "+" or "," depending of using a "||", "&&", "|", "FOR-Loop", "IF-statement".
A little bit curious

I think this is a bit more than curious

If I modify your code I find that the commands within the call() don't change the result, as long as they are syntactically correct. For example I replace ECHO with DIR.

The code inside the () is not executed, but it is parsed, and the parser attempts to call/execute 0 1 2 / + or , if it encounters "||", "&&", "|", "&", "FOR-loop", "IF-statement". Very odd.

Presumably we can use your discovery to probe the order in which a line is parsed.

Dave Benham

[dbenham]

I found a new one for you: () = 3

I reconfigured your test platform to make it easier to work with:

0.bat

Code: Select all

echo 0=^|^|  "%*"  cmdcmdline=%cmdcmdline%

1.bat

Code: Select all

echo 1=^&^& "%*"  cmdcmdline=%cmdcmdline%

2.bat

Code: Select all

echo 2=^|  "%*"  cmdcmdline=%cmdcmdline%

3.bat

Code: Select all

echo 3=()  "%*"  cmdcmdline=%cmdcmdline%

+.bat

Code: Select all

echo +=FOR  "%*"  cmdcmdline=%cmdcmdline%

,.bat

Code: Select all

echo ,=IF  "%*"  cmdcmdline=%cmdcmdline%

test.bat

Code: Select all

@echo off
setlocal enableDelayedExpansion
cls
for /f "delims=" %%c in (test.txt) do (
  set "cmd=%%c"
  call :test
)
exit /b

:test
  echo:
  echo !cmd!
  set "cmd=!cmd:|=^|!"
  set "cmd=!cmd:&=^&!"
  call ( %cmd% )
exit /b

test.txt

Code: Select all

dir || dir
dir && dir
dir | dir
(dir)
dir & dir
for %%x in (1,1,4) do dir
if 1==1 dir
if defined temp dir
dir || dir && dir | dir
dir && dir || dir | dir
(dir && dir) || dir | dir
dir && (dir || dir | dir)
(dir && dir || dir) | dir
(dir && dir || dir | dir)
dir | (dir && dir || dir)
dir && dir | dir
dir && dir || dir & dir | dir
dir | dir >test.out
<missing.txt dir || dir
for %%n in (*) do dir & dir
if 1==1 (dir) else dir
if 1==1 dir & dir
if 1==1 dir && dir
dir && if 1==1 dir & dir
dir && dir & dir

Results:

Code: Select all

dir || dir
0=||  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && dir
1=&& ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir | dir
2=|  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

(dir)
3=()  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir & dir
'/' is not recognized as an internal or external command,
operable program or batch file.

for %%x in (1,1,4) do dir
+=FOR  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

if 1==1 dir
,=IF  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

if defined temp dir
,=IF  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir || dir && dir | dir
0=||  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && dir || dir | dir
0=||  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

(dir && dir) || dir | dir
0=||  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && (dir || dir | dir)
1=&& ""  cmdcmdline="C:\Windows\System32\cmd.exe"

(dir && dir || dir) | dir
2=|  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

(dir && dir || dir | dir)
3=()  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir | (dir && dir || dir)
2=|  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && dir | dir
1=&& ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && dir || dir & dir | dir
'/' is not recognized as an internal or external command,
operable program or batch file.

dir | dir >test.out

<missing.txt dir || dir
The system cannot find the file specified.

for %%n in (*) do dir & dir
+=FOR  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

if 1==1 (dir) else dir
,=IF  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

if 1==1 dir & dir
,=IF  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

if 1==1 dir && dir
,=IF  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && if 1==1 dir & dir
1=&& ""  cmdcmdline="C:\Windows\System32\cmd.exe"

dir && dir & dir
'/' is not recognized as an internal or external command,
operable program or batch file.

Contents of test.out:

Code: Select all

2=|  ""  cmdcmdline="C:\Windows\System32\cmd.exe"

I didn't realize that redirected input and output was handled so early in the process.

The above certainly demonstrates some precedence rules. I think you can make better use of them then I.

Dave Benham

[jeb]

I found a new one for you: () = 3

You can build it also with
call ( (three) )

And I got these two

Code: Select all

set "help=/?"
set LF=^


call (  rem %%HELP%% )
call ( a %%LF%% creates a dot ) 

The Help creates "<", works also with IF /? and FOR /?, I assume as these three are the only ones, as they are the ones who starts their own parser in the special character phase.

Perhaps this can be used to create characters like <TAB> or <CR>.
In my opinion, we see a token-value for different commands/situations.

jeb

[Ed Dyreen]

Perhaps this can be used to create characters like <TAB> or <CR>.

little bit off topic:
Can we create a backspace ?, I don't think so, you think so jeb anyone ?

[dbenham]

In my opinion, we see a token-value for different commands/situations.

Yes, but in a line with multiple token generators, we only get one output. (I had hoped for multiple). Sometimes the order doesn't matter - we always get the same token - establishing precedence: & > || > && > | What that really means I'm not sure. The visible token can be influenced by ().

Perhaps this can be used to create characters like <TAB> or <CR>.

Are you seeing something, or do you mean if we stumble on the right token

Can we create a backspace ?, I don't think so, you think so jeb anyone ?

Ed - you already know how, you responded to that post Re: Problem with delayed expansion after ECHO. or ECHO:

Dave Benham

[Ed Dyreen]

Ed - you already know how

I now learned that:
we can use <CR> to reprint a line
we can use <BS> to reprint a char

but when I use <esc> I get this:
This←Works
Which is nice but when I press <esc> in the console, it erases the entire line like the carriage.
Wonder why