Page 1 of 1

Batch File performance - For Vs. Call

Posted: 09 Aug 2011 07:31
by Acy Forsythe
So while I'm waiting on feedback for my latest script I decided to go clean up some of my older scripts I still use today, starting with my file-indexing script. I ran into something strange (Strange being relative to my own knowledge and what I've learned on this site recently).

I changed this code:

Code: Select all

FOR /F "tokens=5 delims= " %%A IN ('DIR /ON ^|FIND /V "DIR" ^|FIND "%FILEDATE%"') DO ( CALL :MoveFiles %%A)

....
....

:MoveFiles
ECHO Copying File %1
COPY /Y %1 "%WORKDIR%\%NEWDIR%\%1"
IF EXIST "%WORKDIR%\%NEWDIR%\%1" DEL /F /Q "%WORKDIR%\%1"
ECHO File %1 Moved to %WORKDIR%\%NEWDIR%\
GOTO:EOF



To this code:

Code: Select all


FOR /F "tokens=5 delims= " %%A IN ('DIR /ON ^|FIND /V "DIR" ^|FIND "%FILEDATE%"') DO (

    COPY /Y "%WORKDIR%\%%A" "%WORKDIR%\%NEWDIR%\%%A"
    IF EXIST "%WORKDIR%\%NEWDIR%\%%A" DEL /F /Q "%WORKDIR%\%%A"
)


Looks more efficient right? I thought it did, in fact if I had written that bit of script yesterday, that's exactly how I would have written it first! The thought of calling a label 2000+ times would have been right out the window.

Here's the strange bit... That second bit of code is 3x slower than the first bit.

Re: Batch File performance - For Vs. Call

Posted: 10 Aug 2011 05:48
by dbenham
My jaw dropped when I first read this. But I think I have an explanation.

I think you are seeing differences in the time it takes to execute your identical IN() clause, not your modified DO clause. You would think since both versions have identical IN() clauses, the timing for that portion would be consistent. But on my home machine I see extreme variability in the amount of time it takes to launch a new CMD.EXE. It is also quite slow on this machine. On my work machine it is much faster and more consistent. I have never been able to diagnose why my home machine works this way.

I think your IN() clause launches CMD.EXE 3 times - once for the overall command plus once for each pipe.

Try timing either version many times and see how much variability you get.

The DO portion of the 2nd version should be faster than the 1st version. But the difference may pale in comparison to the variability of the timing of the IN() clause.

Also - If you are working on a shared network drive, then the timing of the disk access for your DO clause could also vary significantly depending on the current traffic. I see this in my work environment. Your DO clause (either form) is all about disk access.

Dave Benham

Re: Batch File performance - For Vs. Call

Posted: 10 Aug 2011 10:04
by Acy Forsythe
I thought about that too, and while I only ran it 6 times both ways, the 1st way is consistently 30-50 seconds for 1700 test files and the 2nd version is consistently 90-120 seconds for the same files. It's wierd. I will change my IN() though just to be more efficient. And I still like the look of the all-inclusive FOR loop vs. the FOR / CALL, but I'm stumped on the performance issues.

Re: Batch File performance - For Vs. Call

Posted: 10 Aug 2011 11:31
by dbenham
Is %WORKDIR% set to the current directory :?:

If not then the two scripts are not equivalent :!:

COPY /Y %1 "%WORKDIR%\%NEWDIR%\%1"
vs.
COPY /Y "%WORKDIR%\%%A" "%WORKDIR%\%NEWDIR%\%%A"


Dave Benham

Re: Batch File performance - For Vs. Call

Posted: 10 Aug 2011 11:41
by Acy Forsythe
I was wrong, you were right. Instead of switching back and forth and timing, I wrote another script to undo the copy and ran it back to back to back 10 times and the time really was fluctuating by as much as 60+ seconds.

I switched it and ran it again, and saw that it still fluctuated by 60+ seconds, but on average it was 8 seconds faster.

And %WORKDIR% is current dir, I was just being careful in my re-write just in case it ever isn't current dir.

Anyway Whew! I'm sane again.

Re: Batch File performance - For Vs. Call

Posted: 10 Aug 2011 13:27
by dbenham
Acy Forsyte wrote:Anyway Whew! I'm sane again.
That makes two of us - I was beginning to worry about your results. :D

Acy Forsythe wrote:And %WORKDIR% is current dir, I was just being careful in my re-write just in case it ever isn't current dir.
You might want to modify your IN() clause to include %WORKDIR% as well :wink:

Dave Benham

Re: Batch File performance - For Vs. Call

Posted: 10 Aug 2011 14:20
by Acy Forsythe
I used /A-D to get rid of one of the pipes also.