These conditional tests take far to long - please help me

[alan_b]

This code is to inspect double extensions and it takes over 9 seconds to consider 31 files.
It gets choked after the first dozen.

It takes over 20 seconds to rename 31 files with corrected double extensions, but that is no problem.
I just want it to validate 31 files and take near zero time when nothing needs fixing.

All the files start life with single extension names in the format
???????????????????-00-00.mrimg through to
???????????????????-31-31.mrimg

I wish to rename
???????????????????-00-01.mrimg through to
???????????????????-31-31.inc.mrimg

but to preserve without renaming
???????????????????-00-00.mrimg the initial file
???????????????????-??-??.inc.mrimg previously renamed files
???????????????????-??-??.DIF.mrimg externally renamed files

This is the code to improve

Code: Select all

FOR %%X IN (E*.mrimg) DO ECHO %%X | FIND /I "INC.MriMg" > NUL && (ECHO retain  %%X) || (
  ECHO %%X | FIND /I "dec.MriMg" > NUL && (ECHO retain  %%X) || (
  ECHO %%X | FIND /I "-00-00.MriMg" > NUL && (ECHO Retain FULL %%X) || (
  ECHO Renaming %%X as *.inc.mrimg
  REN %%X *.inc.mrimg
) ) )

Below for your convenience is my complete test code which can be run from the command prompt,
and in the first run it simply creates the 31 files with the single extension in much less than 1 second

When run the second time it renames all excepting the *-00-00.mrimg in over 20 seconds, but I can live with that.

When run the third and subsequent times it takes 9.050 seconds to decide all 31 files need no fixing.

When run with the argument # it deletes all the E*.mrimg files so it can start afresh with 31 single extension files

I developed the timing measurement code when I used XP and before my Win 7 battles fried my brain ! !

Code: Select all

ECHO OFF
ECHO %TIME%
IF %1#==## DEL E*.MRIMG
FOR /F "tokens=1-4 delims=:." %%d in ("%TIME%") do (
  SET /A T1=%%d*360000+1%%e*6000+1%%f*100+1%%g-610100
)
FOR %%X IN (E*.mrimg) DO ECHO %%X | FIND /I "INC.MriMg" > NUL && (ECHO retain  %%X) || (
  ECHO %%X | FIND /I "dec.MriMg" > NUL && (ECHO retain  %%X) || (
  ECHO %%X | FIND /I "-00-00.MriMg" > NUL && (ECHO Retain FULL %%X) || (
  ECHO Renaming %%X as *.inc.mrimg
  REN %%X *.inc.mrimg
) ) )
echo %TIME%
FOR /F "tokens=1-4 delims=:." %%d in ("%TIME%") do (
  SET /A T2=%%d*360000+1%%e*6000+1%%f*100+1%%g-610100-%T1%
)
ECHO DURATION = %T2%0 mSec

IF EXIST E*.mrimg GOTO :EOF
echo creating
FOR /L %%X IN (0,1,9) DO ECHO %TIME% > E4F3BDD38CC36FC5-0%%X-0%%X.mrimg
FOR /L %%X IN (10,1,30) DO ECHO %TIME% > E4F3BDD38CC36FC5-%%X-%%X.mrimg
DIR E*.mrimg
ECHO %TIME%
GOTO :EOF

NB I never got the hang of VBS and think it is too late for me now ! !

Regards
Alan

[Ed Dyreen]

You know that's not true, unless your 80years old. And even then, it's always good to know

NB I never got the hang of VBS and think it is too late for me now ! !

When you use find you should be aware of the fact that this takes time ! It is an external command.
Often I can avoid it with the set command. I think this can work for you . you do know these command don't you ?

Code: Select all

set "MEM=Hello there"
set "?=!mem:hello =!"
if /i ["!mem!"] neq ["!?!"] then we know there is an hello in mem.
set "mem=:~7"
or
set "mem=!mem:*llo=!"

A set if combination is way faster than an external command, but it can't replace it, only sometimes.

also these are 2 writes

Code: Select all

> file  echo.hello
>>file echo.there

And this is 1 write

Code: Select all

> file (
   echo.hello
   echo.there
)

and looks better.

Other than that I don't see much room for improvement though,
You may try creating a macro out of it, but it will have little effect and is hard to understand.

I looked closely at some find commands and yes, they can be replaced by a set if construction.
As you only are interested in a true or false condition &don't need any metacharacters with find.

But I'm sure the next post is gonna be from an expert, making it really fast

[alan_b]

Thanks

You know that's not true, unless your 80years old.

I have just turned the corner and am on the final run.

I think my CPU has enough horsepower to cope with the speed of FIND and everything else in my code because it does 13 files in 380 mSec,
but then it chokes and does nothing for perhaps 4 seconds.
And 31 files takes just over 9 seconds after several fast spurts and long intervals.

A Directory listing of the first 13 files use 1,132 bytes bytes
3 TIMES I use ECHO %%X | FIND

Is there a 4 kByte buffer that needs slowly flushing after each 13 files are processed ?
I have a 64 bit Desktop with 4 GB RAM so it seems unlikely,
but unlikely seems to be Windows 7 speciality ! ! !


I remember the good old days of DOS 3.31 when Billy Gates Bloat was too small to use all of a 1 MByte RAM card,
and we could use either Config.sys or Autoexec.bat to establish a RamDisk and to copy all the Batch files from C:\Bat\* to the RamDisk.
The copies in the RamDisk ran smoking hot with jet aeroplane Afterburner speed.
Is there a RamDisk I could use under Windows 7 ?


You have made me think again about using the command "ECHO %%F | FIND" and only using once instead of 3 times,
PERHAPS then CMD.EXE can handle 3 times as many files before it takes a "comfort break" (I love that euphemism).

I will try experimenting with

Code: Select all

FOR %%X IN (E*.mrimg) DO FOR /F "tokens=1-4 delims=." %%d in ("%%X") do (
    SET DOUBLE_EXTENSIONS=HELP - I HAVE TO THINK ABOUT THIS BIT - THIS WILL BE PAINFUL ! ! !
    IF %DOUBLE_ EXTENSION% GOTO :EOF
    ECHO %%X | FIND /I "-00-00.MriMg" > NUL && (ECHO Retain FULL %%X) || (
      ECHO Renaming %%X as *.inc.mrimg
      REN %%X *.inc.mrimg
  )
)

Regards
Alan

[dbenham]

FINDSTR regular expressions to the rescue!

I think the following will give you what you want. It ignores *-00-00.mrimg and *.*.mrimg

Code: Select all

for /f %%f in (
  'dir /b E*.mrimg ^| findstr /irv /c:"\..*\.mrimg$" /c:"-00-00\.mrimg$"'
) do ren %%f *.inc.mrimg

If it is not perfect, you should be able to modify the regular expressions to suit your needs - Perhaps piping to FINDSTR twice, once for match expression inclusions, and again for match expression exclusions

Dave Benham

[alan_b]

Many thanks

That is a perfect solution.
I increased the range to an upper limit of 99 to process up to 100 files.

It now takes only 250 mSec to skip the *-00-00 and convert to double extension the other 99 single extension files,
and after conversion the next time it runs it takes only 70 mSec to see that nothing needs converting.

N.B. I am now going to test this on my Macrium Disc Partition image backups, which are created as *-nn-nn.mrimg files.
nn is 00 for the first Full image, and increments upwards for each Differential or Incremental backup file.
I create a small Incremental each night, and a less small Differential each weekend,
and I can more easily use Windows Explorer to occasionally prune the older Incrementals if a prepended "inc" distinguishes them.

I think after 3 months it will be time to start a new FULL image at *-00-00 ! ! !

Regards
Alan