need to optimize my batch file [resolved]

[foxidrive]

The actual improvement depends on where in the set of files the string is found.

If the string is found in file 3 of 8068 then the improvement will be greater
than if the string is found in file 8065 of 8068


Do you want to try my suggestion again? I added the /M switch which stops the entire file contents from being processed, even if it is going to NUL. That didn't occur to me.

[zimxavier]

Sorry, i stopped it again. At the end, it treated 445 out of my 1200 strings in 36min.
My original batch treats all of 1200 strings in 18min.
Maybe, the best way is to use my original batch, but with finding a way to add directories (instead of having one folder only)

[foxidrive]

Ok.

Your code can be simplified and the folders specified.

A question I'm curious about:
are you testing Aacini's code and yours and mine with the same files and strings, or does the data change?

Code: Select all

@echo off

set manifest_folder="C:\FOLDER\a";"C:\FOLDER\b"
set "file_list=STRINGS.txt"

   (
     for /f "usebackq delims=" %%a in ("%file_list%") do (
     findstr /l /m /s /c:"%%a" /d:%manifest_folder% *.txt >nul && (
              echo %%a is found
        ) || (echo %%a is not found)
   )) > "IF_STRING_EXISTS.txt"

[zimxavier]

Merciiiiiiiiiiiiiiiiiiiiiiiii foxidrive
It works and it is incredibly fast now !

my batch from internet (1200 strings ; 8068 txt files): 18min05s (124 'is not found')
my batch from internet edited by foxidrive (1200 strings ; 8068 txt files): 18min04s (124 'is not found')
my batch from internet edited by foxidrive (1200 strings ; 2659 txt files): 3min04s (124 'is not found')


And no, no change between my tests, except when i said 8068 text files (one folder) or 2659 text files (4 subdirectories). I'm not able to explain the enormous time differences between the batchs.

(I edited my first post)

[foxidrive]

my batch from internet (1200 strings ; 8068 txt files): 18min05s (124 'is not found')
my batch from internet edited by foxidrive (1200 strings ; 8068 txt files): 18min04s (124 'is not found')
my batch from internet edited by foxidrive (1200 strings ; 2659 txt files): 3min04s (124 'is not found')

I'm glad it's improved so much for you.
If I interpret those results correctly, it seems the improvement is really because of the reduced number of files being processed, as you can specify only the folders you need.

I think the enormous time difference was due to the large number of files - and the way my script handles each file individually, rather than findstr itself processing a whole folder.

[Squashman]

Amazing. The /D option works.