DosTips.com • Batch performance and set variables

Page 1 of 2

Batch performance and set variables

Posted: 04 Aug 2021 13:21

by atfon

I was reading through an old topic on this forum about performance degradation as it relates to setting environmental variables posted by dbenham.

viewtopic.php?f=3&t=2597

There was some interesting discussion here about how the system behaves when a new variable is set. I was wondering if anyone has any suggestions on how best to improve performance in a script that is meant to capture system information into hundreds of variables and echo them to the console and to file. A very very brief example of the type of thing I am doing for a very basic idea:

Code: Select all

@echo off
setlocal enabledelayedexpansion
for /f "tokens=*" %%e in ('wmic timezone get caption 2^>nul^|findstr /r [a-z]^|findstr /v "Caption"') do for /f "tokens=*" %%f in ("%%e") do set "tZone=%%f"
for /f "tokens=*" %%g in ('whoami 2^>nul') do set "whoAM=%%g"
set "ePath=%userprofile%\sys_info.txt"
cls
echo Time Zone: !tZone!
echo User Name: %whoAM%
(
echo Time Zone: !tZone!
echo User Name: %whoAM%
) > %ePath%
pause > nul

In the years since that post, have there been some additional techniques developed to speed up the process of capturing and later expanding a large set of variables?

Edit: For what it's worth, I did try this with a negligible performance benefit:

Code: Select all

for /F "tokens=1 delims==" %%v in ('set') do if /I not "%%v"=="PATH" set "%%v="

Also, I have looked at viewtopic.php?t=7625

Re: Batch performance and set variables

Posted: 06 Aug 2021 10:22

by atfon

I managed to find a performance improvement by doing a few things. I set the full path to findstr.exe as a variable:

Code: Select all

set "fStr=%systemroot%\System32\findstr.exe"

I then attempted to optimize some of my for /f loops better. I did an echo of the wmic class and then used the type command to retrieve just the information I wanted:

Code: Select all

wmic computersystem get /value >"%tmp%\wmi_comp.txt"
for /f "tokens=2 delims==" %%g in ('type "%tmp%\wmi_comp.txt" ^|%fStr% "Manufacturer"') do set "cMan=%%g"

I reduced the number of times I was calling findstr.exe by more judiciously parsing the data. In some cases, I found wmic was padding some of the data. To avoid that, I used the /Format:csv option with "tokens=2 delims=,"

I'm sure there are lots of other things I could be doing, but hopefully that little information might be helpful to someone. I apologize if my initial post was a little too vague for this forum, but I was looking for more generalities than anything. I have been searching through this site and others to find helpful hints. There's a lot of great information here.

Re: Batch performance and set variables

Posted: 06 Aug 2021 11:12

by Squashman

Hmm.
I would have thought that using a FOR /F to parse the WMIC output would be faster.

In regards to using TYPE, I am almost positive that is slower than using redirection.
This should be quicker.

Code: Select all

%fStr% "Manufacturer"<"%tmp%\wmi_comp.txt"

Re: Batch performance and set variables

Posted: 06 Aug 2021 11:44

by aGerman

capture system information into hundreds of variables

What is the point of doing this? If you query the information, just print it. If you're capturing it in a FOR /F loop for formatting reasons, use the FOR variables directly.

I reduced the number of times I was calling findstr.exe

Batch is getting terribly slow whenever you call external commands in the body of a loop. That is, the OS has to schedule and load new processes for each iteration. That's not what you're showing in your examples though. However, this little script tells you which commands are an internal functionality of cmd.exe and which are external tools. It's often way faster to use internal commands in a loop even if thi makes the script more complicated.

Code: Select all

@echo off &setlocal EnableDelayedExpansion
set /a "int=0,ext=0"
for /f %%i in ('help^|findstr /rbc:"[A-Z][ABCDEFGHIJKLMNOPQRSTUVWXYZ]"') do (
  for /f "tokens=1,2 delims=?" %%j in ("%%i.exe?%%i.com") do (
    if "%%~$PATH:j%%~$PATH:k"=="" (
      set /a "int+=1"
      set "line=%%i                                                  "
      echo !line:~,50! - internal
    ) else (
      set /a "ext+=1"
      echo %%i "%%~$PATH:j%%~$PATH:k"
    )
  )
)

echo(
echo internal %int%
echo external %ext%
pause

I would have thought that using a FOR /F to parse the WMIC output would be faster.

It highly depends on which class you're querying. However, a FOR /F loop is implemented as pipe. It's buffering the whole output before it even begins to iterate through the data.

Steffen

Re: Batch performance and set variables

Posted: 06 Aug 2021 12:51

by atfon

Thanks Squashman and Steffen (aGerman)

I would have thought that using a FOR /F to parse the WMIC output would be faster.

I have tried it both ways. I'd have to do some timing tests, but it seems to be faster to just save the whole class and parse what I need than to issue a new wmic query each time.

... using TYPE, I am almost positive that is slower than using redirection.

I'll have to test that out by creating a version doing redirection and checking the timing.

What is the point of doing this?

I know it slows the script down to echo the data to the console, but I like being able to see the information right away and having the option to save the file from the computer in question.

It's often way faster to use internal commands in a loop...

Thank you for the handy script. That makes sense, but I'm still working on ways to eliminate the use of findstr. I've reduced it quite a bit throughout the script.

Re: Batch performance and set variables

Posted: 06 Aug 2021 13:12

by atfon

I tested with

Code: Select all

%fStr% "Manufacturer"<"%tmp%\wmi_comp.txt"

Oddly, that doesn't work. Ive been testing directly on the command line with various findstr searches to parse just the line with "Manufacturer."

findstr "Manufacturer"
Nothing found

findstr /r /C:Manufacturer
Nothing found

findstr /r [a-z]
All items found.

findstr /r M.*
Various items found.

findstr /r Manufacturer.*
Nothing found

I'm still working through this issue. This is still working, so I'm not sure of the difference:

Code: Select all

type "%tmp%\wmi_comp.txt" | findstr "Manufacturer"

Edit: I figured out the issue was the encoding of the file output from WMIC. It encoded the file as UTF-16 LE BOM according to Notepad + or possibly UCS-2 LE BOM. If I type that to another text file, then I can use the redirection format that Squashman suggested. Adding | find /v "" when outputing the wmic command to file is also a workaround:
https://stackoverflow.com/questions/553 ... text-files

Re: Batch performance and set variables

Posted: 07 Aug 2021 13:21

by Squashman

No problem on my end.

Code: Select all

C:\Users\Squashman\Documents>echo Manufacturer>foo.txt

C:\Users\Squashman\Documents>echo Manufacturers>>foo.txt

C:\Users\Squashman\Documents>findstr "Manufacturers" <foo.txt
Manufacturers

C:\Users\Squashman\Documents>findstr "Manufacturer" <foo.txt
Manufacturer
Manufacturers

Re: Batch performance and set variables

Posted: 07 Aug 2021 16:49

by atfon

Hey Squashman. See my edit above in blue. The issue is with the output of wmic. This is how you can work around that issue:

Code: Select all

wmic computersystem get /value  | find /v "" >"%tmp%\wmi_comp.txt"

Re: Batch performance and set variables

Posted: 09 Aug 2021 09:50

by atfon

One more thing to keep in mind is that the output of wmic is going to contain an additional <CR>. That's left even if you use the | find "" or | more before the redirection to file. One option is to type the contents of the resulting file to another file in order to strip those <CR> blank lines. However, you end up with another extra file to deal with here. Another more elegant solution is to use a for loop to strip out those extra returns. I used this based on a solution posted by foxidrive in 2013 here:

https://stackoverflow.com/questions/194 ... put-format

Code: Select all

setlocal enabledelayedexpansion
for /f "tokens=1,* delims==" %%A in ('wmic computersystem get /value ^|findstr /r [a-z0-9]') do (
set "line=%%A=%%B"
set "line=!line:~0,-1!"
echo !line!
)>> "%tmp%\wmic_compsystem.txt"

Re: Batch performance and set variables

Posted: 09 Aug 2021 10:02

by aGerman

The usual way to remove the CR (without an additional external command tool and without delayed expansion) is a nested FOR /F loop. Something like

Code: Select all

for /f "delims=" %%i in ('WMIC ... whatever ...') do for /f "delims=" %%j in ("%%i") do echo %%j

... with the options (delims, tokens ...) customized in the 2nd loop.

Steffen

Re: Batch performance and set variables

Posted: 09 Aug 2021 10:15

by atfon

Hi Steffen

The funny thing is I have been using that method when I wanted to directly capture a specific variable such as:

Code: Select all

for /f "skip=1 tokens=*" %%l in ('wmic baseboard get serialnumber') do for /f "tokens=1" %%m in ("%%l") do set "baseSerial=%%m"

I didn't even think to apply the same methodology when I wanted to output the entire set of values

but this is more efficient and works like a charm:

Code: Select all

for /f "tokens=*" %%g in ('wmic computersystem get /value') do for /f "tokens=1" %%h in ("%%g") do echo %%h >> "%tmp%\wmic_compsystem.txt"

Thanks again!

Re: Batch performance and set variables

Posted: 10 Aug 2021 08:35

by atfon

I have found that when I use this command:

Code: Select all

for /f "tokens=*" %%g in ('wmic computersystem get /value') do for /f "tokens=1" %%h in ("%%g") do echo %%h >> "%tmp%\wmic_compsystem.txt"

It is adding an extra padded space after each property. For example "SystemType=x64-based PC_" where the underscore represents a space. Is there any way to avoid this extra space?

Re: Batch performance and set variables

Posted: 10 Aug 2021 09:20

by aGerman

echo %%h >>

Look what you're doing here. You're echoing [content of %%h][space].
It's also a good idea to write the redirection first. Refer to my best practice tips.
viewtopic.php?f=3&t=10137

Steffen

Re: Batch performance and set variables

Posted: 10 Aug 2021 09:33

by atfon

So I did. Hopefully, we learn from our mistakes. It seems obvious in retrospect. Thanks again, Steffen. I appreciate the pair of Expert eyes and the link.

Re: Batch performance and set variables

Posted: 23 Aug 2021 12:31

by Aacini

atfon wrote: ↑
10 Aug 2021 08:35
I have found that when I use this command:
Code: Select all
for /f "tokens=*" %%g in ('wmic computersystem get /value') do for /f "tokens=1" %%h in ("%%g") do echo %%h >> "%tmp%\wmic_compsystem.txt"
It is adding an extra padded space after each property. For example "SystemType=x64-based PC_" where the underscore represents a space. Is there any way to avoid this extra space?

You should not use >> (append) redirection, but > standard redirection when possible inside loops. The > is processed just one time, but >> is processed in each iteration.

This also fix the problem of your extra space:

Code: Select all

(for /f "tokens=*" %%g in ('wmic computersystem get /value') do for /f "tokens=1" %%h in ("%%g") do echo %%h) > "%tmp%\wmic_compsystem.txt"

Antonio