Time not changing with variable

[OwainEsau]

Hi;

I'm new to Batch and have created a script for work, whilst learning as i go along. (this is not a very elegant script, but it does the job... more or less)

Anyway the idea of the script is to run a program cvimp.exe on a lost of files (this program doesnt dump memory which is why i need to split it up into batches)

This is the batch file so far:

Code: Select all

@ECHO OFF
SETLOCAL enabledelayedexpansion

CALL:resumeParse

:resumeParse
ECHO _______________________ BULK RESUME PARSING - LOG _______________________ > parserStatistics.txt
ECHO. >> parserStatistics.txt

:: Loops through files in file list and calls parser.bat with the file
FOR /F %%a IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file.txt) do (
   :: Variable declaration
   SET a=%a: =%
   SET file=%%a
   
   SET timeBefore=%time%
   timeout 2

   
   FOR /F %%b IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file_parsed.txt) do (
      SET b=%b: =%
      SET fileC=%%b
      IF !file! == !fileC! (
         ECHO [ERR] This file has already been parsed
         ECHO [ERR] Re-Parsing this will result in duplicate data
         ECHO [ERR] Please remove entry from:   dir_file.txt
         ECHO [ERR] Program will now exit
         ECHO [ERR] Resume parser encountered errors file >> parserStatistics.txt
         timeout 10 && EXIT
      )
   )
   :: Call cv importer and wait for end
   ::START /WAIT parser.bat !file!
   
   ECHO [*] Calling cvimporter on file: !file! :: !timeBefore!
   CALL:loggingBefore !timeBefore! !file!
   
   SET timeAfter=%time%

   CALL:loggingAfter !timeAfter! !file!

   ECHO.

   ECHO [+] Cvimporter completed successfully on !file! :: !timeAfter!
   ECHO.
   ECHO [*] Removing file from file list
   TYPE dir_file.txt | findstr !file! >> dir_file_parsed.txt
   
   ECHO.
   ECHO [*] Proceeding to next file
)

ECHO [+] Resume Parser Script ran without errors >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO [+] Resume Parser Script ran without errors

CALL:logInput
EXIT /B

:LogInput
set /p logOpen="Would you like to open the log file (Y / N)? "

If /I "%logOpen%"=="y" (
   start /WAIT notepad.exe parserStatistics.txt
   EXIT
) 
If /I "%logOpen%"=="n" (
   ECHO.
   ECHO [*] Script will exit in 5 seconds
   timeout 5 && exit
) else (
   ECHO.
   ECHO Invalid Input
   CALL:logInput
)
EXIT /B


:loggingBefore
ECHO. >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO   !file!    ::    !timeBefore! - !timeAfter! >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO ######################################################################### >> parserStatistics.txt
ECHO. >> parserStatistics.txt

ECHO. >> parserStatistics.txt
ECHO Parser starting time: !timeBefore! >> parserStatistics.txt
::systeminfo | find "Available Physical Memory"   >> parserStatistics.txt
ECHO. >> parserStatistics.txt
EXIT /B


:loggingAfter
ECHO. >> parserStatistics.txt
ECHO Parser ending time: !timeAfter! >> parserStatistics.txt
::systeminfo | find "Available Physical Memory"   >> parserStatistics.txt
ECHO. >> parserStatistics.txt
EXIT /B

with the two time variables:

SET timeBefore=%time%
SET timeAfter=%time%

The time is the same for every entry in the generated log file.

I was also wanting to have the script pause for 30 minutes or so if the available memory got too low, but i cant figure out how to put the following command into a variable:

systeminfo | find "Available Physical Memory"

Any help on this would be much appreciated!

[penpen]

SET timeBefore=%time%
SET timeAfter=%time%

The time is the same for every entry in the generated log file.

You execute both in a compound statement (also known as block).
In batch a block the percentage expansion is done for the whole block, before executing any statement.
So "%time%" always will be replaced with the same value.

You could use delayed expansion, that is expanded prior execution of the single command, to work araound that issue:
Just use "!time!".

I was also wanting to have the script pause for 30 minutes or so if the available memory got too low, but i cant figure out how to put the following command into a variable:

systeminfo | find "Available Physical Memory"

This topic may help you:
http://www.dostips.com/forum/viewtopic.php?t=5245

penpen

[OwainEsau]

Hi;

thanks for the reply, the time works well now. Im still having issues figuring you the available memory using the link you provided.

When i use the following code:

Code: Select all

@echo off
for /f "tokens=1,2* delims=: " %%i in (
  'systeminfo ^|  findstr /B /C:"OS Name" /C:"Available Physical Memory"'
) do set "%%i_%%j=%%k"
echo %OS_Name%
echo %Available_Physical_Memory%

the OS_Name comes through fine but the available memory doesnt, just says ECHO IS OFF.

[Squashman]

Hi;

thanks for the reply, the time works well now. Im still having issues figuring you the available memory using the link you provided.

When i use the following code:

Code: Select all

@echo off
for /f "tokens=1,2* delims=: " %%i in (
  'systeminfo ^|  findstr /B /C:"OS Name" /C:"Available Physical Memory"'
) do set "%%i_%%j=%%k"
echo %OS_Name%
echo %Available_Physical_Memory%

the OS_Name comes through fine but the available memory doesnt, just says ECHO IS OFF.

You can't use the same code to get both because one has three tokens and the other has 4.

[ShadowThief]

Hi;

thanks for the reply, the time works well now. Im still having issues figuring you the available memory using the link you provided.

When i use the following code:

Code: Select all

@echo off
for /f "tokens=1,2* delims=: " %%i in (
  'systeminfo ^|  findstr /B /C:"OS Name" /C:"Available Physical Memory"'
) do set "%%i_%%j=%%k"
echo %OS_Name%
echo %Available_Physical_Memory%

the OS_Name comes through fine but the available memory doesnt, just says ECHO IS OFF.

That's because "Available Physical Memory" is three words and you made a variable out of the first two. I think you'll find that the variable %Available_Physical% has the value Memory: something MB

[OwainEsau]

Hi;

Ok so ive managed to get this far:

Code: Select all

:checkMemory
CLS
ECHO [*] Running analysis on current memory usage ... Please wait

:: Create variable for current available memory
for /f "tokens=4-5 delims=: " %%i in ('systeminfo ^|  findstr /B /C:"Available Physical Memory"') do ( 
   set availMemory=%%i
   echo !availMemory!
)
pause

EXIT /B

which does output just the value, however it has a comma.
I am unable to remove the comma using the same method i did for the extra spaces at the end of the file names

SET a=%a: =%

SET availMemory=%i:,=%

when i do this it just outputs :,=

[Aacini]

Code: Select all

@echo off
setlocal EnableDelayedExpansion

for /F "tokens=1,2 delims=:" %%i in (
   'systeminfo ^| findstr /B /C:"OS Name" /C:"Available Physical Memory"'
) do (
   set "name=%%i"
   for /F "tokens=*" %%k in ("%%j") do set "!name: =_!=%%k"
)
echo %OS_Name%
echo %Available_Physical_Memory%

Antonio

[OwainEsau]

Thanks for that Aacini. I did get that part of the script working, and i have managed to pull the comma out of the value by putting it outside the loop.

Here is my script so far (i split it into two batch files so i can have it check memory as the main batch file is waiting on the program to finish)


main.bat

Code: Select all

@ECHO OFF
SETLOCAL enabledelayedexpansion

CALL:resumeParse

:resumeParse
ECHO _______________________ BULK RESUME PARSING - LOG _______________________ > parserStatistics.txt
ECHO. >> parserStatistics.txt

:: Loops through files in file list and calls parser.bat with the file
FOR /F %%a IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file.txt) do (
   :: Variable declaration
   SET a=%a: =%
   SET file=%%a
   SET timeBefore=!time!
   pause
   
   FOR /F %%b IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file_parsed.txt) do (
      SET b=%b: =%
      SET fileC=%%b
      IF !file! == !fileC! (
         ECHO [ERR] This file has already been parsed
         ECHO [ERR] Re-Parsing this will result in duplicate date
         ECHO [ERR] Please remove !fileC! from:   dir_file.txt
         ECHO [ERR] Program will now exit
         ECHO [ERR] Resume parser encountered errors file >> parserStatistics.txt
         timeout 10 && EXIT
      )
   )
   :: Call cv importer and wait for end
   ::START /WAIT parser.bat !file!
   
   ECHO [*] Calling cvimporter on file: !file! :: !timeBefore!
   CALL:loggingBefore !timeBefore! !file!
   
   SET timeAfter=!time!

   CALL:loggingAfter !timeAfter! !file!

   ECHO.

   ECHO [+] Cvimporter completed successfully on !file! :: !timeAfter!
   ECHO.
   ECHO [*] Removing file from file list
   TYPE dir_file.txt | findstr !file! >> dir_file_parsed.txt
   
   ECHO.
   ECHO [*] Proceeding to next file
)

ECHO [+] Resume Parser Script ran without errors >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO [+] Resume Parser Script ran without errors

CALL:logInput
EXIT /B

:LogInput
set /p logOpen="Would you like to open the log file (Y / N)? "

If /I "%logOpen%"=="y" (
   start /WAIT notepad.exe parserStatistics.txt
   EXIT
) 
If /I "%logOpen%"=="n" (
   ECHO.
   ECHO [*] Script will exit in 5 seconds
   timeout 5 && exit
) else (
   ECHO.
   ECHO Invalid Input
   CALL:logInput
)
EXIT /B

:loggingBefore
ECHO. >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO   !file!    :: >> parserStatistics.txt
ECHO. >> parserStatistics.txt
ECHO ######################################################################### >> parserStatistics.txt
ECHO. >> parserStatistics.txt

ECHO. >> parserStatistics.txt
ECHO Parser starting time: !timeBefore! >> parserStatistics.txt
::systeminfo | find "Available Physical Memory"   >> parserStatistics.txt
ECHO. >> parserStatistics.txt
EXIT /B


:loggingAfter
ECHO. >> parserStatistics.txt
ECHO Parser ending time: !timeAfter! >> parserStatistics.txt
::systeminfo | find "Available Physical Memory"   >> parserStatistics.txt
ECHO. >> parserStatistics.txt
EXIT /B

checkMemory.bat

Code: Select all

@ECHO OFF
SETLOCAL enabledelayedexpansion

:A
:: Create variable for current available memory
for /f "tokens=4-5 delims=: " %%i in ('systeminfo ^|  findstr /B /C:"Available Physical Memory"') do ( 
   set availMemory=%%i
   echo !availMemory!
)
set availMemory=%availMemory:,=%
IF !availMemory! LSS 500 (
   ECHO ######################### CRITICAL ERROR #########################
   ECHO Available Memory is too low :: !availMemory!
   ECHO.
   ECHO [*] Killing process cvimp.exe
   
   TASKKILL /IM CVImp.exe /F

   ECHO [+] Process will resume when available memory is greater than 500
   
   pause
   CALL:A
)
timeout 30
CALL:A
EXIT /B

Theres only one more thing that i need before i can run this, which is the error checking on the resumeParser:

Code: Select all

   FOR /F %%b IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file_parsed.txt) do (
      SET b=%b: =%
      SET fileC=%%b
      IF !file! == !fileC! (
         ECHO [ERR] This file has already been parsed
         ECHO [ERR] Re-Parsing this will result in duplicate date
         ECHO [ERR] Please remove !fileC! from:   dir_file.txt
         ECHO [ERR] Program will now exit
         ECHO [ERR] Resume parser encountered errors file >> parserStatistics.txt
         timeout 10 && EXIT
      )
   )

The problem with it is is that when this is run live there may be up to 800,000 records being run in batches of 5 - 10k lists. If the memory does fall to low and the CVIMP process is killed it might have already processed hundreds of files, in which case someone (me) would have to run this script hundreds of times and manually pull out each entry from the list.

Is there a way to actually remove the line from the text file rather than have it copied to a new file? So i can just have an IF statement at the start of the script that either does or doesnt create a new file list.

[penpen]

As i actually noticed it, you should use "tokens=* delims=", when reading complete lines out of files:

Code: Select all

FOR /F "tokens=* delims=" %%a IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file.txt) do (
(...)
   FOR /F "tokens=* delims=" %%b IN (C:\Users\owain.esau\Desktop\BulkResumeParser\dir_file_parsed.txt) do (

Reason:
If you don't use this, then the default delimiters (space and horitzontal tabulator) are used, so a line containing
"C:\dir with space\or file with space" are crippled to "C:\dir" (all without doublequotes).

In addition for a similar reason you should compare filenames using doublequotes:

Code: Select all

 IF "%%~a" == "%%~b" (

I'm not sure if i understand your goal right:
Maybe in your case you should not kill your batch file using taskkill; using some kind of signaling, so that the batch exits iself in a controlled way may be more usefull (and don't use parallel processing using start):
You could create a file "terminate" (in the home directory of the batch/any other "well known" (to your the batch files) directory);
Check its existance after processing each file, so it is ensured, that parsing is completely done for each file, and if this file exists delete it and exit the batch using the "exit" command.

When resuming, you then could just ignore files that have been parsed.


penpen