Extracting required columns from the file

[foxidrive]

Taking this task in steps, this code will strip out any log items where the task is STOPPED.

It uses a helper batch file called `findrepl.bat` - download from: https://www.dropbox.com/s/rfdldmcb6vwi9xc/findrepl.bat
Place `findrepl.bat` in the same folder as the batch file or on the path.

Code: Select all

findrepl /V "^REPLICAT.*STOPPED" /e:"^$" <file.log >newfile.log

You mention that it should check for lag and checkpoint lag - I can see the checkpoint lag but I don't see a term for a different lag.


findrepl can search further in the log and is very quick for large files, where plain batch is quite slow, if you can outline a bit more of the items you need to check. I can see that checkpoint lag seems to be listed in hr:min:sec so do you need to check if the hour figure exceeds 5:59:59 ?

[bipul049]

Hello Foxidrive,

Thanks for your prompt response.
I will put in here what i am looking for through this implementation.

1. If you see here, 3rd line which is 00:00:00 is normal lag(this can't exceed more than 06:00:00) and then the one inside () is checkpoint lag i.e. 00:00:09 here, cant exceed more than 00:10:00

Code: Select all

REPLICAT   ABCDEF  Last Started 2014-04-28 13:05   Status STOPPED
Description          My description here for the process
Checkpoint Lag       00:00:00 (updated 00:00:09 ago)
Log Read Checkpoint  File ./dirdat/abc002345
                     First Record  RBA 443058

2. If any one lag timing goes beyond the particular threshold,I am writing it to an output file.
3. Using powershell, I am checking if my output file contains anything. If yes, then sending mail otherwise no.
4. Now while checking lag i need to check the status of process also. If its STOPPED, i don't want anything to be checked(and hence no output to the output file and hence no mail alert). If process is not stopped then lag and checkpoint lag can be checked and then normal process.

Let me know what modification to the current code I should be doing to implement this.

[foxidrive]

Do you only want a yes or no response if any section that is not STOPPED exceeds either of the two lag figures?
Or do you also want the log entries that contain the lag figures that trigger the email?

[bipul049]

Foxidrive,

Thanks for trying to help me out.
I am very new to batch scripting. If you can tell me where to fit in this code of findrepl in my current code which is running without any issues(the first one I posted today). This code only needs to be modified for status checking and it will work flawlessly.
Please let me know if I can modify my code to get this done without using findrepl.

My output file will not contain yes or no. If process is not STOPPED, my output file will contain whole text like:" Replicat ABCDEF have lag of xx:xx:xx" and so on.

[foxidrive]

EDIT1

Change this line:

Code: Select all

for /f "tokens=1-5" %%a in ('findstr /R /C:"^REPLICAT .*" /C:"^Checkpoint Lag .*" "%INPUT_FILE%"') do (

to this:

Code: Select all

for /f "tokens=1-5" %%a in ('type "%INPUT_FILE%" ^| findrepl /V "^REPLICAT.*STOPPED" /e:"^$" ^| findstr /R /C:"^REPLICAT .*" /C:"^Checkpoint Lag .*" ') do (

and it does use findrepl.

[foxidrive]

This might also work without findrepl but the service names cannot have spaces in them.

EDIT2: corrects false match and calculation error

Code: Select all

@echo off
setlocal enableDelayedExpansion

set "INPUT_FILE=input.txt"
set "OUTPUT_FILE=output.txt"
set lag_limit=21600
set checkpoint_limit=600
(
   for /f "tokens=1-8" %%a in ('findstr /R /C:"^REPLICAT .*" /C:"^Checkpoint Lag .*" "%INPUT_FILE%"') do (
   
      if "%%a" == "REPLICAT" (
                 if "%%h"=="STOPPED" (set skip=1) else (set skip=)
      )

      if not defined skip (

      if "%%a" == "REPLICAT" (
         set "NAME=%%b"
      ) else (
          if "%%a" == "Checkpoint" ( 
              for /f "tokens=1-3 delims=:" %%f in ("%%c") do (
                 set /a "SECONDS=((1%%f-100)*3600)+((1%%g-100)*60)+1%%h-100"
              )
              for /f "tokens=1-3 delims=:" %%i in ("%%e") do (
                 set /a "CHECKPOINTLAG=((1%%i-100)*3600)+((1%%j-100)*60)-100+1%%k-100"
              )
              if !SECONDS! GTR !lag_limit! (
                 echo REPLICAT !NAME! have lag of: %%c.
              )
              if !CHECKPOINTLAG! GTR !checkpoint_limit! (
                 echo REPLICAT !NAME! have Checkpoint lag of: %%e.
              )
          )
      ) 

      )
   )   
) > %OUTPUT_FILE%
endlocal
goto:eof

Output:

Code: Select all

REPLICAT ABCDEF have lag of: 07:00:00.
REPLICAT ABCDEF have Checkpoint lag of: 00:12:01.
REPLICAT ABCPQR have Checkpoint lag of: 00:30:11.

Older code: Edited to try and handle each record

Code: Select all

@echo off
setlocal enableDelayedExpansion

set "INPUT_FILE=input.txt"
set "OUTPUT_FILE=output.txt"
set lag_limit=21600
set checkpoint_limit=600
(
   for /f "tokens=1-8" %%a in ('findstr /R /C:"^REPLICAT .*" /C:"^Checkpoint Lag .*" "%INPUT_FILE%"') do (
   
      if "%%a" == "REPLICAT" (
                 if "%%h"=="STOPPED" (set skip=1) else (set skip=)
      )

      if not defined skip (

      if "%%a" == "REPLICAT" (
         set "NAME=%%b"
      ) else (
         for /f "tokens=1-3 delims=:" %%f in ("%%c") do (
            set /a "SECONDS=(%%f*3600)+(%%g*60)+%%h"
         )
    for /f "tokens=1-3 delims=:" %%i in ("%%e") do (
            set /a "CHECKPOINTLAG=(%%i*3600)+(%%j*60)+%%k"
         )
     if !SECONDS! GTR !lag_limit! (
       echo REPLICAT !NAME! have lag of: %%c.
     )
          if !CHECKPOINTLAG! GTR !checkpoint_limit! (
      echo REPLICAT !NAME! have Checkpoint lag of: %%e.
     )
   
      )
   
   )
 )   
) > %OUTPUT_FILE%
endlocal
goto:eof

[bipul049]

I tried running your code Foxidrive. This is what i did:

Input file content:

Code: Select all

REPLICAT   ABCDEF  Last Started 2014-04-28 16:05   Status RUNNING
Description          Description here
Checkpoint Lag       07:00:00 (updated 00:12:01 ago)
Log Read Checkpoint  File ./dirdat/rg000260
                     2014-04-28 16:10:32.812000  RBA 445370

REPLICAT   ABCGHI Last Started 2014-04-28 16:05   Status STOPPED
Description          Description here
Checkpoint Lag       09:00:00 (updated 00:20:03 ago)
Log Read Checkpoint  File ./dirdat/rg000260
                     2014-04-28 16:10:32.812000  RBA 445370

REPLICAT   ABCJKL Last Started 2014-04-28 16:05   Status RUNNING
Description          Description here
Checkpoint Lag       05:59:20 (updated 00:09:59 ago)
Log Read Checkpoint  File ./dirdat/rh010248
                     2014-04-28 16:57:47.721000  RBA 268193613

REPLICAT   ABCMNO  Last Started 2014-04-28 16:05   Status STOPPED
Description          Description here
Checkpoint Lag       40:00:00 (updated 00:00:07 ago)
Log Read Checkpoint  File ./dirdat/rh010248
                     2014-04-28 16:57:59.377333  RBA 268870283

REPLICAT   ABCPQR  Last Started 2014-04-28 16:05   Status RUNNING
Description          Description here
Checkpoint Lag       00:00:13 (updated 00:30:11 ago)
Log Read Checkpoint  File ./dirdat/rh010248
                     2014-04-28 16:57:44.378666  RBA 268029200

and when i ran your code without modifying anything in that(the one without findrepl), i got the below output which doesn't seem to be correct.

Code: Select all

REPLICAT ABCDEF have lag of: 07:00:00.
REPLICAT ABCDEF have Checkpoint lag of: 00:12:01.
REPLICAT ABCDEF have lag of: 09:00:00.
REPLICAT ABCDEF have Checkpoint lag of: 00:20:03.
REPLICAT ABCGHI  have Checkpoint lag of: 00:09:59.
REPLICAT ABCGHI  have lag of: 40:00:00.
REPLICAT ABCPQR have Checkpoint lag of: 00:30:11.

Whereas my correct output in this case should be:

Code: Select all

REPLICAT ABCDEF have lag of: 07:00:00.
REPLICAT ABCDEF have Checkpoint lag of: 00:12:01.
REPLICAT ABCJKL have Checkpoint lag of: 00:09:59.
REPLICAT ABCPQR have Checkpoint lag of: 00:30:11.

[foxidrive]

I edited my last post - see if it helps now.

[foxidrive]

The code now provides the output you pasted, but the original code has an issue and aborts here.

Code: Select all

Invalid number.  Numeric constants are either decimal (17),
hexadecimal (0x11), or octal (021).

The calculation routine is encountering a leading zero and failing.

[bipul049]

Oh it looks perfectly alright now. Thank you so much Foxidrive for taking out time for me to get this done. I really appreciate that. As of now its working as expected. I will cross check few more test cases and will get back to you for any clarification.
Thanks alot alot alot for this.

Yes it does give that error. I am not very much sure how this error will affect my code. I think its not affecting anywhere, since I am getting desired output.

[foxidrive]

I think I've solved the calculation error and it now also removes the false match at 00:09:59

The modified code is in the same post above, at the top of the post. The older code remains at the bottom of the post.

[bipul049]

Thank you so much foxidrive. You are just amazing at your work. I will test these things here which I am sure will work flawlessely. If anything , will post it here.
You really are time saver.

Thanks a lot

[penpen]

Hi, there.
Sorry i had no time yesterday, and also
for the calculation error, somehow i havn't noticed it:
I Fixed it in my above code.

penpen