sorry folks... it was a long week of beating my head against the wall and my brain was fried.
New week, new efforts.
Purpose:To generate a script that runs through a folder full of large log files, pull out the last two error messages (as seen in example below) along with 50 lines of text before/after and place them in a new log file to be emailed or *something* later on.
Problem: When performing a FINDSTR query on the log file to determine the line number where the error occurs it seems to return a different number than the actual line number where the error occurs.
Example Log file:Code: Select all
2016-09-21 08:18:54,209 [12] DEBUG
2016-09-21 08:18:54,213 [12] DEBUG Bla bla bla
------------------------------------------------------------------------------------------------------------------------
-- Procedure Name:
-- Generated by
-- Copyright:
------------------------------------------------------------------------------------------------------------------------
CREATE PROCEDURE [xxx].[xxx]
@id_ nvarchar(32)
, @ts_PublicID nvarchar(50)
, @ts_Name nvarchar(80)
, @ts_Name2 nvarchar(80)
, @ts_Name3 nvarchar(80)
, @ts_Description ntext
etc
2016-09-21 08:25:17,405 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,406 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,406 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,406 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,407 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,407 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,407 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,408 [12] INFO xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,408 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,408 [12] INFO xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,408 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,409 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,409 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,409 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,410 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,410 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:25:17,410 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2016-09-21 08:24:55,159 [15] ERROR xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(String sRegistryKey, String sKeyTag, String sDataTag, Encoding encoding)
at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(String value)
at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGetCSConnectionString()
at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxGetConnectionString()
etc
2016-09-21 08:25:17,149 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute Insert
2016-09-21 08:25:17,150 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute FindByKey
2016-09-21 08:25:17,150 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute Insert
2016-09-21 08:25:17,151 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute FindByKey
2016-09-21 08:25:17,151 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute Insert
2016-09-21 08:25:17,152 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute FindByKey
2016-09-21 08:25:17,152 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute Insert
2016-09-21 08:25:17,153 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute FindByKey
2016-09-21 08:25:17,153 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute Insert
2016-09-21 08:25:17,154 [12] DEBUG xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - Execute FindByKey
etc x 11ty billion
** Disclamer ** I've made no efforts to clean any of this up yet, I realize there is likely better ways to go about certain things (setting variables = '.' for instance); I'm open for any suggestions.
Code: Select all
CLS
@ECHO OFF
setlocal enabledelayedexpansion
:: Look in each file in the directory
FOR %%f IN (*) DO (
> nul FINDSTR /N /c:"] ERROR" %%f && (
::Error message was found
ECHO.
REM Set the variables equal to . so they can be referenced later
REM as it was problematic to reference null
SET SecondLastError=.
SET LastError=.
::Loop through the lines containing Errors and record them
FOR /F "delims=:" %%a IN ('FINDSTR /n /c:"] ERROR" %%f') DO (
::If a new error is found reference the previous one
SET SecondLastError=!LastError!
::Reference the line the error was found on.
SET LastError=%%a
REM ECHO %%a >> C:\Automation\Logs\ERRORS-%%f
)
ECHO %%f Last error^(s^) on line^(s^) !SecondLastError! !LastError!
::Call the function to determine what range of lines to record
CALL:CalcRange %%f !LastError! !SecondLastError!
)
)
This bit works as intended; for instance for one log file I get the following:
LOG_FILE_NAME.txt.1 Last error(s) on line(s) 106982 107264
The call to 'CalcRange' then does its thing (plus a whole bunch of other logic to determine if the errors are close enough the padded values overlap, are close to the start/end of the file, etc) and spits out:
Second Last error range - From: 106932 To: 107032
Last error range - From: 107214 To: 107314
This is also right. - For each of the last two error message ranges another function is called. Where I run in to the problems is pulling that data out of the log files consistently and in a timely fashion. I've tried a large number of different options (which I've apparently not kept very good track of....)
The following
DOES work, it's just extremely slow.
For the sake of making things more simple I've hard coded some values here (This is the last function) just to look at the large log file that's giving me issues:
Code: Select all
CLS
@ECHO OFF
setlocal enabledelayedexpansion
:: Setup values to skip to (saves looking at lines at the first of files which don't contain what we're after)
SET StartTime=%time%
SET /A start=106932 - 1
SET count=106932
:: For each line (Skipping to first error) Check to see if it is in the range we want to record
REM FOR /f "tokens=* skip=%start%" %%a IN ('find /n /v "" ^< "%~1"') DO (
REM FOR /f "delims=* skip=%start%" %%a IN ('find /n /v "" ^< "%~1"') DO (
REM for /f %%i in ('type "%file%"^|FINDSTR /N /c:"] ERROR"') do
REM FOR /f %%a in ('type "REMEMConfig.txt.1"^|findstr /v /c:"] ERROR"') do (
REM for /f %%i in ('type "%~1"^|FIND /v " "') do (
FOR /f "delims=* skip=%start%" %%a IN ('find /n /v "" ^< "LOG_FILE_NAME.txt.1"') DO (
IF !count! GEQ 106932 (
IF !count! LEQ 107032 (
::If in the range we want to record do so
ECHO %%a >> C:\Automation\Logs\Sum-TESTLOG.log
::If more than the range of lines we want exit
IF !count! GEQ 107032 GOTO:TESTER
)
)
SET /a count+=1
)
:TESTER
ECHO Ended
ECHO Start Time: %startTime%
ECHO Finish Time: %time%
This works but it takes a pile of time...
Ended
Start Time: 8:51:10.44
Finish Time: 9:00:45.24I've been trying to replace the FIND with a FINDSTR without any luck; I previously had one call which executed in a timely fashion but it was ignoring blank line spaces for some reason, this resulted in the wrong lines being returned.
Hopefully what I've provided is adequate this time around for you folks to understand what I'm trying to do.
