FINDSTR very slow - other options?

[SIMMS7400]

Hi Folks -

I have a log file that is 80,000kb so it's quite large.

This log file lets me know when a certain TM1 (IBM) server goes down and then comes back up.

What happens is I kick off an import process, and the last step it restarts the server. Once the server restarts, I need to run a data export. However, the server can take anywhere from 4-8 minutes to restart so instead of checking manually, I want to capture dynamically.

My intent is to search the log file right before it downs down for the last occurrence of the following string: "TM1 Server is ready" and set the entire line in a variable.
Then, kepe scanning the file checking for the LAST instance of that string and set in a new variable. If the variables DO NOT equal each other, that means the server is now up.

My issue is that FINDSTR is very slow. Is there another method?

Here is my script I was working on:

Code: Select all

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

PUSHD "\\server\IBMCognos\ASCEND\LOG"
ECHO %TIME%

SET "LASTOCCURENCE="
SET "NEWOCCURENCE="

SET /A "CNT=0"

:LOOP
for /F "delims=" %%G in ('FINDSTR /C:"TM1 Server is ready" tm1server.log') DO (
	IF "%CNT%"=="0" SET "LASTOCCURENCE=%%G"
	IF "%CNT%" NEQ "0" SET "NEWOCCURENCE=%%G"

	IF "!LASTOCCURENCE!" NEQ "!NEWOCCURENCE!" GOTO STARED
	SET /A "CNT+=1" & GOTO LOOP
)

:STARTED
ECHO TM1 APP has been started
ECHO %TIME%
POPD

echo "!LASTOCCURENCE!" 
echo "!NEWOCCURENCE!"
pause

Here is a sample log file ( see last night) :

Code: Select all

16104   []   INFO   2018-01-11 04:40:46.920   TM1.Process   Process "ASCEND_TOUCH_FILE_MDM" run from process "ASCEND_MDM_EXPORT_WRAPPER" by user "R*EXPORT_WRAPPER"
16104   []   INFO   2018-01-11 04:40:48.748   TM1.Process   Process "ASCEND_TOUCH_FILE_MDM":  finished executing normally, elapsed time 1.83 seconds
16104   []   INFO   2018-01-11 04:40:48.748   TM1.Process   Process "ASCEND_MDM_EXPORT_WRAPPER":  finished executing normally, elapsed time 178.13 seconds
16104   []   INFO   2018-01-11 04:40:48.748   TM1.Process   Process ASCEND_EXPORT_CC_CMSO_MASTERDATA executed by chore EXPORT_WRAPPER
16104   []   INFO   2018-01-11 04:40:49.842   TM1.Process   Process "ASCEND_EXPORT_CC_CMSO_MASTERDATA":  finished executing normally, elapsed time 1.09 seconds
16104   []   INFO   2018-01-11 04:40:49.842   TM1.Process   Process ASCEND_EXPORT_CC_MDM_MASTERDATA executed by chore EXPORT_WRAPPER
16104   []   INFO   2018-01-11 04:40:54.732   TM1.Process   Process "ASCEND_EXPORT_CC_MDM_MASTERDATA":  finished executing normally, elapsed time 4.89 seconds
16104   []   INFO   2018-01-11 04:40:54.732   TM1.Process   Process ASCEND_EXPORT_CC_FULL executed by chore EXPORT_WRAPPER
16104   []   INFO   2018-01-11 04:41:01.092   TM1.Process   Process "ASCEND_EXPORT_CC_FULL":  finished executing normally, elapsed time 6.36 seconds
16104   []   INFO   2018-01-11 04:41:01.092   TM1.Process   Process ASCEND_EXPORT_CE_MASTERDATA executed by chore EXPORT_WRAPPER
16104   []   INFO   2018-01-11 04:41:01.983   TM1.Process   Process "ASCEND_EXPORT_CE_MASTERDATA":  finished executing normally, elapsed time 0.89 seconds
16104   []   INFO   2018-01-11 04:41:01.998   TM1.Process   Process ASCEND_EXPORT_SAP_IO_REF_MAP executed by chore EXPORT_WRAPPER
16104   []   INFO   2018-01-11 04:41:04.842   TM1.Process   Process "ASCEND_EXPORT_SAP_IO_REF_MAP":  finished executing normally, elapsed time 2.84 seconds
16104   []   INFO   2018-01-11 04:41:04.920   TM1.Chore   Chore "EXPORT_WRAPPER" time = 299.98 seconds
16128   []   INFO   2018-01-11 04:41:04.920   TM1.Server   Starting PseudoElement thread.  Thread Id: 16128
1848   []   INFO   2018-01-11 04:41:04.920   TM1.Server   Starting RebuildFacetModel thread.  Thread Id: 1848
9744   []   INFO   2018-01-11 04:41:04.920   TM1.Server   Starting spy thread.  Thread Id: 9744
16104   []   INFO   2018-01-11 04:41:04.920   TM1.Server   TM1 Server load time (secs) = 537
8368   []   INFO   2018-01-11 04:41:05.717   TM1.Server   TM1 Server is ready

Does JREPL do "finds"?

Thanks!

[pieh-ejdsch]

Hi Simms,
Findstr is not slow in your case.
first the network connection has to be established. Is there possibly an error?
If it takes longer -then it makes sense just to search for the value but outside a for loop.
this is written in a temporary file and then evaluated in the for loop.

your for loop does not guarantee in the first or only pass whether the last log entry and the first are the same.
It's always different from the first time anyway.
NEWOCCURENCE will never be any different than a load if this entry is only once in the log or several times. last is empty before.
Goto fails if the brand can not be found.
If the server writes this log and you only search this line, the loop will not start again.
If nothing is found, the loop will not start again.

I would just check if the last entry in the log contains this line. maybe this is enough.
but you do not need to start a For loop for this.

With this super-long log the question is, if the server always writes a new log and if so, what if the log is still there and the server is down anyway?
How do you check that?

If the server writes the log as long as it keeps writing open until it is finished, maybe it will be enough to check if another write process to the file is possible.
type nul>> logfile.log

[SIMMS7400]

Thank you for your reply!

My intent with the for loops is to be able to grab the LAST occurrence of my string and set as variable. Then, when I scan the file again, I can compare the other instance. if both are the same, the server hasn't restarted yet. If different, it means a NEWER version of my search string is present, indicating it's restarted.

Is my approach wrong? It's taking a long time to open the file, I believe.