Looking for some help greping a large logfile and split the logs in separate logs per hour

[rcmpayne]

Looking to use JREPL to grep a very large file (5-15GB or more) file and split the file to per hour. the example below would be the start of every logline.

2017-04-24T01:26:58.728-0400 -
2017-04-24T03:26:58.728-0400 -
2017-04-24T03:26:58.728-0400 -
2017-04-24T04:26:58.728-0400 -
2017-04-24T04:26:58.728-0400 -
2017-04-24T05:26:58.728-0400 -
2017-04-24T05:26:58.728-0400 -

[Squashman]

If every line starts with a date and time stamp, you certainly wouldn't need to use JREPL to do that.

[rcmpayne]

Trying something like this works but not very sufficient. any tips would be great

set InputFile=ALL_UOS.txt
set OutputFile=ALL_UOS.txt

Cant get this to work

Code: Select all

rem set "CoreRegex=\r?\n(?!\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d\.\d\d\d-\d{4})"
rem cat ALL_UOS_ACCS.txt | grep \d{4}-\d\d-\d\dT05:\d\d:\d\d\.\d\d\d-\d{4}
rem cat ALL_UOS_ACCS.txt | grep -E "\d{4}-\d\d-\d\dT05:\d\d:\d\d\.\d\d\d-\d{4}"

These work if i run one for every time hour (T00, T01-T23)

Code: Select all

cat %InputFile% | grep T00: > T00_%OutputFile%
cat %InputFile% | grep T01: > T01_%OutputFile%
cat %InputFile% | grep T02: > T02_%OutputFile%
cat %InputFile% | grep T03: > T03_%OutputFile%
cat %InputFile% | grep T04: > T04_%OutputFile%
cat %InputFile% | grep T05: > T05_%OutputFile%
cat %InputFile% | grep T06: > T06_%OutputFile%
cat %InputFile% | grep T07: > T07_%OutputFile%
cat %InputFile% | grep T08: > T08_%OutputFile%
cat %InputFile% | grep T09: > T09_%OutputFile%
cat %InputFile% | grep T10: > T10_%OutputFile%
cat %InputFile% | grep T11: > T11_%OutputFile%
...