sub strings extraction from multiline output

[Mahendra]

i request you to please help to extract the sub strings from the multiline output. Thanks in advance

we have a command in MQ in display the channel status. that command display the status of different types of channels ( SVRCONN/CLUSSDR/CLUSRCVR/SDR/RCVR). all the types channels does not have all common properties. only few of the properties are common. i want to extract the value of property.

command :
dis chs(*) chltype chstada chstati msgs sslpeer

The above command displays the data in below format.( in multiple lines). here i have written sample outputs.

CHANNEL(ABCD) CHLTYPE(SVRCONN)
CHSTADA(10-12-2013) CHSTATI(10:20:30)
connAME(10.20.30.40) STATUS(RETRYING)
msgs(1009) sslpeer(kjhdsfkjsd)
substate(Recive)



CHANNEL(DEFG) CHLTYPE(SDR)
CHSTADA(10-12-2013) CHSTATI(10:20:30)
connAME(10.20.30.40) STATUS(RETRYING)
substate(MQGet) msgs(10020)
sslpeer(kjhdsfkjsdfhjkshfjksfdhsjdfhsfsdhjfhsdjfhjkfhhfkfjkhfjkdsjfsdjkhdsjkfhsdjkfhdskjfhkj)
XMITQ(ABCD.TEST)


CHANNEL(GHIJ) CHLTYPE(CLUSRCVR)
CHSTADA(10-12-2013) CHSTATI(10:20:30)
connAME(10.20.30.40) STATUS(RETRYING)
current mcauser(ahghj)
substate(Recive) sslpeer(kjhdsfkjsdfhjkshfjkjkhdsjkfhsdjkfhdskjfhkj)


i want to extract the property values

channel | chltype | chstada |chstati |status |msgs |sslpeer
ABCD | SVRCONN | 10-12-2013| 10:20:30|running|1009| kjhdsfkjsd
DEFG | SVRCONN | 10-12-2013| 10:20:30|running|1009| kjhdsfkjsd
GHIJwewe | SVRCONN | 10-12-2013| 10:20:30|running|1009| kjhdsfkjsd



some channels have some extra properties and may not end all type channels with same property. for example SDR channel having extra property xmitq. but all the channels display start with channel name.

The proprty value may not standard length and position. it varies from channel to channel. suppose channel name can have max allowed upto 20 chars..

could you please some one guide me on this. with the below thread solution i tried ....
viewtopic.php?f=3&t=5227

[dbenham]

Using REPL.BAT

Code: Select all

@echo off
setlocal disableDelayedExpansion
set "channel="

:: Define $D to contain output delimiter. Any string could be used.
set "$D=|"

:: Define the output format
set "$header=channel!$D!chltype!$D!chstada!$D!chstati!$D!status!$D!msgs!$D!sslpeer"
set "$data=!%$header:!=!!%!"

:: Process the command output
setlocal enableDelayedExpansion
>output.txt (
  echo %$header%
  for /f "tokens=1,2 delims=()" %%A in (
    'dis chs(*) chltype chstada chstati msgs sslpeer^|repl "\) *" "\n" x'
  ) do (
    if /i %%A equ CHANNEL if defined CHANNEL echo %$data%
    set "%%A=%%B"
  )
  if defined channel echo %$data%
)

The code assumes you do not have ! in your data. It should be easy to toggle delayed expansion on and off if you need to support ! in data.

EDIT - Simplified logic and added a few comments

Dave Benham

[Aacini]

The Batch file below use FindRepl.bat program.

Code: Select all

@echo off
set "caret=^"
setlocal EnableDelayedExpansion

rem Get parameters and assemble from they:
rem 1- The regexp to search for multiple terms
rem 2- The list of submatched subexpressions
rem 3- The headers of the columns
rem The first parameter "channel" is pre-inserted
set "regexp=(channel\([!caret!\)]*)\)"
set "subexprs=:1"
set header[1]=channel
set n=1
:nextParam
   set "regexp=!regexp!|%1\(([!caret!\)]*)\)"
   set /A n+=1
   set subexprs=%subexprs%:%n%
   set header[%n%]=%1
   shift
if "%1" neq "" goto nextParam

rem Get the command output
dis chs(*) %* > data.txt

rem Show headers
set "line="
for /L %%i in (1,1,%n%) do (
   rem Insert here code for width column alignment
   set "line=!line!  !header[%%i]!  |"
)
echo !line:~0,-1!

rem Perform the search. FindRepl program return one line per SUBEXPRESSION (%n% lines per channel)
rem with %n% values in each line enclosed in quotes: all values empty excepting the submatched one

for /L %%i in (1,1,%n%) do set var[%%i]=Undefined
for /F "delims=" %%a in ('^< data.txt FindRepl /I "!regexp!" /$%subexprs%') do (
   set i=0
   for %%b in (%%a) do (
      set /A i+=1
      if %%b neq "" (
         for /F "tokens=1,2 delims=(" %%c in (%%b) do set "next=%%c" & set "channel=%%d"
         if /I "!next!" equ "channel" (
            rem Data for next channel started: show the previous one, if any
            if "!var[1]!" neq "Undefined" (
               set "line="
               for /L %%i in (1,1,%n%) do (
                  rem Insert here code for width column alignment
                  set "line=!line!  !var[%%i]!  |"
               )
               echo !line:~0,-1!
            )
            rem Restart data for next channel
            set "var[1]=!channel!"
            for /L %%i in (2,1,%n%) do set var[%%i]=Undefined
         ) else (
            set "var[!i!]=%%~b"
         )
      )
   )
)

set "line="
for /L %%i in (1,1,%n%) do (
   rem Insert here code for width column alignment
   set "line=!line!  !var[%%i]!  |"
)
echo !line:~0,-1!

For example:

Code: Select all

C:\> test chltype chstada  chstati  status  msgs sslpeer
  channel  |  chltype  |  chstada  |  chstati  |  status  |  msgs  |  sslpeer
  ABCD  |  SVRCONN  |  10-12-2013  |  10:20:30  |  RETRYING  |  1009  |  kjhdsfkjsd
  DEFG  |  SDR  |  10-12-2013  |  10:20:30  |  RETRYING  |  10020  |  kjhdsfkjsdfhjkshfjksfdhsjdfhsfsdhjfhsdjfhjkfhhfkfjkhfjkdsjfsdjkhdsjkfhsdjkfhdskjfhkj
  GHIJ  |  CLUSRCVR  |  10-12-2013  |  10:20:30  |  RETRYING  |  Undefined  |  kjhdsfkjsdfhjkshfjkjkhdsjkfhsdjkfhdskjfhkj

Antonio

[berserker]

awk script

Code: Select all

function strip( string ){
    # remove anything from beginning to the first ( ,
    # and remove anything from last bracket ) till the end
    gsub(  /.*\(|\).*/ , "", string )   
    return string
}

BEGIN{ 
    RS="\n\n+"      # set record separator to be more than 1 newlines
    FS="[ \n]+"     # set field separator to be space or newline
    IGNORECASE=1    # case insensitive search
    sep       ="|"  # output delimiter
}
{
    for( i=1; i<=NF; i++){
        printf  $i ~ /CHANNEL/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /CHLTYPE/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /CHSTADA/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /CHSTATI/ ? (strip( $i ) sep ) : ""
        printf  $i ~ /STATUS/  ? (strip( $i ) sep ) : ""
        printf  $i ~ /msgs/    ? (strip( $i ) sep ) : ""
        printf  $i ~ /sslpeer/ ? (strip( $i ) sep ) : ""
        
    }
    print ""

}

result

Code: Select all

C:\>awk -f  myScript.awk  myStatusFile.txt
ABCD|SVRCONN|10-12-2013|10:20:30|RETRYING|1009|kjhdsfkjsd|
DEFG|SDR|10-12-2013|10:20:30|RETRYING|10020|kjhdsfkjsdfhjkshfjksfdhsjdfhsfsdhjfhsdjfhjkfhhfkfjkhfjkdsjfsdjkhdsjkfhsdjkfhdskjfhkj|
GHIJ|CLUSRCVR|10-12-2013|10:20:30|RETRYING|kjhdsfkjsdfhjkshfjkjkhdsjkfhsdjkfhdskjfhkj|

you can also pass your status to the script using pipe

Code: Select all

`your MQ command` | awk -f myScript.awk 

[Mahendra]

is there anyway without using any third party tools..i.e pure batch script.

[berserker]

is there anyway without using any third party tools..i.e pure batch script.

then have you tried the batch solutions others posted?

[dbenham]

is there anyway without using any third party tools..i.e pure batch script.

sure

Code: Select all

@echo off
setlocal disableDelayedExpansion
set "channel="

:: Define $D to contain output delimiter. Any string could be used.
set "$D=|"

:: Define the output format
set "$header=channel!$D!chltype!$D!chstada!$D!chstati!$D!status!$D!msgs!$D!sslpeer"
set "$data=!%$header:!=!!%!"

:: Define LF to contain linefeed character (0x10)
set ^"LF=^

^" Above empty line is critical - DO NOT REMOVE

:: Process the command output
setlocal enableDelayedExpansion
>output.txt (
  echo %$header%
  for %%L in ("!LF!") do for /f "delims==" %%X in (
    'dis chs(*) chltype chstada chstati msgs sslpeer'
  ) do (
    set "ln=%%X"
    for %%L in ("!LF!") do set "ln=!ln:) =%%~L!"
    for /f "tokens=1,2 delims=()" %%A in ("!ln:) =%%~L!") do (
      if /i %%A equ CHANNEL if defined CHANNEL echo %$data%
      set "%%A=%%B"
    )
  )
  if defined channel echo %$data%
)

Dave Benham

[foxidrive]

i request you to please help to extract the sub strings from the multiline output. Thanks in advance

The quote in blue really means: "I am too lazy to come back and tell you if your solution worked, and to personally thank you for writing free code for me, in your spare time"

Don't mind me, it's just one of my petty annoyances.

[berserker]

Don't mind me, it's just one of my petty annoyances.

lol, foxi i think this happens quite common in forums.

[foxidrive]

lol, foxi i think this happens quite common in forums.

It does. I sometimes rant about it.

[Squashman]

is there anyway without using any third party tools..i.e pure batch script.

So what don't you like about REPL.bat and FINDREPL.bat? Sure they are hybrid scripts but they are still native to Windows. It isn't technically a 3rd party utility.

[Mahendra]

due to limitations, The extra scripts are not allowed to copy in the release bundle with out documentation and information to all the teams. so, it will be preferable single script.

The solution provide in the earlier conversations is not working for all the scenarios. it is working for the first channel only.

here i have a actual output of the command. few minor property values changed. HIGHLIGHTED FEW OF THE COMPONENTS FOR DIFFERENCE. there is no space in the command output.

command output:

AMQ8417:Display Channel Status details
CHANNEL(MQSTE.SVCS.S) CHLTYPE(SVRCONN)
CHSTADA(2014-01-08) CHSTATI(01.02.46)
CONNAME(143.34.1.234) CURRENT
LSTMSGDA(2014-01-08) LSTMSGTI(06.03.17)
MSGS(234) SSLPEER(CN=mqt.abc.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US)
STATUS(RUNNING) SUBSTATE(RECEIVE)
AMQ8417:Display Channel Status details
CHANNEL(SDR.MQSTE.SVCS.S) CHLTYPE(CLUSSDR)
CHSTADA(2014-01-08) CHSTATI(01.02.46)
CONNAME(143.34.12.234(1234)) CURRENT
LSTMSGDA(2014-01-08) LSTMSGTI(06.03.17)
MSGS(2) SSLPEER(CN=mqt.asdfghjj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US)
STATUS(RUNNING) SUBSTATE(MQGET)
XMITQ(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8417:Display Channel Status details
CHANNEL(RCVR.MQSTE.SVCS.S) CHLTYPE(CLUSRCVR)
CHSTADA(2014-01-08) CHSTATI(01.02.46)
CONNAME(143.34.12.23) CURRENT
LSTMSGDA(2014-01-08) LSTMSGTI(06.03.17)
MSGS(5) SSLPEER(CN=mqt.ghjjghyj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US)
STATUS(RUNNING) SUBSTATE(RECEIVE)


required format:

CHANNEL|CHLTYPE|CHSTADA|CHSTATI|MSGS|STATUS | SSLPEER

MQSTE.SVCS.S | SVRCONN | 2014-01-08 | 01.02.46 | 234| RUNNING | CN=mqt.abc.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US
SDR.MQSTE.SVCS.S | CLUSSDR | 2014-01-08 | 01.02.46 | 2| RUNNING | CN=mqt.asdfghjj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US
RCVR.MQSTE.SVCS.S | CLUSRCVR | 2014-01-08 | 01.02.46 | 2| RUNNING | CN=mqt.ghjjghyj.com,OU=mqtif,O=Form Tech Corporation,L=Newyork,ST=North carolina,C=US

[Squashman]

due to limitations, The extra scripts are not allowed to copy in the release bundle with out documentation and information to all the teams. so, it will be preferable single script.

Plenty of documentation for Repl.bat
viewtopic.php?f=3&t=3855

And for FindRepl.bat
viewtopic.php?f=3&t=4697

[Mahendra]

There other limitations to use...., like...it will always preferable single script. because to reduce the dependency of other script. size of the release bundle.....

[dbenham]

I had a silly bug in my original pure batch solution. It also had a redundant FOR loop.

Below is fixed code, with the delimiter set to " | " instead of "|". I tested with a static file containing the sample output instead of command output.

Code: Select all

@echo off
setlocal disableDelayedExpansion

:: Define $D to contain output delimiter. Any string could be used.
set "$D= | "

:: Define the output format
set "$header=channel!$D!chltype!$D!chstada!$D!chstati!$D!status!$D!msgs!$D!sslpeer"
set "$data=!%$header:!=!!%!"

:: Define LF to contain linefeed character (0x10)
set ^"LF=^

^" Above empty line is critical - DO NOT REMOVE

:: Process the command output
setlocal enableDelayedExpansion
set "channel="
>output.txt (
  echo %$header%
  for %%L in ("!LF!") do for /f "delims=" %%X in (
    'dis chs(*) chltype chstada chstati msgs sslpeer'
  ) do (
    set "ln=%%X"
    for /f "tokens=1,2 delims=()" %%A in ("!ln:) =%%~L!") do (
      if /i %%A equ CHANNEL if defined CHANNEL echo %$data%
      set "%%A=%%B"
    )
  )
  if defined channel echo %$data%
)

Dave Benham