DosTips.com • Pulling my hair out.. simple stringlen counter breaks script

Page 1 of 3

Pulling my hair out.. simple stringlen counter breaks script

Posted: 01 Aug 2013 20:03

by MKANET

The below script (thanks so much to PenPen on the forum!!!) runs perfectly against the below stunnel.log file. As soon as I insert a stringlength counter (reserved for a future function) , the added lines of code produces a, "The syntax of the command is incorrect." error. I have tried and tried (more than I'd like to admit :cry:

:cry:

), but I can't figure out how to make the stringlength counter compatible.

I had also tried using the macros technique to get string length (mentioned on this forum); but, I had even more problems with that. All I want, is the fastest possible way to get the string lenghth of %print% variable IN the run-log.cmd script.

If I put the stringlength counter in a separate test script (or even macros stringlength technique, they work perfectly. I'm hoping someone much more experienced than me would know how I can make stringlength counting work.

run-log.cmd

Code: Select all

@ECHO OFF
setlocal enableDelayedExpansion
set "log=stunnel.log"
set "logx=stunnelx.log"

(
   for /f "tokens=1-3* delims= " %%a in ('findstr /V /I /C:"127.0.0" /C:"192.168" /C:"byte" /C:"WSAECONNRESET" /C:"Peer suddenly disconnected" "%log%"') do (
   
   
    set "dateValue=%%a"
      set "dateValue=!dateValue:~5,2!/!dateValue:~8,2!/!dateValue:~2,2!"

      for /F "tokens=1* delims=:" %%e in ("%%b") do (
         set /A "timeValue=%%e"
         if !timeValue! == 0 (
            set "timeValue=12:%%fam"
         ) else if !timeValue! Lss 12 (
            set "timeValue=%%e:%%fam"
         ) else if !timeValue! == 12 (
            set "timeValue=12:%%fpm"
         ) else (
            set /A "timeValue-=12"
            set "timeValue=0!timeValue!"
            set "timeValue=!timeValue:~-2!:%%fpm"
         )
      )

      set "info=%%d"
     
      set "ipPort="
      for %%e in (!info!) do set "ipPort=%%e"

      for /F "tokens=1,2 delims=:" %%f in ("!ipPort!") do (
         if not defined IP[%%f] for /f "tokens=2 delims= " %%h in ('nslookup %%f 2^> nul ^| findstr /I /C:"Name" ') do set "IP[%%f]= %%h"
         if not defined IP[%%f] set "IP[%%f]= "

         set "host=!IP[%%f]:~1!"
         if defined host (
            set "info=!info: %%f:%%g=!: !host! [%%f]"
         ) else (
            set "info=!info: %%f:%%g=!: [%%f]"
         )
      )


       set "location=- United States of America"
       set "print=%dateValue% %timeValue% %info% %location%"

       ::: If I add the below lines, the batch file produces, "The syntax of the command is incorrect." error.
       :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
       call :strlen result print
       set /a charlength=%result%
       if !charlength! GTR 190 goto :endcounter
       set /a padding = 190 - %charlength%
       set counter=0
       :counter
       if !counter!==!padding! goto :endcounter
       set /a counter=%counter%+1
       set "location= %location%"
       goto counter
       :endcounter
       :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
       
         echo(!dateValue! !timeValue! !info! !location!
       )     
) > "%logx%"

endlocal
goto :eof


:strlen
(   
    setlocal EnableDelayedExpansion
    set "s=!%~2!#"
    set "len=0"
    for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
        if "!s:~%%P,1!" NEQ "" ( 
            set /a "len+=%%P"
            set "s=!s:~%%P!"
        )
    )
)
( 
    endlocal
    set "%~1=%len%"
    exit /b
)

stunnel.log

Code: Select all

2013.07.23 10:16:00 LOG5[10152:15136]: Service [stunnel-sslh] connected remote server from 24.12.152.129:58773
2013.07.23 10:16:00 LOG3[10152:15136]: SSL_read: Connection reset by peer (WSAECONNRESET) (10054)
2013.07.23 10:16:00 LOG5[10152:832]: Connection reset: 272 byte(s) sent to SSL, 96 byte(s) sent to socket
2013.07.23 10:17:53 LOG5[10152:4000]: Service [stunnel-sslh] accepted connection from 71.194.51.232:5535
2013.07.23 10:17:53 LOG5[10152:16008]: connect_blocking: connected 24.12.152.129:7777
2013.07.23 10:17:53 LOG5[10152:4000]: Service [stunnel-sslh] connected remote server from 24.12.152.129:58799
2013.07.23 10:17:53 LOG5[10152:13212]: Service [stunnel-sslh] accepted connection from 71.194.51.232:5508
2013.07.23 10:17:53 LOG5[10152:3348]: Service [stunnel-sslh] accepted connection from 71.194.51.232:5509
2013.07.23 10:17:53 LOG5[10152:2884]: Service [stunnel-sslh] accepted connection from 71.194.51.232:5519

stringlen-test.cmd (works perfectly outside of the above script)

Code: Select all

@ECHO OFF
setlocal EnableDelayedExpansion

set "timeValue=11:44:32pm "
set "dateValue=07/23/13 "
set "info=Service [stunnel-sslh] accepted connection from: r74-194-126-35.jsbrcmta02.jnboar.lr.dh.suddenlink.net [74.194.126.35]"
set "location=- United States of America "

set "print=%dateValue% %timeValue% %info% %location%"
call :strlen result print
set /a charlength=%result%
if !charlength! GTR 190 goto :endcounter
set /a padding = 190 - %charlength%
set counter=0
:counter
if !counter!==!padding! goto :endcounter
set /a counter=%counter%+1
set "location= %location%"
goto counter
:endcounter

echo "%dateValue% %timeValue% %info% %location%"

goto :end
:strlen
(   
    setlocal EnableDelayedExpansion
    set "s=!%~2!#"
    set "len=0"
    for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
        if "!s:~%%P,1!" NEQ "" ( 
            set /a "len+=%%P"
            set "s=!s:~%%P!"
        )
    )
)
( 
    endlocal
    set "%~1=%len%"
    exit /b
)
:end

"High-performance" stringlength counter using macros (only seems to work outside of the my batch file):

Code: Select all

@echo off
call :loadMacros
set "myVar=abcdefg"
%$strlen%  result myVar
echo %result%
exit /b


:loadMacros
set LF=^


::Above 2 blank lines are required - do not remove
set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"
:::: StrLen pResult pString
set $strLen=for /L %%n in (1 1 2) do if %%n==2 (%\n%
        for /F "tokens=1,2 delims=, " %%1 in ("!argv!") do (%\n%
            set "str=A!%%~2!"%\n%
              set "len=0"%\n%
              for /l %%A in (12,-1,0) do (%\n%
                set /a "len|=1<<%%A"%\n%
                for %%B in (!len!) do if "!str:~%%B,1!"=="" set /a "len&=~1<<%%A"%\n%
              )%\n%
              for %%v in (!len!) do endlocal^&if "%%~b" neq "" (set "%%~1=%%v") else echo %%v%\n%
        ) %\n%
) ELSE setlocal enableDelayedExpansion ^& set argv=,

exit /

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 01 Aug 2013 21:21

by Magialisk

I don't have your log files, so I can't say for sure, but this appears to run. I ran a few tests, for example swapping your first FOR statement with a simple "FOR /L %%a in (1,1,5) DO" and the code ran 5 times, gave some missing operand errors (since there's no data coming in from the log), and dumped some garbage into "stunnelx.log". As far as I can tell, given the correct log inputs, if your other logic is sound, it should do what you want.

Code: Select all

@ECHO OFF
setlocal enableDelayedExpansion
set "log=stunnel.log"
set "logx=stunnelx.log"
for /f "tokens=1-3* delims= " %%a in ('findstr /V /I /C:"127.0.0" /C:"192.168" /C:"byte" /C:"WSAECONNRESET" /C:"Peer suddenly disconnected" "%log%"') do (
   set "dateValue=%%a"
   set "dateValue=!dateValue:~5,2!/!dateValue:~8,2!/!dateValue:~2,2!"
      for /F "tokens=1* delims=:" %%e in ("%%b") do (
         set /A "timeValue=%%e"
         if !timeValue! == 0 (
            set "timeValue=12:%%fam"
         ) else if !timeValue! Lss 12 (
            set "timeValue=%%e:%%fam"
         ) else if !timeValue! == 12 (
            set "timeValue=12:%%fpm"
         ) else (
            set /A "timeValue-=12"
            set "timeValue=0!timeValue!"
            set "timeValue=!timeValue:~-2!:%%fpm"
         )
      )
      set "info=%%d"    
      set "ipPort="
      for %%e in (!info!) do set "ipPort=%%e"
      for /F "tokens=1,2 delims=:" %%f in ("!ipPort!") do (
         if not defined IP[%%f] for /f "tokens=2 delims= " %%h in ('nslookup %%f 2^> nul ^| findstr /I /C:"Name" ') do set "IP[%%f]= %%h"
         if not defined IP[%%f] set "IP[%%f]= "
         set "host=!IP[%%f]:~1!"
         if defined host (
            set "info=!info: %%f:%%g=!: !host! [%%f]"
         ) else (
            set "info=!info: %%f:%%g=!: [%%f]"
         )
      )
   set "location=- United States of America"
   set "print=!dateValue! !timeValue! !info! !location!"
   call:strlen result print
   set /a charlength=!result!
   if !charlength! GTR 190 set skippadding=1
   IF NOT DEFINED skippadding (
      set /a padding = 190 - !charlength!
      FOR %%l IN (0,1,!padding!) DO set "location= %location%"
      set skippadding=
   )
   echo(!dateValue! !timeValue! !info! !location! >> "%logx%"
) 
GOTO:EOF

:strlen
(   
    setlocal EnableDelayedExpansion
    set "s=!%~2!#"
    set "len=0"
    for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
        if "!s:~%%P,1!" NEQ "" (
            set /a "len+=%%P"
            set "s=!s:~%%P!"
        )
    )
)
(
    endlocal
    set "%~1=%len%"
    exit /b
)

The main change was getting rid of the labels and gotos in the middle of the for loop. I'm not sure whether that's allowed or not, I don't think so, but either way I re-wrote it a little cleaner. Hopefully I didn't break the logic in the process

. The other thing I changed was the > "%logx%" at the end I made >>, as I'm assuming you want to append lines to the log on each pass, not overwrite the contents (leaving it with only the last line written).

By the way, the operation you're doing with the padding. I have two macros called LeftCol and RightCol that I added to dbenham's macroLib_String library that do that same operation. They pad the left or right side of a string with spaces until the string is a certain length. If the string is already longer than the given length (ie: "helloworld" with length input 7) they chop the string to length from the left or right (ie: "hellowo" or "loworld"). I don't have the code with me at this second, but I was planning on posting those macros anyway in the string encryption thread tomorrow. I can post them here if you'd like to replace both the StrLen and padding operations with macros for speed.

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 01 Aug 2013 21:24

by Magialisk

I'm sorry, I didn't realize that you did in fact post a sample log file. I just copied it down, tested the code I posted above against it, and got the following output:

Code: Select all

07/23/13 10:16:00am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]   
07/23/13 10:17:53am Service [stunnel-sslh] accepted connection from: c-71-194-51-232.hsd1.il.comcast.net [71.194.51.232]   
07/23/13 10:17:53am connect_blocking: connected: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]   
07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]   
07/23/13 10:17:53am Service [stunnel-sslh] accepted connection from: c-71-194-51-232.hsd1.il.comcast.net [71.194.51.232]   
07/23/13 10:17:53am Service [stunnel-sslh] accepted connection from: c-71-194-51-232.hsd1.il.comcast.net [71.194.51.232]   
07/23/13 10:17:53am Service [stunnel-sslh] accepted connection from: c-71-194-51-232.hsd1.il.comcast.net [71.194.51.232]

Hopefully that's what you were expecting

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 08:55

by MKANET

Thanks Magialisk!

The script works perfectly. There was a small mistake (the !location! variable wasn't showing in stunnelx.log). I sometimes miss delayed variable expansion too.

Its seems to be working fine now!

BTW: If you (or anyone) can you figure out how to use the "high-performance" macros stringlength method (working script example in first post) to count string length in my batch file instead of the slower binary search method? I could never get it to work, I had problems with local delayed expanded variables. Also, any other methods to speed up/optimize performance in the script would be greatly appreciated!

Code: Select all

   IF NOT DEFINED skippadding (
      set /a padding = 190 - !charlength!
      FOR %%l IN (0,1,!padding!) DO set "location= %location%"
      set skippadding=
   )

Code: Select all

Should be:
   IF NOT DEFINED skippadding (
      set /a padding = 190 - !charlength!
      FOR %%l IN (0,1,!padding!) DO set "location= !location!"
      set skippadding=
   )

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 09:50

by penpen

Magialisk wrote:The other thing I changed was the > "%logx%" at the end I made >>, as I'm assuming you want to append lines to the log on each pass, not overwrite the contents (leaving it with only the last line written).

The for Loop is encapsulated within (), so the > "%logx%" part does not override the file content:

Code: Select all

(
   for ... do (
   :: ...
   )
) > "logx"

Additionally using > in this way is faster than using >> because then only one file output stream is referenced, instead of one file output stream per processed text line.

@MKANET
The goto command breaks the Loop, as i have writtten on my post with timestamp 31 Jul 2013 18:58: http://www.dostips.com/forum/viewtopic.php?f=3&t=4786

penpen

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 10:06

by penpen

The fastest padding is the following:

Code: Select all

set "padding=          "
set "string=Test."


echo "%leftPadded%":left padding
set "leftPadded=%padding%%string%"
set "leftPadded=%leftPadded:~-10%"

:right padding
set "rightPadded=%string%%padding%"
set "rightPadded=%rightPadded:~0,10%"

echo "%rightPadded%"

The padding variable should only contain at minimum the number of padded spaces.
The disadvantage is, that the string is shortened, if the padding is shorter than the string.

penpen

Edit a Little workaraound is:

Code: Select all

setlocal enableDelayedExpansion
if "!string:~0,10!" == "!string!" (
   set "leftPadded=!padding!!string!"
   set "leftPadded=!leftPadded:~-10!"

   set "rightPadded=!string!!padding!"
   set "rightPadded=!rightPadded:~0,10!"
) else (
   set "leftPadded=!padding!"
   set "rightPadded=!padding!"
)
endlocal

Not much slower.

Edit2: Corrected the errors in the workaround.

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 10:14

by MKANET

Hi Penpen, that's the first thing tried to do.. but, unfortunately with what I'm trying to do, I need to be able to control the number of spaces (padding) preceeding the string. So, unfortunately, something like

set "leftPadded=%leftPadded:~-%padding%%"

Doesn't work. At least it didn't work for me. Padding size can vary greatly (for what I'll ultimately need).. so truncating the string isn't an option.

penpen wrote:The fastest padding is the following:
Code: Select all
set "padding=          "
set "string=Test."

:left padding
set "leftPadded=%padding%%string%"
set "leftPadded=%leftPadded:~-10%"

:right padding
set "rightPadded=%string%%padding%"
set "rightPadded=%rightPadded:~0,10%"

echo "%leftPadded%"
echo "%rightPadded%"
The padding variable should only contain at minimum the number of padded spaces.
The disadvantage is, that the string is shortened, if the padding is shorter than the string.

penpen

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 10:30

by penpen

MKANET wrote:At least it didn't work for me. Padding size can vary greatly (for what I'll ultimately need).. so truncating the string isn't an option.

That should not be a problem, just store the padding spaces of the maximum count into it:

Code: Select all

@echo off
:: example max padding 200
set "padding=                                                                                                                                                                                                        "
set "string=Test."

setlocal enableDelayedExpansion
:: secure left padding so that the length of padding and string is 200 (only if the string is shorter than 200 chars)
if "%string:~0,200%" == "%string%" (
   set "leftPadded200=%padding%%string%"
   set "leftPadded200=!leftPadded200:~-200!"
) else (
   set "leftPadded200=%string%"
)

:: secure left padding so that the length of padding and string is 10 (only if the string is shorter than 10 chars)
if "%string:~0,10%" == "%string%" (
   set "leftPadded10=%padding%%string%"
   set "leftPadded10=!leftPadded10:~-10!"
) else (
   set "leftPadded10=%string%"
)

echo "%leftPadded10%"
echo "%leftPadded200%"
endlocal

penpen

Edit: Corrected the errors.
Edit: Corrected more errors, should never view tv while coding

.

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 10:53

by MKANET

Hi Penpen, I tried the last script you posted. But, it's giving me an error.

( was unexpected at this time.

Code: Select all

:: example max padding 200
set "padding=                                                                                                                                                                                                        "
set "string=Test."

:: secure left padding so that the length of padding and string is 200 (only if the string is shorter than 200 chars)
if "%string:0,200%" == "%string%" (
   set "leftPadded200=%padding%%string%"
   set "leftPadded200=%leftPadded:~-200%"
) else (
   set "leftPadded200=%padding%"
)

:: secure left padding so that the length of padding and string is 10 (only if the string is shorter than 10 chars)
if "%string:0,10%" == "%string%" (
   set "leftPadded10=%padding%%string%"
   set "leftPadded10=%leftPadded:~-10%"
) else (
   set "leftPadded10=%padding%"
)

echo "%leftPadded10%"
echo "%leftPadded200%"

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 11:02

by penpen

Sorry, i've tricked myself out, so the correction took a while.
I have corrected the code above, i hope it works now.

penpen

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 12:06

by MKANET

Penpen, I've been thinking and thinking about how your suggested method would apply to my intended scenario; but, I just dont see how it would work.

Maximum length = 170

%print% = 07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129] - United States of America

%location% = - United States of America"

-------------------------------------------------------------------------------------------------------------------------

stunnelx.log (without any padding applied)

Code: Select all

07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129] - United States of America

stunnelx.log (with padding to equal 170 characters)

Code: Select all

07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]                    - United States of America

Also, Magialisk's padding code doesnt work either; even if I fix the delayed expansion. It doesnt add any padding no matter what. I'm guessing he wasn't sure how it should look. I probably should have made it clear. Hopefully its clear what I'm intending to do now.

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 13:12

by penpen

Do you mean something like this?

Code: Select all

:: padding to 170 chars assumed that info.length + location.length <=170
set "padding=                                                                                                                                                                          "
set "info=07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]"
set "location= - United States of America"
set "info.length="

:: assumed that getStringLength is defined and assigns the length of the string stored in the variable info to the variable info.length
call :getStringLength "info.length" "info"

set "print=%padding%%location%"
set "print=%print:~-170%"
call set "print=%info%%%print:~%info.length%%%"

echo "%print%"

penpen

Edit: Today is not my day.. corrected call set line length -> info.length
Edit 2. closed unterminated string in call line

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 13:57

by MKANET

Penpen, it doesnt look like your suggested script does what you intended it to do. Either that, or you didn't understand what I'm trying to do.
EDIT: After reading my previous post, I can see why you were confused. I had said "stunnelx.log (with padding to equal 170 characters)". I meant (with padding included to equal 170 characters). Sorry!

Your proposed script:

Code: Select all

@echo off

:: padding to 170 chars assumed that info.length + location.length <=170
set "padding=                                                                                                                                                                          "
set "info=07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]"
set "location= - United States of America"
set "info.length="

:: assumed that getStringLength is defined and assigns the length of the string stored in the variable info to the variable info.length
call :getStringLength "info.length" "info"

set "print=%padding%%location%"
set "print=%print:~-170%"
call set "print=%info%%%print:~%length%%%

echo %print%


goto:eof

:getStringLength
(   
    setlocal EnableDelayedExpansion
    set "s=!%~2!#"
    set "len=0"
    for %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
        if "!s:~%%P,1!" NEQ "" ( 
            set /a "len+=%%P"
            set "s=!s:~%%P!"
        )
    )
)
( 
    endlocal
    set "%~1=%len%"
    exit /b
)

Output:

Code: Select all

07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]                                                                                                                                                - United States of America

=294 characters total.

The output should be this:

Code: Select all

07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]                    - United States of America

=170 characters total. If not enough room, then no padding.

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 14:25

by penpen

It is to hot here in Germany... i corrected the error in the script above.
Now it creates 170 char lines.

penpen

Re: Pulling my hair out.. simple stringlen counter breaks sc

Posted: 02 Aug 2013 14:41

by penpen

I didn't notice the non padding option on first reading your post, so the whole code should be something like this:

Code: Select all

set "padding=                                                                                                                                                                          "
set "info=07/23/13 10:17:53am Service [stunnel-sslh] connected remote server from: c-24-12-152-129.hsd1.il.comcast.net [24.12.152.129]"
set "location= - United States of America"
set "info.length="
set "location.length="

call :getStringLength "info.length" "info"
call :getStringLength "loction.length" "location"

set /A "min=%location.length%+%info.length%"

if %min% GEQ 170 goto: noPadding
set "print=%padding%%location%"
set "print=%print:~-170%"
call set "print=%info%%%print:~%info.length%%%"
goto :out

:noPadding
set "print=%info%%location%"

:out
echo "%print%"

penpen