DosTips.com

A Forum all about DOS Batch
https://www.dostips.com/forum/

FINDSTR + literal strings/double quotes and DelayedExpansion

https://www.dostips.com/forum/viewtopic.php?f=3&t=5442

Page 1 of 2

FINDSTR + literal strings/double quotes and DelayedExpansion

Posted: 15 Mar 2014 09:29
by ©opy[it]®ight
Greetings DosTips addicts :-)

After hours of trial-and-error i've (once again) come to this place to hopefully get an explanation or solution for my FINDSTR batch troubles.

I have a script that has both CommandExtensions and DelayedExpansion enabled.

The purpose of the script is to search through a file for a matching string and show an errorlevel/exit code based on the result.


I've been trying it with and without a FOR /F loop and eventually managed to get it working, BUT i doubt i did it correctly and even don't know why it works, so hopefully one of you batch-guru's can assist/educate me.


The string to look for is in a .json file:

Code: Select all

"selectedProfile": "


the script:

Code: Select all

@ECHO OFF

SETLOCAL EnableExtensions EnableDelayedExpansion

SystemPath=!SystemRoot!\System32\

SET PROFILES=example.json

REM continues below


doesn't work:

Code: Select all

"!SystemPath!FINDSTR" /IC:"\"selectedProfile\": \"" "!PROFILES!">NUL
ECHO '!ERRORLEVEL!'


works:

Code: Select all

"!SystemPath!FINDSTR" /IC:^"\"selectedProfile\": \"" "!PROFILES!">NUL
ECHO '!ERRORLEVEL!'



All i did was add a caret, and i could even make the script work by putting the caret somewhere else.

I really don't get it :?

- ©opyitright

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 09:37
by foxidrive
This may work for you.


Code: Select all

findstr /i ".selectedProfile.: .*," "file.json" >nul && echo found it

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 09:51
by ©opy[it]®ight
Thanks foxidrive for the quick answer/solution.

I'm only afraid your code isn't strict enough, as you used some wildcards (.) which will make any character pass through as valid.
To decrease the chance of false-positives, "selectedProfile": " should be treated as a literal search string =/

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 12:02
by ©opy[it]®ight
How about this:

Code: Select all

FOR /F "delims=" %%L in ('!SystemPath!FINDSTR /IC:"\"selectedProfile\": \"^" "!PROFILES!"') do (IF %ERRORLEVEL% EQU 0 ECHO FOUND)


Code: Select all

"!SystemPath!FINDSTR" /IC:"\"selectedProfile\": \"^" "!PROFILES!">NUL && ECHO FOUND IT


What do you think?

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 12:58
by penpen
©opy[it]®ight wrote:After hours of trial-and-error i've (once again) come to this place to hopefully get an explanation or solution for my FINDSTR batch troubles.
(...)
BUT i doubt i did it correctly and even don't know why it works, so hopefully one of you batch-guru's can assist/educate me.

The problem is, that 2 parsers are involved:
- the cmd parser, and
- the findstr parser.

The tokenizer is affected by the strings, to make these affects visible i use this notation (per character):
- open a string: /
- close a string: \
- within a string: +
- outside a string: -
Within a string all characters (except closing character) is handled as a normal character, so space, '>', have no special meaning.
You could say that after tokenization, each character belongs to one token.
We assume the tokens are numbered starting with 1.
So the following displays the strings and tokenization:

Code: Select all

Case1:
/+++++++++++++++++++\-----/+\----------------/+++\/+\----------/++++
"!SystemPath!FINDSTR" /IC:"\"selectedProfile\": \"" "!PROFILES!">NUL
11111111111111111111122222222222222222222222222222222222222222222222
Token 1: ["!SystemPath!FINDSTR"]
Token 2: [ /IC:"\"selectedProfile\": \"" "!PROFILES!">NUL]

Case2:
/+++++++++++++++++++\--------/++++++++++++++++\---/\-/++++++++++\----
"!SystemPath!FINDSTR" /IC:^"\"selectedProfile\": \"" "!PROFILES!">NUL
111111111111111111111222222222222222222222222223333344444444444445555

Token 1: ["!SystemPath!FINDSTR"]
Token 2: [ /IC:^"\"selectedProfile\"]
Token 3: [: \""]
Token 4: [ "!PROFILES!"]
Token 5: [>NUL]
The command and all its parameters are passed to findstr, but the redirection [>NUL] is not part of it, so the following command lines are passed to findstr
(as i don't know the content of your variables i still use them, but they are replaced when findstr is called;
in addition i use the same format to show string content and tokens as the findstr parser "sees" it;
note that the ^ is resolved):

Code: Select all

Case1:
/+++++++++++++++++++\-----/+++++++++++++++++++++++\-/+++++++++++++++
"!SystemPath!FINDSTR" /IC:"\"selectedProfile\": \"" "!PROFILES!">NUL
11111111111111111111122222222222222222222222222222233333333333333333

Token 1: ["!SystemPath!FINDSTR"]
Token 2: [ /IC:"\"selectedProfile\": \""]
Token 3: [ "!PROFILES!">NUL]

Case2:
/+++++++++++++++++++\-----/+++++++++++++++++++++++\-/++++++++++\
"!SystemPath!FINDSTR" /IC:"\"selectedProfile\": \"" "!PROFILES!"
1111111111111111111112222222222222222222222222222223333333333333

Token 1: ["!SystemPath!FINDSTR"]
Token 2: [ /IC:"\"selectedProfile\": \""]
Token 3: [ "!PROFILES!"]
So they are both valid findstr parameter strings: Token 3 is interpreted as filename (from both findstr instances).
In case 1 this filename does not exist (i assume), so the result is an error.
In addition, it does, what you wanted it to do (if i see it right :D ); search for the string (without []): ["selectedProfile": "].

A simple solution that should work is, to write the >NUL at the front of the command (line string), where it is recognized correctly:
[code]
>NUL "!SystemPath!FINDSTR" /IC:"\"selectedProfile\": \"" "!PROFILES!"
>NUL "!SystemPath!FINDSTR" /IC:^"\"selectedProfile\": \"" "!PROFILES!"

penpen

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 15:57
by ©opy[it]®ight
Oh my, you're a walking encyclopedia/handbook :D

I greatly appreciate what you did penpen, this truly helps me a lot :) (where to give +rep?)

Thanks :wink:

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 17:06
by ©opy[it]®ight
Hmm.. i've been testing the code on a collection of json profiles, but i've run into a parsing problem where a profile has escaped double quotes in it (i totally forgot about this :oops: )

partly content of the json file:

Code: Select all

{
  "profiles": {
    "\"test\"": {
      "name": "\"test\""
    }
  },
  "selectedProfile": "\"test\"",


the script:

Code: Select all

FOR /F "delims=" %%L in ('!SystemPath!FINDSTR /IC:^"\"selectedProfile\": \"" "!PROFILES!"') do SET SELECTED_PROFILE=%%L

IF DEFINED SELECTED_PROFILE (

REM strip unnecessary string parts to get the selected profile "\"test\""
  SET SELECTED_PROFILE=!SELECTED_PROFILE:~21,-1!

REM look for a matching profile container "\"test\"": { 
  >NUL "!SystemPath!FINDSTR" /IC:!SELECTED_PROFILE!: {" "!PROFILES!"
  ECHO '!ERRORLEVEL!'

)


But it fails with errorlevel/exit code 1

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 15 Mar 2014 19:13
by Squashman
©opy[it]®ight wrote:Greetings DosTips addicts :-)

After hours of trial-and-error i've (once again) come to this place to hopefully get an explanation or solution for my FINDSTR batch troubles.

Probably help if you spent a little more time on the forums reading some of the questions and solutions and assisting when you can instead of being gone for months and years at a time.

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 16 Mar 2014 08:26
by penpen
©opy[it]®ight wrote:Hmm.. i've been testing the code on a collection of json profiles, but i've run into a parsing problem where a profile has escaped double quotes in it (i totally forgot about this :oops: )
It seems you have also forgotten that findstr has its own parser, so you need an unescaped version of the text string to search for.
But to be honest your case is a little bit special: normally a ["] character is escaped using [\"], but in some cases this doesn't work;
at least in xp: actually i have no access to other windows versions.

But ["] also can be escaped using [\\\"] which works in more cases;
in some cases you have to escape mixing these two versions... but your case is not that bad, so the following should work (as mentioned, at least using xp):

Code: Select all

@echo off
cls
setlocal enableDelayedExpansion
set "PROFILES=json.txt"
:: ------------------------------------------

FOR /F "delims=" %%L in ('!SystemPath!FINDSTR /IC:^"\"selectedProfile\": \"" "!PROFILES!"') do SET SELECTED_PROFILE=%%L
:: @

IF DEFINED SELECTED_PROFILE (

REM strip unnecessary string parts to get the selected profile "\"test\""
  SET SELECTED_PROFILE=!SELECTED_PROFILE:~21,-1!

REM apply escaping
  SET SELECTED_PROFILE=!SELECTED_PROFILE:\=\\!
  SET SELECTED_PROFILE=!SELECTED_PROFILE:"=\\\"!

REM look for a matching profile container "\"test\"": { 
  >nul "FINDSTR" /R /C:"!SELECTED_PROFILE!: {" "json.txt"
  ECHO '!ERRORLEVEL!'

)

:: ------------------------------------------
endlocal
goto :eof

penpen

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 19 Mar 2014 06:10
by ©opy[it]®ight
Squashman wrote:Probably help if you spent a little more time on the forums reading some of the questions and solutions and assisting when you can instead of being gone for months and years at a time.
True. But you must know that i've been googling A LOT and have been on many sites before i came here.
Perhaps i could've done a better search on the forum but as it was kind of late (or early) that didn't happen ;)

@ penpen: It took me some time to let everything get '(in)to' me, but its mostly clear to me so kudos for that! :)


edit: I run Win 7 and to escape the backslashes i had to enter them four times instead of two times as in your example, but then other already escaped characters in the json file (i.e. \u0026, "&" unescaped) won't parse correctly so i don't know what to do next (give up? :mrgreen: )

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 19 Mar 2014 07:55
by penpen
©opy[it]®ight wrote:I run Win 7 and to escape the backslashes i had to enter them four times instead of two times as in your example, but then other already escaped characters in the json file (i.e. \u0026, "&" unescaped) won't parse correctly so i don't know what to do next (give up? )
The '&' character should be no special character for findstr, so it sound a problem with the cmd(.exe) (line) interpreter; could you give an example string that goes wrong?
(But i only can test on WinXp the next days, it will take at minimum 2 days, until i get access to a Win7 (32bit) system.)

penpen

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 19 Mar 2014 10:20
by ©opy[it]®ight
For the record: I'm trying to parse a json file that can hold escaped strings/characters :!:

I don't think FINDSTR /R will work as i've noticed that strings can have [ ] brackets as well, where /R would see them as a RegEx Character class.

With DisableDelayedExpansion all characters are preserved, but then i'm unable to do percentage sign string-modification.

Here's a string i'm trying to parse but fails (for me):

Json (escaped):

Code: Select all

"selectedProfile": "!@#$%^\u0026*()\u003d{}[]|\\;:\u0027\",./\u003c\u003e?",

How to escape a pre-escaped string (snicker)

The unescaped version looks like this:

Code: Select all

!@#$%€^&*()={}[]|\;:'",./<>?

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 19 Mar 2014 18:52
by penpen
It seems you don't realize the power of escaping strings.
Create this text file:
json.txt wrote:"selectedProfile": "!@#$%^\u0026*()\u003d{}[]|\\;:\u0027\",./\u003c\u003e?",
!@#$%€^&*()={}[]|\;:'",./<>?

Then this code should work (at least for winxp, as i have no win 7 to test):

Code: Select all

setlocal enableDelayedExpansion
setlocal disableDelayedExpansion
"FINDSTR"    /C:^"\\\^"selectedProfile\\\^": \\\^"!@#$%%^^\\u0026*^(^)\\u003d{}[]^|\\\\;:\\u0027\\\\\^",./\\u003c\\u003e?\\\^",^" "json.txt"
"FINDSTR" /R /C:^"\\\^"selectedProfile\\\^": \\\^"!@#\$%%^^\\u0026\*^(^)\\u003d{}\[\]^|\\\\;:\\u0027\\\\\^",./\\u003c\\u003e?\\\^",^" "json.txt"
"FINDSTR"    /C:^"!@#$%%€^^^&^*^(^)={}[]^|\\;:'\\\^",./^<^>?^" "json.txt"
"FINDSTR" /R /C:^"!@#$%%€^^^&^\*^(^)={}\[\]^|\\;:'\\\^",./^<^>?^" "json.txt"
endlocal
"FINDSTR"    /C:^"\\\^"selectedProfile\\\^": \\\^"^^^!@#$%%^^^^\\u0026*^(^)\\u003d{}[]^|\\\\;:\\u0027\\\\\^",./\\u003c\\u003e?\\\^",^" "json.txt"
"FINDSTR" /R /C:^"\\\^"selectedProfile\\\^": \\\^"^^^!@#$%%^^^^\\u0026\*^(^)\\u003d{}\[\]^|\\\\;:\\u0027\\\\\^",./\\u003c\\u003e?\\\^",^" "json.txt"
"FINDSTR"    /C:^"^^^!@#$%%€^^^^^&^*^(^)={}[]^|\\;:'\\\^",./^<^>?^" "json.txt"
"FINDSTR" /R /C:^"^^^!@#$%%€^^^^^&^\*^(^)={}\[\]^|\\;:'\\\^",./^<^>?^" "json.txt"
endlocal
So escaping should be possible in all modes, and for all strings.
Just escape the string in the format that is supported by findstr.

In addition:
It doesn't matter if a string is in the (un)escaped form of a third party program.
Findstr doesn't care of their (the third party programs) escaping rules, as it has its own (that are not depending on delayed expansion) to unambiguously describe the search string.

If findstr is executed from within a batch file (or from command line) you also have to apply the cmd interpreter escaping rules to the findstr command (these rules are varying depending on the mode; for example escape of ! is only needed if delayed expansion is enabled).

penpen

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 20 Mar 2014 06:09
by penpen
Sorry, it was too late (for me) yesterday, so i've forgotten to post an findstr example where to apply the escape sequence (may be incomplete):

Code: Select all

@echo off
cls
setlocal disableDelayedExpansion
rem seacrh string 1: "selectedProfile": "!@#$%^\u0026*()\u003d{}[]|\\;:\u0027\",./\u003c\u003e?",
rem seacrh string 2: "selectedProfile": "!@#$%^\u0026*()\u003d{}[]|\\;:\u0027\",./\u003c\u003e?",

rem cmd line interpreter escaped search strings in disabled delayed expansion style:
rem The aterisk (*) should be double escaped (^^^*), too, as it is a problematic character on string replacement
set ^"string1=^"selectedProfile^": ^"!@#$%%^^\u0026^^^*^(^)\u003d{}[]^|\\;:\u0027\^",./\u003c\u003e?^",^"
set ^"string2=!@#$%%€^^^&^^^*^(^)={}[]^|\;:'^",./^<^>?^"

setlocal enableDelayedExpansion
set ^"string3=^"selectedProfile^": ^"^^^!@#$%%^^^^\u0026^^^^*^(^)\u003d{}[]^|\\;:\u0027\^",./\u003c\u003e?^",^"
set ^"string4=^!@#$%%€^^^&^^^*^(^)={}[]^|\;:'^",./^<^>?^"

rem apply findstr default escapes
for %%a in (1 2 3 4) do (
   set "searchString%%a=!string%%a:\=\\!"
   set "searchString%%a=!searchString%%a:"=\\\^"!"
   set ^"searchString%%a=^^^"!searchString%%a!^^^"^"
   set "searchString%%a=!searchString%%a:^*=*!"
)

rem apply findstr regular expression escapes
for %%a in (1 2 3 4) do (
   set "searchStringR%%a=!string%%a:\=\\!"
   set "searchStringR%%a=!searchStringR%%a:"=\\\^"!"
   set ^"searchStringR%%a=^^^"!searchStringR%%a!^^^"^"
   set "searchStringR%%a=!searchStringR%%a:[=\[!"
   set "searchStringR%%a=!searchStringR%%a:]=\]!"
   set "searchStringR%%a=!searchStringR%%a:^*=\*!"
   set "searchStringR%%a=!searchStringR%%a:$=\$!"
)

"FINDSTR"    /C:!searchString1!  "json.txt"
"FINDSTR" /R /C:!searchStringR1! "json.txt"
"FINDSTR"    /C:!searchString3!  "json.txt"
"FINDSTR" /R /C:!searchStringR3! "json.txt"

"FINDSTR"    /C:!searchString2!  "json.txt"
"FINDSTR" /R /C:!searchStringR2! "json.txt"
"FINDSTR"    /C:!searchString4!  "json.txt"
"FINDSTR" /R /C:!searchStringR4! "json.txt"

endlocal
endlocal
goto :eof
This works at least on my win xp (32 bit) (win7 not tested).

penpen

Re: FINDSTR + literal strings/double quotes and DelayedExpan

Posted: 20 Mar 2014 08:31
by ©opy[it]®ight
omg.. I would never be able to do this on my own, penpen :shock: (reading this, Squashman? :wink: ).
And it even works under Win 7 :) I hope this stuff didn't keep you from sleeping? :P

Does this escape all known (to-be-escaped) characters or is there more to it?

I still have to learn a lot, but thanks to you (and others) i can study this a lot better! 8)

(they really need to add a reputation/thanks button here)

Vielen Dank!
All times are UTC-06:00
Page 1 of 2
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/