DosTips.com • Question About Quotation Marks

Page 1 of 2

Question About Quotation Marks

Posted: 28 Feb 2013 18:53

by Queue

As jeb's answer here: viewtopic.php?f=3&t=4345 highlighted, quotation marks in env vars often make if statements explode. We wrap each side of the comparison in quotation marks to avoid other problems.

So when I was working on something to check if a batch was run via double-click / ShellExecute / etc. versus run directly via command line, I got confused by situations where quotation marks in env vars WASN'T making things explode.

Please forgive the use of " as the name of the env var I work with; it makes things more confusing to read, but I wanted to show exactly what I was working with:

Code: Select all

set CMDCMDLINE=%CMDCMDLINE%
set ""=%0"
set ""=%":~2,1%%CMDCMDLINE:~-2,1%%CMDCMDLINE:~10,1%"
set ""=%":"=%"
if /i not "%~0"=="%~f0" set ""="
if "%"%"==": :" ( set ""="" ) else set ""="
if not defined ^" echo on & prompt;::$S$T$_

Most noteworthy is set ""=%0" which, even if %0 contains quotation marks, doesn't break. But I don't understand why. Is set just special and matches the first quotation mark to the final quotation mark, rather than to the first intermediate quotation mark it encounters? Does elsewhere in batch behave this way, because it sure doesn't seem to in if statements where it would be useful.

Also, set ""=" to clear the " env var had to end a line as I was having it choke if there was whitespace after it.

Aaand, how viable of a check is this? It works for me on XP SP3 but I haven't tested it anywhere else yet. The "%~0"=="%~f0" comparison was an afterthought and I intend to work it into the test a little more fluidly.

Queue

Re: Question About Quotation Marks

Posted: 01 Mar 2013 06:29

by jeb

Hi Queue,

I like your tests

But in this case there isn't any special at %0.

You could test it with a file named:
cat&dog.bat

Code: Select all

@echo on
rem # %0 #
set ""=%0"

Now start it with

Code: Select all

cat^&dog.bat

It fails with an interesting effect :shock:

:shock:

,
but in the REM output you can see that the real content in %0 was cat&dog.bat

jeb

Re: Question About Quotation Marks

Posted: 01 Mar 2013 15:48

by Queue

Haha, yeah, I know some of the other special characters make this explode.

I'm just confused by the quotation marks though. As a simpler example:

Code: Select all

set ""="hello jeb""

Why does this work? Why doesn't the first quotation mark pair with the next quotation mark (and break everything)?

Am I misunderstanding how quotation marks work? In an if statement, it's always seemed like a quotation mark pairs with the next quotation mark it finds, like:
if not " "yes" "=="no" echo not a match
Where I color-coded what I think are the paired quotation marks. In my previous example, it seems to be:
set ""="hello jeb""

I expect I can remove set ""=%":"=%" and move the whole thing over to delayed expansion to avoid other special characters.

Queue

Re: Question About Quotation Marks

Posted: 01 Mar 2013 22:30

by Queue

jeb wrote:Now start it with
Code: Select all
cat^&dog.bat
It fails with an interesting effect :shock:,

jeb

Ok, that is madness.

Queue

Re: Question About Quotation Marks

Posted: 02 Mar 2013 01:02

by jeb

Queue wrote:Why does this work? Why doesn't the first quotation mark pair with the next quotation mark (and break everything)?

Am I misunderstanding how quotation marks work? In an if statement, it's always seemed like a quotation mark pairs with the next quotation mark it finds, like:
if not " "yes" "=="no" echo not a match
Where I color-coded what I think are the paired quotation marks. In my previous example, it seems to be:
set ""="hello jeb""

1. You are right

Quotation marks are always paired with the next one

2. Why should it fail? The quotes are only for determing to escape or not escape characters.
How the line is handled is defined by the command of the line.
In this case it's a SET command, their is the rule, if the variable is prefixed with a quoteation mark all characters are used until the last quotation mark, the rest is removed.
The SET command doesn't care about special characters, these are handled before by the standard parser.

Therefore you only get problems when special characters are outside of the quoted regions here.

But the IF command uses an own parser, this parser splits the tokens at spaces and other delimiters,
that's why an IF breaks without quotes.

jeb

Re: Question About Quotation Marks

Posted: 03 Mar 2013 03:52

by Queue

Got it, and now I see what a fluke it was that using " as my env var name was making set ""=%0" work when %0 had enclosing quotation marks.

Since putting an & in the name of the .bat file as per your example certainly shows where things break, I've gone with that for testing (though I went with test27&echo.bat).

Right now I've moved to a delayed expansion test of cmdcmdline first, and only if that passes inspection do I then attempt a set ""="%~0"" and if THAT passes inspection, I then attempt a set ""=%0". I know it's a bit crazy, but that's sort've the point. After I iron out a few wrinkles, I'll toss it up here in case you want to break my next iteration as well.

Edit - test28&echo.bat

Code: Select all

@setlocal
:: @echo off
@prompt;::$S$T$_

setlocal enabledelayedexpansion
set ""=^" & set CMDCMDLINE=!CMDCMDLINE!
if not "!CMDCMDLINE:~10,1!"==":" goto:chk
if not "!CMDCMDLINE:~-2,1!"==" " goto:chk
set ""="%~0""
if not ^"!":~2,1!"==":" set ""=^" & goto:chk
set ""=%0"
if not ^"!":~2,1!"==":" set ""=^" & goto:chk
set ""=:^"
:chk
endlocal & if not "%"%"=="" ( set ""="" ) else set ""=^"

echo [%"%]
if defined ^" pause & cls

Queue

Re: Question About Quotation Marks

Posted: 03 Mar 2013 11:08

by jeb

Queue wrote:I know it's a bit crazy, but that's sort've the point. After I iron out a few wrinkles, I'll toss it up here in case you want to break my next iteration as well.

I rejoice in breaking your attempt.

I always start my batch files with this

Code: Select all

"&"^&\..\test.bat

Now, I'm waiting to see your solution for this one

jeb

Re: Question About Quotation Marks

Posted: 04 Mar 2013 04:21

by Queue

To be fair, the CMDCMDLINE checks would prevent that from being a problem under normal circumstances; I had to make a second batch file that I run via double-click that then calls the original batch file via "&"^&\..\ to get it to explode.

I also went further and named it test30^=&echo.bat (and have test30^=&rem.bat run it directly via @"&"^&\..\test30^^^=^&echo.bat test).

Anyhow, here's my next attempt, test30^=&echo.bat

Code: Select all

@setlocal
@echo off
:: @prompt;::$S$T$_

setlocal enabledelayedexpansion
set ""=^" & set CMDCMDLINE=!CMDCMDLINE!
if not "!CMDCMDLINE:~10,1!"==":" goto:chk
if not "!CMDCMDLINE:~-2,1!"==" " goto:chk
set BAKCMDLINE=!CMDCMDLINE!
set CMDCMDLINE=
call(!CMDCMDLINE:*:=*!!CMDCMDLINE:~0,1!
call call(%%%%CMDCMDLINE:**=*%%0%%%%
set CMDCMDLINE=!CMDCMDLINE:^^^^=^^!
if "!CMDCMDLINE:~3,1!"==":" set ""=:^"
set CMDCMDLINE=
call(!CMDCMDLINE:~0,1!%%CMDCMDLINE:**=!BAKCMDLINE!%%
call(!CMDCMDLINE:^^^^=^^!
:chk
endlocal & if not "%"%"=="" ( set ""="" ) else set ""=^"

echo [%"%]

pause
cls

Queue

Re: Question About Quotation Marks

Posted: 04 Mar 2013 07:35

by jeb

Ok, I needed a bit of time to create some nasty errors.

Try to name your files like this

Code: Select all

test& if exists 2 %%0XYZ.bat
or 
test %%~*

Both outputs an error but the batch still works.

jeb

Re: Question About Quotation Marks

Posted: 04 Mar 2013 15:16

by Queue

Whew, those are devilish. This isn't a complete solution, but it might be on the right track; if nothing else, it makes things more readable and makes spotting where things are choking easier:

test32&if;exists;2.bat

Code: Select all

@setlocal
@echo off
:: @prompt;::$S$T$_

setlocal enabledelayedexpansion
set ""=^"
set {=!CMDCMDLINE!
if not "!{:~10,1!"==":" goto:chk
if not "!{:~-2,1!"==" " goto:chk
call(:^&rem;!CMDCMDLINE:*:=*!_error_1
call(:^&rem;!CMDCMDLINE:~0,1!_error_2
call call(%%%%CMDCMDLINE:**=*%%0%%%%_error_3
set }=!CMDCMDLINE:^^^^=^^!
if "!}:~3,1!"==":" set ""=:^"
call(:^&rem;!CMDCMDLINE:~0,1!_error_4
call(:^&rem;%%CMDCMDLINE:**=!{!%%_error_5
call(:^&rem;!CMDCMDLINE:^^^^=^^!_error_6
if !CMDCMDLINE!==!{! echo CMDCMDLINE restored
:chk
endlocal & if not "%"%"=="" ( set ""="" ) else set ""=^"

echo [%"%]

if not defined ^" goto:jmp
"&"^&\..\test32^&if^;exists^;2.bat test
:jmp

pause
cls

REM obviously isn't protection from the likes of %%~, and I haven't worked out what my options are for the doubled call (error 3), but it seems more robust than wrapping the guts of the call( in two ^^^"'s.

Code: Select all

call(^^^"!CMDCMDLINE:*:=*!_error_1^^^"

Edit - I suppose I could just cheat and suppress the remaining error messages, but I don't like that solution (but I partially added it in for the version below; errors 1 and 7 would also need to be suppressed). Regardless, here's an iteration that fixes CMDCMDLINE restoration in case there's a % in the file name; the placeholder character is a TAB (which are showing up as 3 spaces for me in this post):

test33&%%~.bat

Code: Select all

@setlocal
@echo off
:: @prompt;::$S$T$_

setlocal enabledelayedexpansion
set ""=^"
set {=!CMDCMDLINE!
if not "!{:~10,1!"==":" goto:chk
if not "!{:~-2,1!"==" " goto:chk
call(:^&rem;!CMDCMDLINE:*:=*!_error_1
call(:^&rem;!CMDCMDLINE:~0,1!_error_2
2>nul call call(%%%%CMDCMDLINE:**=*%%0%%%%_error_3
set }=!CMDCMDLINE:^^^^=^^!
if "!}:~3,1!"==":" set ""=:^"
call(:^&rem;!CMDCMDLINE:~0,1!_error_4
call(:^&rem;%%CMDCMDLINE:**=!{:%%=   !%%_error_5
call(:^&rem;!CMDCMDLINE:^^^^=^^!_error_6
call(:^&rem;!CMDCMDLINE:   =%%!_error_7
if !CMDCMDLINE!==!{! echo CMDCMDLINE restored
:chk
endlocal & if not "%"%"=="" ( set ""="" ) else set ""=^"

echo [%"%]

if not defined ^" goto:jmp
"&"^&\..\test33^&%%%%~.bat test
:jmp

pause
cls

Any advice on a better placeholder?

Queue

Re: Question About Quotation Marks

Posted: 05 Mar 2013 03:57

by jeb

Hi Queue,

I don't understand what you try with

Code: Select all

call(:^&REM

The ^& will not work, you can escape an ampersand here, but in the second parser run it will completely cancel the CALL,
as all special characters will fail in the second run.
So your REM has no effect, you could also use a simple echo or anything else

This will never work

Code: Select all

call echo one ^& echo two

Why you use two CALL's here?

Code: Select all

call call(%%%%CMDCMDLINE:**=*%%0%%%%_error_3

Why not only

Code: Select all

call(%%CMDCMDLINE:**=*%0%%_error_3

jeb

Re: Question About Quotation Marks

Posted: 05 Mar 2013 04:55

by Queue

The double call is to break up parsing into 3 passes.

The first pass is carefully controlled and we precisely know the output:

Code: Select all

call(%%CMDCMDLINE:**=*%0%%_error_3

Otherwise, during this first pass, if %0 had already been expanded, inserted quotation marks, ampersands, etc. could blow everything up.

At the end of the first pass, one call is removed, and the second pass occurs following rules of expansion during a call (no delayed expansion, carets are doubled). The %0 is expanded, the %'s around CMDCMDLINE are halved and prepared for the next pass.

At the end of the second pass, the second call is removed, and the third pass occurs, also following rules of expansion during a call. The ( at the start of the call should be dummying out the call attempt (not actually tested) and this is when CMDCMDLINE gets modified and any control characters make stuff blow up. For whatever reason, things blowing up at this point doesn't make cmd.exe close, so that's nice.

I think I have all of that right. Maybe stuff blows up during the second pass and not the third; it's 3 AM and I kinda forget at the moment. Honestly, just change the double call to the reduced single call you suggest and observe the difference in errors.

double call:

Code: Select all

The following usage of the path operator in batch-parameter
substitution is invalid: %~.bat"%_error_3

For valid formats type CALL /? or FOR /?
CMDCMDLINE restored
["]
CMDCMDLINE restored
[]
Press any key to continue . . .

single call:

Code: Select all

The following usage of the path operator in batch-parameter
substitution is invalid: %~.bat"%_error_3

For valid formats type CALL /? or FOR /?
CMDCMDLINE restored
["]
'\..\test33' is not recognized as an internal or external command,
operable program or batch file.
'%%~.bat%_error_3' is not recognized as an internal or external command,
operable program or batch file.
CMDCMDLINE restored
[]
Press any key to continue . . .

Notice how the single call gives 2 "not recognized as an internal or external command" which indicate attempted file access. A parsing error is a much less serious error. The downside is calling a call has the same problem as calling any internal command where it searches %path% for a file named call. Since it's all wrapped in a setlocal anyway, I could just add

Code: Select all

path; & dpath; & set pathext=;

but I haven't gotten to that yet.

As to the weird REM's, I was attempting to trigger REM parsing rules during the second pass, but I think the results I was getting were a placebo. However, I was only able to get parsing errors (due to %~), and didn't see any bad command errors. Replacing :^&rem; with :^& serves the same purpose.

Edit - Here's a further iteration to play with. Unfortunately, it's getting more and more esoteric as it goes along. I'm not doing this to make it harder to read; just experimenting with different approaches to trigger variable expansion and avoid errors. At this point, changing to call set might be viable, but I want to exhaust abuse of CMDCMDLINE first.

test34&%~.bat

Code: Select all

@setlocal
@echo off
:: @prompt;::$S$T$_

setlocal enabledelayedexpansion
path; & dpath; & set pathext=;
set ""=^" & set "{=!CMDCMDLINE!"
if not "!{:~10,1!"==":" goto:chk
if not "!{:~-2,1!"==" " goto:chk
(::
:;: !CMDCMDLINE:*:=*!!CMDCMDLINE:~0,1!)
2>nul call call(%%%%CMDCMDLINE:**=*%%0%%%%
set "}=!CMDCMDLINE:^^^^=^^!"
if "!}:~3,1!"==":" set ""=:^"
(::
:;: !CMDCMDLINE:~0,1!)
call(:^&%%CMDCMDLINE:**=!{:%%=   !%%
(::
:;: !CMDCMDLINE:^^^^=^^!!CMDCMDLINE:   =%%!)
if !CMDCMDLINE!==!{! echo CMDCMDLINE restored
:chk
endlocal & if not "%"%"=="" ( set ""="" ) else set ""=^"

echo [%"%]

if not defined ^" goto:jmp
"&"^&\..\test34^&%%~.bat test
:jmp

pause
cls

I trade out most of the calls for () label parsing rules. Only the two that are reliant on second+ passes still have calls. The placeholder characters for the %'s are TAB characters again. There must be a better placeholder I can use.

Queue

Re: Question About Quotation Marks

Posted: 05 Mar 2013 15:44

by jeb

Nice try again

But now you can fight against this one :twisted:

:twisted:

Code: Select all

qTest&%%0&;&This.bat

Queue wrote:The placeholder characters for the %'s are TAB characters again. There must be a better placeholder I can use.

You could use one of the characters which can't be in the filename, like ?.

But perhaps it's much simpler to access %0 in a different way, with the REM technic (do you know this?)

SO: How to receive even the strangest command line parameters?

Then you don't need the calls at all.

jeb

Re: Question About Quotation Marks

Posted: 05 Mar 2013 17:03

by Queue

I hadn't seen that solution, and I'm happy to, because I had early on tried unsuccessfully to redirect a REM. I had read part of that DosTips thread related to it though, about the redirection madness.

Keep in mind that the point of this big mess is simply to safely check if the 3rd character of %0 is a :. I don't need to successfully read the entirety of %0, but I do need the batch to not abort if the file (or more importantly, a parent folder) has a crazy name, or if it's called in a weird way.

I'm not a big fan of temporary files, but for a final solution, not having to mess with a bug in CMDCMDLINE will likely be worth it.

As for qTest&%%0&;&This.bat, a 2>nul redirection before the second call silences the ''& was unexpected at this time.'' messages it spits out, and the final results are still accurate.

A ? does seem like a good idea, but CMDCMDLINE can contain a ? (or a TAB for that matter). During testing this, though, I found another crazy situation. Run my most recent iteration via something like:

Code: Select all

cmd /c ^""%~dp0test35&%%%%0&;&this.bat" "/?" ^"

(I just put it in a second batch file to call the main one).

That /? triggers the help text on the call that attempts to restore CMDCMDLINE!

Queue

Re: Question About Quotation Marks

Posted: 05 Mar 2013 17:23

by jeb

Queue wrote:A ? does seem like a good idea, but CMDCMDLINE can contain a ? (or a TAB for that matter).

But this case shouldn't occour, as if you start a batch from the explorer it hasn't parameters.

Queue wrote:That /? triggers the help text on the call that attempts to restore CMDCMDLINE!

I know

A /? anywhere in the line of a CALL triggers the help, except the /? is quoted or expanded in the call itself.
Even when there are spaces or other delimiters between / and ?.

Code: Select all

call set var=/?
call set var=/  ?
call set var=/;=, ?
setlocal EnableDelayedExpansion
set help=/?
call set var=!help!

But this one works

Code: Select all

set help=/?
call set var=%%help%%

jeb