why does this code output such result

[sincos2007]

code:

Code: Select all

set "^=escapeOnly"
set hi=hi001
set str1="a^!bcd^!efg^^^^^^^^^^^^^!^^^^^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!"


echo ----- original ----
echo  !str1!


echo ----- other 4 ----- 2x
echo " %str1%

output:

Code: Select all

----- original ----
 "a!bcd!efg^^^^^^!^^^^^!^^^^^^^escapeOnly "

----- other 4 ----- 2x
" "aefg^!^^escapeOnly "

my confusion is in other 4, I tried to do expand and escape operations to build output in my mind, but I can not understand why my code output this.

thanks

[jeb]

The question is: What do you expect?

- Simplify your test to only show the unexpected part.
- Use "set str" to show the unmodified content after the set str=... command, to be sure that the correct content is there

[sincos2007]

look at this link: viewtopic.php?f=3&t=9137

I want to research how the example in link above works when I assign const string to variable str1

thanks

[dbenham]

Your code is a goofy test of parsing - vary hard to follow. But it gives the expected result.

I assume you are familiar with the various phases involved when parsing batch scripts, including a detailed description of the phase 1 and phase 5 rules for percent and delayed expansion. If not, then you need to read them, especially phases 1, 2, and 5.

Starting with this command: echo " %str1%

After phase 1 percent expansion we have:
echo " "a!bcd!efg^^^^^^!^^^^^!^^^^^^^escapeOnly "

Now in phase 2, none of the characters are quoted after the 2nd quote (up until the last quote).The unquoted ^ are all special. I've color coded the escape pairs above in red and blue to make the phase 2 result easier to see. Each ^^ becomes ^ and ^! becomes ! and ^e becomes e

After phase 2, we have:
echo " " a!bcd!efg^^^!^^!^^^escapeOnly "

You have delayed expansion enabled, and there are ! characters, so phase 5 delayed expansion becomes important. I have color coded the escape pairs above in red and blue again. Each pair simply results in a single character literal. Each ^^ becomes ^ and ^! becomes ! and ^e becomes e. The !bcd! expands to nothing because there is no bcd environment variable. The ! is stripped because there is no ! or : following it.

After phase 5 we have:
echo " "aefg^!^^escapeOnly "

The line is executed in phase 7, and you get your final result.

Your oddly named ^ variable is never used.


Dave Benham

[sincos2007]

Hi Dave,

Thank you for your help. The content is very detailed.

One question: after phase 2, why third ! and ^^ and fourth ! does not expand to value of variable ^^, but only stripe fourth ! .

thanks

[sincos2007]

Is it because third exclamation mark is escaped and lost its special meaning, so it and fourth exclamation mark cannot be used to expand variable?

[dbenham]

Yes - I thought that is what I effectively demonstrated in my explanation.

But even if the 3rd ! was not escaped for round 5, it still would not expand to escapeOnly because you have !^^!, not !^!.

If you want to get escapeOnly in round 5, then you need to remove the round 5 escape before the 3rd !, and remove one of the round 5 ^ after it. In your code that is achieved by removing ^^ before and ^^^^ after in the original definition of str1.

Code: Select all

set str1="a^!bcd^!efg^^^^^^^^^^^!^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!"

I will leave it up to you to figure out why the above "works".

Your testing will be much easier if you define str1 with delayed expansion disabled. Then you don't need to worry about phase 5 escaping and expansion during the definition.

Use SET STR1 to see the original definition.

Then you should first ECHO the percent expansion with delayed expansion disabled so you can see the result of phase 2.

Then do the ECHO with delayed expansion enabled so you can see the result of phase 5.


Dave Benham

[sincos2007]

I do not understand clearly from your explanation why code:

Code: Select all

echo “ %str1%

need phase 2, but code:

Code: Select all

echo %str1%

does not.

output of code: echo %str1%

Code: Select all

"aefg^^^^^^escapeOnly "

thanks

[dbenham]

I never said that, so I'm not sure why you are reaching that conclusion.

Of course both forms pass through phase 2.

My original explanation attempts to show a command, and then illustrate the intermediate result of each phase.

And here is an example of what I intended in my last post:

Code: Select all

@echo off
setlocal disableDelayedExpansion

set "^=(ExpandedCaret)"
set "str1=!^^!-"!^^!""
set "str2=^^!^^^^^^!-"^^!^^^^^^!""

:: Show values of variables
set ^^
set str

:: Show command that we are evaluating by doubling all % and doubling all unquoted ^
echo Evaluating:        echo ExpandedInPhase1:%%^^%%  ExpandedInPhase5:%%str1%%  NotExpanded:%%str2%%

:: Show result of Phases 1 and 2
echo AfterPhases1and2:  ExpandedInPhase1:%^%  ExpandedInPhase5:%str1%  NotExpanded:%str2%

setlocal enableDelayedExpansion
:: Result of phases 1 and 2 the same, so difference shows result of phase 5
echo AfterPhases1,2,5:  ExpandedInPhase1:%^%  ExpandedInPhase5:%str1%  NotExpanded:%str2%

--OUTPUT--

Code: Select all

^=(ExpandedCaret)
str1=!^^!-"!^!"
str2=^^!^^^^^^!-"^!^^^!"
Evaluating:        echo ExpandedInPhase1:%^%  ExpandedInPhase5:%str1%  NotExpanded:%str2%
AfterPhases1and2:  ExpandedInPhase1:(ExpandedCaret)  ExpandedInPhase5:!^!-"!^!"  NotExpanded:^!^^^!-"^!^^^!"
AfterPhases1,2,5:  ExpandedInPhase1:(ExpandedCaret)  ExpandedInPhase5:(ExpandedCaret)-"(ExpandedCaret)"  NotExpanded:!^!-"!^!"

Remember, there is escaping happening in both phases 2 and 5, which is why str1 and str2 are defined the way they are.

To preserve ^ literal in phase 2 you need "^" if quoted, and ^^ if not quoted. A ! literal just needs itself in phase 2.

To preserve ^ literal in phase 5 you need ^^ and ! literal requires ^!, regardless whether the strings are quoted or not.

But the string must pass through phase 2 before it gets to phase 5. So unquoted ^ and ! require ^^^^ ^^! and quoted requires "^^ ^!"

Here is effectively the same test, but without using the intermediate str1 and str2 variables:

Code: Select all

@echo off
setlocal disableDelayedExpansion

set "^=(ExpandedCaret)"

:: Show values of variables
set ^^

:: Show command that we are evaluating by doubling all % and doubling all unquoted ^
echo Evaluating:        echo ExpandedInPhase1:%%^^%%  ExpandedInPhase5:!^^^^!-"!^!"  NotExpanded:^^^^!^^^^^^^^^^^^!-"^!^^^!"

:: Show result of Phases 1 and 2
echo AfterPhases1and2:  ExpandedInPhase1:%^%  ExpandedInPhase5:!^^!-"!^!"  NotExpanded:^^!^^^^^^!-"^!^^^!"

setlocal enableDelayedExpansion
:: Result of phases 1 and 2 the same, so difference shows result of phase 5
echo AfterPhases1,2,5:  ExpandedInPhase1:%^%  ExpandedInPhase5:!^^!-"!^!"  NotExpanded:^^!^^^^^^!-"^!^^^!"

--OUTPUT--

Code: Select all

^=(ExpandedCaret)
Evaluating:        echo ExpandedInPhase1:%^%  ExpandedInPhase5:!^^!-"!^!"  NotExpanded:^^!^^^^^^!-"^!^^^!"
AfterPhases1and2:  ExpandedInPhase1:(ExpandedCaret)  ExpandedInPhase5:!^!-"!^!"  NotExpanded:^!^^^!-"^!^^^!"
AfterPhases1,2,5:  ExpandedInPhase1:(ExpandedCaret)  ExpandedInPhase5:(ExpandedCaret)-"(ExpandedCaret)"  NotExpanded:!^!-"!^!"

I hope the above makes things clearer. I don't have any more explanation to give you. I understand this is complicated. It has taken me years to develop my understanding.


Dave Benham

[sincos2007]

Sorry, I have not state my question in accuracy.

This is another question.

I want to know why following two commands have different output:

Code: Select all

echo " %str1%

Code: Select all

echo %str1%

if definition of str1 is:

Code: Select all

set str1="a^!bcd^!efg^^^^^^^^^^^!^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!"

or

if definition of str1 after removing quotes is:

Code: Select all

set str1=a^!bcd^!efg^^^^^^^^^^^!^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!

thanks

[dbenham]

Of course adding the quote makes a profound difference - it inverts what is quoted within the value of %str%. Quoted carets don't escape anything in phase 2 - they are simply string literals.

Dropping the variable and working just with strings.

echo "^A"^B"^A"^B yields "^A"B"^A"B because the A strings are quoted, and the B strings are not.

Add an extra quote, and things are reversed.

echo " "^A"^B"^A"^B yields " "A"^B"A"^B because the B strings are quoted, and the A strings are not. In addition, the space between the first pair of quotes is also quoted.

Another way to invert what is quoted is to escape the initial quote.

echo ^"^A"^B"^A"^B yields "A"^B"A"^B because the first quote is escaped, so the A strings are NOT quoted, and the B strings are quoted.

The first quote turns quoting on, unless it is escaped. But once quoting starts, the very next quote always turns it off because you cannot escape anything within a quoted string, not even the closing quote.

I'll add some more carets to demonstrate.

echo "^A^"^B"^A^"^B yields "^A^"B"^A^"B, again because the A strings are quoted.

But escape the first quote, and the result is dramatically different.

echo ^"^A^"^B"^A^"^B yields "A"B"^A^"B. Only the last A string is quoted. The first two quotes are escaped, so the first A and B strings are NOT quoted. Of course the last B string is also NOT quoted.


Dave Benham

[sincos2007]

Very appreciate for your help and patience. Your explanation is valuable and rare. Very clear and helpful and into detail. Hope this could help others.

Thanks.