Page 1 of 1
why does this code output such result
Posted: 28 May 2019 15:43
by sincos2007
code:
Code: Select all
set "^=escapeOnly"
set hi=hi001
set str1="a^!bcd^!efg^^^^^^^^^^^^^!^^^^^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!"
echo ----- original ----
echo !str1!
echo ----- other 4 ----- 2x
echo " %str1%
output:
Code: Select all
----- original ----
"a!bcd!efg^^^^^^!^^^^^!^^^^^^^escapeOnly "
----- other 4 ----- 2x
" "aefg^!^^escapeOnly "
my confusion is in other 4, I tried to do expand and escape operations to build output in my mind, but I can not understand why my code output this.
thanks
Re: why does this code output such result
Posted: 29 May 2019 04:40
by jeb
The question is: What do you expect?
- Simplify your test to only show the unexpected part.
- Use "set str" to show the unmodified content after the set str=... command, to be sure that the correct content is there
Re: why does this code output such result
Posted: 29 May 2019 06:05
by sincos2007
look at this link:
viewtopic.php?f=3&t=9137
I want to research how the example in link above works when I assign const string to variable str1
thanks
Re: why does this code output such result
Posted: 29 May 2019 08:42
by dbenham
Your code is a goofy test of parsing - vary hard to follow. But it gives the expected result.
I assume you are familiar with
the various phases involved when parsing batch scripts, including a detailed description of the
phase 1 and phase 5 rules for percent and delayed expansion. If not, then you need to read them, especially phases 1, 2, and 5.
Starting with this command:
echo " %str1%
After phase 1 percent expansion we have:
echo " "a!bcd!efg^^^^^^!^^^^^!^^^^^^^escapeOnly "
Now in phase 2, none of the characters are quoted after the 2nd quote (up until the last quote).The unquoted
^ are all special. I've color coded the escape pairs above in red and blue to make the phase 2 result easier to see. Each
^^ becomes
^ and
^! becomes
! and
^e becomes
e
After phase 2, we have:
echo " " a!bcd!efg^^^!^^!^^^escapeOnly "
You have delayed expansion enabled, and there are ! characters, so phase 5 delayed expansion becomes important. I have color coded the escape pairs above in red and blue again. Each pair simply results in a single character literal. Each
^^ becomes
^ and
^! becomes
! and
^e becomes
e. The
!bcd! expands to nothing because there is no bcd environment variable. The
! is stripped because there is no ! or : following it.
After phase 5 we have:
echo " "aefg^!^^escapeOnly "
The line is executed in phase 7, and you get your final result.
Your oddly named
^ variable is never used.
Dave Benham
Re: why does this code output such result
Posted: 29 May 2019 22:19
by sincos2007
Hi Dave,
Thank you for your help. The content is very detailed.
One question: after phase 2, why third ! and ^^ and fourth ! does not expand to value of variable ^^, but only stripe fourth ! .
thanks
Re: why does this code output such result
Posted: 30 May 2019 03:13
by sincos2007
Is it because third exclamation mark is escaped and lost its special meaning, so it and fourth exclamation mark cannot be used to expand variable?
Re: why does this code output such result
Posted: 30 May 2019 05:41
by dbenham
Yes - I thought that is what I effectively demonstrated in my explanation.
But even if the 3rd ! was not escaped for round 5, it still would not expand to escapeOnly because you have !^^!, not !^!.
If you want to get escapeOnly in round 5, then you need to remove the round 5 escape before the 3rd !, and remove one of the round 5 ^ after it. In your code that is achieved by removing ^^ before and ^^^^ after in the original definition of str1.
Code: Select all
set str1="a^!bcd^!efg^^^^^^^^^^^!^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!"
I will leave it up to you to figure out why the above "works".
Your testing will be much easier if you define str1 with delayed expansion disabled. Then you don't need to worry about phase 5 escaping and expansion during the definition.
Use SET STR1 to see the original definition.
Then you should first ECHO the percent expansion with delayed expansion disabled so you can see the result of phase 2.
Then do the ECHO with delayed expansion enabled so you can see the result of phase 5.
Dave Benham
Re: why does this code output such result
Posted: 31 May 2019 03:57
by sincos2007
I do not understand clearly from your explanation why code:
need phase 2, but code:
does not.
output of code: echo %str1%
thanks
Re: why does this code output such result
Posted: 31 May 2019 06:51
by dbenham
I never said that, so I'm not sure why you are reaching that conclusion.
Of course both forms pass through phase 2.
My original explanation attempts to show a command, and then illustrate the intermediate result of each phase.
And here is an example of what I intended in my last post:
Code: Select all
@echo off
setlocal disableDelayedExpansion
set "^=(ExpandedCaret)"
set "str1=!^^!-"!^^!""
set "str2=^^!^^^^^^!-"^^!^^^^^^!""
:: Show values of variables
set ^^
set str
:: Show command that we are evaluating by doubling all % and doubling all unquoted ^
echo Evaluating: echo ExpandedInPhase1:%%^^%% ExpandedInPhase5:%%str1%% NotExpanded:%%str2%%
:: Show result of Phases 1 and 2
echo AfterPhases1and2: ExpandedInPhase1:%^% ExpandedInPhase5:%str1% NotExpanded:%str2%
setlocal enableDelayedExpansion
:: Result of phases 1 and 2 the same, so difference shows result of phase 5
echo AfterPhases1,2,5: ExpandedInPhase1:%^% ExpandedInPhase5:%str1% NotExpanded:%str2%
Code: Select all
^=(ExpandedCaret)
str1=!^^!-"!^!"
str2=^^!^^^^^^!-"^!^^^!"
Evaluating: echo ExpandedInPhase1:%^% ExpandedInPhase5:%str1% NotExpanded:%str2%
AfterPhases1and2: ExpandedInPhase1:(ExpandedCaret) ExpandedInPhase5:!^!-"!^!" NotExpanded:^!^^^!-"^!^^^!"
AfterPhases1,2,5: ExpandedInPhase1:(ExpandedCaret) ExpandedInPhase5:(ExpandedCaret)-"(ExpandedCaret)" NotExpanded:!^!-"!^!"
Remember, there is escaping happening in both phases 2 and 5, which is why str1 and str2 are defined the way they are.
To preserve ^ literal in phase 2 you need "^" if quoted, and ^^ if not quoted. A ! literal just needs itself in phase 2.
To preserve ^ literal in phase 5 you need ^^ and ! literal requires ^!, regardless whether the strings are quoted or not.
But the string must pass through phase 2 before it gets to phase 5. So unquoted ^ and ! require ^^^^ ^^! and quoted requires "^^ ^!"
Here is effectively the same test, but without using the intermediate str1 and str2 variables:
Code: Select all
@echo off
setlocal disableDelayedExpansion
set "^=(ExpandedCaret)"
:: Show values of variables
set ^^
:: Show command that we are evaluating by doubling all % and doubling all unquoted ^
echo Evaluating: echo ExpandedInPhase1:%%^^%% ExpandedInPhase5:!^^^^!-"!^!" NotExpanded:^^^^!^^^^^^^^^^^^!-"^!^^^!"
:: Show result of Phases 1 and 2
echo AfterPhases1and2: ExpandedInPhase1:%^% ExpandedInPhase5:!^^!-"!^!" NotExpanded:^^!^^^^^^!-"^!^^^!"
setlocal enableDelayedExpansion
:: Result of phases 1 and 2 the same, so difference shows result of phase 5
echo AfterPhases1,2,5: ExpandedInPhase1:%^% ExpandedInPhase5:!^^!-"!^!" NotExpanded:^^!^^^^^^!-"^!^^^!"
Code: Select all
^=(ExpandedCaret)
Evaluating: echo ExpandedInPhase1:%^% ExpandedInPhase5:!^^!-"!^!" NotExpanded:^^!^^^^^^!-"^!^^^!"
AfterPhases1and2: ExpandedInPhase1:(ExpandedCaret) ExpandedInPhase5:!^!-"!^!" NotExpanded:^!^^^!-"^!^^^!"
AfterPhases1,2,5: ExpandedInPhase1:(ExpandedCaret) ExpandedInPhase5:(ExpandedCaret)-"(ExpandedCaret)" NotExpanded:!^!-"!^!"
I hope the above makes things clearer. I don't have any more explanation to give you. I understand this is complicated. It has taken me years to develop my understanding.
Dave Benham
Re: why does this code output such result
Posted: 01 Jun 2019 01:54
by sincos2007
Sorry, I have not state my question in accuracy.
This is another question.
I want to know why following two commands have different output:
if definition of str1 is:
Code: Select all
set str1="a^!bcd^!efg^^^^^^^^^^^!^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!"
or
if definition of str1 after removing quotes is:
Code: Select all
set str1=a^!bcd^!efg^^^^^^^^^^^!^^^^^^^!^^^^^^^^^^!hi^^^^^^^^^!^^^^!^!
thanks
Re: why does this code output such result
Posted: 01 Jun 2019 06:08
by dbenham
Of course adding the quote makes a profound difference - it inverts what is quoted within the value of %str%. Quoted carets don't escape anything in phase 2 - they are simply string literals.
Dropping the variable and working just with strings.
echo "^A"^B"^A"^B yields "^A"B"^A"B because the A strings are quoted, and the B strings are not.
Add an extra quote, and things are reversed.
echo " "^A"^B"^A"^B yields " "A"^B"A"^B because the B strings are quoted, and the A strings are not. In addition, the space between the first pair of quotes is also quoted.
Another way to invert what is quoted is to escape the initial quote.
echo ^"^A"^B"^A"^B yields "A"^B"A"^B because the first quote is escaped, so the A strings are NOT quoted, and the B strings are quoted.
The first quote turns quoting on, unless it is escaped. But once quoting starts, the very next quote always turns it off because you cannot escape anything within a quoted string, not even the closing quote.
I'll add some more carets to demonstrate.
echo "^A^"^B"^A^"^B yields "^A^"B"^A^"B, again because the A strings are quoted.
But escape the first quote, and the result is dramatically different.
echo ^"^A^"^B"^A^"^B yields "A"B"^A^"B. Only the last A string is quoted. The first two quotes are escaped, so the first A and B strings are NOT quoted. Of course the last B string is also NOT quoted.
Dave Benham
Re: why does this code output such result
Posted: 01 Jun 2019 14:31
by sincos2007
Very appreciate for your help and patience. Your explanation is valuable and rare. Very clear and helpful and into detail. Hope this could help others.
Thanks.