IF curioisity: Delayed expansion of undefined variable = 0

[dbenham]

Very curious this
I'm testing on Vista 64. I haven't tried other versions yet.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "undefined="
if !undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)

Of course I get the expected result if I add another character to the comparison

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "undefined="
if "!undefined!" equ "0" (echo undefined = 0) else (echo undefined ^<^> 0)

Dave Benham

[aGerman]

I testet under Win7 x86 with the same result.
You don't get a syntax error because undefined was not expanded to nothing before cmd.exe parsed the line due to the delayed variable expansion. Actually it seems to be the normal behaviour that an undefined variable is set to 0 in a numeric context. Another test would be:

Code: Select all

set "undefined="
set /a n=undefined
echo %n%

You'll get back 0.
SET /A with only the variable name has a similar behaviour like delayed expansion. !undefined! would fail though. I think the whole thing happens on a deeper level of the cmd processing.

Regards
aGerman

[foxidrive]

FWIW it has the same behaviour under Windows 8 32 bit

[dbenham]

@aGerman - I too wondered if it might be related to SET /A handling of undefined variables, but then I reasoned it does not.

SET /A computes 0 if it encounters an unexpanded variable name that is undefined. But IF does not do the same thing with the variable name - the variable must be expanded for IF to see the value.

Also, SET /A does not treat delayed expansion of an undefined variable as 0

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "undefined="
set "n="
set /a "n=!undefined!"
echo n=!n!
set /a "n=1/!undefined!"
echo n=!n!

Result:

Code: Select all

Missing operand.
n=
Missing operand.
n=

If SET /A treated !undefined! as 0 then I would expect the first SET /A to return 0, and the second SET /A to generate a divide by 0 error. Instead, both generate the Missing operand error and n fails to get a value.

Also, in Rules for how CMD.EXE parses numbers I demonstrate how SET /A and IF treat number parsing differently.

I suppose it is still possible that the IF undefined behavior is related to SET /A, but my guess is it is not. I don't think we can ever know for sure without seeing the source code.


Dave Benham

[Liviu]

I'm testing on Vista 64. I haven't tried other versions yet.

Confirmed under XP 32b as well.

Actually it seems to be the normal behaviour that an undefined variable is set to 0 in a numeric context.

Indeed, and that's (obliquely) documented in the set/? help "if an environment variable name is specified but is not defined in the current environment, then a value of zero is used".

I don't think we can ever know for sure without seeing the source code.

Even then, only assuming the infinite acumen and dedication to follow through whatever mess is responsible for this

Code: Select all

@echo off & setlocal enableDelayedExpansion & set "undefined="
if !undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
if !undefined! == 0 (echo undefined = 0) else (echo undefined ^<^> 0)
if ==!undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
if =!undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
echo ...silent error
if !undefined!== equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
echo ...noisy error
if !undefined!= equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
echo ...never reached

Code: Select all

undefined = 0
undefined <> 0
undefined = 0
undefined = 0
...silent error
...noisy error
= was unexpected at this time.

Liviu

[dbenham]

I don't think we can ever know for sure without seeing the source code.

Even then, only assuming the infinite acumen and dedication to follow through whatever mess is responsible for this

Code: Select all

@echo off & setlocal enableDelayedExpansion & set "undefined="
if !undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
if !undefined! == 0 (echo undefined = 0) else (echo undefined ^<^> 0)
if ==!undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
if =!undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
echo ...silent error
if !undefined!== equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
echo ...noisy error
if !undefined!= equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
echo ...never reached

Code: Select all

undefined = 0
undefined <> 0
undefined = 0
undefined = 0
...silent error
...noisy error
= was unexpected at this time.

Liviu

Actually, most of those make "perfect" sense

Code: Select all

if !undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> undefined = 0

My original point of this thread, and the only thing that really surprises me.

Code: Select all

if !undefined! == 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> undefined <> 0

This does not surprise me since == is always a string comparison, never numeric. So the undefined 0 equivalence is strictly numeric based.

Code: Select all

if ==!undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> undefined = 0
if =!undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> undefined = 0

Leading unquoted token delimiters are converted to space and ignored. They are not part of the condition, as evidenced by

Code: Select all

if  ==   ,,  ;;  A equ A echo match
  --> match

Code: Select all

if !undefined!== equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> no output

Compares an empty string with equ. If it evaluated to TRUE then would attempt to execute 0 and the remainder would be parameters to the command. Easily proven by inserting NOT:

Code: Select all

if not !undefined!== equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> '0' is not recognized as an internal or external command, operable program or batch file.

Code: Select all

if !undefined!= equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> = was unexpected at this time.

Almost the same as the prior example, except the == comparison operator is malformed, yielding the syntax error.


Dave Benham

[Liviu]

Actually, most of those make "perfect" sense

Sure makes alot more sense with a good dose of hindsight

Code: Select all

if !undefined! equ 0 (echo undefined = 0) else (echo undefined ^<^> 0)
  --> undefined = 0

My original point of this thread, and the only thing that really surprises me.

If I were to guess, and keeping in mind the rather cryptic if/? description of equ "these comparisons are generic, in that if both string1 and string2 are both comprised of all numeric digits, then the strings are converted to numbers and a numeric comparison is performed"... I'd guess that the parser sees the right-hand value as a literal un-quoted number, and before evaluating the (delayed-expanded) left-hand value it decides to "bet" that it's a numeric comparison. Later, when the left-hand value is evaluated (and found missing), and since it's been "decided" that it must be a number, it's resolved to 0.

Leading unquoted token delimiters are converted to space and ignored. They are not part of the condition, as evidenced by

Code: Select all

if  ==   ,,  ;;  A equ A echo match
  --> match

Right. Also, it's not just leading, and not all delimiters appear to be equal in this context.

Code: Select all

if  ==A;; ,,equ,= ==A== (echo match) else (echo no match)
:: -->  match
if  ==A;;  ,equ=  ==A== (echo match) else (echo no match)
:: -->  equ= was unexpected at this time.

Liviu

[dbenham]

I'd guess that the parser sees the right-hand value as a literal un-quoted number, and before evaluating the (delayed-expanded) left-hand value it decides to "bet" that it's a numeric comparison. Later, when the left-hand value is evaluated (and found missing), and since it's been "decided" that it must be a number, it's resolved to 0.

I don't think that is quite right. The decision to do a numeric or string comparison is determined at run time, not by the parser. The IF command (not the CMD parser, unless the parser has another phase after delayed expansion) first attempts to parse both sides as numbers. If either fails, then it reverts to a string comparison.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "less=#"
set "more=a"
set "ten=10"
if !less! lss 9 (echo # ^< 9) else (echo # ^> 9)
if !more! lss 9 (echo a ^< 9) else (echo a ^> 9)
if !ten! lss 9 (echo 10 ^< 9) else (echo 10 ^> 9)

output

Code: Select all

# < 9
a > 9
10 > 9

Dave Benham

[Liviu]

I don't think that is quite right. The decision to do a numeric or string comparison is determined at run time, not by the parser. The IF command (not the CMD parser, unless the parser has another phase after delayed expansion) first attempts to parse both sides as numbers. If either fails, then it reverts to a string comparison.

You have a point. Confirmed by the octal vs. decimal comparisons below. Well, at least 3 out of 4

Code: Select all

set "n011=011"
set "n11=11"
set "n011_=011 "
set "n_011= 011"
if !n011! equ 9 (echo [!n011!] == 9) else (echo [!n011!] ^<^> 9)
if 0!n11! equ 9 (echo [0!n11!] == 9) else (echo [0!n11!] ^<^> 9)
if !n011_! equ 9 (echo [!n011_!] == 9) else (echo [!n011_!] ^<^> 9)
if !n_011! equ 9 (echo [!n_011!] == 9) else (echo [!n_011!] ^<^> 9)

Code: Select all

[011] == 9
[011] == 9
[011 ] <> 9
[ 011] == 9

Liviu

P.S. [ Edit ] Just to point the (now) "obvious" but case #3 above also has these funny side effects.

Code: Select all

set "n011_=011 "
if !n011_! equ 9 (echo [!n011_!] == 9) else (echo [!n011_!] ^<^> 9)
if %n011_% equ 9 (echo [%n011_%] == 9) else (echo [%n011_%!] ^<^> 9)
if %n011_% equ !n011_! (echo [%n011_%] == [!n011_!]) else (echo [%n011_%] ^<^> [!n011_!])
if !n011_! equ %n011_% (echo [!n011_!] == [%n011_%]) else (echo [!n011_!] ^<^> [%n011_%])

Code: Select all

[011 ] <> 9
[011 ] == 9
[011 ] <> [011 ]
[011 ] <> [011 ]

[jeb]

Interesting thread.

I don't think that is quite right. The decision to do a numeric or string comparison is determined at run time, not by the parser. The IF command (not the CMD parser, unless the parser has another phase after delayed expansion) first attempts to parse both sides as numbers. If either fails, then it reverts to a string comparison.
Code:

Leading unquoted token delimiters are converted to space and ignored. They are not part of the condition, as evidenced by

I suppose there are multiple different phases while parsing an IF statement.
The leading delimiters are parsed by the standard batch parser and removed or converted to space.
But the IF command itself also removes leading spaces and TAB's but not the other delimiters.
Trailing spaces are only removed by the batch parser but not by the IF-command parser.

Code: Select all

setlocal EnableDelayedExpansion
set "hexNum1= 0x10"
set "hexNum2=;0x10"
if !hexNum1! EQU 16 (echo hexNum1 equal 16) ELSE (echo hexNum1 NOT equal 16) 
if !hexNum2! EQU 16 (echo hexNum2 equal 16) ELSE (echo hexNum2 NOT equal 16) 
if %hexNum2% EQU 16 (echo hexNum2 equal 16) ELSE (echo hexNum2 NOT equal 16) 

hexNum1 equal 16
hexNum2 NOT equal 16
hexNum2 equal 16

jeb

[Aacini]

I suppose that IF command uses the classical method to convert a string to number before the numeric comparison:

Code: Select all

set string=12345
set number=0
:nextDigit
   if not defined string goto completed
   set /A number=number*10 + %string:~0,1%
   set string=%string:~1%
   goto nextDigit
:completed

Previous method return zero if the string is empty. Of course, the real conversion should be programmed in assembly language, but I could bet that the method is the same.

Antonio

[dbenham]

Not surprisingly, the same results can be achieved with an empty FOR variable value.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "undefined="
set "forVar=%%%%~A"
if !undefined! equ 0 (echo Number: ^^!undefined^^! = 0) else echo Number: fail
if !undefined!. equ . (echo String: ^^!undefined^^!. = . ) else echo String: fail
for %%A in ("") do (
  if %%~A equ 0 (echo Number: !forVar! = 0) else echo Number: fail
  if %%~A. equ . (echo String: !forVar!. = .) else echo Number: fail
)

Resuts:

Code: Select all

Number: !undefined! = 0
String: !undefined!. = .
Number: %%~A = 0
String: %%~A. = .

Dave Benham

[DigitalSnow]

This behavior seems more like an error prevention mechanism of Extensions' if command that is only visible with variable expansion on command execution. Here is my reasoning.

Variable behavioral refresher...

Variable expansion using percent signs %var% happens when the entire command line is read.

Variable expansion using exclamation marks !var! happens when the specific commands using the variable are executed.

Variable expansion for loop variables %%var happens when the specific commands using the variables are executed.

CMD BATCH behavioral refresher... (not conclusive)

1. Line is read (from command or batch) and percent sign %var% variables are expanded.

A line includes everything that is contained within the command's scope. This can span multiple lines when parentheses scope modifiers are used.

Code: Select all

if %var%==true (
    set var=false
    echo %var%
)

For example in the example above, both of the %var% will be expanded since they are both in the same line of scope for the if command.

2. Validation for proper syntax is performed on the line.

3. As the individual commands are executed on the line, the %%var and !var! variables are expanded.

The variables being expanded upon execution of the commands get to bypass the syntax validation for their values. The validation is not needed since the commands were already able to be properly parsed. So the value of the variables get passed directly to the commands being executed.

Variable Expansion Code Illustration

Code: Select all

@echo off
setlocal EnableExtensions EnableDelayedExpansion

:: Illustrates command execution expansion vs line read expansion.

set "X=%%%%~A"
set "Y=%%A%%"
set "Z=%%B%%"

:: Set A to a character known for a script error issue.
set "A=()"
set "B="
for %%A in ("()") do (
   echo !X! = %%~A
   echo ^^!A^^!  = !A!
   echo !Y! will cause a scope close error due to expansion upon read.
   rem echo %A%
   set "B=[]"
   for %%A in ("%B%") do (
      echo Sub !X! from !Z! = %%~A
   )
   for %%A in ("!B!") do (
      echo Sub !X! from ^^!B^^! = %%~A
   )
   echo !X! = %%~A
)

echo No issues outside of scope for the parentheses.
echo %A%
echo %B%

endlocal
pause >nul

Therefore, this issue is not really an issue, but the undefined behavior of Extensions' if command. As Liviu already quoted, there is a vague explanation in the if /? documentation.

These comparisons are generic, in that if both string1 and string2 are both comprised of all numeric digits, then the strings are converted to numbers and a numeric comparison is performed.

So when a nul value is passed to the if command, instead of throwing an error stating that the input is invalid, it just uses the base default value for the type comparison needed.

My Mental Disassembly of the IF EQU Code

Code: Select all

bool equ(string sLeft, string sRight)
{
    // A string is numeric unless it contains characters other than 0-9.
    // This allows blank strings to be both numeric and alpha.
    if (IsNumeric(sLeft) && IsNumeric(sRight))
    {
        // ToNumber returns 0 when the string is blank.
        int nLeft(sLeft.ToNumber());
        int nRight(sRight.ToNumber());
        return (nLeft == nRight);
    }

    // A blank string has a string weight of null "/0".
    return (0 == sLeft.ASCIICompare(sRight));
}

Regular variable expansion does not exhibit this undefined behavior because undefined variables are evaluated to blank strings and they get caught at step 2, the validation of the command.

So when the only string specified is a number, the if command performs a numeric comparison with a base type value of 0.

When the only string specified is not a number, the if command performs a string comparison with a null string.

Here is my Tester Script (Contains SOH character, 0x01 Hex value)

Code: Select all

@echo off
setlocal EnableExtensions EnableDelayedExpansion
echo THE IF COMMAND AND DELAYED EXPANSION
:: Create an undefined variable.
set "undefined="
:: String Comparison [ Blank Strings are Evaluated to Null ]
if !undefined! geq 
 (echo ^^!undefined^^! ^>= SOH ) else (echo ^^!undefined^^! ^< SOH )
if !undefined! lss 
 (echo ^^!undefined^^! ^< SOH ) else (echo ^^!undefined^^! ^>= SOH )
if !undefined! equ !undefined! (echo ^^!undefined^^! = ^^!undefined^^! ) else (echo ^^!undefined^^! = ? )
:: Number Comparison [ Blank Strings are Evaluated to Zero ]
if !undefined! lss 1 (echo ^^!undefined^^! ^< 1 ) else (echo ^^!undefined^^! ^>= 1 )
if !undefined! equ 0 (echo ^^!undefined^^! = 0 ) else (echo ^^!undefined^^! ^<^> 0 )
if !undefined! gtr -1 (echo ^^!undefined^^! ^> -1 ) else (echo ^^!undefined^^! ^<= -1 )
:: It does not matter which side the undefined variable is on.
set "X=%%%%~A"
for %%A in ("") do (
    rem String Comparison [ Blank Strings are Evaluated to Null ]
   if %%~A geq 
 (echo !X! ^>= SOH ) else (echo !X! ^< SOH )
   if %%~A lss 
 (echo !X! ^< SOH ) else (echo !X! ^>= SOH )
   if %%~A equ %%~A (echo !X! = !X! ) else (echo !X! = ? )
   rem Number Comparison [ Blank Strings are Evaluated to Zero ]
   if %%~A lss 1 (echo !X! ^< 1 ) else (echo !X! ^>= 1 )
   if %%~A equ 0 (echo !X! = 0 ) else (echo !X! ^<^> 0 )
   if %%~A gtr -1 (echo !X! ^> -1 ) else (echo !X! ^<= -1 )
)
echo Done
endlocal
pause >nul

I also noticed, that this behavior only appears within BATCH scripts and does not happen on the command line directly (Except with the for loop command as pointed out by dbenham below). The reasoning for this is, on the command line directly, when a variable is undefined, the variable cannot be expanded and therefore, no expansion is performed on the variable name and it is used as a string directly. See the code snippet below.

Code: Select all

cmd /E:ON /V:ON
set "undefined="
echo %undefined%
echo !undefined!
set "undefined=0"
echo %undefined%
echo !undefined!

TLDR: This is the undefined behavior of the IF command with EXTENSIONS. This is only visible due to the ability of variables expanded on command execution to bypass the line syntax validation step that regular variable expansion has to pass.

David Ruhmann

[dbenham]

I also noticed, that this behavior only appears within BATCH scripts and does not happen on the command line directly.

But it does appear on the command line if FOR variables are used

Code: Select all

D:\test>for %A in ("") do if %~A equ 0 echo match

D:\test>if  EQU 0 echo match
match

This is the undefined, but apparent behavior of the IF command with EXTENSIONS. This is only visible due to the ability of variables expanded on command execution to bypass the line syntax validation step that regular variable expansion has to pass.

I think fairly self evident, especially since EQU is itself a command extension.


In my mind, this behaviour is very similar to the result of executing a delayed expanded variable that results in an empty string. An empty string as a command equates to a no-op (remark) that always succeeds. The words after the no-op are treated as parameters to the no-op.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "undefined="
!undefined! This text is ignored && (echo success) || (echo failure)
for %%A in ("") do %%~A This text is ignored && (echo success) || (echo failure)

Dave Benham

[DigitalSnow]

I think fairly self evident, especially since EQU is itself a command extension.

Well I try not, to take any of the readers' knowledge for granted. Since there are many people who seem to read these forums, but who may not know all the intricacies of CMD BATCH. My statement was trying to make sure that the reader knew that it did not just apply to the equ extension alone, but to all of the extensions for IF.

In my mind, this behaviour is very similar to the result of executing a delayed expanded variable that results in an empty string. An empty string as a command equates to a no-op (remark) that always succeeds. The words after the no-op are treated as parameters to the no-op.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "undefined="
!undefined! This text is ignored && (echo success) || (echo failure)
for %%A in ("") do %%~A This text is ignored && (echo success) || (echo failure)

I would not equate the two situations, because the if is the command being executed and not the blank variable value itself. The behavior of the variable expansion is the same for both situations, but the lines are quite different. The if command is actually being called with a null parameter not a null command being called with junk parameters. That is my perspective on it anyway.