Rules for label names vs GOTO and CALL

[dbenham]

So I've been thinking about my recent experiments, and I realize that there must be two different phases where a label can be recognized so that it is not executed.

I had already updated jeb's cmd parsing rules on StackOverflow to show that a command token that begins with : is detected as a label in phase 2, and the command is aborted - the later phases are not executed.

But my recent experiments show that some labels with preceding redirection actually execute the redirection in phase 5.5, yet the label still is not executed (no error message). This implies that command execution in phase 7 must be able to recognize that commands that begin with : are labels, and should not be executed.

I did a bunch of additional tests, and came up with the following rules:

There are two major types of labels:

• Unexecuted Label - A label that is fully detected in phase 2
  • The command is aborted, the remainder of the line is not parsed, and the subsequent phases do not take place for the label. Edit - Except ^ line continuation still occurs
  • Generally, if a command token discovered in phase 2 begins with a colon, then it is an Unexecuted Label.
    But there are exceptions listed under the Executed Label section
  • An Unexecuted Label within a parenthesized block will result in a fatal syntax error in phase 2 unless it is immediately followed by a command or executed label on the very next line
• Executed Label - A label that is not fully detected until phase 7
  All prior phases are applied to the label, but then during execution, the label is ignored (not really executed), or else results in an error, or in rare cases, changes the current volume.
  There are two basic classes of executed labels
  • Delayed label - The command token did not begin with a colon in phase 2. The leading colon only appears after FOR expansion in phase 4 or delayed expansion in phase 5
    .
  • Phase 2 Executed Label exceptions - A command token that begins with : in phase 2 will be executed under any of the following circumstances:
    • The command token begins with : and there is redirection that precedes the command token, and there is &, &&, or || command concatenation or a | pipe on the line
    • The command token begins with : and there is redirection that precedes the command token, and the command is within a parenthesized block
    • The command token begins with : and it is the very first command on a line within a parenthesized block, and the line immediately above ended with an unexecuted label
    The following occurs when an executed label is discovered in phase 2
    • The label, its arguments, and its redirection are all excluded from any echo output in phase 3
    • Any subsequent concatenated commands on the line are fully parsed and executed

In order to fully describe the behavior of an Executed Label, phase 7 must be expanded as follows.
Steps that are important to labels are in bold italics.
Note that there are slight differences between batch mode and command line mode that are important for executed labels.

Phase 7) Execute the command

• 7.1 - Execute internal command - If the command token is not quoted and it is an internal command, then execute the internal command
  .
• 7.2 - Execute volume change - Else if the command token does not begin with a quote, is exactly two characters long, and the 2nd character is a colon, then change the volume
  • All argument tokens are ignored
  • If the volume specified by the first character cannot be found, then abort with an error.
  • A command token of :: will always result in an error unless SUBST is used to define a volume for ::
    If SUBST is used to define a volume for :: then the volume will be changed, it will not be treated as a label
• 7.3 - Execute external command - Else try to treat the command as an external command
  • If the 2nd character is a colon, then verify the volume specified by the 1st character can be found.
    If the volume cannot be found, then abort with an error
  • If in batch mode and the command token is a label that begins with : then goto 7.4
    Note that if the label begins with :: then this will not be reached because the preceding step will have issued an error unless SUBST is used to define a volume for ::
  • Identify the external command to execute
    This is a complex process that can involve the current volume, current directory, PATH variable, PATHEXT variable, and file associations
    If a valid external command cannot be identified, then abort with an error
  • If in command line mode and the command token is a label that begins with : then goto 7.4
    Note that this is rarely reached because the preceding step will have identified the label as an error unless the command token begins with :: and SUBST is used to define a volume for :: and the entire command token is a valid path to an external command
  • Execute the external command
• 7.4 - Ignore a label - Ignore the command and all its arguments if the command token begins with :
  Rules in 7.2 and 7.3 may prevent a label from reaching this point

I've updated jeb's parsing rules to reflect the rules above.

The remainder of this post is evidence for and/or examples of the rules I outlined above for the batch context only.
In the future I may do a similar set of tests for command line mode.

I have two methods to probe whether a label is parsed beyond phase 2 (unexecuted label), or continues on to phase 7 (executed label)

• Test whether delayed expansion phase 5 is performed
  At first I didn't think I could probe this because delayed expansion never results in a fatal syntax error, and labels never have any output.
  But then I remembered that find/replace and substring operations on the pseudo CMDCMDLINE variable actually modifies the underlying value.
  So I simply add an argument after the label that uses delayed expansion to modify CMDCMDLINE.
  If the value changes, then phase 5 must have been performed. If not, then phase 5 must not have been performed.
• Test whether redirection phase 5.5 is performed
  I create a text file with length greater than 0 prior to executing the label command.
  Then for the label "command" I redirect stdout to the file.
  A label "command" never has any output, so if the file size goes to 0, then redirection must have occurred. If not, then redirection did not occur.

If neither phase 5 nor 5.5 is executed, then the presumption is that the label stopped at phase 2.
If both phase 5 and 5.5 are executed, then the presumption is that the label is processed all the way through phase 7.

I don't want to permanently mess with the CMDCMDLINE value for my current cmd session, so I execute each test in a new cmd.exe process.

I built a test harness that allows me to perform multiple tests within a single script, each test within its own cmd process.

If there is a fatal syntax error, then I cannot probe the delayed expansion because the test cmd.exe process will terminate and I will lose my CMDCMDLINE value.
But the file still exists after a fatal syntax error, so I check the file size after the test cmd.exe process closes.

If no arguments are given, then all 26 tests are run.
If a single START integer argument is given, then the test starts with the START test, and continues to the end.
If both START and STOP are given, then the test starts with the START test, and continues through the STOP test.

Here is my set of batch mode tests:

Code: Select all

@echo off
setlocal
set "beg=%~1"
set "end=%~2"
if not defined beg set "beg=1"
if not defined end set "end=26"

for /l %%N in (%beg% 1 %end%) do if %0=="%~dp0Test%%N\..\%~nx0" (
  echo(
  set "print="
  for /f "usebackq delims=" %%A in ("%~f0") do (
    for /f %%B in ("%%A") do if "%%B"=="@exit" (set print=) else if "%%B"==":test%%N" set print=1
    if defined print for /f "eol=@ delims=" %%B in ("%%A") do echo(%%B
  )
  <nul set /p "=------------------------------------------"
  echo content >test1.txt
  echo content >test2.txt
  setlocal enableDelayedExpansion
  break "!cmdcmdline:*.bat""= 1 2!"
  echo on
  call :test%%N
  @echo off
  echo cmdCmdLine=!cmdcmdline!
  exit /b
)

cls
prompt echo$g 
for /l %%N in (%beg% 1 %end%) do (
  cmd /c ^""%~dp0Test%%N\..\%~nx0""
  for %%F in (test1.txt test2.txt) do <nul set /p "=%%F size=%%~zF  "
  echo(
)
prompt
exit /b


:test1 - A normal label is not executed
@echo(
:label !cmdcmdline:1=Mod1! >test1.txt & echo ignored
@exit /b

:test2 - A normal label is still not executed if concatenated to a command
echo exec & :label !cmdcmdline:1=Mod1! >test1.txt & echo ignored
@exit /b

:test3 - This delayed label is executed because it doesn't exist until phase 4
for %%A in (:label) do %%A !cmdcmdline:1=Mod1! >test1.txt & echo executed
@exit /b

:test4 - This delayed label is executed because it doesn't exist until phase 5
set "var=:label" & !var! !cmdcmdline:1=Mod1! >test1.txt & echo executed
@exit /b

:test5 - A label with preceding redirection is not executed if no command concatenation
@echo(
>test1.txt :label !cmdcmdline:1=Mod1!
@exit /b

:test6 - A label with preceding redirection is executed if it is concatenated to a command
echo executed & >test1.txt :label !cmdcmdline:1=Mod1!
@exit /b

:test7 - A label with preceding redirection is also executed if a concatenated command follows
>test1.txt :label !cmdcmdline:1=Mod1! & echo exec
@exit /b

:test8 - A label with preceding redirection is executed even if the concatenated command is an unexecuted label
>test1.txt :label !cmdcmdline:1=Mod1! & :label !cmdcmdline:2=Mod2! >test2.txt
@exit /b

:test9 - Of course the label with preceding redirection is executed if sandwiched between executing commands
echo executed & >test1.txt :label !cmdcmdline:1=Mod1! & echo executed
@exit /b

:test10 - Both concatenated labels with preceding redirection are executed
>test1.txt :label !cmdcmdline:1=Mod1! & >test2.txt :label !cmdcmdline:2=Mod2!
@exit /b

:test11 - Executed label beginning with :: normally gives an error
>test1.txt ::label !cmdcmdline:1=Mod1! & echo executed
@exit /b

:test12 - Executed label beginning with :: is OK if volume :: is defined via SUBST
subst :: . & >test1.txt ::label !cmdcmdline:1=Mod1! & subst :: /d
@exit /b

:test13 - Executed :: "label" with volume :: defined is actually an executed change volume command
subst :: . & >test1.txt :: !cmdcmdline:1=Mod1!
cd & subst :: /d
@exit /b

:test14 - label with preceding redirection and conditional && concatenation also forces label execution
>test1.txt :label !cmdcmdline:1=Mod1! && echo executed label SUCCESS
@exit /b

:test15 - label with preceding redirection and conditional || concatenation also forces label execution
>test1.txt ::label !cmdcmdline:1=Mod1! || echo executed label beginning with :: FAILED
@exit /b

:test16 - Within parens: unexecuted label without immediately following command gives fatal syntax error
@echo(
(
  :label !cmdcmdline:1=Mod1! >test1.txt  Syntax Error
)
@exit /b

:test17 - Within parens: unexecuted label without immediately following command gives fatal syntax error
@echo(
(
  :label !cmdcmdline:1=Mod1! >test1.txt
  
  echo Syntax Error
)
@exit /b

:test18 - Within parens: uexecuted label immediately followed by command is valid syntax
(
  :validLabel !cmdcmdline:1=Mod1! >test1.txt
  echo exec
)
@exit /b

:test19 - Within parens: 1st label not executed, 2nd label is executed, yielding valid syntax
(
  :label 1 !cmdcmdline:1=Mod1! >test1.txt
  :label 2 !cmdcmdline:2=Mod2! >test2.txt
)
@exit /b

:test20 - As expected, ::label2 is executed, but normally gives an error
(
  ::label1 !cmdcmdline:1=Mod1! >test1.txt
  ::label2 !cmdcmdline:2=Mod2! >test2.txt
)
@exit /b

:test21 - As expected, defining volume :: allows ::label2 to execute without error
(
  subst :: .
  ::label1 !cmdcmdline:1=Mod1! >test1.txt
  ::label2 !cmdcmdline:2=Mod2! >test2.txt
  subst :: /d
)
@exit /b

:test22 - Within parens: delayed label is executed, so it is valid syntax even though no following command
(
  set "var=:label"
  !var! !cmdcmdline:1=Mod1! >test1.txt
)
@exit /b

:test23 - Within parens: lone label with preceding redirection is executed without fatal syntax error, even without concatenated command
(
  >test1.txt :label !cmdcmdline:1=Mod1!
)
@exit /b

:test24 - Within parens: lone label without preceding redirection is not executed, no following command gives fatal syntax error
@echo(
(
  :label !cmdcmdline:1=Mod1! >test1.txt 
)
@exit /b

:test25 - Within parens: unexecuted label2 is not followed by executed command, so fatal syntax error
@echo(
(
  >test1.txt :label1 !cmdcmdline:1=Mod1! & :label2 !cmdcmdline:2=Mod2! >test2.txt
)
@exit /b

:test26 - Within parens: unexecuted label2 is followed by a command, so valid syntax
(
  >test1.txt :label1 !cmdcmdline:1=Mod1! & :label2 !cmdcmdline:2=Mod2! >test2.txt
  echo executed
)
@exit /b

Below is the output of each test. I put each test result in its own command block so that you don't have to scroll any test.
I run each test with ECHO ON so you can see how labels affect phase 3 output.

Each test output contains the following components:

• The test number and lesson learned
• The actual test code
• A ------------------------- separator
• The test results
  • Phase 3 output (ECHO ON output) follows the echo> prompt
    .
  • The CMDCMDLINE probe can have four results. Only the first two are valid if there is only one label:
    • cmdCmdLine= 1 2 - no label was executed
    • cmdCmdLine= Mod1 2 - label 1 was executed
    • cmdCmdLine= 1 Mod2 - label 2 was executed
    • cmdCmdLine= Mod1 Mod2 - both label 1 and label 2 were executed
    If the CMDCMDLINE value is missing, then there was a fatal syntax error.
    .
  • The redirection probe can have four results. Only the first two are valid if there is only one label:
    • test1.txt size=10 test2.txt size=10 - no label was executed
    • test1.txt size=0 test2.txt size=10 - label 1 was executed
    • test1.txt size=10 test2.txt size=0 - label 2 was executed
    • test1.txt size=0 test2.txt size=0 - both label 1 and label 2 were executed

Code: Select all

:test1 - A normal label is not executed
:label !cmdcmdline:1=Mod1! >test1.txt & echo ignored
------------------------------------------
cmdCmdLine= 1 2
test1.txt size=10  test2.txt size=10

Code: Select all

:test2 - A normal label is still not executed if concatenated to a command
echo exec & :label !cmdcmdline:1=Mod1! >test1.txt & echo ignored
------------------------------------------
echo> echo exec
exec
cmdCmdLine= 1 2
test1.txt size=10  test2.txt size=10

Code: Select all

:test3 - This delayed label is executed because it doesn't exist until phase 4
for %%A in (:label) do %%A !cmdcmdline:1=Mod1! >test1.txt & echo executed
------------------------------------------
echo> for %A in (:label) do %A !cmdcmdline:1=Mod1!   1>test1.txt  & echo executed

echo>  & echo executed
executed
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test4 - This delayed label is executed because it doesn't exist until phase 5
set "var=:label" & !var! !cmdcmdline:1=Mod1! >test1.txt & echo executed
------------------------------------------
echo> set "var=:label"   & !var! !cmdcmdline:1=Mod1!   1>test1.txt  & echo executed
executed
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test5 - A label with preceding redirection is not executed if no command concatenation
>test1.txt :label !cmdcmdline:1=Mod1!
------------------------------------------
cmdCmdLine= 1 2
test1.txt size=10  test2.txt size=10

Code: Select all

:test6 - A label with preceding redirection is executed if it is concatenated to a command
echo executed & >test1.txt :label !cmdcmdline:1=Mod1!
------------------------------------------
echo> echo executed   &
executed
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test7 - A label with preceding redirection is also executed if a concatenated command follows
>test1.txt :label !cmdcmdline:1=Mod1! & echo exec
------------------------------------------
echo>  & echo exec
exec
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test8 - A label with preceding redirection is executed even if the concatenated command is an unexecuted label
>test1.txt :label !cmdcmdline:1=Mod1! & :label !cmdcmdline:2=Mod2! >test2.txt
------------------------------------------
echo>
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test9 - Of course the label with preceding redirection is executed if sandwiched between executing commands
echo executed & >test1.txt :label !cmdcmdline:1=Mod1! & echo executed
------------------------------------------
echo> echo executed   &  & echo executed
executed
executed
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test10 - Both concatenated labels with preceding redirection are executed
>test1.txt :label !cmdcmdline:1=Mod1! & >test2.txt :label !cmdcmdline:2=Mod2!
------------------------------------------
echo>  &
cmdCmdLine= Mod1 Mod2
test1.txt size=0  test2.txt size=0

Code: Select all

:test11 - Executed label beginning with :: normally gives an error
>test1.txt ::label !cmdcmdline:1=Mod1! & echo executed
------------------------------------------
echo>  & echo executed
The system cannot find the drive specified.
executed
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test12 - Executed label beginning with :: is OK if volume :: is defined via SUBST
subst :: . & >test1.txt ::label !cmdcmdline:1=Mod1! & subst :: /d
------------------------------------------
echo> subst :: .   &  & subst :: /d
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test13 - Executed :: "label" with volume :: defined is actually an executed change volume command
subst :: . & >test1.txt :: !cmdcmdline:1=Mod1!
cd & subst :: /d
------------------------------------------
echo> subst :: .   &

echo> cd   & subst :: /d
::\
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test14 - label with preceding redirection and conditional && concatenation also forces label execution
>test1.txt :label !cmdcmdline:1=Mod1! && echo executed label SUCCESS
------------------------------------------
echo>  && echo executed label SUCCESS
executed label SUCCESS
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test15 - label with preceding redirection and conditional || concatenation also forces label execution
>test1.txt ::label !cmdcmdline:1=Mod1! || echo executed label beginning with :: FAILED
------------------------------------------
echo>  || echo executed label beginning with :: FAILED
The system cannot find the drive specified.
executed label beginning with :: FAILED
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test16 - Within parens: unexecuted label without immediately following command gives fatal syntax error
(
  :label !cmdcmdline:1=Mod1! >test1.txt  Syntax Error
)
------------------------------------------
) was unexpected at this time.

echo> )
test1.txt size=10  test2.txt size=10

Code: Select all

:test17 - Within parens: unexecuted label without immediately following command gives fatal syntax error
(
  :label !cmdcmdline:1=Mod1! >test1.txt

  echo Syntax Error
)
------------------------------------------
The syntax of the command is incorrect.

echo>
test1.txt size=10  test2.txt size=10

Code: Select all

:test18 - Within parens: uexecuted label immediately followed by command is valid syntax
(
  :validLabel !cmdcmdline:1=Mod1! >test1.txt
  echo exec
)
------------------------------------------
echo> (echo exec )
exec
cmdCmdLine= 1 2
test1.txt size=10  test2.txt size=10

Code: Select all

:test19 - Within parens: 1st label not executed, 2nd label is executed, yielding valid syntax
(
  :label 1 !cmdcmdline:1=Mod1! >test1.txt
  :label 2 !cmdcmdline:2=Mod2! >test2.txt
)
------------------------------------------
echo> ()
cmdCmdLine= 1 Mod2
test1.txt size=10  test2.txt size=0

Code: Select all

:test20 - As expected, ::label2 is executed, but normally gives an error
(
  ::label1 !cmdcmdline:1=Mod1! >test1.txt
  ::label2 !cmdcmdline:2=Mod2! >test2.txt
)
------------------------------------------
echo> ()
The system cannot find the drive specified.
cmdCmdLine= 1 Mod2
test1.txt size=10  test2.txt size=0

Code: Select all

:test21 - As expected, defining volume :: allows ::label2 to execute without error
(
  subst :: .
  ::label1 !cmdcmdline:1=Mod1! >test1.txt
  ::label2 !cmdcmdline:2=Mod2! >test2.txt
  subst :: /d
)
------------------------------------------
echo> (
subst :: .

 subst :: /d
)
cmdCmdLine= 1 Mod2
test1.txt size=10  test2.txt size=0

Code: Select all

:test22 - Within parens: delayed label is executed, so it is valid syntax even though no following command
(
  set "var=:label"
  !var! !cmdcmdline:1=Mod1! >test1.txt
)
------------------------------------------
echo> (
set "var=:label"
 !var! !cmdcmdline:1=Mod1!  1>test1.txt
)
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test23 - Within parens: lone label with preceding redirection is executed without fatal syntax error, even without concatenated command
(
  >test1.txt :label !cmdcmdline:1=Mod1!
)
------------------------------------------
echo> ()
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Code: Select all

:test24 - Within parens: lone label without preceding redirection is not executed, no following command gives fatal syntax error
(
  :label !cmdcmdline:1=Mod1! >test1.txt
)
------------------------------------------
) was unexpected at this time.

echo> )
test1.txt size=10  test2.txt size=10

Code: Select all

:test25 - Within parens: unexecuted label2 is not followed by executed command, so fatal syntax error
(
  >test1.txt :label1 !cmdcmdline:1=Mod1! & :label2 !cmdcmdline:2=Mod2! >test2.txt
)
------------------------------------------
) was unexpected at this time.

echo> )
test1.txt size=10  test2.txt size=10

Code: Select all

:test26 - Within parens: unexecuted label2 is followed by a command, so valid syntax
(
  >test1.txt :label1 !cmdcmdline:1=Mod1! & :label2 !cmdcmdline:2=Mod2! >test2.txt
  echo executed
)
------------------------------------------
echo> ( & echo executed )
executed
cmdCmdLine= Mod1 2
test1.txt size=0  test2.txt size=10

Dave Benham

[jeb]

Hi Dave,

great work

But some annotations.

When scanning the file for the label name, I see that there are 5 (maybe 6) stop characters for the label name instead of two: <+>, <:>, <space>, <tab>, <LF>, and possibly <CR>. The other token delimiters are not stop characters

I retested and I came to the same conclusion, CR is a hard stop character, too.

When scanning, the caret seems to work as an escape character, it can append even stop characters to the label, but then these labels can't be called anymore, as the CALL or GOTO itself can't escape the stop characters.

The command is aborted, the remainder of the line is not parsed, and the subsequent phases do not take place for the label.

Not quite correct, at least multiline carets still work, despite of REM where even carets are not parsed (when they aren't in the first argument token).

Code: Select all

:label This caret still works ^
echo this line is part of the label

Phase 2 Executed Label exceptions - A command token that begins with : in phase 2 will be executed under any of the following circumstances:

The command token begins with : and there is redirection that precedes the command token, and there is &, &&, or || command concatenation on the line

It's also "executed" for pipe "|", but then typically an error occours, unless you construct your label into something like this (label.bat must exist and subst has to be executed for ::)

Code: Select all

<nul ::\label.bat | more

My test suite

Code: Select all

@echo Off
SETLOCAL EnableDelayedExpansion

call :prepare

call :noLabelEvaluation
call :LabelEvaluation

call :noRedirectEvaluation
call :redirectEvaluation

call :clearEnviroment
exit /b


:noLabelEvaluation
call :resetCmdline
<nul :label !cmdcmdline:*#=1#!
call :checkCmdline - "Label will not be evaluated without any operator &, &&, ||, |"
exit /b

:LabelEvaluation
for /F "tokens=1,2" %%C in ("label &!\n!label &&!\n!label ||!\n!:\dummy.bat |") DO (
    call :resetCmdline
    set "labelName=%%~C"
    set "testExpr=%%~D"
    call :test_LabelEvaluation 2>nul
    call :checkCmdline 1 "label ':!labelName!' is evaluated with appended '!testExpr!'"
)    
exit /b
:test_LabelEvaluation
<nul :%labelName% !cmdcmdline:*#=1#! %testExpr% set do=nothing
exit /b


:redirectEvaluation
for /F "tokens=1,2" %%C in ("label &!\n!label &&!\n!label ||!\n!:\dummy.bat |") DO (
    call :resetCmdline
    set "labelName=%%~C"
    set "testExpr=%%~D"
    call :test_redirectEvaluation 2>nul
    call :checkCmdline 1 "Redirection with label ':!labelName!' is evaluated with appended !testExpr!"
)    
exit /b
:test_redirectEvaluation
<"!cmdcmdline:*#=1#!" :%labelName% %testExpr% set do=nothing
exit /b

:noRedirectEvaluation
call :resetCmdline
<"!cmdcmdline:*#=1#!" :label
call :checkCmdline - "Redirection with only label is not evaluated"
exit /b


@echo off
echo cmdcmdline !cmdcmdline!
echo END
exit /b

:resetCmdline
rem. change cmdcmdline !cmdcmdline:"C=-#!
rem. change cmdcmdline !cmdcmdline:*#=-#!
rem. change cmdcmdline !cmdcmdline:~0,2!
exit /b

:checkCmdline <expected value> <text>
set "cmdl=!cmdcmdline!"
set "result=FAIL"
if "!cmdl:~0,1!" == "%~1" set "result=Okay"
set "msg=%~2"
echo !result!: !msg!
exit /b

:prepare
(set \n=^
%=EMPTY LINE=%
)

subst /d :: 2>nul 1>nul

subst :: C:\temp
(
    echo @echo off
    echo This is %%~f0
) > ::\Dummy.bat
exit /b

:clearEnviroment
del ::\Dummy.bat
subst /d ::
exit /b

Okay: Label will not be evaluated without any operator &, &&, ||, |
Okay: label ':label' is evaluated with appended '&'
Okay: label ':label' is evaluated with appended '&&'
Okay: label ':label' is evaluated with appended '||'
Okay: label '::\dummy.bat' is evaluated with appended '|'
Okay: Redirection with only label is not evaluated
Okay: Redirection with label ':label' is evaluated with appended &
Okay: Redirection with label ':label' is evaluated with appended &&
Okay: Redirection with label ':label' is evaluated with appended ||
Okay: Redirection with label '::\dummy.bat' is evaluated with appended |

[dbenham]

Thanks for the corrections jeb.

I've updated the SO post to reflect the corrected label rules.

I also cleaned up some issues that I had created when I initially tried to refine phase 4. My fix also expands phases 2 and 7. I think it is good now.

But I have some additional concerns unrelated to labels. So I am starting a new generalized central thread for parsing rules discussions: viewtopic.php?f=3&t=8355


Dave Benham

[dbenham]

Just for yucks, I decided to create a :label that calls itself. I've come up with two different strategies.
All conceivable cases rely on jeb's discovery that any character can appear in the first position of a label line, and the label can still be valid.

Code: Select all

@echo off
setlocal
call :test1 normal
call :test2 normal
exit /b

:test1
echo(
<nul set /p"=1) Redirect self call - "
findstr "self1" "%~f0"|findstr /v "findstr"
<:self1 <nul call :self1 self_call
echo %%0 = %0    %%* = %*
exit /b

:test2
echo(
<nul set /p"=2) Variable self call - "
findstr "self2" "%~f0"|findstr /v "findstr"
set ":=call :%%%%"
%:%self2 self_call
echo %%0 = %0    %%* = %*
exit /b

--OUTPUT--

Code: Select all

1) Redirect self call - <:self1 <nul call :self1 self_call
%0 = :self1    %* = self_call
%0 = :test1    %* = normal

2) Variable self call - %:%self2 self_call
%0 = :%self2    %* = self_call
%0 = :test2    %* = normal

By far my "favorite" strategy is the definition of the colon variable, as the label name only appears once on the line.

All well and good. But then I tested on a Windows 10 machine and both self calls fail

I did limited testing, and it appears that on Windows 10, a label with an odd character in position 1 on the line is only recognized as a label if the CALL (or GOTO) is on the physical line directly above the label line. The exact same CALL or GOTO placed anywhere else in the file fails to find the label.

So now we have different parsing rules for windows versions


Edit - Actually, there is a self call option that doesn't rely on an odd character in position 1 - but I consider it cheating because it uses line continuation to split the call across two physical lines

Code: Select all

@echo off
call :test0 normal
exit /b

:test0
call ^
:self0 self_call
echo %%0 = %0    %%* = %*
exit /b

--OUTPUT--

Code: Select all

%0 = :self0    %* = self_call
%0 = :test0    %* = normal

Dave Benham

[jeb]

By far my "favorite" strategy is the definition of the colon variable, as the label name only appears once on the line.

All well and good. But then I tested on a Windows 10 machine and both self calls fail

I did limited testing, and it appears that on Windows 10, a label with an odd character in position 1 on the line is only recognized as a label if the CALL (or GOTO) is on the physical line directly above the label line. The exact same CALL or GOTO placed anywhere else in the file fails to find the label.

Your colon variable definition is really nice

I tested today with Windows 7 and Windows 10 and I got the same results for both.
I can't find any differences in the behaviour.
Enabling or disabling DelayedExpansion doesn't change anything.
Disabling the extensions simply fails, as expected.
Perhaps you have some strange test environment on your win10

Btw. Your findstr construct can be simplyfied to

Code: Select all

findstr "self[1]" "%~f0"

[dbenham]

Btw. Your findstr construct can be simplyfied to

Code: Select all

findstr "self[1]" "%~f0"

I like it

I tested today with Windows 7 and Windows 10 and I got the same results for both.
I can't find any differences in the behaviour.
Enabling or disabling DelayedExpansion doesn't change anything.
Disabling the extensions simply fails, as expected.
Perhaps you have some strange test environment on your win10

I may have fooled myself into thinking there is a Win10 difference. But I have discovered the trigger that causes the label with an odd character in position 1 to fail. If the batch script uses \n line terminators instead of \r\n, then I get the weird behavior on Win10. I haven't had a chance to do the same test on Win7 yet. Hopefully it fails there as well.

If the batch script terminates lines with \r\n, then Win10 exhibits the expected behavior - allowing the odd character in position 1.

The whole business of allowing the odd character in position 1 has always mystified me - it makes no sense to allow that. I'm thinking the difference between \r\n vs. \n might be a clue as to the derivation of the behavior, but I'm drawing blanks.


Dave Benham

[jeb]

I may have fooled myself into thinking there is a Win10 difference. But I have discovered the trigger that causes the label with an odd character in position 1 to fail. If the batch script uses \n line terminators instead of \r\n, then I get the weird behavior on Win10. I haven't had a chance to do the same test on Win7 yet. Hopefully it fails there as well.

On windows 7 it's the same behaviour. Good to know that the world is still a reliable place.

I've build some tests to discover the CR line ending rules

Code: Select all

@echo off
setlocal EnableDelayedExpansion
for /f %%a in ('copy /Z "%~dpf0" nul') do set "\r=%%a"
(set \n=^
%=empty=%
)


call :buildTest ":test1" ":test1"

call :buildTest ":test10" "^!\r^!:test10"
call :buildTest ":test11" "#^!\r^!:test11"
call :buildTest ":test12" "##^!\r^!:test12"
call :buildTest ":test13" "^!\r^!#:test13"
call :buildTest ":test14" "^!\r^!^!\n^!#:test14"

call :buildTest ":test20" "^!\n^!:test20"
call :buildTest ":test21" "#^!\n^!:test21"
call :buildTest ":test22" "##^!\n^!:test22"
call :buildTest ":test23" "^!\n^!#:test23"
call :buildTest ":test24" "^!\n^!#^!\r^!:test24"


exit /b

:buildTest
setlocal disabledelayedExpansion 
echo calling "%~2"
setlocal EnableDelayedExpansion
set "labelname=%~1"
set "fullLabel=%~2"
(
  echo @echo off
  echo call !labelname!
  echo exit /b
  echo REM !\r!!\n!!fullLabel!
  echo echo Function "%%0" called
  echo exit /b
) > testGenerated.bat
call testGenerated.bat
echo(
exit /b

calling ":test1"
Function ":test1" called

calling "!\r!:test10"
Function ":test10" called

calling "#!\r!:test11"
Function ":test11" called

calling "##!\r!:test12"
Das Sprungziel - test12 wurde nicht gefunden.

calling "!\r!#:test13"
Das Sprungziel - test13 wurde nicht gefunden.

calling "!\r!!\n!#:test14"
Function ":test14" called

calling "!\n!:test20"
Function ":test20" called

calling "#!\n!:test21"
Function ":test21" called

calling "##!\n!:test22"
Function ":test22" called

calling "!\n!#:test23"
Das Sprungziel - test23 wurde nicht gefunden.

calling "!\n!#!\r!:test24"
Das Sprungziel - test24 wurde nicht gefunden.

It's strange, in the CR only tests (test10-test14), the "extra" character is no longer allowed in front of the colon, but instead it's allowed to put one "extra" character in front of the CR, but not two.

But with the newline tests (test20-test24), I can't get it working with "extra" characters at all.
Test21 and test22 only demonstrates that the newline splits the lines, but the CR character doesn't work the same, as test12 fails.

jeb

[dbenham]

Your test plan gave me some ideas on how to structure some additional tests.
And now I believe I have it figured out

\r functions as a token delimiter - any number of \r can appear before the :label, just like spaces, commas, etc.

The first character of a :label line may be any character (other than colon or newline) if and only if:

A) The :label is at or below the current file position and all lines from the current file position until the :label line are all terminated by \r\n. It is okay if the :label line itself is not terminated by \r\n.

OR

B) The :label is above the current file position and all lines from the file beginning until the :label line are all terminated by \r\n. It is okay if the :label line itself is not terminated by \r\n. It is also okay if lines from the current file position to the end of file are not terminated by \r\n.

This is another case where I can't believe this behavior was intentional, yet I can't imagine any sane design that would inadvertently result in this behavior

I ran a series of tests to determine exactly when the "any character" is allowed. I chose @ as my "any character" because @:label is never executed.

Code: Select all

@echo off
cls
setlocal DisableDelayedExpansion
for %%n in (^"^
%=empty=%
^") do for /f %%r in ('copy /Z "%~dpf0" nul') do (
  set "\r=%%r"
  set "\n=%%~n"
  set "\r\n=%%r%%~n"
)
set "col=:"

for %%A in (

  @:below0_0

  !\r\n!@:below1_0
  !\n!@:below1_1

  !\r\n!!\r\n!@:below2_0
  !\n!!\r\n!@:below2_1
  !\r\n!!\n!@:below2_2

  !\r\n!!\r\n!!\r\n!@:below3_0
  !\n!!\r\n!!\r\n!@:below3_1
  !\r\n!!\n!!\r\n!@:below3_2
  !\r\n!!\r\n!!\n!@:below3_3

) do call :belowTest %%A

for %%A in (

  :above0_0

  !\r\n!@:above1_0
  !\n!@:above1_1

  !\r\n!!\r\n!@:above2_0
  !\n!!\r\n!@:above2_1
  !\r\n!!\n!@:above2_2

  !\r\n!!\r\n!!\r\n!@:above3_0
  !\n!!\r\n!!\r\n!@:above3_1
  !\r\n!!\n!!\r\n!@:above3_2
  !\r\n!!\r\n!!\n!@:above3_3

) do call :aboveTest %%A

exit /b

:belowTest
echo calling "%~1"
for /f "tokens=2 delims=:" %%A in (".%~1") do (
  setlocal EnableDelayedExpansion
  set "labelName=:%%A"
)
set "fullLabel=%~1"
(
  echo @echo off
  <nul set /p "=call !labelName!&exit /b!\n!"
	echo !fullLabel!!\n!
  echo echo Function "%%0" called
  echo exit /b
) > testGenerated.bat
call testGenerated.bat
echo(
exit /b

:aboveTest
echo calling "%~1"
for /f "tokens=2 delims=:" %%A in (".%~1") do (
  setlocal EnableDelayedExpansion
  set "labelName=:%%A"
)
set "fullLabel=%~1"
(
  echo !fullLabel!!\n!
  echo @if defined begun echo Function "%%0" called^&exit /b
  echo @echo off
  echo set "begun=1"
  echo call !labelName!!\n!!\n!
  <nul set /p "=exit /b"
) > testGenerated.bat
call testGenerated.bat
echo(
exit /b

calling "@:below0_0"
Function ":below0_0" called

calling "!\r\n!@:below1_0"
Function ":below1_0" called

calling "!\n!@:below1_1"
The system cannot find the batch label specified - below1_1

calling "!\r\n!!\r\n!@:below2_0"
Function ":below2_0" called

calling "!\n!!\r\n!@:below2_1"
The system cannot find the batch label specified - below2_1

calling "!\r\n!!\n!@:below2_2"
The system cannot find the batch label specified - below2_2

calling "!\r\n!!\r\n!!\r\n!@:below3_0"
Function ":below3_0" called

calling "!\n!!\r\n!!\r\n!@:below3_1"
The system cannot find the batch label specified - below3_1

calling "!\r\n!!\n!!\r\n!@:below3_2"
The system cannot find the batch label specified - below3_2

calling "!\r\n!!\r\n!!\n!@:below3_3"
The system cannot find the batch label specified - below3_3

calling ":above0_0"
Function ":above0_0" called

calling "!\r\n!@:above1_0"
Function ":above1_0" called

calling "!\n!@:above1_1"
The system cannot find the batch label specified - above1_1

calling "!\r\n!!\r\n!@:above2_0"
Function ":above2_0" called

calling "!\n!!\r\n!@:above2_1"
The system cannot find the batch label specified - above2_1

calling "!\r\n!!\n!@:above2_2"
The system cannot find the batch label specified - above2_2

calling "!\r\n!!\r\n!!\r\n!@:above3_0"
Function ":above3_0" called

calling "!\n!!\r\n!!\r\n!@:above3_1"
The system cannot find the batch label specified - above3_1

calling "!\r\n!!\n!!\r\n!@:above3_2"
The system cannot find the batch label specified - above3_2

calling "!\r\n!!\r\n!!\n!@:above3_3"
The system cannot find the batch label specified - above3_3

Dave Benham

[siberia-man]

Waw! Interesting thread. 4 years ago I have performed some investigations in this direction. I did it in the frame of the correct colorizing of labels in FAR3-embedded editor, soI don't pretend that my little work is 100% complete but it has some fruitful conclusions.

The following link shows these findings - http://forum.script-coding.com/viewtopi ... 366#p80366. That text is written in Russian, so I am really sorry if you feel some troubles with this but Google Translate can help you. Here I just share the translated conclusions:

There is set of 100% allowable characters. The allowable characters are assumed if
-- the character could be placed anywhere in the label name (in the beginning, in the midst, in the end)
-- the label is correct independently of the setlocal mode
-- the label is accessible by any method (directly via goto/call :LABEL or indirectly via goto/call :%1)

The label name could consist of:
-- Latin letters
-- digits
-- underscore character
-- punctuation characters: "." (dot), "-" (dash), "[" and "]" (square brackets), "{" and "}" (curly brackets), "/" slash and "\" backslash
-- "'" (apostrophe) and "`" (backtrick), "~" (tilde), "@" (at), "#" (diez), "$" (dollar)

Other characters can not be considered as 100% valid label characters because they could lead to error or require additional escaping.