Rules for label names vs GOTO and CALL

[dbenham]

The "anonymous subroutine" called by a certain "anonymous call" is the code placed below the "call :%%anonymous%%" line, so there is no need to distinguish one "anonymous subroutine" from another one:

Thanks Aacini - that is exactly what I had in mind. But it didn't dawn on me that not everyone would realize this fact.


Dave Benham

[npocmaka_]

Oohh
Right.

Next thing. Can the anonymous functions be nested?
Because whatever I try it does not work.

[OperatorGK]

_npocmaka, I tried with different %anonymous% variable value, but this thing prints nothing for me:

Code: Select all

@echo off
setlocal 
set "anonymous=/?"
for /l %%N in (1 1 5) do (
   set /a N=%%N*2
  call :%%anonymous%% a b c 3>&1 >nul
)
if "%0" == ":%anonymous%" (
  set "anonymous#=/?#"
  for /l %%M in (1 1 5) do (
    set /a M=%%M*2
    call :%%anonymous#%% %1 %2 %3 3>&1 >nul
  )
  if "%0" == ":%anonymous#%" (
      echo %1 %2 %3 %N% %M%
      exit /b
  )>&3
  exit /b
)>&3

[Aacini]

This is a bit tricky!

Code: Select all

@echo off
setlocal 

set "anonymous=/?"

for /l %%N in (1 1 3) do (
   set "line=Line %%N: "
   for /l %%M in (1 1 7) do (
      call :%%anonymous%% ONE %%M 3>&1 >nul
   )
   if "%1" neq "ONE" call :%%anonymous%% TWO 3>&1 >nul
)

      if "%1" equ "ONE" (
         set "line=%line% %2"
         exit /b
      )>&3

   if "%1" equ "TWO" (
      echo %line%
      exit /b
   )>&3

I used this strange arrangement because the purpose of my test was to show results generated inside FOR loops without using delayed expansion. For this reason, the "anonymous subroutines" here are placed outside any code block.

If the code in the subroutines does not need to be outside code blocks, then it can be placed nested inside another one. The problem here is find an example that makes sense...

Antonio

[dbenham]

If the code in the subroutines does not need to be outside code blocks, then it can be placed nested inside another one.

I think not
Nesting one anonymous function within the same code block as another is the cause why OperatorGK's attempt does not work. The CALL will parse and execute the next code that follows the currently executing (already parsed) code block. If the 2nd anonymous function is embedded, then it is already parsed, so cannot execute.

Here is a fixed version of OperatorGK's code. One can argue as to whether the anonymous functions are truly nested: The code is not physically nested, but the CALLs are logically nested. I believe this is the only way to have one anonymous function CALL another.

Code: Select all

@echo off
setlocal 
set "anonymous=/?"
for /l %%N in (1 1 5) do (
  set /a N=%%N*2
  call :%%anonymous%% a b c 3>&1 >nul
)
if "%0" == ":%anonymous%" (
  for /l %%M in (1 1 5) do (
    set /a M=%%M*2
    call :%%anonymous%% %1 %2 %3 3>&1 >nul
  )
  exit /b
)>&3
if "%0" == ":%anonymous%" (
  echo %1 %2 %3 %N% %M%
  exit /b
)>&3

One good outcome of this arrangement is there is no need to change the name of the 2nd anonymous function, as the EXIT /B in the first function prevents it from falling through into the 2nd.

Dave Benham

[npocmaka_]

If the code in the subroutines does not need to be outside code blocks, then it can be placed nested inside another one.

I think not
Nesting one anonymous function within the same code block as another is the cause why OperatorGK's attempt does not work. The CALL will parse and execute the next code that follows the currently executing (already parsed) code block. If the 2nd anonymous function is embedded, then it is already parsed, so cannot execute.

Here is a fixed version of OperatorGK's code. One can argue as to whether the anonymous functions are truly nested: The code is not physically nested, but the CALLs are logically nested. I believe this is the only way to have one anonymous function CALL another.

Code: Select all

@echo off
setlocal 
set "anonymous=/?"
for /l %%N in (1 1 5) do (
  set /a N=%%N*2
  call :%%anonymous%% a b c 3>&1 >nul
)
if "%0" == ":%anonymous%" (
  for /l %%M in (1 1 5) do (
    set /a M=%%M*2
    call :%%anonymous%% %1 %2 %3 3>&1 >nul
  )
  exit /b
)>&3
if "%0" == ":%anonymous%" (
  echo %1 %2 %3 %N% %M%
  exit /b
)>&3

One good outcome of this arrangement is there is no need to change the name of the 2nd anonymous function, as the EXIT /B in the first function prevents it from falling through into the 2nd.

Dave Benham

Great

[Aacini]

This is strange. In my opinion, this should work:

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "anonymous=/?"
set level=0

for /L %%N in (1 1 5) do (
   echo Inside for cycle: %%N
   set /A level+=1
   call :%%anonymous%% 3>&1 >nul
   if !level! == 1 (
      echo Inside "anonymous sub", level=!level!, for cycle=%%N
      set /A level-=1
      exit /B
   )>&3
   echo After "anonymous sub"
)

echo Outside for

After the "exit /B" is executed inside the sub, the next line to execute is the "if !level! == 1 " and then, the "echo After..." should be executed and pass to next for iteration; however, the process end at this point.

The most strange thing is that a "FOR /L" is broken!

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "anonymous=/?"

for /l %%N in (1 1 5) do (
   echo Inside for cycle: %%N
   if %%N == 4 (
      rem Break the FOR /L, BUT ALSO THE PROGRAM
      call :%%anonymous%% 3>&1 >nul
      exit /b
   )>&3
   echo After "anonymous sub"
)

echo Outside for

Antonio

[dbenham]

Try this slight variation, and all should be clear

Code: Select all

@echo off
setlocal EnableDelayedExpansion

set "anonymous=/?"
set level=0

for /L %%N in (1 1 5) do (
   echo Inside for cycle: %%N
   set /A level+=1
   call :%%anonymous%% 3>&1 >nul
   if !level! == 1 (
      echo Inside "anonymous sub", level=!level!, for cycle=%%N
      set /A level-=1
      exit /B
   )>&3
   echo After "anonymous sub"
)

echo Outside for >&3

As I said before, the anonymous block cannot be in the same parsed block that CALLs it.


Dave Benham

[Aacini]

[ . . .]

As I said before, the anonymous block cannot be in the same parsed block that CALLs it.

Dave Benham

Excuse me Dave, but the important point here is: why? A subroutine can be placed in the same parsed block than its CALL, as you can see in this code:

Code: Select all

@echo off
setlocal EnableDelayedExpansion

for /L %%N in (1 1 5) do (
   echo For cycle: %%N
   set /A var=%%N*3
   call :inSub !var!
   if 0 == 1 (
      :inSub
      echo Parameter: %1, var: %var%
      exit /B
   )
)

So the question is: why the CALL command behaves differently in the case of an "annonymous subroutine"? In the original SO thread about this topic we concluded that "CALL execute normally": the "call part" push in the call stack the return address and then execute the "goto :/?" part. This one show the GOTO help screen and does not return any error, so the "call" part continue normally. If this is true, my code above should work! Also, note that that code fail in the EXIT /B, not in the CALL!

I think there are some points about this mechanism we can not discover yet...

Antonio

[dbenham]

So the question is: why the CALL command behaves differently in the case of an "annonymous subroutine"? In the original SO thread about this topic we concluded that "CALL execute normally": the "call part" push in the call stack the return address and then execute the "goto :/?" part. This one show the GOTO help screen and does not return any error, so the "call" part continue normally. If this is true, my code above should work! Also, note that that code fail in the EXIT /B, not in the CALL!

I think there are some points about this mechanism we can not discover yet...

I can't claim that there is nothing left to discover, but jeb's original analysis fully explains the currently observed behavior.

At the time of the CALL, the current file pointer is already at the end of the entire block - batch always parses the entire block before any code is executed. The CALL first pushes the call stack and sets the %0 argument to :/?, and then implicitly invokes GOTO. Normally GOTO would begin scanning for a label, starting at the current file position. But GOTO sees the /? and prints out the help message instead. Upon completion of the help (still within the context of the call) the parser resumes at the current file position (remember - it is already at the end of the block ) The remaining code is executed until EXIT /B, or GOTO :EOF, or end of file, upon which the CALL returns and pops the stack, thus resetting the file pointer to the end of the block with the original CALL. The remainder of the code is executed normally.

I see no mystery - It is not possible to CALL an anonymous function that resides within a code block that is shared by the CALL.


Dave Benham

[dbenham]

At some point I'd like to get a single comprehensive post that describes all the ins and outs of :label parsing and CALL and GOTO behavior.

I see a couple things that don't look correct in jeb's first post:

Now the label names.
They also use STOP characters, but not the same, I only know two :!:

Code: Select all

:+

When scanning the file for the label name, I see that there are 5 (maybe 6) stop characters for the label name instead of two: <+>, <:>, <space>, <tab>, <LF>, and possibly <CR>. The other token delimiters are not stop characters

So these labels are equivalent

Code: Select all

:one
:one:two
:one+two
:one^

That last :one^ looks incorrect to me - I can never manage to CALL or GOTO that label. I suppose that may be what you were driving at in This post

I also cannot find a way to call any of the following labels:

Code: Select all

:one^<space>
:one^<tab>
:one;
:one,
:one=
:one^+
:one^:

It appears to me that the label scanner does escape label stop characters, such that they are included in the label name. But the GOTO and CALL label parsers do not escape any of the stop characters, even though the escaped characters are included within the token. I think that would explain why the above labels are inaccessible.

One thing that appears to be missing from this discussion is details as to how the batch parser recognizes labels so that they are never executed. I've attempted to partially describe that in a modified Phase 2 in jeb's SO post. The label detection happens late in phase 2, so there can be some surprising lines that are interpreted as un-executable labels.

One consequence of those rules is redirection can exist prior to the label, and the label will still prevent the line from ever being executed. The redirection will also prevent the label from ever being CALLed (or GOTOed - sorry about the grammer). Note that the redirection never actually takes place.

Code: Select all

@echo off
>test.txt :label this inaccessible label can never be executed or CALLed

This behavior has been used before. It is critical to the WSF hybrid scripting technique

More trivial is the fact that any number of expansion terms can exist prior to the label and still make it unexecutable and inaccessible as long as they expand to nothing or token delimiters.

Code: Select all

@echo off
set local enableDelayedExpansion
%= undefined =% :Cannot be executed or accessed
!= undefined =% :Cannot be executed or accessed

Something that I have never seen demonstrated before, the unexecutable and inaccessible label can appear on the same line as a valid command, after command concatenation. This also is explained by my modifed Phase 2 rules.

Code: Select all

@echo off
echo This is printed & :The remainder of the line is ignored & echo So this command is never executed

One thing that is not explained by my modifed Phase 2 rules: Introduction of redirection prior to the label enables subsequent command concatenation after the inline label. Also, the redirection now takes place.

Code: Select all

@echo off
>test.txt :This is ignored except redirection takes place - the file is overwritten and made empty & echo This is printed
echo Part 1 is printed & >test2.txt :This is ignored but redirection takes place & echo Part 2 is printed

This all can have some interesting effects on labels / comments within parenthesized blocks. But I am too tired to write these out (I may have the flu - not a good New Year's Eve).

But I do have one quick point to make about comment labels within blocks.

You can use :; instead of :: as a comment. The inaccessible label is legal / never treated as a path. So you can safely use symmetric :; label comments as long as they are paired

Code: Select all

(
  :;This is a safe comment
  :;And so is this because they are paired
)

Dave Benham

[dbenham]

I have confirmed that <CR> functions as a stop character in all cases - for a CALL name, a GOTO name, and also for the label name when scanning for the label.

I also believe I have discovered the source of the "bug" that leads to the "anonymous" CALL behavior that npocmaka discovered.

When CALL executes the GOTO to jump to the :label, it expects a non zero return code if the GOTO failed. This happens as expected if the label token is missing, or if the label is not found.

I always thought that requesting help on a command always returned a 1 return code, but it is not that simple:

Using HELP does indeed return 1:

Code: Select all

prompt>(call )

prompt>help goto >nul

prompt>echo %errorlevel%
1

But using the /? option does not set the errorlevel to 1 unless the || operator is used

Code: Select all

prompt>(call )

prompt>goto /? >nul

prompt>echo %errorlevel%
0

prompt>goto /? >nul || rem

prompt>echo %errorlevel%
1

But the CALL execution of the subroutine occurs before the || gets a chance to set the ERRORLEVEL. So CALL never gets the message that the GOTO "failed". The && fires because the last command of the "CALLed" routine was successful:

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "var=/?"
(call )
call :%%var%% 3>&1 1>nul &&echo SUCCESS errorlevel=!errorlevel!||echo FAIL errorlevel=!errorlevel!
>&3 echo %%0=[%0]  errorlevel=[%errorlevel%]
exit /b

--OUTPUT--

Code: Select all

%0=[:/?]  errorlevel=[0]
SUCCESS errorlevel=0
%0=[test.bat]  errorlevel=[0]

Dave Benham

[dbenham]

I have some clarification about the following:

One consequence of those rules is redirection can exist prior to the label, and the label will still prevent the line from ever being executed. The redirection will also prevent the label from ever being CALLed (or GOTOed - sorry about the grammer). Note that the redirection never actually takes place.

Code: Select all

@echo off
>test.txt :label this inaccessible label can never be executed or CALLed
This behavior has been used before. It is critical to the WSF hybrid scripting technique

...

One thing that is not explained by my modifed Phase 2 rules: Introduction of redirection prior to the label enables subsequent command concatenation after the inline label. Also, the redirection now takes place.

Code: Select all

@echo off
>test.txt :This is ignored except redirection takes place - the file is overwritten and made empty & echo This is printed
echo Part 1 is printed & >test2.txt :This is ignored but redirection takes place & echo Part 2 is printed

Well, the above only works in batch mode. It is not true for command line mode - the command line attempts to execute the "label". Also, the redirection always takes place (evidence not shown)

Code: Select all

prompt>>nul :ThisIsNotALabel
The filename, directory name, or volume label syntax is incorrect.

prompt>echo Part 1&>nul :ThisIsNotALabel&echo Part 2
Part 1
The filename, directory name, or volume label syntax is incorrect.
Part 2

With FOR you can look at the ECHO ON output of the DO clause to see how it is parsed, even in command line mode.

Code: Select all

prompt>for %a in (1 2) do echo Part %a.1&>nul :ThisIsNotALabel&echo Part %a.2

prompt>echo Part 1.1  &  & echo Part 1.2
Part 1.1
The filename, directory name, or volume label syntax is incorrect.
Part 1.2

prompt>echo Part 2.1  &  & echo Part 2.2
Part 2.1
The filename, directory name, or volume label syntax is incorrect.
Part 2.2

I find it interesting that the ECHO of the DO clause "knows" that the "label" is not a command, but during execution, the "label" is executed anyway.

But in a batch file, the label is never executed. Here is a batch file that demonstrates this, along with a few more quirks about the batch mode:

test.bat

Code: Select all

@echo on
echo Original Content >test.txt

@echo(
>test.txt :ThisIsALabel_NoRedirection
type test.txt

@echo(
for %%a in (1 2) do 2>nul echo part %%a.1&>test.txt :ThisIsALabel_WithRedirection & 2>nul echo Part %%a.2
type test.txt

--OUTPUT--

Code: Select all

prompt>test

prompt>echo Original Content  1>test.txt


prompt>type test.txt
Original Content


prompt>for %a in (1 2) do echo part %a.1 2>nul  &  & echo Part %a.2 2>nul

prompt>echo part 1.1 2>nul  &  & echo Part 1.2 2>nul
part 1.1
Part 1.2

prompt>echo part 2.1 2>nul  &  & echo Part 2.2 2>nul
part 2.1
Part 2.2

prompt>type test.txt

prompt>

In the first test, I show that the label is not echoed, it is not executed, and the redirection does not take place (the content of test.txt is not cleared)

In the second test with FOR, I show that the label is still not echoed or executed, but the redirection does take place (the content was cleared), even though the redirection is not echoed.


Dave Benham

[jeb]

I didn't think of testing it in the command line context.
That was an interresting idea.

But I get other results than you.

Tested on the command line

Code: Select all

echo Original > out.txt
> out.txt :label & dir out.txt

I get

Die Syntax für den Dateinamen, Verzeichnisnamen oder die Datenträgerbezeichnung ist falsch.
Volume in Laufwerk C: hat keine Bezeichnung.
Volumeseriennummer: 88BA-1CA5

Verzeichnis von c:\temp\labels

18.01.2018 20:05 0 out.txt
1 Datei(en), 0 Bytes

Then I tried the subst trick (I suppose npocmaka had that idea)

Code: Select all

subst :: C:\temp
echo dummy > dummy.bat
> out.txt ::\dummy.bat & echo works without error

It works, but dummy.bat will not be called, but it will be checked for existence.

[dbenham]

Hi jeb. I was hoping you would show up for this

At first I was confused by your post, but I think I understand your point now.

My statement that the command line attempts to execute the ":label" is wrong. Instead, the command line verifies that the ":label" has valid path syntax for an external command, and issues the error when it doesn't.

You demonstrate how to use SUBST to force it to be a valid command, so the error message is eliminated, but then the true :label behavior kicks in and the command is not executed.

Very interesting

I have an additional test that proves your statement about the command line verifying existence of the ":label" external command.

I go through a progression that shows various types of errors, eventually proving that the command line verifies the ":label" external command actually exists.

Code: Select all

prompt><nul :\notExists.bat & echo OK
The filename, directory name, or volume label syntax is incorrect.
OK

prompt><nul ::\notExists.bat & echo OK
The system cannot find the drive specified.
OK

prompt>subst :: c:\test

prompt><nul ::\notExists.bat & echo OK
'::\notExists.bat' is not recognized as an internal or external command,
operable program or batch file.
OK

How about the rest of what I documented in my prior 3 posts in this thread? Does that all look good?

If so, then I hope to create a single post that tries to capture all the label rules in one place.
I'm not sure if I will put that in this thread, or else as a new answer to the SO cmd.exe Parsing Rules question. I think it would be too large to fold into any of the existing answers.

Speaking of that SO Q&A, how do my edits look to you? I made significantly more additions to your answer, and I always feel better when I get the jeb seal of approval.


Dave Benham