A killer : in a comment

[jfl]

Help!

Earlier today I posted a reply to an old thread initiated by dbenham on StackOverflow.
The small batch script I posted contains lots of comments.
The bizarre problem I have is that if the FIRST comment in the for loop contains a ':' anywhere, then the script fails when invoked with the "\" argument.

Code: Select all

#2 E:on V:off C:\JFL\Temp> t \progra~1\window~1
C:\Program Files\Windows Defender

#2 E:on V:off C:\JFL\Temp> t \
& was unexpected at this time.

#2 E:on V:off C:\JFL\Temp>

As usual, I used :# for comments, which I prefer over the :: that most people use. But this is not the issue:
If I use :: instead of :#, the problem persists.
If I remove ALL comments except the first two in the for loop, the problem persists.
And if I change both to rem instead of :# or ::, the problem persists if there's a ':' anywhere within the first comment!
For example this modified script reproduces the problem: (Notice the word "sh:rt" instead of "short" in the first comment.)

Code: Select all

@echo off

:GetLongPathname %1=PATHNAME %2=Output variable name
setlocal EnableDelayedExpansion
set "FULL_SHORT=%~fs1"           
set "FULL_SHORT=%FULL_SHORT:~3%" 
set "FULL_LONG=%~d1"
echo set "FULL_SHORT=!FULL_SHORT!"
if defined FULL_SHORT (
  for %%x in ("%FULL_SHORT:\=" "%") do ( rem Loop on all sh:rt components.
    set "ATTRIB_OUTPUT=" & rem
    for /f "delims=" %%l in ('attrib "!FULL_LONG!\%%~x" 2^>NUL ^| findstr /v /c:" - %~d1"') do set "ATTRIB_OUTPUT=%%l"
    if defined ATTRIB_OUTPUT (
      for %%f in ("!ATTRIB_OUTPUT:*\=\!") do set "LONG_NAME=%%~nxf"
    ) else (
      set "LONG_NAME=%%~x"
    )
    set "FULL_LONG=!FULL_LONG!\!LONG_NAME!"
  )
) else (
  set "FULL_LONG=%~d1\"
)
endlocal & if not "%~2"=="" (set "%~2=%FULL_LONG%") else echo %FULL_LONG%
exit /b

Changing back the ':' to an 'o' in the comment fixes the issue.

Obviously this is linked to the %FULL_SHORT:\=" "% variable expansion when that variable is empty.
But I don't understand the relationship with a ':' anywhere in the following comment!
Anybody has clues?

[ShadowThief]

:: is only a comment if it's at the very beginning of the line, otherwise it's a syntax error
I also see two lines where you don't use a & between the line and the comment, which is also causing issues.

Labels inside of loops have always been precarious at best. Personally, I either switch to REM for in-loop comments or don't comment the inside of loops at all.

[jfl]

Well I've always used :# comments indented inside loops, so obviously they're not a syntax error.

As far as I know, the only case that never works is when the :# comment is on the last line before a closing parenthesis.
But then they fail with a different symptom. Ex, this code:

Code: Select all

@echo off
if 1==1 (
  echo yes
  :# I said yes
)
exit /b

fails with this output:

Code: Select all

C:\JFL\Temp>t
) was unexpected at this time.

C:\JFL\Temp>

[aGerman]

Just tried to shorten the code to something that still reproduce the failure.

Code: Select all

@set "a="
for %%i in (%a:~0%) do ( rem foo:bar
  echo whatever>nul
)

I used a simpler string manipulation to ensure the quotes have nothing to do with it. Also the & isn't necessarily what eventually was unexpected. It's the > in this code.
And even if you remove the special character (the redirection in this case) you'll still face a syntax error.

Code: Select all

@set "a="
for %%i in (%a:~0%) do ( rem foo:bar
  echo whatever
)

Codes that work:
1) a is defined

Code: Select all

@set "a=1"
for %%i in (%a:~0%) do ( rem foo:bar
  echo whatever>nul
)

2) without string manipulation

Code: Select all

@set "a="
for %%i in (%a%) do ( rem foo:bar
  echo whatever>nul
)

3) without colon in the comment

Code: Select all

@set "a="
for %%i in (%a:~0%) do ( rem foobar
  echo whatever>nul
)

4) comment in the next line

Code: Select all

@set "a="
for %%i in (%a:~0%) do (
rem foo:bar
  echo whatever>nul
)

Steffen

[dbenham]

No, :# will always generate a syntax error within a parenthesized block if it is not immediately followed by a valid command on the next line.

For example:

Code: Select all

@echo off
(
  echo 1
  :# Syntax error
  
  echo 2
)

But move the 2nd echo to immediately follow the :# and all is good

Code: Select all

@echo off
(
  echo 1
  :# This comment is OK
  echo 2
  
)

You can use a 2nd :# as an "executing" label (The label is not recognized during the initial parsing phase, but rather at the very end during the execution phase)

Code: Select all

@echo off
(
  echo 1
  :# This comment is OK
  :# This comment functions as an "executed" label (no operation), so all is OK
  echo 2  
)

:: cannot be used for the 2nd "comment" because during "execution" of the label, it is interpreted as a drive specification - the character before the : is the drive "letter", which just happens to be :

This is an old, and pretty well explored topic on this site. See the long thread at Rules for label names vs GOTO and CALL. I've attempted to summarize the end results of the investigations by myself and many others at viewtopic.php?f=3&t=3803&p=55405#p55405 on the 4 page of that thread.


Dave Benham

[aGerman]

You're right. But can't remember I've ever seen the issue related to string manipulation.
Even shorter:

Code: Select all

@set "a="
echo %a:~0%&rem foo:bar

It doesn't produce an error but the output is weird.

Code: Select all

D:\test>echo ~0bar
~0bar

Steffen

[aGerman]

If I apply this observation to my first code, it'll become

Code: Select all

@set "a="
for %%i in (~0bar
  echo whatever>nul
)

... which reproduces exactly the same error message. So this seems to be the reason. Still weird though.

Steffen

[dbenham]

Obviously this is linked to the %FULL_SHORT:\=" "% variable expansion when that variable is empty.
But I don't understand the relationship with a ':' anywhere in the following comment!
Anybody has clues?

It is well understood

The percent expansion does not parse in an intuitive way when the variable does not exists and the expansion involves string manipulation. This is fully described at https://stackoverflow.com/a/7970912/1012053

The beginning of your intended FOR loop is

Code: Select all

for %%x in ("%FULL_SHORT:\=" "%") do ( rem Loop on all sh:rt components.

%%x is converted into %x, and then FULL_SHORT is not defined, so %FULL_SHORT: is removed, yielding:

Code: Select all

for %x in ("\=" "%") do ( rem Loop on all sh:rt components.

Now the parser thinks there is another variable to expand between the % and :, again not defined. The non-existent variable name is ") do ( rem Loop on all sh. The expansion yields:

Code: Select all

for %x in ("\=" "rt components.

The FOR command no longer has a closed IN() clause, so it thinks the next line is another file specification, not a command to be executed:

Code: Select all

set "ATTRIB_OUTPUT=" & rem

The unescaped & naturally causes the syntax error at this point.


Without the : in the REM, there is no 2nd variable expansion, rather the lone % is simply stripped. So the closing IN() clause parenthesis is preserved and all is well. The loop line becomes:

Code: Select all

for %x in ("\=" "") do ( rem Loop on all short components.

The important lesson is to never allow undefined variables when you use %var% expansion with string manipulation. Always detect the undefined variable before hand and don't allow the undefined expansion line to be parsed. If you fail to do this, then you can very easily run into some hard to diagnose failures. Remember that the entire parenthesized block is parsed all at once. The problem expansion could be anywhere within the block!


Dave Benham

[aGerman]

Good to see it doesn't break the expansion rules you discovered once. But I admit every time I read the SO thread I can't recall even half of it when I reach the end

[dbenham]

I remember a lot of it, but believe me, I frequently reference the rules I wrote because I can't remember everything either. At least I usually remember enough to know when to look it up

[jeb]

And this behaviour is a useful feature!

Code: Select all

%LOAD_ONLY_ONCE:call "%~dp0\libBase.cmd" := %

If LOAD_ONLY_ONCE is undefined, the call is executed and there the LOAD_ONLY_ONCE macro is defined.
Any further such line will execute only the macro.

jeb

[jfl]

Thanks a lot for the investigations and explanations. I now understand the issue well enough to avoid it in the future.
(And I would never have understood it without dbenham's patient accumulation of knowledge on that thorny subject!)

Note that I had actually already seen a few incomprehensible failures of :# comments in () blocks. But this had always been in huge scripts at work. Until yesterday, I had never managed to reproduce the problem in a short script that I could share here. Every time the easy solution had been to use rem instead of :#, and forget about it. All this was rare enough that it did not deter me from using :# all over the place.

One way to avoid the issue will actually be to use delayed expansion more often. For example, this works, contrary to the version using (%a:~0%):

Code: Select all

@echo off
setlocal EnableDelayedExpansion
set "a="
for %%i in (!a:~0!) do ( rem foo:bar
  echo whatever
)
exit /b

For the sake of beauty, I've done that in the post on StackOverflow, which allowed to change the single rem back to a :#.

:: cannot be used for the 2nd "comment" because during "execution" of the label, it is interpreted as a drive specification - the character before the : is the drive "letter", which just happens to be :

One more reason to prefer :# over :: for all comments!

[dbenham]

Probably the best option for comments within loops is to use %= UNDEFINED VARIABLE AS COMMENT =%. This can safely be used absolutely anywhere as long as the comment does not contain a % or : character. It even works as an inline comment. It is safe because it is impossible to define a variable with an = in the name, and dynamic pseudo variables never have an = anywhere other than the first character. I use = at beginning and end just for symmetry and better visual appearance.

I don't like :# because that is a valid label that can be CALLed or jumpted to by GOTO. The :: can never function as a valid label because the : is a stop character for the label name.

Code: Select all

@echo off
echo 1
goto :#
echo 2 skipped
:#
echo 3

Dave Benham

[dbenham]

And this behaviour is a useful feature!

Code: Select all

%LOAD_ONLY_ONCE:call "%~dp0\libBase.cmd" := %

If LOAD_ONLY_ONCE is undefined, the call is executed and there the LOAD_ONLY_ONCE macro is defined.
Any further such line will execute only the macro.

jeb

Oooh - clever

But I think only useful if you want to make your code inscrutable.

I prefer to separate the macro definition from the execution. I just put an IF DEFINED MACRO at the top of the definition script to EXIT /B or GOTO to prevent multiple runs. Not clever, but the intent is much more obvious.


Dave Benham

[Aacini]

Just as a personal preference, I used to use %/* C STYLE COMMENT */%...

Antonio