Variable expansion inside :: comments?

[Sponge Belly]

Hello All!

I was trying to catch errors from overflowing variables (is it possible, btw?) when I noticed the following:

Code: Select all

@echo off & setlocal enableextensions disabledelayedexpansion

:: create variable containing 4k hashes
set "hashFile=%tmp%\hashFile.tmp"
if not exist "%hashFile%" (
    <nul set /p "=#" >"%hashFile%"
    for /l %%I in (1 1 12) do type "%hashFile%" >>"%hashFile%"
)
for /f "usebackq delims=" %%A in ("%hashFile%") do set "hashStr=%%A"

:: next line parses ok
:: %hashStr:#=$%
echo(works ok

:: next line throws 2 errors
:: %hashStr:#=$$%
echo(never reached

endlocal & goto :EOF

It seems there is some form of variable expansion going on inside :: comments after all!

- SB

[pieh-ejdsch]

Hello,
the content of this variable has Parenthesis!?

Please note that these lines are read in completely. %Variable% and Parameter %1 ... %9 %* as well as special characters are treated as on a line to be executed (see Jebs declaration). The alone is already the reason to write these commentary lines carefully. These :: comment lines can be written as multilines, meaning that the (unmasked) caret at the end of the line, the following line is considered as continuation of the comment. Thus, several successive comment lines without a double point are possible.

https://stackoverflow.com/questions/43410288/batch-setting-rem-as-variable-to-make-code-look-different
use rem instead

edit
you may use

Code: Select all

if . == ^%var:~8192,1%. (echo var OK) else echo var bad

\edit
Phil

[dbenham]

@Sponge Belly - Absolutely

The same thing will happen if you put the expansion within a REM (instead of ::).

The REM fatal error is predicted by the cmd.exe batch parsing rules documented by jeb, because variable expansion in phase 1 occurs before REM detection in phase 2.

I believe label detection also occurs early in phase 2 (the bit that prevents labels from executing, not the CALL or GOTO search for the label). But as of yet, that is missing from jeb's post.

I've documented two other expansion fatal errors in my % expansion rules, both of which can occur within REM and label comments.

All of the following will result in a fatal error:

Code: Select all

:: %~ Missing digit following the tilde gives fatal error
REM %~ Missing digit following the tilde gives fatal error

set DefinedVar=1
:: %DefinedVar:=  Missing find string before = gives fatal error
:: %DefinedVar:*=  Missing find string before = gives fatal error
REM %DefinedVar:=  Missing find string before = gives fatal error
REM %DefinedVar:*=  Missing find string before = gives fatal error

The only mechanism for safely embedding an unrestricted comment without fear of a batch fatal error is to put the comment somewhere where it can never be executed. For example, after EXIT /B, or use GOTO to bypass the comment.


Dave Benham

[aGerman]

or use GOTO to bypass the comment.

Where that means you don't even need to bypass the whole line. It's sufficient to have it in the same line along with the label.

Code: Select all

goto :comment
:comment %hashStr:#=$$%
echo(works, too

Finding a label using GOTO and CALL only sees the first token.

Steffen

[dbenham]

Nice clarification. Thanks

[Sponge Belly]

@Dave

Thanks for the refresher!

@Phil

Something like:

Code: Select all

set "var=%var:#=$$%"

could potentially double a variable in size. And if the variable is over 4095 characters to begin with, it will overflow and cause a fatal error.

Is there any way to catch such errors and take remedial action?

- SB

[pieh-ejdsch]

Hello,
I have a bit in my old things around and found something.
The FOR loop goes only up to ca. 8190 characters + "" and then goes out.
The census of the foreseeable length is also clear.

Code: Select all

@echo off
setlocal enableextensions disabledelayedexpansion
call :setAllMacros
:: create variable containing 4k hashes
set "hashFile=%tmp%\hashFile.tmp"
del %hashfile%
if not exist "%hashFile%" (
    <nul set /p "=#" >"%hashFile%"
    for /l %%I in (1 1 12) do type "%hashFile%" >>"%hashFile%"
)
for /f "usebackq delims=" %%A in ("%hashFile%") do set "hashStr=%%A"

:: next line parses ok
:: %hashStr:#=$%
echo(works ok

 rem one method
 %checkString(var):var=hashStr%
if NOT errorlevel 1 echo Lenght is OK

echo  # ^> $$
 %checkString(var):var=hashStr:#=$$%
if     errorlevel 1 echo can not be changed from # to $$

 rem other method
setlocal enabledelayedexpansion
%strLen(var):var=!hashStr!%
set /a "hashStrLength = len"

%strLen(var):var=!hashStr:#=!%
set /a reducedLength = len
set /a increasedLength = (hashStrLength -reducedLength) *2
set /a newLength = increasedLength +reducedLength

echo HashString Length = %hashStrLength%
echo    reduced Length = %reducedLength%
echo  increased Length = %increasedLength%
echo        new Length = %newLength%
if %newLength% gtr 8189 echo is to long from # to $$
pause

exit /b

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:setAllMacros
:: define LF as a Line Feed (newline) character
set ^"LF=^

^" Above empty line is required - do not remove

:: define a newline with line continuation
set ^"\n=^^^%LF%%LF%^%LF%%LF%^^"

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:strLen.var
@for %%T in ("%temp%\%~n0.tmp.cmd") do @(
 @ >%%T (
  echo(@set strLen(var^)=(%%\n%%
  echo( set "str=Avar"%%\n%%
  echo( set "len=0"%%\n%%
  for /l %%i in (12 -1 0) do @(
   echo( set /a "len|=1<<%%i"%%\n%%
   echo( for %%%%# in (!len!^) do if .!str:~%%%%#^^^^^^,1!==. set /a "len&=~1<<%%i"%%\n%%
  )
  echo(^)
 )
 call %%T
 del %%T
)

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
@set checkString(var)=(%\n%
 setlocal enabledelayedexpansion%\n%
 ( for /f tokens^^^=*delims^^^=^^^ eol^^^= %%V in (%\n%
  "!var!") do @%\n%
 ) ^|^| call%\n%
 endlocal%\n%
 if     errorlevel 1 echo string is BAD!%\n%
 if NOT errorlevel 1 echo string is ok%\n%
)
set "LF="
set "\n="
exit /b

Phil

[jeb]

Is there any way to catch such errors and take remedial action?

I see no way to catch the error with batch itself, as this kind of syntax errors stops the batch file immediately, even the setlocal stack remains unchanged.
This can enable delayed expansion for the command line!

To catch it you can use a subprocess with cmd /c, FOR /F seems not to work here, as it stops but I can't see how to fetch the error

Code: Select all

@echo off
setlocal EnableDelayedExpansion
rem if "%~1" NEQ "" exit /b
if "%~1" == ":longFail" (
    call %~1 %~2
    exit /b
)

call :test 12
call :test 1
exit /b

:test
echo(
echo test %1 %~f0
(
    cmd /c %~f0 :longFail %1 
) || echo ###### CATCH FAILURE ######
echo ERRORLEVEL=!ERRORLEVEL!
echo END
exit /b

:longFail
set "long=-"
echo %~0 started %1

for /L %%n in (%1 1 13) do set "long=!long:~0,4000!!long:~0,4000!"

echo %long%%long%
  
echo %~0 END
exit /b

[Aacini]

Is there any way to catch such errors and take remedial action?

I see no way to catch the error with batch itself, as this kind of syntax errors stops the batch file immediately, even the setlocal stack remains unchanged.
This can enable delayed expansion for the command line!

I tried to make good use of this behavior and wrote this program, intended to leave the Delayed Expansion status Enabled at the command line:

Code: Select all

@echo off
setlocal EnableDelayedExpansion
rem %~ Missing digit...

However, it don't works... I also tested all possible errors described in this thread, but none of them leave Delayed Expansion on... Windows 8.1 here.

Antonio

[jeb]

However, it don't works... I also tested all possible errors described in this thread, but none of them leave Delayed Expansion on... Windows 8.1 here.

I retested it with Win7 and found that this only works when a call is involved.

Code: Select all

@echo off
setlocal EnableDelayedExpansion
call :func

:func
rem %~ Missing digit...

And this enables directly the possibility to supress the error message

Code: Select all

call :func 2> nul

[Aacini]

Ok. This works (I called it EDE.bat):

Code: Select all

@echo off
setlocal EnableDelayedExpansion
call :func 2> nul

:func
rem %~ Missing digit...

Thanks, jeb!

Antonio