Automating the generation of macros usable within other macros
Posted: 16 Feb 2024 09:51
I'm starting a new thread, because the discussion in Macro for Dummies is diverging way beyond the original subject.
I've extended the four macros shown in post #15 in the above thread, to define four additional %%variables after 1 to 4 %expansions%:
Here are the extended FOR!, FOR!2, FOR!3 and FOR!4 macros:
Example of use:
You'll notice how the ABORT and ABORT2 macros are generated with identical source code. Yet ABORT2 is usable within another macro, whereas ABORT is only useable directly.
This common source code is also relatively straightforward, despite all the tricky characters (! > &) involved.
The test runs twice, with delayed expansion enabled then disabled, and its output is:
Notice how the EDE_TEST macro has exactly the same content in the two cases, whereas it was generated with delayed expansion disabled in the first case, and enabled in the second.
In the first case, running with delayed expansion disabled, the ABORT2 macro fires and outputs the abort message as intended before exiting.
In the second case, the test shows the issue that I reported in Macros for Dummies post #14.
This validates the workaround I implemented, but I still don't understand why things work this way.
Anyway, I'm well aware that these four FOR!* macros aren't general enough to cover all macro inclusion cases.
For example they will generate too many carets inside quoted strings. (Workaround in simple cases: Define quoted strings with ^" at both ends.)
The only bullet-proof solution would be to write a batch parser in batch, and use it to automatically create new macros based on original ones, adding the right number of ^ based on the parser state.
Maybe writing such a parser is easier to do now, with the efficient macros for looping, and breaking out of loops, that we have now!
In the short term, any idea for improving that set of macros for generating other macros, or propose alternatives, is welcome.
One side issue I'm still investigating is a good way to define a %%variable that generates a %.
Using a %%p variable is easy, but this might interfere with a user-defined %%p variable.
I tried using the same syntax with a caret, as for the other tricky characters: for %%^% in (1 2) do ...
This does loop twice as intended, proving that % itself can be used as a %%variable.
But I've not managed to use the %%^% value as expected.
I've extended the four macros shown in post #15 in the above thread, to define four additional %%variables after 1 to 4 %expansions%:
- %%^> will generate a '>' character
- %%^< will generate a '<' character
- %%^& will generate a '&' character
- %%^| will generate a '|' character
Here are the extended FOR!, FOR!2, FOR!3 and FOR!4 macros:
Code: Select all
:# Define a %FOR!% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after one %expansion%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do ^
for %%~ in ("") do for %%+ in (%%1%%~~) do for /f "tokens=2" %%- in ("!=! %%1 %%+%%+") do ^
set FOR%%!=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%1 . %%1%%1%%1%%1%%1%%!=%%1%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%+^&) do for %%p%%1^| in (%%-^|) do
:# Define a %FOR!2% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after two %expansions%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do for %%2 in (%%1%%1%%1%%1) do ^
for %%~ in (%%1) do for %%+ in (%%2%%~~) do for /f "tokens=2" %%- in ("!=! %%1%%1%%1 %%+%%+") do ^
set FOR%%!2=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%1%%1 %%2%%2%%2%%2%%~~%%~~%%~~%%!=%%~~%%~~%%~~%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%-^&) do for %%p%%1^| in (%%-^|) do
:# Define a %FOR!3% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after three %expansions%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do for %%2 in (%%1%%1%%1%%1) do for %%3 in (%%2%%2%%2%%2) do ^
for %%~ in (%%2%%1) do for %%+ in (%%3%%~~) do for /f "tokens=2" %%- in ("!=! %%2%%1%%1%%1 %%+%%+") do ^
set FOR%%!3=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%2 %%3%%3%%3%%3%%~~%%~~%%~~%%!=%%~~%%~~%%~~%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%-^&) do for %%p%%1^| in (%%-^|) do
:# Define a %FOR!4% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after four %expansions%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do for %%2 in (%%1%%1%%1%%1) do for %%3 in (%%2%%2%%2%%2) do for %%4 in (%%3%%3%%3%%3) do ^
for %%~ in (%%3%%2%%1) do for %%+ in (%%4%%~~) do for /f "tokens=2" %%- in ("!=! %%2%%2%%2%%1%%1%%1 %%+%%+") do ^
set FOR%%!4=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%2%%2 %%4%%4%%4%%4%%~~%%~~%%~~%%!=%%~~%%~~%%~~%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%-^&) do for %%p%%1^| in (%%-^|) do
Code: Select all
@echo off
goto :dostips1.main
:dostips1.test
:# Two versions of the same ABORT macro. The macro argument is an error message to display on stderr before exiting.
%FOR!% set ABORT=for %%# in (1 2) do if %%#==2 ^(echo Error.%%!ERRMSG%%! Aborting.%%^>%%^&2 %%^& exit /b 1^) else setlocal EnableDelayedExpansion %%^& set ERRMSG=
%FOR!2% set ABORT2=for %%# in (1 2) do if %%#==2 ^(echo Error.%%!ERRMSG%%! Aborting.%%^>%%^&2 %%^& exit /b 1^) else setlocal EnableDelayedExpansion %%^& set ERRMSG=
(set \n=^^^
%# This creates an escaped Line Feed for use in multiline macros - DO NOT ALTER #%
)
:# Test the ABORT macro within another macro.
:# The EDE_TEST macro tests a trick issue about delayed expansion.
%FOR!% set EDE_TEST=(%\n%
if not "%%!%%!"=="" %ABORT2% Must be used with enabled delayed expansion.%\n:# Check the EDE prerequisite =%
for /f "tokens=2" %%t in ("%%!%%! one two three four") do echo Test 1 token 2 = %%t %\n%
for /f "tokens=2" %%t in ("%%!%%! one two%%!%%! three four") do echo Test 2 token 2 = %%t %\n%
for /f "tokens=2" %%t in ("%%!%%! one two%%!=%%! three four") do echo Test 3 token 2 = %%t %\n%
for /f "tokens=2" %%t in ("%%!=%%! one two%%!%%! three four") do echo Test 4 token 2 = %%t %\n%
for /f "tokens=2" %%t in ("%%!=%%! one two%%!=%%! three four") do echo Test 5 token 2 = %%t %\n%
)
set EDE_TEST
%EDE_TEST%
exit /b
:dostips1.init
:# Define a %FOR!% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after one %expansion%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do ^
for %%~ in ("") do for %%+ in (%%1%%~~) do for /f "tokens=2" %%- in ("!=! %%1 %%+%%+") do ^
set FOR%%!=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%1 . %%1%%1%%1%%1%%1%%!=%%1%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%+^&) do for %%p%%1^| in (%%-^|) do
:# Define a %FOR!2% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after two %expansions%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do for %%2 in (%%1%%1%%1%%1) do ^
for %%~ in (%%1) do for %%+ in (%%2%%~~) do for /f "tokens=2" %%- in ("!=! %%1%%1%%1 %%+%%+") do ^
set FOR%%!2=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%1%%1 %%2%%2%%2%%2%%~~%%~~%%~~%%!=%%~~%%~~%%~~%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%-^&) do for %%p%%1^| in (%%-^|) do
:# Define a %FOR!3% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after three %expansions%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do for %%2 in (%%1%%1%%1%%1) do for %%3 in (%%2%%2%%2%%2) do ^
for %%~ in (%%2%%1) do for %%+ in (%%3%%~~) do for /f "tokens=2" %%- in ("!=! %%2%%1%%1%%1 %%+%%+") do ^
set FOR%%!3=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%2 %%3%%3%%3%%3%%~~%%~~%%~~%%!=%%~~%%~~%%~~%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%-^&) do for %%p%%1^| in (%%-^|) do
:# Define a %FOR!4% macro, itself defining %%^^=^, %%!=!, %%^>=>, %%^<=<, %%^&=&, %%^|=| after four %expansions%
for %%p in (%%) do for /f %%! in ("! =! ^^^!") do for /f %%1 in ("^ ^^^^ !!") do for %%2 in (%%1%%1%%1%%1) do for %%3 in (%%2%%2%%2%%2) do for %%4 in (%%3%%3%%3%%3) do ^
for %%~ in (%%3%%2%%1) do for %%+ in (%%4%%~~) do for /f "tokens=2" %%- in ("!=! %%2%%2%%2%%1%%1%%1 %%+%%+") do ^
set FOR%%!4=for /f %%%%! in ^(^"%%! =%%! %%+%%+%%+%%!^"^) do for /f "tokens=2" %%%%1%%1 in ^(^"%%!=%%! %%2%%2 %%4%%4%%4%%4%%~~%%~~%%~~%%!=%%~~%%~~%%~~%%!^"^) do^
for %%p%%1^> in (%%-^>) do for %%p%%1^< in (%%-^<) do for %%p%%1^& in (%%-^&) do for %%p%%1^| in (%%-^|) do
exit /b
:dostips1.main
for %%x in (Disable Enable) do (
echo.
setlocal %%xDelayedExpansion
for /f "tokens=2" %%e in ("!! Dis En") do echo Expansion is %%eabled
call :dostips1.init
call :dostips1.test
endlocal
)
exit /b
This common source code is also relatively straightforward, despite all the tricky characters (! > &) involved.
The test runs twice, with delayed expansion enabled then disabled, and its output is:
Code: Select all
Expansion is Disabled
EDE_TEST=(
if not "!!"=="" for %# in (1 2) do if %#==2 (echo Error.!ERRMSG! Aborting.>&2 & exit /b 1) else setlocal EnableDelayedExpansion & set ERRMSG= Must be used with enabled delayed expansion.
for /f "tokens=2" %t in ("!! one two three four") do echo Test 1 token 2 = %t
for /f "tokens=2" %t in ("!! one two!! three four") do echo Test 2 token 2 = %t
for /f "tokens=2" %t in ("!! one two!=! three four") do echo Test 3 token 2 = %t
for /f "tokens=2" %t in ("!=! one two!! three four") do echo Test 4 token 2 = %t
for /f "tokens=2" %t in ("!=! one two!=! three four") do echo Test 5 token 2 = %t
)
Error. Must be used with enabled delayed expansion. Aborting.
Expansion is Enabled
EDE_TEST=(
if not "!!"=="" for %# in (1 2) do if %#==2 (echo Error.!ERRMSG! Aborting.>&2 & exit /b 1) else setlocal EnableDelayedExpansion & set ERRMSG= Must be used with enabled delayed expansion.
for /f "tokens=2" %t in ("!! one two three four") do echo Test 1 token 2 = %t
for /f "tokens=2" %t in ("!! one two!! three four") do echo Test 2 token 2 = %t
for /f "tokens=2" %t in ("!! one two!=! three four") do echo Test 3 token 2 = %t
for /f "tokens=2" %t in ("!=! one two!! three four") do echo Test 4 token 2 = %t
for /f "tokens=2" %t in ("!=! one two!=! three four") do echo Test 5 token 2 = %t
)
Test 1 token 2 = two
Test 2 token 2 = four
Test 3 token 2 = three
Test 4 token 2 = two
Test 5 token 2 = two
In the first case, running with delayed expansion disabled, the ABORT2 macro fires and outputs the abort message as intended before exiting.
In the second case, the test shows the issue that I reported in Macros for Dummies post #14.
This validates the workaround I implemented, but I still don't understand why things work this way.
Anyway, I'm well aware that these four FOR!* macros aren't general enough to cover all macro inclusion cases.
For example they will generate too many carets inside quoted strings. (Workaround in simple cases: Define quoted strings with ^" at both ends.)
The only bullet-proof solution would be to write a batch parser in batch, and use it to automatically create new macros based on original ones, adding the right number of ^ based on the parser state.
Maybe writing such a parser is easier to do now, with the efficient macros for looping, and breaking out of loops, that we have now!
In the short term, any idea for improving that set of macros for generating other macros, or propose alternatives, is welcome.
One side issue I'm still investigating is a good way to define a %%variable that generates a %.
Using a %%p variable is easy, but this might interfere with a user-defined %%p variable.
I tried using the same syntax with a caret, as for the other tricky characters: for %%^% in (1 2) do ...
This does loop twice as intended, proving that % itself can be used as a %%variable.
But I've not managed to use the %%^% value as expected.