Reinterpreting macros in command-line context

[mataha]

I have a script with an alias definition like this:

Code: Select all

set ^"$true=(call )"

doskey ^!^! = (^
    if "^!^"=="^!" (^
        (echo(Delayed expansion is ON.)^
    ) else (^
        (echo(Delayed expansion is OFF.)^
    )^
) ^>con ^& %$true%

It obviously works. However, I would like to reinterpret it in order to get rid of all the indentation so that it's easier to parse visually from the command-line (via e.g. doskey /macros | findstr /birc:"!!="). Well then:

Code: Select all

setlocal EnableDelayedExpansion
if /i "%~f0"=="%~dpnx0" (
    set ^"$p=%%<nul"
) else (
    set ^"$p=^%<nul"
)
for /f "tokens=1,* delims== " %$p%l in ('doskey /macros:cmd.exe') do (
    set "label=%$p%~l"
    set "macro=%$p%~m"
    set "macro=!macro:    =!"
    doskey !label! = !macro!
)
endlocal

Two problems arise:

1. Hazardous characters: ^ and ! obviously have to be escaped (else illegal macro definition). Normally I would use goto in order to escape these in an eager expansion scope (DisableDelayedExpansion), but...
2. I would like to source the script via pipe to cmd.exe (as another program will be in charge of producing it). This means I will be operating in command-line context, thus setlocal is out of the window.

Is this even possible to achieve? output_script | cmd /v:on >nul can only get me so far.
For reference my use-case can be found here.

[jeb]

Hi mataha,

it's a bit unclear what you want to achieve.

Currently, I assume, you only want to compress already defined doskey macros.
If this is the case, I would opt to define them without the indention, at the first definition, not redefine them later.
I don't understand, if you try to define them in a command line or a batch file context?
If you define them with a pipe, you could just call another batch file on the right side, that's much simpler than handle all the command line syntax quirks.

Btw. You could define your doskey-macros with visible indention, but without adding them.
Doesn't look perfect, but better than no indention at all.

Code: Select all

doskey ^!^! = (^
%= =%if "^!^"=="^!" (^
%=     =%(echo(Delayed expansion is ON.)^
%= =%) else (^
%=     =%(echo(Delayed expansion is OFF.)^
%= =%)^
) ^>con ^& %$true%

[mataha]

Hello!

My immediate goals are to:

• be able to execute any Batch code from command-line context
• have readable alias definitions obtained from doskey /macros, ideally with syntax highlighting

The first goal would let me execute scripts in a similar way POSIX shells do (and I'm very close - those percent definitions were key!), e.g.:

Code: Select all

curl -fsSL https://some.script.on.a.site | cmd /d >nul

I just need to perfect the following:

1. sanitizing hazardous characters in a command-line context without toggling delayed expansion (I can either run cmd /v:on or cmd /v:off - there's no middle ground)
2. create a check for command-line context that would work in every scope; "%~f0"=="%~dpnx0" is good, but doesn't work when the code has been already evaluated in a parentheses block (e.g. goto 2>nul); I think calling a label is better? Though I can't reliably do that when I'm piping a script into cmd.exe nor when Command Extensions are disabled... maybe using goto is better?
3. obtaining the caller script, if any, from any context - currently that works only for %~f0 in top-level scope; if I call it from a subroutine it returns the label's name instead... do I just move up the scopes until the first character is not a colon?

My second goal would allow me to view macros in a format that's fast to parse visually - like this, with syntax highlighting and stuff. However, I would like to both view macros with indentation and sparkles AND have them stripped of all unnecessary whitespace within DOSKEY (faster execution).

Because I want to use this for debugging, I would like to have that as a DOSKEY macro - without relying on a presence of a script in a potentially unfamiliar environment.

Whew, that was a bit off-topic, but I hope the picture is a bit clearer now.

[jeb]

First, a minor problem.
- I would use a different context detection

Code: Select all

if "%=^%=" == "%=%=" ( echo batch-context ) else echo cmd-context

Because "%~f0"=="%~dpnx0" can be disturbed, by

Code: Select all

set "~f0"=="=& "^&  XXXXXXXXXXXXX"
if "%~f0"=="%~dpnx0" ( echo batch-context ) else echo cmd-context
< no output at all >

My immediate goals are to:

be able to execute any Batch code from command-line context

It's not possible, because CALL/GOTO can't call any function, SETLOCAL doesn't work, ...
Without these, your only way to loop anything, is to use FOR-loops, it's like the (batch-)macro technique.

I just need to perfect the following:

1. sanitizing hazardous characters in a command-line context without toggling delayed expansion (I can either run cmd /v:on or cmd /v:off - there's no middle ground)
2. create a check for command-line context that would work in every scope; "%~f0"=="%~dpnx0" is good, but doesn't work when the code has been already evaluated in a parentheses block (e.g. goto 2>nul); I think calling a label is better? Though I can't reliably do that when I'm piping a script into cmd.exe nor when Command Extensions are disabled... maybe using goto is better?
3. obtaining the caller script, if any, from any context - currently that works only for %~f0 in top-level scope; if I call it from a subroutine it returns the label's name instead... do I just move up the scopes until the first character is not a colon?

1. sanitizing hazardous characters in a command-line context without toggling delayed expansion
- When and what do you want to sanitize? Examples needed

2. create a check for command-line context ... but doesn't work when the code has been already evaluated in a parentheses block (e.g. goto 2>nul)
- I don't understand how your code is written in that case. Even if it's already evaluated, it still should be correct. Please show an example

3a. obtaining the caller script, if any, from any context...
- It can only work in batch context, what do you expect in cmdline-context?
3b. do I just move up the scopes until the first character is not a colon?
- Yes, that's the common way (common for the four people in the world who uses this at all)

[mataha]

First, a minor problem.
- I would use a different context detection

Code: Select all

if "%=^%=" == "%=%=" ( echo batch-context ) else echo cmd-context

Because "%~f0"=="%~dpnx0" can be disturbed, by

Code: Select all

set "~f0"=="=& "^&  XXXXXXXXXXXXX"
if "%~f0"=="%~dpnx0" ( echo batch-context ) else echo cmd-context
< no output at all >

I forgot about that. This is ingenious, thanks.

It's not possible, because CALL/GOTO can't call any function, SETLOCAL doesn't work, ...
Without these, your only way to loop anything, is to use FOR-loops, it's like the (batch-)macro technique.

Macros won't work either as setlocal doesn't work.
With functions, one way of "emulating" their behaviour that I've considered is:

Code: Select all

call :is_batch_context || (
    ...do some stuff...
)

...irrelevant code...

:is_batch_context > Result
    exit /b 0 "If this is callable, then we're operating in a batch context"

But you're right - not having access to subroutines hurts.

1. sanitizing hazardous characters in a command-line context without toggling delayed expansion
- When and what do you want to sanitize? Examples needed

CMDCMDLINE, DOSKEY macros etc. Say I wanted to parse the following and prepend opening parentheses with indentation within a DOSKEY alias:

Code: Select all

pwd!=(for /f "skip=9 tokens=1,2,*" %j in ('"fsutil reparsepoint query ."') do @(if "%~j"=="Print" if "%~k"=="Name:" if not "%~l"=="" (echo(%~l))) || (chdir)

The end result would be:

Code: Select all

pwd!=(
    for /f "skip=9 tokens=1,2,*" %j in (
        '"fsutil reparsepoint query ."'
    ) do @(
        if "%~j"=="Print" if "%~k"=="Name:" if not "%~l"=="" (
            echo(%~l
        )
    )
) || (
    chdir
)

Impossible without variable substitution.
Or examining CMDCMDLINE in a code snippet curled into cmd.exe:

Code: Select all

set "cmd=!CMDCMDLINE!"
for /f "tokens=1,* delims=/" %p in ("!cmd://=!") do (
    ...do some stuff...
)

If I don't use delayed expansion here then every metacharacter (<>|&) is dangerous. If I do, then ^ and ! get trimmed. I can't mutate CMDCMDLINE either because of its questionable behaviour - I have to copy/clone it first. Honestly I have no idea how to proceed.

2. create a check for command-line context ... but doesn't work when the code has been already evaluated in a parentheses block (e.g. goto 2>nul)
- I don't understand how your code is written in that case. Even if it's already evaluated, it still should be correct. Please show an example

Code: Select all

:exit (exit_code: number?) > Abort
set "exit_code=%~1"

(
    (goto) & (goto)
    call :exit %exit_code% || (
        title %0 &@rem Here, I need top-level context name
        "%ComSpec%" /d @exit /b %exit_code% 2>nul
    )
) 2>nul

Too early to save %0 to a variable, too late to retrieve it. title here can be any command - I just want to operate on %0 here.
Something that crossed my mind just now - maybe call set between the gotos is the way?

3a. obtaining the caller script, if any, from any context...
- It can only work in batch context, what do you expect in cmdline-context?
3b. do I just move up the scopes until the first character is not a colon?
- Yes, that's the common way (common for the four people in the world who uses this at all)

My bad - I meant label context here, not command-line context (this doesn't make sense in command-line context anyway). I simply want to know what the global %0 is from within a label in order to reliably obtain the title with or without parentheses. Currently I use the code above, but it's a one-way ride.

[jeb]

1. sanitizing hazardous characters in a command-line context without toggling delayed expansion (I can either run cmd /v:on or cmd /v:off - there's no middle ground)
...
Examining CMDCMDLINE in a code snippet curled into cmd.exe

I would always opt for using a helper batch file, that would simplify things a lot, but in this case it could also be done in commandline-context
Simple by using the batch-macro technique.
Double all carets, prepend one caret to every exclamation mark

Code: Select all

set "cmd=!CMDCMDLINE!"

REM *** Test string only, to test hardcore input
set "cmd=;Say: Harry & Caret^^ Bang^! & single "quote with ^^^> redirect " !

for /F %P in ("%") do @(
  set "cmd=!cmd:"=""!"
  set "cmd=!cmd:^=^^^^!"
  call set "cmd=%Pcmd:!=^^^!%P"
  set "cmd=!cmd:""="!"
  set "cmd=!cmd:^^=^!"
  for /f "delims=" %L in (^""!cmd://=!"^") do (
    set "line=%~L" !
    echo ### !line!
  )
)

3. obtaining the caller script, if any, from any context
...
Too early to save %0 to a variable, too late to retrieve it. title here can be any command - I just want to operate on %0 here.
Something that crossed my mind just now - maybe call set between the gotos is the way?

Yes, that's the way

Code: Select all

@echo off

call :func1
echo NEVER
exit /b

:func1
call :func2
echo NEVER 
exit /b

:func2
(
    set "skip="
    for /L %%n in (1 1 20) do (
      if not defined skip (
        call set "arg0=%%0"
        call set "first=%%arg0:~0,1%%"
        call set skip=%%first::=%%
        call echo Stack %%arg0%%
        (goto)
      )
    )
) 2>nul
echo NEVER

But if you only need the batch file name itself, it's really simple

Code: Select all

echo Batch filename %~f0 independent if used in nested func, current func name is %0

jeb

[mataha]

Code: Select all

set "cmd=!CMDCMDLINE!"

REM *** Test string only, to test hardcore input
set "cmd=;Say: Harry & Caret^^ Bang^! & single "quote with ^^^> redirect " !

for /F %P in ("%") do @(
  set "cmd=!cmd:"=""!"
  set "cmd=!cmd:^=^^^^!"
  call set "cmd=%Pcmd:!=^^^!%P"
  set "cmd=!cmd:""="!"
  set "cmd=!cmd:^^=^!"
  for /f "delims=" %L in (^""!cmd://=!"^") do (
    set "line=%~L" !
    echo ### !line!
  )
)

That's a pretty clever hack; and here I thought I had a solid grasp on the language. I have a few questions though as I don't think I understand this fully:

• That call set serves as a substitute for DisableDelayedExpansion, right?
• I'm not sure what purpose do ^" serve in the second for /f - is it to remove unneeded carets in tandem with that trailing exclamation mark?
• I believe I have a slight improvement over the first for /f:

Code: Select all

for %P in (%) do @(
    ...
    call set "cmd=%Pcmd:!=^^^!%P"
    ...
)

This works even with Command Extensions disabled (not that it matters in this case).

But if you only need the batch file name itself, it's really simple

Code: Select all

echo Batch filename %~f0 independent if used in nested func, current func name is %0

I guess one way of tackling this is limiting for /l to... what was the maximum for subroutines, 200? Then just doing goto over and over.

That said... My main quip is %0 won't work inside a label as the script might have been called with quotes or fully qualified path or other weird stuff:

Code: Select all

::: C:\Users\user\context.cmd

@call :label
@(echo(%0) & @(echo(%~0) & (echo(%~f0)

@pause >nul & exit /b

:label
@(echo(%0) & @(echo(%~0) & (echo(%~f0)

Code: Select all

C:\Users\user> "..\user\.\"context""
:label
:label
C:\Users\user\context.cmd
"..\user\.\"context""
..\user\.\"context"
C:\Users\user\"context"

Why is this relevant? Because the title will have %0 appended to it, in this case "..\user\.\"context"". But I can check if the first character is (not) a colon and then refer to %0 in a similar fashion to your example.

[jeb]

That call set serves as a substitute for DisableDelayedExpansion, right?
I'm not sure what purpose do ^" serve in the second for /f - is it to remove unneeded carets in tandem with that trailing exclamation mark?
I believe I have a slight improvement over the first for /f:

1. That call set serves as a substitute for DisableDelayedExpansion, right?
"CALL SET" is a substitute for DisableDelayedExpansion, but my reason for using it is quite different!
To replace an exclamation mark, the only way is to use the percent replace syntax, but as we are already in a code block, percent expansion doesn't work, so the only way is to use CALL to force a second dynamically parse step for the percent expansion.

2. I'm not sure what purpose do ^" serve in the second for /f - is it to remove unneeded carets in tandem with that trailing exclamation mark?
It's for the correct quoting. With the first caret, only the inner block "!cmd://=!" is quoted, without the caret the inner block wouldn't be quoted and delimiter characters (<space>,;=) could be nasty.

3. I believe I have a slight improvement over the first for /f: for %P in (%) do @( ...
Looks good, I don't see any problems here

-----------------
Behaviour of %0
As I understand, you want the original value of %0, not the sanitized version (like in %~f0)?
IMHO, then the only way is to use the (goto) technique, but perhaps you should consider to use instead the sanitized value for the title

[mataha]

"CALL SET" is a substitute for DisableDelayedExpansion, but my reason for using it is quite different!
To replace an exclamation mark, the only way is to use the percent replace syntax, but as we are already in a code block, percent expansion doesn't work, so the only way is to use CALL to force a second dynamically parse step for the percent expansion.

I see! In that step, only unescaped exclamation marks are an issue, and e.g. carets are fine as long as they're quadruple-escaped?

It's for the correct quoting. With the first caret, only the inner block "!cmd://=!" is quoted, without the caret the inner block wouldn't be quoted and delimiter characters (<space>,;=) could be nasty.

Because we're using for /f, the inner quotes would serve only as a pointer that we're processing a string, and that string could still be subject to tokenization?

Behaviour of %0
As I understand, you want the original value of %0, not the sanitized version (like in %~f0)?
IMHO, then the only way is to use the (goto) technique, but perhaps you should consider to use instead the sanitized value for the title

I know, but unfortunately it's %0 which gets appended to the title and tasklist /v will happily show that when queried!

[jeb]

jeb wrote: ↑
Wed Jun 21, 2023 10:53 am
"CALL SET" is a substitute for DisableDelayedExpansion, but my reason for using it is quite different!
To replace an exclamation mark, the only way is to use the percent replace syntax, but as we are already in a code block, percent expansion doesn't work, so the only way is to use CALL to force a second dynamically parse step for the percent expansion.

I see! In that step, only unescaped exclamation marks are an issue, and e.g. carets are fine as long as they're quadruple-escaped?

The first part is correct, each exclamation mark need one caret, but I add two.
Carets are NOT ok when quadrupled, they just need to be doubled!
BUT the exclamation mark replace with CALL SET can only add carets in a multiple of two.
Look at this sample

Code: Select all

@echo off
CALL echo Test1# ^ 1Caret
CALL echo Test2# ^^ 2Caret
CALL echo Test3# ^^^ 3Caret
CALL echo Test4# ^^^^ 4Caret
CALL echo Test5# "^ 1Caret"
CALL echo Test6# "^^ 2Caret"
CALL echo Test7# "^^^ 3Caret"
CALL echo Test8# "^^^^ 4Caret"

Test1# 1Caret
Test1# ^ 2Caret
Test1# ^ 3Caret
Test1# ^^ 4Caret
Test1# "^^ 1Caret"
Test1# "^^^^ 2Caret"
Test1# "^^^^^^ 3Caret"
Test1# "^^^^^^^^ 4Caret"

If the caret is inside quotes, it's (nearly) impossible to create a single quote, and we need the quotes to be safe against special characters.

Therefore, I replace a single caret with four carets, and an exclamation mark with "^^!" and later, I replace all double carets to single carets.

Code: Select all

set "prompt=$G "

set "cmd=!CMDCMDLINE!"
set "cmd=;Say: Harry & Caret^^ Bang^! & single "quote with ^^^> redirect " !
for /F %P in ("%") do @(
  set /p "=Step#0 : " <nul & set cmd
set "cmd=!cmd:"=""!"
  set /p "=Step#1 : " <nul & set cmd
set "cmd=!cmd:^=^^^^!"
  set /p "=Step#2 : " <nul & set cmd
call set "cmd=%Pcmd:!=^^^!%P"
  set /p "=Step#3 : " <nul & set cmd
set "cmd=!cmd:""="!"
  set /p "=Step#4 : " <nul & set cmd
set "cmd=!cmd:^^=^!"
  set /p "=Step#5 : " <nul & set cmd
for /f "delims=" %L in (^""!cmd://=!"^") do (
    set "line=%~L" !
    echo ### !line!
)
)
REM ** ENDE **

Step#0 : cmd=;Say: Harry & Caret^ Bang! & single "quote with > redirect
Step#1 : cmd=;Say: Harry & Caret^ Bang! & single ""quote with > redirect
Step#2 : cmd=;Say: Harry & Caret^^^^ Bang! & single ""quote with > redirect
Step#3 : cmd=;Say: Harry & Caret^^^^ Bang^^! & single ""quote with > redirect
Step#4 : cmd=;Say: Harry & Caret^^^^ Bang^^! & single "quote with > redirect
Step#5 : cmd=;Say: Harry & Caret^^ Bang^! & single "quote with > redirect

### ;Say: Harry & Caret^ Bang! & single "quote with > redirect

There is one more tricky part, perhaps you saw the dangling exclamation mark in

Code: Select all

set "line=%~L" !

Most of the time it has no meaning, but in case when the cmd contains carets but no exclamation marks, it's important to reduce the carets in the last step.

jeb wrote: ↑
Wed Jun 21, 2023 10:53 am
It's for the correct quoting. With the first caret, only the inner block "!cmd://=!" is quoted, without the caret the inner block wouldn't be quoted and delimiter characters (<space>,;=) could be nasty.

Because we're using for /f, the inner quotes would serve only as a pointer that we're processing a string, and that string could still be subject to tokenization?

Correct

jeb wrote: ↑
Wed Jun 21, 2023 10:53 am
Behaviour of %0
As I understand, you want the original value of %0, not the sanitized version (like in %~f0)?
IMHO, then the only way is to use the (goto) technique, but perhaps you should consider to use instead the sanitized value for the title

I know, but unfortunately it's %0 which gets appended to the title and tasklist /v will happily show that when queried!

Now I understand your requirement