foolproof counting of arguments

[Liviu]

Yet another variation on the same theme... The code below collects all parameters in a temp file, with the stop condition based on the size of the file. Then, it reads the file back in one shot.

EDIT: code below has changed vs. the one posted earlier today. At the time I edited, there were no followups, so I hope no one minds too much. More details at the bottom.

2nd EDIT: nested the "do rem" parts inside a "setlocal disableextensions" block to protect against accidental expansion of %~a %~dpa etc.

Note that the enclosing (...) is still evaluated at the outer level with extensions enabled, which allows %* to work correctly, but the for loop executes with extensions disabled so %~a, %~dpa etc are no longer expanded.

Code: Select all

@echo off
setlocal disableDelayedExpansion
set "tab=   "

:: can't call with wacky %%*
:: must goto and rely on hardcoded return to :main
::
:: upon return, the following variables are set
::   args is the complete command line
::   argc is the number of arguments
::   arg1, arg2, etc are the individual arguments
::   argsLen, arg1len, arg2len etc are the respective string lengths
:: inside a nested setlocal enableDelayedExpansion
::
goto :getArgs

:: body of actual code would go under :main below
::
:main

echo.
echo ---- copied from %%? ----
echo.
echo args %tab% %tab%{ !args! }
for /l %%n in (1, 1, %argc%) do (
  echo arg%%n %tab% %tab%{ !arg%%n! }
)

:: this part is only needed to get the original cmd line offsets
:: e.g. in order to preserve whitespace with !args:~%arg2off%!
::
:: requires the args, argc, arg?len variables be set by :getArgs
:: sets arg?off upon return to the offset of arg? within args
::
call :getOffs

:: only uses arg1, arg2 etc for verification
::
echo.
echo ---- chopped off %%* ----
echo.
@rem entire command line
echo args %tab%~0,%argsLen% %tab%{ !args! }
set /a delimOff = 0
for /l %%n in (1, 1, %argc%) do (
  @rem whitespace before %%n'th argument
  set /a delimLen = !arg%%noff! - !delimOff!
  for /f "tokens=1*" %%p in ("!delimOff! !delimLen!") do (
    echo %tab%~!delimOff!,!delimLen! %tab%{ !args:~%%p,%%q! }
  )
  set /a delimOff = !arg%%noff! + !arg%%nLen!
  @rem %%n'th argument
  for /f "tokens=1*" %%p in ("!arg%%noff! !arg%%nlen!") do (
    echo arg%%n %tab%~!arg%%noff!,!arg%%nlen! %tab%{ !args:~%%p,%%q! }
    @rem reality check
    if not "!args:~%%p,%%q!"=="!arg%%n!" (
      echo arg%%n ??? %tab%{ !args:~%%p,%%q! } ^^!= { !arg%%n! } 1>&2
    )
  )
)
set /a delimLen = !argsLen! - !delimOff!
if defined args (
  @rem whitespace after last argument
  echo %tab%~!delimOff!,!delimLen! %tab%{ !args:~%delimOff%,%delimLen%! }
)
set "delimOff="
set "delimLen="

goto :eof

::.............................................................................

:getArgs
set "remArgs=%temp%\%random%.tmp"
set prompt=@
>"%remArgs%" (
  setlocal disableextensions
  echo on
  for %%a in (%%a) do rem { %* }
  @echo off
  endlocal
)
call :getFileSize "%remArgs%" sizeOld
set /a argsLen = %sizeOld% - 14

set /a argc = 1
:nextArg
>>"%remArgs%" (
  setlocal disableextensions
  echo on
  for %%a in (%%a) do rem { %1 }
  @echo off
  endlocal
)
call :getFileSize "%remArgs%" sizeNew
set /a argLen = %sizeNew% - %sizeOld% - 14
if %argLen% leq 0 (
  set "sizeOld="
  set "sizeNew="
  set "argLen="
  goto :lastArg
)

set /a arg%argc%len = %argLen%
set /a sizeOld = %sizeNew%
set /a argc += 1
shift /1
goto :nextArg

:lastArg
set /a argc -= 1
prompt

setlocal enableDelayedExpansion
<"%remArgs%" (
  set /p "args="
  set /p "args="
  set "args=!args:~7,-3!"
  for /l %%n in (1, 1, %argc%) do (
    set /p "arg%%n="
    set /p "arg%%n="
    set "arg%%n=!arg%%n:~7,-3!"
  )
)
del "%remArgs%"

goto :main

:getFileSize
set /a %~2 = %~z1
goto :eof

::.............................................................................

:getOffs
set /a argsOff = 0
for /l %%n in (1, 1, %argc%) do (
  call :getOff args !argsOff! arg%%noff
  set /a argsOff = !arg%%noff! + !arg%%nlen!
)
set "argsOff="
goto :eof

:getOff
setlocal enableDelayedExpansion
set /a argOff = %~2
:nextOff
set "chrNext=!%~1:~%argOff%,1!"
if "!chrNext!"=="" goto :lastOff
if not "!chrNext!"==" " ^
if not "!chrNext!"=="%tab%" ^
if not "!chrNext!"=="," ^
if not "!chrNext!"==";" ^
if not "!chrNext!"=="=" goto :lastOff
set /a argOff += 1
goto :nextOff
:lastOff
endlocal & set /a %~3 = %argOff%
goto :eof

Still don't like it completely, because it shifts parameters out thus destroying the original %1/%2/etc. It also leaves all variables stuck in a setlocal block without an easy way to send them back to the enclosing context. But, other than that, it appears to be working as advertised.

Liviu

EDIT: I tidied up some code (the 'for' loop when reading "%remArgs%" back), and added some inline comments (since I seem to be misplacing/forgetting web references way more often than I lose local batch files).

Also added the :getOffs counterpart, which essentially recovers the separators that cmd ignores from the full command line. Nice thing about it is that it basically tokenizes the command line while leaving the actual token parsing to cmd itself. In other words, as long as (1) cmd uses the same escaping logic between %* and %1/%2/etc, and (2) the delimiters remain the known space/tab/comma/semicolon/equal ones, this "manual reparsing" should match cmd's own, regardless of whatever the rules are inside a token.

Below is a sample output using the updated code.

Code: Select all

C:\>eccoargs ; %a %0 %~1 %cd^% !cd! a^|b a^&b "a&b" ^<"&"^> ^^;,=

---- copied from %? ----

args            { ; %a %0 %~1 %cd% !cd! a|b a&b "a&b" <"&"> ^;,= }
arg1            { %a }
arg2            { %0 }
arg3            { %~1 }
arg4            { %cd% }
arg5            { !cd! }
arg6            { a|b }
arg7            { a&b }
arg8            { "a&b" }
arg9            { <"&"> }
arg10           { ^ }

---- chopped off %* ----

args    ~0,46   { ; %a %0 %~1 %cd% !cd! a|b a&b "a&b" <"&"> ^;,= }
        ~0,2    { ;  }
arg1    ~2,2    { %a }
        ~4,1    {   }
arg2    ~5,2    { %0 }
        ~7,1    {   }
arg3    ~8,3    { %~1 }
        ~11,1   {   }
arg4    ~12,4   { %cd% }
        ~16,1   {   }
arg5    ~17,4   { !cd! }
        ~21,1   {   }
arg6    ~22,3   { a|b }
        ~25,1   {   }
arg7    ~26,3   { a&b }
        ~29,1   {   }
arg8    ~30,5   { "a&b" }
        ~35,1   {   }
arg9    ~36,5   { <"&"> }
        ~41,1   {   }
arg10   ~42,1   { ^ }
        ~43,3   { ;,= }

C:\>

[jeb]

Hi,

Pretty sure I've seen this noted before, though I can't find a reference handy now... One other corner case is that the 'for' loop would save the incorrect %1 when passed %a or similar as an argument, presumably because of some loop variable name clash. Didn't look into it much yet, but for whatever reason replacing it with
Code:
for %%a in (%%a) do rem . %1.

appears to work better under xp.sp3 at least. Again, not fully understood or tested on my part.

Good idea, but it fails too

The main problem is to use a FOR-Parameter which can't be expanded.
That's Dave speciality, he made somewhere a list.

As now your code would fail with

Code: Select all

foolProofParam.bat %a...%~a...%~dpa

jeb

[dbenham]

Hi,

for %%a in (%%a) do rem . %1.

Good idea, but it fails too

The main problem is to use a FOR-Parameter which can't be expanded.
That's Dave speciality, he made somewhere a list.

As now your code would fail with

Code: Select all

foolProofParam.bat %a...%~a...%~dpa

Good catch
I should have seen that one coming once the problem with %a was pointed out

Unfortunately, I think we've reached a dead end

The list of special cases for FOR variables you were referring to is on StackOverflow. There are characters that cannot be directly defined as FOR variables, yet can be expanded. But they don't do us any good.

What we need is a character that can be defined as a FOR variable, but can't be expanded. I've never come up with any. The closest we have is the one you showed me: %

Code: Select all

for %%%% in (%%~%%) do rem . %1.

Command line arguments %% and %~% will be preserved.

But arguments like %~a% %~dp% %~path:% etc. will fail

So % has the fewest possible corner cases. But I don't think those cases are the least likely to occur. We might be better off choosing some extremly rare character for a command line argument. Perhaps a control character like <BackSpace> 0x08.

I believe we can't ever have a "perfect" solution involving FOR.

The only other way I know to capture the echoed command line in a file involves CALL, and that is no good because it can corrupt the command line and it isn't safe.

So it seems we are at a dead end. I don't see how we can ever have an absolutely foolproof method to capture any argument...


unless...

We could attempt to capture the complete set of arguments at least twice, using a different character for the FOR statement in each attempt. We could then compare the results and if we find discrepencies, we can use logic to try to reconstruct the full argument list. But it sounds like a complicated process. I don't think I want to try to attempt it. I'm not even sure if that could ever be foolproof.

Dave Benham

[Liviu]

Good idea, but it fails too
...
As now your code would fail with

Code: Select all

foolProofParam.bat %a...%~a...%~dpa

Thanks for pointing that. Think I have a workaround, see the "2nd EDIT" comments above.

What we need is a character that can be defined as a FOR variable, but can't be expanded. I've never come up with any. The closest we have is the one you showed me: %
...
So % has the fewest possible corner cases. But I don't think those cases are the least likely to occur. We might be better off choosing some extremly rare character for a command line argument. Perhaps a control character like <BackSpace> 0x08.

Agreed. I'll remember <BackSpace> as a fallback if the disableextensions workaround proves to be insufficient.

unless...

We could attempt to capture the complete set of arguments at least twice, using a different character for the FOR statement in each attempt. We could then compare the results and if we find discrepencies, we can use logic to try to reconstruct the full argument list. But it sounds like a complicated process. I don't think I want to try to attempt it. I'm not even sure if that could ever be foolproof.

Crossed my mind, too, but I wouldn't go there, either

Liviu

[dbenham]

Note that the enclosing (...) is still evaluated at the outer level with extensions enabled, which allows %* to work correctly, but the for loop executes with extensions disabled so %~a, %~dpa etc are no longer expanded.

disableExtensions - perfect

I always forget that option.

Dave Benham

[Liviu]

Now I understand why jeb had that # in his REM statement, to protect against /?

And only now do I understand why he had the space after # as well... REM appears to look for a '/?' all across the first argument (unless quoted) i.e. 'REM */?*' and 'REM a^&b%%c^(^!^<^|^>/^?...' both work just as 'REM /?'.

[jeb]

Now I understand why jeb had that # in his REM statement, to protect against /?

And only now do I understand why he had the space after # as well... REM appears to look for a '/?' all across the first argument (unless quoted) i.e. 'REM */?*' and 'REM a^&b%%c^(^!^<^|^>/^?...' both work just as 'REM /?'.

Ok, perhaps I should comment a bit more

Note that the enclosing (...) is still evaluated at the outer level with extensions enabled, which allows %* to work correctly, but the for loop executes with extensions disabled so %~a, %~dpa etc are no longer expanded.

I didn't understand why you disable extensions, as the REM never use delayed expansion, or better it's hard to see it.

The redirection is done to the output of the echo phase (before the delayed expansion phase).

Code: Select all

setlocal EnableDelayedExpansion
set quest=/?
rem !quest!
rem results to nothing

EDIT: Reading is helpful
I didn't realize that you used the word "EXTENSION" not "EXPANSION"

Now, I understand why you disable extensions , a really clever trick

[dbenham]

EDIT: Reading is helpful
I didn't realize that you used the word "EXTENSION" not "EXPANSION"

You would never catch me failing to read properly... noooooo, Not me

Dave Benham

[Liviu]

EDIT: Reading is helpful
I didn't realize that you used the word "EXTENSION" not "EXPANSION"

You would never catch me failing to read properly... noooooo, Not me

In fact, I am pretty sure I've noticed that during an unrelated search, and it must have somehow stuck "disableextensions" in my memory cache Had never used it before, or thought I'd (need to) ever use it, until this case here where it fit just right. Thanks to both of you, again, and to dostips contributors at large.

Liviu

[jeb]

In my opinion there are only two problems left.

We need a temporary file, it would be nicer if we could avoid this.

The REM technic currently fails with embedded linefeeds in the parameters %*.

I create a completly other way of capturing the parameters into a file

Code: Select all

@echo on
@echo Hello1 1>stream1.txt 2>stream2.txt 3>stream3.txt 4>stream4.txt
rem # %* #
@echo off
type stream3.txt > con
exit /b

This is adapted from the code of walid2me SO: Commenting multiple lines in DOS batch file

The only problem is to restore the standard handles
As after the code even the prompt isn't visible anymore

jeb

[Liviu]

In my opinion there are only two problems left.

We need a temporary file, it would be nicer if we could avoid this.

Right. The other unfortunate side effect of using a temp file is that characters outside the current code page cannot be handled (while a straightforward "set arg1=%1" works, barring of course poison characters in %1).

Code: Select all

C:\>eccoargs ‹αß©∂€›

---- copied from %? ----

args            { <αßc??> }
arg1            { <αßc??> }

---- chopped off %* ----

args    ~0,7    { <αßc??> }
        ~0,0    {  }
arg1    ~0,7    { <αßc??> }
        ~7,0    {  }

The REM technic currently fails with embedded linefeeds in the parameters %*.

True, and worth noting. Though in practice it only becomes a problem in rather uncommon cases.

I create a completly other way of capturing the parameters into a file
...
The only problem is to restore the standard handles
As after the code even the prompt isn't visible anymore

Oddly enough, piping the command "| more" brings back the prompt and the ">con" output. That said, I know (and found) virtually no information on handles 3 and 4, so I can't even begin to guess how cmd uses them internally.

Liviu

[dbenham]

Oddly enough, piping the command "| more" brings back the prompt and the ">con" output.

I can't figure out what you mean. What is the exact syntax in context?

Dave Benham

[Liviu]

What is the exact syntax in context?

Save Jeb's snippet as, say, stream3.cmd. Running it as such results in...

Code: Select all

C:\>stream3

C:\>rem #  #
cls
exit

...where the prompt remains in an odd input state. The last two lines were typed in, 'cls' did nothing, and 'exit' does in fact close the console window once Enter is pressed.

However, piping into "| more" returns to the cmd prompt normally.

Code: Select all

C:\>stream3 | more

C:\>rem #  #


C:\>

Liviu

[jeb]

...where the prompt remains in an odd input state. The last two lines were typed in, 'cls' did nothing, and 'exit' does in fact close the console window once Enter is pressed.

However, piping into "| more" returns to the cmd prompt normally.

But piping do the same as the exit, as the pipe itself creates two seperate sub-tasks for each side.
The prompt disappears in the child task which is never visible.

So the pipe doesn't solve the problem this way.

jeb

[Liviu]

But piping do the same as the exit, as the pipe itself creates two seperate sub-tasks for each side.
The prompt disappears in the child task which is never visible.

Haven't followed this in any depth, but it doesn't look quite the same. The pipe closes both ends without any obvious 'exit' being typed in and/or piped through.

So the pipe doesn't solve the problem this way.

Streams 3/4 redirection is foggy enough to me that I can't even spell out what the "problem" might be, let alone solve it

Liviu