Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
koko
Posts: 38
Joined: 13 Oct 2016 00:40

Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#1 Post by koko » 29 Sep 2019 11:05

After reaching a maximum setlocal recursion level in one of my scripts due to going over a certain number of inputs I decided to re-face the abyss that is batch escaping and getting past endlocal barriers to see if I could find or come up with some alternative method that fulfilled the following aims:

- Is self-contained and requires no temp files or secondary batch scripts
- Works with any input path/filename, regardless of special characters or Unicode.

After trying various things suggested here and elsewhere (SO, etc) it seemed that even if one part of what I tried was successful some other part made it fail (eg: an otherwise working escaped replacement suddenly behaving differently or not working at all when placed in a necessary for loop). Quite frustrating :!:

After racking my head for ideas (not helped by my limited proficiency of batch :)) I thought why not instead transport the length of each input past the endlocal barrier instead of the actual string itself, then trim the original argument? It seems that the trim function doesn't interfere with the original characters and so I tried it and to my delight it worked with any input filename I tried!

Now granted I'm not an uber-expert like some here but figured I'd share for thoughts and also in case it's useful for others in a similar situation. There may also be some better method that fulfills those aims that I just didn't find. In the version I made it assumes a cmd.exe path with the /c option and the batch script path in the arguments (ie: the script launched from File Explorer/Send To/Run). I read on SS64.org that these two things are omitted when launched directly from the command line which could be accounted for in with a conditional if/else but I didn't here.

---

Sample filenames to test with (also tested special characters in batch script path and in batch script name). See this post for additional test paths specifically with ampersands in no space paths.

Code: Select all

Example [! !§$%&()=`´'_;,.-#+´^ßöäüÖÄÜ°^^#].png
Example [עִבְרִית].png
Example [ファイル名の例].png
Example&Another&&.png
Update: the only character combination so far I've found it can't handle is &( if the path has no other spaces. A limitation of cmd.exe itself (see this post below regarding this). Other combinations of ampersands in no space paths I've tested work fine, including funny enough &).

---

Overview of scripts embedded below and in the following post:

- New-method.bat = the method which splits just the batch script path itself and the input arguments into separate variables
- New-method-with-sub-path-example.bat = same example but showing how sub-paths (eg: %%~pi, %%~ni) can be obtained as well.
- Standard-method-1.bat = typical method of obtaining batch script path and input arguments (using %1, etc). Fails with problematic special characters.
- Standard-method-2.bat = simple cmdcmdline method but not accounting for special characters. Setlocal DisableDelayedExpansion could be used but leads to recursion issues with enough inputs if not paired with an endlocal.

New-method.bat (commented):

Code: Select all

@echo off
setlocal enableextensions enabledelayedexpansion

rem New line variable (requires the two empty lines beneath)
set lf=^


set "count=0"
rem Batch script self path
for %%f in ("!cmdcmdline!") do (
    setlocal disabledelayedexpansion
    set scr="%~f0"
    setlocal enabledelayedexpansion
    call :len scr scrlen
    for /f "delims=" %%a in ("!scrlen!") do endlocal & endlocal & set "scrlen=%%a"
    )
rem Trim the cmd.exe path and /c (assumes launched from File Explorer/Send To/Run)
set "cmd=!cmdcmdline:~32!" & call set scr=%%cmd:~0,!scrlen!%%
rem Trim script path and trailing quote from arguments variable
for %%b in (!scrlen!) do set "args=!cmd:~%%b,-1!" & set "args=!args:* =!"

rem Account for input paths both with and without spaces 
set args="!args:"=!"
for /f "tokens=1 delims=:" %%d in (!args!) do (
    rem Split into new lines based on drive letter
    for %%l in ("!lf!") do (
        set args=!args:%%d:\=%%l%%d:\!
        )
    )
set "args=!args: "="!"
rem Remove empty first new line and leading double quote
set "args=!args:~2!"

for %%f in (!args!) do (
    set /a "count+=1"
    setlocal disabledelayedexpansion
    set infull="%%~f"
    setlocal enabledelayedexpansion
    rem Prepare length
    call :len infull infulllen
    rem Send past endlocal barrier
    for /f "delims=" %%a in ("!infulllen!") do (
        endlocal & endlocal
        for %%i in (!count!) do set "infulllen[%%i]=%%a"
        )
    for %%i in (!count!) do (
        rem Obtain each input's full path by trimming the full arguments variable using its length value
        for %%l in (!infulllen[%%i]!) do set "infull[%%i]=!args:~0,%%l!"
        rem Trim space left over from previous argument
        set "args=!args:~1!"
        rem Trim the same length from !args! after each input, for the next
        for %%l in (!infulllen[%%i]!) do set "args=!args:~%%l!"
        rem Remove left-over new line character
        set "infull[%%i]=!infull[%%i]:~1!"
        set "infull[%%i]=!infull[%%i]:"=!"
        set infull[%%i]="!infull[%%i]!"
        )
    )

rem Test output
echo Script path: !scr! & echo.
for /l %%i in (1,1,!count!) do (
    echo Input %%i -------------------------------- & echo.
    echo !infull[%%i]! & echo. & echo.
    )

pause
exit /b

:len
    set "s=!%~1!#"
    set "len=0"
    for %%p in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
        if "!s:~%%p,1!" neq "" ( 
            set /a "len+=%%p"
            set "s=!s:~%%p!"
        )
    )
    endlocal
    set "%~2=!len!"
    exit /b

endlocal
Last edited by koko on 06 Oct 2019 23:06, edited 16 times in total.

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#2 Post by koko » 29 Sep 2019 11:13

Getting 500 server error messages each time I either edit or submit a post. Anyone else getting these? Tried to edit the OP twice subsequent to the first edit and it failed.

Edit: perhaps in part due to some post character limit (though I also get server errors editing posts in general as well). Copied the remaining scripts to this post instead.

New-method-with-sub-path-example.bat (commented)

Code: Select all

@echo off
setlocal enableextensions enabledelayedexpansion

rem New line variable (requires the two empty lines beneath)
set lf=^


set "count=0"
rem Batch script self path
for %%f in ("!cmdcmdline!") do (
    setlocal disabledelayedexpansion
    set scr="%~f0"
    setlocal enabledelayedexpansion
    call :len scr scrlen
    for /f "delims=" %%a in ("!scrlen!") do endlocal & endlocal & set "scrlen=%%a"
    )
rem Trim the cmd.exe path and /c (assumes launched from File Explorer/Send To/Run)
set "cmd=!cmdcmdline:~32!" & call set scr=%%cmd:~0,!scrlen!%%
rem Trim script path and trailing quote from arguments variable
for %%b in (!scrlen!) do set "args=!cmd:~%%b,-1!" & set "args=!args:* =!"

rem Account for input paths both with and without spaces 
set args="!args:"=!"
for /f "tokens=1 delims=:" %%d in (!args!) do (
    rem Split into new lines based on drive letter
    for %%l in ("!lf!") do (
        set args=!args:%%d:\=%%l%%d:\!
        )
    )
set "args=!args: "="!"
rem Remove empty first new line and leading double quote
set "args=!args:~2!"

for %%f in (!args!) do (
    set /a "count+=1"
    setlocal disabledelayedexpansion
    set infull="%%~f"
    for %%i in ("%%~f") do (
        set "indr=%%~di"
        set "inpa=%%~pi"
        set "inna=%%~ni"
        set "inex=%%~xi"
        )
    setlocal enabledelayedexpansion
    rem Prepare lengths of individual strings
    call :len infull infulllen
    call :len indr indrlen
    call :len inpa inpalen
    call :len inna innalen
    call :len inex inexlen
    rem Concatenate values so it can be passed as one variable
    set "inlen-all=!infulllen! !indrlen! !inpalen! !innalen! !inexlen!"
    rem Sent past endlocal barrier
    for /f "delims=" %%a in ("!inlen-all!") do (
        endlocal & endlocal
        for %%i in (!count!) do set "inlen-all[%%i]=%%a"
        )
    for %%i in (!count!) do (
        rem Split and store the variables back to their matching originals
        for /f "tokens=1-5 delims= " %%a in ("!inlen-all[%%i]!") do (
            set "infulllen[%%i]=%%a"
            set "indrlen[%%i]=%%b"
            set "inpalen[%%i]=%%c"
            set "innalen[%%i]=%%d"
            set "inexlen[%%i]=%%e"
            )
        rem Obtain each input's full path by trimming the full arguments variable using its length value
        for %%l in (!infulllen[%%i]!) do set "infull[%%i]=!args:~0,%%l!"
        rem Trim space left over from previous argument
        set "args=!args:~1!"
        rem Trim the same length from !args! after each input, for the next
        for %%l in (!infulllen[%%i]!) do set "args=!args:~%%l!"
        rem Split each input path into the sub-path variables
        rem Remove left-over new line character
        set "infull[%%i]=!infull[%%i]:~1!"
        rem Remove surrounding quotes to match sub-path lengths
        set "infull[%%i]=!infull[%%i]:"=!"
        for %%d in (!indrlen[%%i]!) do (
            set "indr[%%i]=!infull[%%i]:~0,%%d!"
            for %%p in (!inpalen[%%i]!) do (
                set "inpa[%%i]=!infull[%%i]:~%%d,%%p!"
                )
            )
        for %%e in (!inexlen[%%i]!) do (
            set "inex[%%i]=!infull[%%i]:~-%%e!"
            set /a "innalen[%%i]=!innalen[%%i]!+%%e"
            for %%n in (!innalen[%%i]!) do (
                set "inna[%%i]=!infull[%%i]:~-%%n!"
                set "inna[%%i]=!inna[%%i]:~0,-%%e!"
                )
            )
        set infull[%%i]="!infull[%%i]!"
        )
    )

rem Test output
echo Script path: !scr! & echo.
for /l %%i in (1,1,!count!) do (
    echo Input %%i -------------------------------- & echo.
    echo !infull[%%i]! & echo.
    echo Drive:       !indr[%%i]!
    echo Sub-path:    !inpa[%%i]!
    echo Filename:    !inna[%%i]!
    echo Extension:   !inex[%%i]! & echo. & echo.
    )

pause
exit /b

:len
    set "s=!%~1!#"
    set "len=0"
    for %%p in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
        if "!s:~%%p,1!" neq "" ( 
            set /a "len+=%%p"
            set "s=!s:~%%p!"
        )
    )
    endlocal
    set "%~2=!len!"
    exit /b

endlocal
Standard-method-1.bat

Code: Select all

@echo off
setlocal enableextensions enabledelayedexpansion

set "count=0"
set scr="%~f0"

for %%f in (%*) do (
    set /a "count+=1"
    set infull[!count!]="%%~f"
    )

rem Test output
echo Script path: !scr! & echo.
for /l %%i in (1,1,!count!) do (
    echo Input %%i -------------------------------- & echo.
    echo !infull[%%i]! & echo. & echo.
    )

pause

endlocal
Standard-method-2.bat

Code: Select all

@echo off
setlocal enableextensions enabledelayedexpansion

set "count=0"
set scr="%~f0"
set "cmd=!cmdcmdline:*%~f0=!"
set "args=!cmd:~0,-1! "
set "args=!args:* =!"

for %%f in (!args!) do (
    set /a "count+=1"
    set infull[!count!]="%%~f"
    )

rem Test output
echo Script path: !scr! & echo.
for /l %%i in (1,1,!count!) do (
    echo Input %%i -------------------------------- & echo.
    echo !infull[%%i]! & echo. & echo.
    )

pause

endlocal
Last edited by koko on 06 Oct 2019 22:08, edited 6 times in total.

jfl
Posts: 226
Joined: 26 Oct 2012 06:40
Location: Saint Hilaire du Touvet, France
Contact:

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#3 Post by jfl » 01 Oct 2019 02:40

koko wrote:
29 Sep 2019 11:13
Getting 500 server error messages each time I either edit or submit a post. Anyone else getting these? Tried to edit the OP twice subsequent to the first edit and it failed.
Yes, I got these as well when I tried posting a message about my updated version of which.exe last week.
I retried several times, apparently unsuccessfully every time... Only to see several instances of the post suddenly appear three days later.
Apparently the dostips.com server is still seriously ill: If I click on the "Private messages" link at the top, I get a "Disk full" error message.

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#4 Post by koko » 01 Oct 2019 05:26

jfl wrote:
01 Oct 2019 02:40
Yes, I got these as well when I tried posting a message about my updated version of which.exe last week.
Looks like it's working without errors currently, so hopefully it's sorted :)

In the meantime realized I hadn't accounted for paths without spaces in the initial 'new method' scripts (which obviously changes whether the cmdcmdline variable wraps the path in double quotes and hence affects the length of the overall variable) so updated the scripts above to account for it.

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#5 Post by aGerman » 01 Oct 2019 09:39

I'll send an email to the forum admin. Maybe it needs his attention.

Steffen

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#6 Post by koko » 05 Oct 2019 07:20

Updated two lines to differently trim the args variable so it works properly for paths with ampersands and no spaces (a test I forgot to try until I was reminded of it by this topic).

Edit: ampersands and alphanumeric by themselves are fine however after testing a path with no spaces with !§$%&()=`´'_;,.-#+´^ßöäüÖÄÜ°^^# it exits entirely while with !§$%()=`´'_;,.-#+´^ßöäüÖÄÜ°^^# it splits the variable incorrectly. Will need to look more into this aspect, clearly. Would be nice to account for all cases (perhaps wishful thinking with a no spaces path but eh, wishful thinking brought me to the alternate solution of this topic in the first place :)).

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#7 Post by aGerman » 05 Oct 2019 08:43

perhaps wishful thinking
Yes, unfortunately. The problem you are facing is still caused by the ampersand bug. If you drag/drop a file with the name you provided to a script that consists of nothing but @pause, the cmd will still crash. You don't even have any chance to access the cmdcmdline variable.

Steffen

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#8 Post by koko » 05 Oct 2019 15:35

aGerman wrote:
05 Oct 2019 08:43
Yes, unfortunately. The problem you are facing is still caused by the ampersand bug. If you drag/drop a file with the name you provided to a script that consists of nothing but @pause, the cmd will still crash. You don't even have any chance to access the cmdcmdline variable.
Updated the scripts to account for the second problematic filename (the one that incorrectly split) by pre-wrapping all arguments in double quotes prior to loop rather than handling it within it. Nice thing about the path to the script itself is it always appears to be enclosed in double quotes even if in a path without spaces.

Found that &( is the deadly culprit. All other ampersand character combinations I tried without spaces worked, among them:

Code: Select all

C:\Users\koko\Desktop\&.png
C:\Users\koko\Desktop\&).png
C:\Users\koko\Desktop\%&.png
C:\Users\koko\Desktop\&%.png
C:\Users\koko\Desktop\&!.png
C:\Users\koko\Desktop\!&.png
C:\Users\koko\Desktop\!&%.png
C:\Users\koko\Desktop\$%&.png
C:\Users\koko\Desktop\$%&.png
C:\Users\koko\Desktop\!§$%&.png
C:\Users\koko\Desktop\Example&Another&&.png
C:\Users\koko\Desktop\!§$%()=`´'_;,.-#+´^ßöäüÖÄÜ°^^#.png
Presumably also other ampersand combinations but some automated test would need to be tried to be positive. I'm glad this method seems to work on every path I've tried except that deadly two character combination, although I'd imagine it's a more common two character combination that many other obscure ones I've been testing. Would certainly be interested if there were any other problematic paths it didn't handle correctly.

---

As a side note the new block of code added to pre-wrap all the arguments in double quotes required each set being on a new line. If I combined the lines from the second set onward in a single line using & to join them the double quote string manipulations behaved differently, even though the syntax used was set "var=foo". I'm guessing this is something due to the replacement of double quotes and batch becoming confused when on a single line.

So this was fine:

Code: Select all

set args="!args!"
set "args=!args: =//!"
set "args=!args:"//"=" "!"
set "args=!args://"=" "!"
set "args=!args:"//=" "!"
set "args=!args://= !"
set "args=!args:""="!"
But not this:

Code: Select all

set args="!args!"
set "args=!args: =//!" & set "args=!args:"//"=" "!" & set "args=!args://"=" "!" & set "args=!args:"//=" "!" & set "args=!args://= !" & set "args=!args:""="!"
If I had worked I would have combined them on the same line to save a little bit of vertical code space but better to have it functional than not :)

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#9 Post by aGerman » 05 Oct 2019 16:42

Found that &( is the deadly culprit.
Interesting! I'll keep that in mind.

There's still something wrong in you current version.
scrnsht.png
scrnsht.png (44.56 KiB) Viewed 26401 times
The first argument consists of several paths.

Steffen

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#10 Post by koko » 05 Oct 2019 17:06

aGerman wrote:
05 Oct 2019 16:42
Interesting! I'll keep that in mind.

There's still something wrong in you current version.

The first argument consists of several paths.
Oops, must have been I tested the double quote pre-wrapping manipulation with a certain selection of those inputs that didn't reveal this mistake (I've also found that depending on which was the actual file dragged and dropped onto a script, among a selection of highlighted files, changes the order of the arguments). Will look into what I can change of that section a bit later.

Thanks for bringing it to my attention.

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#11 Post by aGerman » 05 Oct 2019 17:16

(I've also found that depending on which was the actual file dragged and dropped onto a script, among a selection of highlighted files, changes the order of the arguments).
That's absolutely okay since it also changes the order in cmdcmdline. You can't do anything to prevent this rearrangement.

Steffen

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#12 Post by koko » 06 Oct 2019 04:09

aGerman wrote:
05 Oct 2019 16:42
There's still something wrong in you current version. The first argument consists of several paths.
Updated both scripts. It now divides the input paths into new lines within the single arguments variable, then wraps each line in double quotes regardless of whether the path has spaces or not.

The position the new lines are divided is determined by the drive letter colon delimiter, since it seems from what I've read that batch only supports drive letters for batch argument paths (non-mapped network paths for example aren't supported for arguments according to SS64.org unless they're temporarily mapped but that would be done within the script itself rather than being part of the cmdcmdline variable) so it seemed like the best way to determine each input regardless of paths with spaces/no spaces.

Let me know if you run into any other issues or if I've overlooked any different use cases with the input new line division concept.

pieh-ejdsch
Posts: 240
Joined: 04 Mar 2014 11:14
Location: germany

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#13 Post by pieh-ejdsch » 06 Oct 2019 05:39

Hi,

At that time I had simply forgotten an endlocal which I had now inserted.
Therefore, the pairs should always be turned on and off exactly.
Of course, here it goes.

Code: Select all

@echo off
@set prompt=$g$s
if . equ .!! setlocal disabledelayedexpansion

 rem Anwendung ohne Dateiname
set program= echo Verarbeite

::-------------------------------
call :setMacros program
set "drag="
setlocal enabledelayedexpansion
for /f usebackqeol^=^:tokens^=2*delims^=^" %%h in (
 '!cmdcmdline!'
) do ( endlocal
 if "%~0" equ "%%h" (
  set drag="%%i"
))

if not defined drag exit /b
set /a n=0
call :readDrag
>nul pause|set /p=%n% verarbeitet
exit

:readDrag
if . neq .!! setlocal enabledelayedexpansion
if !drag! equ "" exit /b 

for /f eol^=^:tokens^=1-2*delims^=^" %%h in (!drag!) do (
 endlocal
 set   drag="%%j"
 for /f "eol=:tokens=*" %%q in ("%%h") do (
  set "withoutQ=%%q"
  if defined withoutQ (
   setlocal enabledelayedexpansion
   for %%? in ("!withoutQ: =" "!") do ( if . equ .!! endlocal
    if %%? neq "" %ExpFile%
 )))
 if "%%i" neq ""  %ExpFile:?=i%
)
goto :readDrag

:setMacros
(set \n=^^^

)
set ExpFile=( ^<nul (if NOT ? == i (set/p"=NOT Quoted ") else set/p"= in Quotes  ")%\n%
 for /f %%n in ('"set/an+=1"') do ( set /an=%%n%\n%
  echo Nr:  %%n - "%%~?"%\n%
  )%\n%
  rem echo Drive:    %%~d?%\n%
  rem echo Path:     %%~p?%\n%
  rem echo Name:     %%~n?%\n%
  rem echo Ext:      %%~x?%\n%
)


aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#14 Post by aGerman » 06 Oct 2019 06:44

That seems to be working (both scripts).
@koko consider to add an EXIT or EXIT /B after PAUSE in order to protect you from running the :len routine even if you are actually already done.

Steffen

koko
Posts: 38
Joined: 13 Oct 2016 00:40

Re: Thoughts on this alternative method of obtaining cmdcmdline arguments (safe for all characters?)

#15 Post by koko » 06 Oct 2019 22:08

pieh-ejdsch wrote:
06 Oct 2019 05:39
Hi,

At that time I had simply forgotten an endlocal which I had now inserted.
Therefore, the pairs should always be turned on and off exactly.
Very interesting. Quite a bit over my head :) Would someone mind explaining in which part the variables could be set for use under enableddelayedexpansion elsewhere in such a script, outside of being echo'd in the call functions? As I tried echoing elsewhere various variables that appear to be set in the script but obviously I must be missing how this script is operating since they weren't defined. Or is the script intended to solely echo the paths within the call function in order to be captured by some separate script? (I think I've seen that suggested in an SO topic but I may be completely misunderstanding the script).
aGerman wrote:
06 Oct 2019 06:44
That seems to be working (both scripts).
@koko consider to add an EXIT or EXIT /B after PAUSE in order to protect you from running the :len routine even if you are actually already done.
Mmm. I have that set for my own script but didn't consider adding it for these test scripts. Will update the test scripts for good measure.

Post Reply