I suppose that !k doesn't see any "E" with a cyrillic font.
But perhaps you could be right, that shutdown always have an unlocaized part.
jeb (sleeping already)
new functions: :chr, :asc, :asciiMap
Moderator: DosItHelp
Re: new functions: :chr, :asc, :asciiMap
Good morning, jeb 
I couldn't figure out whether the E is language independent, but now I know that shutdown /? doesn't work in general.
The output on the XP machine at work:
There is no TAB at all
Regards
aGerman
I couldn't figure out whether the E is language independent, but now I know that shutdown /? doesn't work in general.
The output on the XP machine at work:
Code: Select all
Usage: shutdown [-i | -l | -s | -r | -a] [-f] [-m \\computername] [-t xx] [-c "comment"] [-d up:xx:yy]
No args Display this message (same as -?)
-i Display GUI interface, must be the first option
-l Log off (cannot be used with -m option)
-s Shutdown the computer
-r Shutdown and restart the computer
-a Abort a system shutdown
-m \\computername Remote computer to shutdown/restart/abort
-t xx Set timeout for shutdown to xx seconds
-c "comment" Shutdown comment (maximum of 127 characters)
-f Forces running applications to close without warning
-d [u][p]:xx:yy The reason code for the shutdown
u is the user code
p is a planned shutdown code
xx is the major reason code (positive integer less than 256)
yy is the minor reason code (positive integer less than 65536)
There is no TAB at all
Regards
aGerman
Re: new functions: :chr, :asc, :asciiMap
Well I finally did what should have been obvious, and downloaded a free Hex editor. (thanks Jeb)
It's much easier to investigate and work things out now.
I added two additional related functions:
:hex2str converts a string of hex digits into a string
(for example 414243 -> ABC)
:str2hex converts a string into a string of hex digits
(for example ABC -> 414243)
The code is at the end of this post.
Using Jeb's tips (with a few modifications) the functions now support all but the following two characters: NUL 0x00 and LF 0x0A.
Most of the added characters are embedded directly in the batch file. I reconfigured my editor to preserve Tabs (I use Context - it was configured to convert tabs into 2 spaces).
By default, only the characters that are embedded in the batch file are available. The optional /X switch adds support for CR 0x0D and SUB 0x1A by adding them programatically. I structured things this way because the /X option takes significantly longer to build the ASCII map, and variables with CR can only be accessed with delayed expansion. I don't want to penalize calls that don't need CR or SUB.
The /X option requires writing and reading a temporary file. I used the DOS Tips :Unique function plus %random% to generate a temporary file name that should prevent collisions even during concurrent use on a shared drive.
I would love to add support for LF 0x0A. I tried many variations of Jeb's for loop idea, but I could never get it to work when combined with the full ASCII map. I think if I restrict the code to :asc and :chr I could make things work with ugly code. But :hex2str really causes problems.
The problem I have is when trying to pass a string that requires delayed expansion across the function return boundry. I need a technique that will allow me to pass any combination of characters across the boundry. Usually the problem involves the for loop parsing the string into two or more strings such that the loop has multiple iterations. The technique I am currently using splits the string at each LF but works for all other characters.
Some notes on what I discovered:
1) CR passes through my variation of the function for loop return technique as long as it is not the last character in the string. If it is the last character it gets dropped. I solved this by simply appending an extra CR on the end if and only if the string ends in CR.
2) I ended up not using this, but I tested Jeb's code to generate a BS 0x08 and I found one small flaw that is easy to fix. Jeb's current DEL variable actually has length 3. It consists of <BS><Space><BS> (hex 08 20 08). A simple substring operation trims it down to the desired single character.
The prompt $H works the way it does because there is no guarantee there will be a subsequent character after the $H. The first <BS> moves the screen cursor back one position, but does not erase the character there. The <Space> "erases" the character but moves the cursor forward, and the final <BS> moves the cursor back again.
-----
And now the revised code. As before, the test cases are at the top. There are two tests. First loop through all valid ASCII codes and convert to char and back to code using :chr and :asc. The end code should match the original code. The second test builds the ASCII map, converts the entire ASCII map to a hex string and then back to a map string using :asciiMap, :str2hex and :hex2str. The starting map should match the ending map. The tests are run both with and without the /X option. I won't include the output because it is so long, but it does give the expected results on my Vista machine.
The /X option would be much faster if the ASCII map were preserved as a global variable, but I want each function to stand on its own without depending on global variables (in other words no side effects).
WARNING - I believe the web site is corrupting the ASCII map in the code block. The 10th character after the = should be a TAB but I think it is getting converted into 4 spaces. Below is the actual line (hopefully intact)
set %~1= !^"#$%%^&'^(^)*+,-./0123456789:;^<=^>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`abcdefghijklmnopqrstuvwxyz{^|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ
It's much easier to investigate and work things out now.
I added two additional related functions:
:hex2str converts a string of hex digits into a string
(for example 414243 -> ABC)
:str2hex converts a string into a string of hex digits
(for example ABC -> 414243)
The code is at the end of this post.
Using Jeb's tips (with a few modifications) the functions now support all but the following two characters: NUL 0x00 and LF 0x0A.
Most of the added characters are embedded directly in the batch file. I reconfigured my editor to preserve Tabs (I use Context - it was configured to convert tabs into 2 spaces).
By default, only the characters that are embedded in the batch file are available. The optional /X switch adds support for CR 0x0D and SUB 0x1A by adding them programatically. I structured things this way because the /X option takes significantly longer to build the ASCII map, and variables with CR can only be accessed with delayed expansion. I don't want to penalize calls that don't need CR or SUB.
The /X option requires writing and reading a temporary file. I used the DOS Tips :Unique function plus %random% to generate a temporary file name that should prevent collisions even during concurrent use on a shared drive.
I would love to add support for LF 0x0A. I tried many variations of Jeb's for loop idea, but I could never get it to work when combined with the full ASCII map. I think if I restrict the code to :asc and :chr I could make things work with ugly code. But :hex2str really causes problems.
The problem I have is when trying to pass a string that requires delayed expansion across the function return boundry. I need a technique that will allow me to pass any combination of characters across the boundry. Usually the problem involves the for loop parsing the string into two or more strings such that the loop has multiple iterations. The technique I am currently using splits the string at each LF but works for all other characters.
Some notes on what I discovered:
1) CR passes through my variation of the function for loop return technique as long as it is not the last character in the string. If it is the last character it gets dropped. I solved this by simply appending an extra CR on the end if and only if the string ends in CR.
2) I ended up not using this, but I tested Jeb's code to generate a BS 0x08 and I found one small flaw that is easy to fix. Jeb's current DEL variable actually has length 3. It consists of <BS><Space><BS> (hex 08 20 08). A simple substring operation trims it down to the desired single character.
Code: Select all
:BL.String.CreateDEL_ESC
:: Creates two variables with one character DEL=Ascii-08 and ESC=Ascii-27
:: DEL and ESC can be used with and without DelayedExpansion
setlocal
for /F "tokens=1,2 delims=#" %%a in ('"prompt #$H#$E# & echo on & for %%b in (1) do rem"') do (
ENDLOCAL
set "DEL=%%a"
set "DEL=%DEL:~0,1%
set "ESC=%%b"
goto :EOF
)
goto :eof
The prompt $H works the way it does because there is no guarantee there will be a subsequent character after the $H. The first <BS> moves the screen cursor back one position, but does not erase the character there. The <Space> "erases" the character but moves the cursor forward, and the final <BS> moves the cursor back again.
-----
And now the revised code. As before, the test cases are at the top. There are two tests. First loop through all valid ASCII codes and convert to char and back to code using :chr and :asc. The end code should match the original code. The second test builds the ASCII map, converts the entire ASCII map to a hex string and then back to a map string using :asciiMap, :str2hex and :hex2str. The starting map should match the ending map. The tests are run both with and without the /X option. I won't include the output because it is so long, but it does give the expected results on my Vista machine.
The /X option would be much faster if the ASCII map were preserved as a global variable, but I want each function to stand on its own without depending on global variables (in other words no side effects).
WARNING - I believe the web site is corrupting the ASCII map in the code block. The 10th character after the = should be a TAB but I think it is getting converted into 4 spaces. Below is the actual line (hopefully intact)
set %~1= !^"#$%%^&'^(^)*+,-./0123456789:;^<=^>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`abcdefghijklmnopqrstuvwxyz{^|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ
Code: Select all
@echo off
:top
setlocal disableDelayedExpansion
for /l %%n in (0,1,255) do (
call :chr %1 %%n char
if not errorlevel 1 (
call :asc %1 char 0 n
call echo "%%n:%%char%%:%%n%%"
) else echo chr %%n produced above error
)
echo:
call :asciiMap %1 map
call :strLen map len
set map
echo len=%len%
echo:
call :str2hex %1 map hex
set hex
echo:
call :hex2str %1 hex str
set str
echo:
setlocal enableDelayedExpansion
if "!map!"=="!str!" (echo map matches str) else (echo map does not match str)
if "%1"=="" call :top /X
exit /b
:str2hex [/X] StrVar [RtnVar]
::
:: Converts the string contained within variable StrVar into a string of
:: ASCII codes, with each code represented as a pair of hexadecimal digits.
:: The length of the result will always be exactly twice the length of
:: the original string.
::
:: Sets RtnVar=result
:: or displays result if RtnVar not specified
::
:: If one of the following problematic characters appears within StrVar
:: then the corresponding hex pair is set to 00 and errorlevel is set to 1.
::
:: C H A R A C T E R S U P P O R T E D ?
:: Dec Hex Oct Char Normal /X Option
:: --- ---- ---- ---- ------ ---------
:: 0 0x00 00 NUL (null) No No
:: 10 0x0A 012 LF (line feed) No No
:: 13 0x0D 015 CR (carriage return) No Yes
:: 26 0x1A 032 SUB (substitute) No Yes
::
:: If the case insensitive /X option is specified then CR (0x0D) and
:: SUB (0x1A) are supported properly as input.
::
:: Note: The /X option requires writing and reading a small temporary file.
:::
::: Dependencies - :asciiMap, :strLen, :Unique
:::
setlocal disableDelayedExpansion
if /i "%~1"=="/X" (
set option=%1
shift /1
) else set option=
call :asciiMap %option% asciiMap
set "hexMap=0123456789ABCDEF"
call :strLen %~1 len
set /a len-=1
set rtn=
set err=
setlocal enableDelayedExpansion
for /l %%n in (0,1,%len%) do (
set c=!%~1:~%%n,1!
set d=0
for /l %%n in (0,1,255) do if "!asciiMap:~%%n,1!"=="!c!" set d=%%n
set /a h1=d/16, h2=d%%16
for %%a in (!h1!) do for %%b in (!h2!) do set rtn=!rtn!!hexMap:~%%a,1!!hexMap:~%%b,1!
)
( endlocal
endlocal
if "%~2" neq "" (set %~2=%rtn%) else echo:%rtn%
exit /b %err%
)
exit /b
:hex2str [/X] HexVar RtnVar
::
:: Converts a string of hexadecimal digits contained within variable HexVar
:: into a string, where each pair of hex digits in the input represents the
:: ASCII code of a character in the result.
::
:: Stores the result in the variable RtnVar.
::
:: If one of the following problematic characters is specified within the Hex
:: string then a space is substituted and errorlevel is set to 1:
::
:: C H A R A C T E R S U P P O R T E D ?
:: Dec Hex Oct Char Normal /X Option
:: --- ---- ---- ---- ------ ---------
:: 0 0x00 00 NUL (null) No No
:: 10 0x0A 012 LF (line feed) No No
:: 13 0x0D 015 CR (carriage return) No Yes
:: 26 0x1A 032 SUB (substitute) No Yes
::
:: If the case insensitive /X option is specified then CR (0x0D) and
:: SUB (0x1A) may appear in the output. If the result contains CR (0x0D)
:: then RtnVar should only be accessed via delayed expansion.
::
:: Note: The /X option requires writing and reading a small temporary file.
::
:: If an invalid hexadecimal digit is detected within the Hex string then the
:: corresponding result character is set to a space and errorlevel is set
:: to 2.
::
:: Aborts with an error message to stderr and errorlevel 3 if the Hex string
:: length is not divisible by 2.
:::
::: Dependencies - :asciiMap, :strLen, :Unique
:::
setlocal disableDelayedExpansion
if /i "%~1"=="/X" (
set option=%1
shift /1
) else set option=
call :asciiMap %option% map
call :strLen %1 len
set /a mod=len%%2
if %mod%==1 1>&2 echo "ERROR: Hex string length not a multiple of 2" & exit /b 3
set rtn=
set /a len-=1
setlocal enableDelayedExpansion
set err=0
for /l %%n in (0,2,%len%) do (
2>nul set /a d=0x!%~1:~%%n,2! || (set d=32&set err=2)
for %%d in (!d!) do set c=^!map:~%%d,1!
if "!c!"==" " if not !d!==32 if !err!==0 set err=1
set "rtn=!rtn!!c!"
)
if defined option if "!rtn:~-1!"=="!map:~13,1!" set "rtn=!rtn!!map:~13,1!"
for /f "delims=" %%s in ("!rtn!") do (
endlocal
endlocal
if "%~2" neq "" set %~2=%%s
exit /b %err%
)
exit /b
:asc [/X] StrVar IntVal [RtnVar]
::
:: Computes the ASCII code for a specified character within the string
:: contained by variable StrVar. The position within the string is specified
:: by the IntVal argument. A non-negative value is relative to the beginning
:: of the string, with 0 specifiying the first character. A negative value is
:: relative to the end of the string, with -1 specifying the last character.
::
:: Sets RtnVar=result
:: or displays result if RtnVar not specified
::
:: IntVal may be passed as a variable without enclosing the name in percent
:: symbols.
::
:: If one of the following problematic characters is specified then RtnVar
:: will be undefined and errorlevel will be set to 1.
::
:: C H A R A C T E R S U P P O R T E D ?
:: Dec Hex Oct Char Normal /X Option
:: --- ---- ---- ---- ------ ---------
:: 0 0x00 00 NUL (null) No No
:: 10 0x0A 012 LF (line feed) No No
:: 13 0x0D 015 CR (carriage return) No Yes
:: 26 0x1A 032 SUB (substitute) No Yes
::
:: If the case insensitive /X option is specified then CR (0x0D) and
:: SUB (0x1A) may successfully be specified as input.
::
:: Note: The /X option requires writing and reading a small temporary file.
::
:: If StrVar is not defined then aborts with an error message to stderr and
:: errorlevel 2.
::
:: If IntVal is greater than or equal to the length of the string then aborts
:: with an error message to stderr and errorlevel 3.
::
:: Negative IntVal values will never result in errorlevel 2: Positions earlier
:: than the 1st character are treated as the 1st character.
:::
::: Dependencies - :asciiMap, :Unique
:::
setlocal disableDelayedExpansion
if /i "%~1"=="/X" (
set option=%1
shift /1
) else set option=
set /a n=%~2 2>nul
if errorlevel 1 1>&2 echo "ERROR: Invalid numeric value"&exit /b 11
if not defined %~1 1>&2 echo "ERROR: Variable not defined"&exit /b 2
call :asciiMap %option% ascii
setlocal enableDelayedExpansion
set "chr=!%~1:~%n%,1!"
if not defined chr 1>&2 echo "ERROR: String position not found"&exit /b 3
if "!chr!"==" " (set /a rtn=32) else (
if defined rtn set rtn=
for /l %%n in (0,1,255) do if "!ascii:~%%n,1!"=="!chr!" set rtn=%%n
)
if defined rtn (set err=0) else set err=1
(endlocal & rem -- return values
endlocal
if "%~3" neq "" (set %~3=%rtn%) else (echo:%rtn%)
exit /b %err%
)
exit /b
:chr [/X] IntVal [RtnVar]
::
:: Converts ASCII code IntVal into the corresponding character.
::
:: Sets RtnVar=result
:: or displays result if RtnVar not specified
::
:: IntVal must be a value between 0 and 255.
::
:: Aborts with an error message to stderr and errorlevel 11 if IntVal is not
:: a valid ASCII code.
::
:: Aborts with an error message to stderr and errorlevel 1 if IntVal
:: corresponds to one of the following problematic characters:
::
:: C H A R A C T E R S U P P O R T E D ?
:: Dec Hex Oct Char Normal /X Option
:: --- ---- ---- ---- ------ ---------
:: 0 0x00 00 NUL (null) No No
:: 10 0x0A 012 LF (line feed) No No
:: 13 0x0D 015 CR (carriage return) No Yes
:: 26 0x1A 032 SUB (substitute) No Yes
::
:: If the case insensitive /X option is specified then CR (0x0D) and
:: SUB (0x1A) are supported. If the result is CR (0x0D) then RtnVar should
:: only be accessed via delayed expansion.
::
:: Note: The /X option requires writing and reading a small temporary file.
::
:: IntVal may be passed as a variable without enclosing the name in percent
:: symbols.
:::
::: Dependencies - :asciiMap, :Unique
:::
setlocal disableDelayedExpansion
if /i "%~1"=="/X" (
set option=%1
shift /1
) else set option=
set /a n=%~1 2>nul
if errorlevel 1 1>&2 echo "ERROR: Invalid ASCII Code"&exit /b 11
if %n%==32 set "c= "&setlocal enableDelayedExpansion&goto :chr.end
if %n% lss 0 1>&2 echo "ERROR: Invalid ASCII Code"&exit /b 11
if %n% gtr 255 1>&2 echo "ERROR: Invalid ASCII Code"&exit /b 11
call :asciiMap %option% ascii
setlocal EnableDelayedExpansion
set "c=!ascii:~%n%,1!"
if "!c!"==" " (
1>&2 echo "ERROR: Problematic ASCII Code"&exit /b 1
)
if %n%==13 set "c=!c!!c!"
:chr.end
for /f "delims=" %%c in ("!c!") do (
endlocal
endlocal
if "%~2" neq "" (set %~2=%%c) else (echo:%%c)
)
exit /b
:asciiMap [/X] rtnVar
::
:: Sets variable rtnVar to a 256 character string containing the complete
:: extended ASCII character set except a space has been substituted for each
:: of the following problematic characters:
::
:: C H A R A C T E R S U P P O R T E D ?
:: Dec Hex Oct Char Normal /X Option
:: --- ---- ---- ---- ------ ---------
:: 0 0x00 00 NUL (null) No No
:: 10 0x0A 012 LF (line feed) No No
:: 13 0x0D 015 CR (carriage return) No Yes
:: 26 0x1A 032 SUB (substitute) No Yes
::
:: If the case insensitive /X option is specified then CR (0x0D) and
:: SUB (0x1A) characters are included in the map. However, a map with
:: these characters can only be accessed via delayed expansion.
::
:: Note: The /X option requires writing and reading a small temporary file.
:::
::: Dependencies - :Unique
:::
setlocal disableDelayedExpansion
if /i "%~1"=="/X" shift /1 & goto :asciiMap.extend
(endlocal
call :asciiMap.setMap %~1
exit /b
)
:asciiMap.extend
call :Unique file
if defined temp (set filePath=%temp%) else if defined tmp (set filePath=%tmp%) else set filePath=.
set file="%filePath%\_asciiMap_%file%_%random%.tmp"
for /f %%a in ('copy /Z "%~dpf0" nul') do set "cr=%%a"
copy /a nul+nul %file% > nul
for /f "usebackq" %%a in (%file%) do set "sub=%%a"
del %file%
call :asciiMap.setMap map
setlocal enableDelayedExpansion
set "map=!map:~0,13!!cr!!map:~14,12!!sub!!map:~27!"
for /f "delims=" %%a in ("!map!") do (
endlocal
endlocal
set %~1=%%a
exit /b
)
:asciiMap.setMap
:: WARNING - THE 10th character after the = should be a single TAB character. I believe the web site is converting the TAB into 4 spaces
set %~1= !^"#$%%^&'^(^)*+,-./0123456789:;^<=^>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^^_`abcdefghijklmnopqrstuvwxyz{^|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ
exit /b
::-----------------------------------------------------------------------------
:: The following are existing dostips functions
::-----------------------------------------------------------------------------
:strLen string len -- returns the length of a string
:: -- string [in] - variable name containing the string being measured for length
:: -- len [out] - variable to be used to return the string length
:: Many thanks to 'sowgtsoi', but also 'jeb' and 'amel27' dostips forum users helped making this short and efficient
:$created 20081122 :$changed 20101116 :$categories StringOperation
:$source http://www.dostips.com
( SETLOCAL ENABLEDELAYEDEXPANSION
set "str=A!%~1!"&rem keep the A up front to ensure we get the length and not the upper bound
rem it also avoids trouble in case of empty string
set "len=0"
for /L %%A in (12,-1,0) do (
set /a "len|=1<<%%A"
for %%B in (!len!) do if "!str:~%%B,1!"=="" set /a "len&=~1<<%%A"
)
)
( ENDLOCAL & REM RETURN VALUES
IF "%~2" NEQ "" SET /a %~2=%len%
)
EXIT /b
:Unique ret -- returns a unique string based on a date-time-stamp, YYYYMMDDhhmmsscc
:: -- ret [out,opt] - unique string
:$created 20060101 :$changed 20080219 :$categories StringOperation,DateAndTime
:$source http://www.dostips.com
SETLOCAL
for /f "skip=1 tokens=2-4 delims=(-)" %%a in ('"echo.|date"') do (
for /f "tokens=1-3 delims=/.- " %%A in ("%date:* =%") do (
set %%a=%%A&set %%b=%%B&set %%c=%%C))
set /a "yy=10000%yy% %%10000,mm=100%mm% %% 100,dd=100%dd% %% 100"
for /f "tokens=1-4 delims=:. " %%A in ("%time: =0%") do @set UNIQUE=%yy%%mm%%dd%%%A%%B%%C%%D
ENDLOCAL & IF "%~1" NEQ "" (SET %~1=%UNIQUE%) ELSE echo.%UNIQUE%
EXIT /b
Re: new functions: :chr, :asc, :asciiMap
Correction - the code block is converting the TAB into 3 spaces. The TAB in the map outside the code block is correct. But the 161st character after the = is supposed to be ASCII decimal 160, but it has been corrupted into a space in both places.
Dave
Dave
Re: new functions: :chr, :asc, :asciiMap
Hi dbenham,
look nice now, but some comments...
the NUL character seems to be impossible to handle with batch, as I suppose cmd.exe works internally with null-terminated strings.
So you can't store a NUL in a variable.
The character searching in :asc or in :str2hex could be enhanced by a map (similar to :strLen)
You only have to care about case sensitive.
The <LF> and exclams should work with a obvious escape sequence
jeb
look nice now, but some comments...
the NUL character seems to be impossible to handle with batch, as I suppose cmd.exe works internally with null-terminated strings.
So you can't store a NUL in a variable.
The character searching in :asc or in :str2hex could be enhanced by a map (similar to :strLen)
Code: Select all
set "ch=a"
set "ascMap=000#101#202#A41#B42"
set "split=!ascMap:*%ch%=!"
set /a "hexVal=0x!split:~0,2!"
You only have to care about case sensitive.
The <LF> and exclams should work with a obvious escape sequence
Code: Select all
@echo off
setlocal EnableDelayedExpansion
call :lfTest result
echo the result is:
echo !result!-- The end
goto :eof
:lfTest
setlocal
set "NotDelayedFlag=!"
setlocal EnableDelayedExpansion
set LF=^
rem TWO Empty lines are neccessary
rem Test result
set "var=one&line!LF!two with exclam^! !LF!three with "quotes^&""
rem ** Prepare for return
if not defined NotDelayedFlag (
for %%a in ("!LF!") do set "var=!var:%%~a=""L!"
set "var=!var:^=^^^^!"
set "var=!var:"=""Q!"
)
if not defined NotDelayedFlag (
set "var=%var:!=^^^^^!%" !
set "var=!var:""Q="!"
for %%a in ("!LF!") do set "var=!var:""L=%%~a!"
)
set ^"var=!var:"=^"!"
set "var=!var:&=^&!"
for %%a in ("!LF!") do set "var=!var:%%~a=^%%~a%%~a!"
(
ENDLOCAL
ENDLOCAL
set ^"%~1=%var%" !
goto :eof
)
jeb
Re: new functions: :chr, :asc, :asciiMap
The character searching in :asc or in :str2hex could be enhanced by a map (similar to :strLen)Code: Select all
set "ch=a"
set "ascMap=000#101#202#A41#B42"
set "split=!ascMap:*%ch%=!"
set /a "hexVal=0x!split:~0,2!"
Great idea Jeb.
I had hoped to use one map that would convert biderectionally, but I think speed improvements using your strategy are worth the added effort of maintaining two maps.
I don't understand how your strategy is similar to :strLen.
In my earlier post I was referring to a failed strategy outlined below that was similar to strLen
(pseudo-code):
Code: Select all
build my original map length 256
set rtn=0
For each 1 bit in a byte going from high to low (
XOR the bit with rtn
if CHR < %map:~rtn,1% then remove the bit (XOR the bit with rtn again)
)
Upon completion of the loop, rtn should be the ASCII code for CHR. The loop has only 8 iterations, so it would be very fast. But the algorithm FAILS because the characters do not sort in ASCII code sequence.
=================
Regarding the 2nd part of your reply:
Close, but I still see problems. I added support for ^ | < > ( ) that was missing in your original. But there are still problems if input contains CR. Also, there is a problem with calling while using delayed expansion if input contains ""L.
Code: Select all
@echo off
setlocal DisableDelayedExpansion
call :lfTest result
setlocal EnableDelayedExpansion
echo The result is:
echo !result!
call :lfTest result
echo The result is:
echo !result!
goto :eof
:lfTest
setlocal
set "NotDelayedFlag=!"
echo:
if defined NotDelayedFlag (echo lfTest was called with Delayed Expansion DISABLED) else echo lfTest was called with Delayed Expansion ENABLED
setlocal EnableDelayedExpansion
for /f %%a in ('copy /Z "%~dpf0" nul') do set "CR=%%a"
set LF=^
rem TWO Empty lines are neccessary
rem Test result
set "var=one&line!LF!two with exclam^! !LF!three with "quotes^&"&"!LF!four with ^^^^ ^| ^< ^> ( ) ^& ^^^! ^"!LF!xxxxxwith CR!CR!five !LF!six with ^"^"Q ^"^"L still six "
echo the input is:
echo !var!
echo:
rem ** Prepare for return
set "var=!var:^=^^!"
if defined NotDelayedFlag goto :NotDelayed
for %%a in ("!LF!") do set "var=!var:%%~a=""L!"
set "var=!var:^=^^^^!"
set "var=!var:"=""Q!"
set "var=%var:!=^^^^^!%" !
set "var=!var:""Q="!"
for %%a in ("!LF!") do set "var=!var:""L=%%~a!"
:NotDelayed
set ^"var=!var:"=^"!"
set "var=!var:&=^&!"
set "var=!var:|=^|!"
set "var=!var:<=^<!"
set "var=!var:>=^>!"
set "var=!var:(=^(!"
set "var=!var:)=^)!"
for %%a in ("!LF!") do set "var=!var:%%~a=^%%~a%%~a!"
for %%a in ("!CR!") do set "var=!var:%%~a=^%%~a%%~a!"
(
ENDLOCAL
ENDLOCAL
set ^"%~1=%var%" !
goto :eof
)
Here is the output from above
Code: Select all
lfTest was called with Delayed Expansion DISABLED
the input is:
one&line
two with exclam!
three with "quotes&"&"
four with ^ | < > ( ) & ! "
five with CR
six with ""Q ""L still six
The result is:
one&line
two with exclam!
three with "quotes&"&"
four with ^ | < > ( ) & ! "
xxxxxwith CRfive
six with ""Q ""L still six
lfTest was called with Delayed Expansion ENABLED
the input is:
one&line
two with exclam!
three with "quotes&"&"
four with ^ | < > ( ) & ! "
five with CR
six with ""Q ""L still six
The result is:
one&line
two with exclam!
three with "quotes&"&"
four with ^ | < > ( ) & ! "
xxxxxwith CRfive
six with ""Q
still six
So it looks like the functions can support CR or LF, but not both.
Unless you have any other ideas?I had already given up on calling functions while delayed expansion is enabled. But it looks like you have discovered a "universal" technique that works as long as returned value does not contain CR or LF! Awsome!
Dave
Re: new functions: :chr, :asc, :asciiMap
I have fixed two bugs in my last posted code- :chr and :hex2str both failed with ; (0x59) due to the default FOR /F "eol=;" option. Unfortunately eol= option cannot be completely disabled like "delims=", there is always an eol character. (The description of eol within "help for" or "for /?" is terrible!)
Fixed :hex2str at end of function:
Fixed :chr at end of function
The bug was masked, but not completely hidden, by a poor choice in variable naming in the test case.
improved test case at top:
I'm still working on Jeb's idea for speeding up :str2hex and :asc. I'm close, but still some things to overcome. So far there is some improved performance, but I'm not sure yet if it's worth the effort.
Dave
Fixed :hex2str at end of function:
Code: Select all
if "!rtn:~0,1!"==";" (set "eol=eol= ") else set "eol="
for /f "%eol%delims=" %%s in ("!rtn!") do (
endlocal
endlocal
if "%~2" neq "" set %~2=%%s
exit /b %err%
)
exit /b
Fixed :chr at end of function
Code: Select all
if "!c!"==";" (set "eol=eol= ") else set "eol="
for /f "%eol%delims=" %%c in ("!c!") do (
endlocal
endlocal
if "%~2" neq "" (set %~2=%%c) else (echo:%%c)
)
exit /b
The bug was masked, but not completely hidden, by a poor choice in variable naming in the test case.
improved test case at top:
Code: Select all
for /l %%n in (0,1,255) do (
call :chr %1 %%n char
if not errorlevel 1 (
call :asc %1 char 0 rtn
call echo "%%n:%%char%%:%%rtn%%"
) else echo chr %%n produced above error
)
I'm still working on Jeb's idea for speeding up :str2hex and :asc. I'm close, but still some things to overcome. So far there is some improved performance, but I'm not sure yet if it's worth the effort.
Dave
Re: new functions: :chr, :asc, :asciiMap
dbenham wrote:Close, but I still see problems. I added support for ^ | < > ( ) that was missing in your original. But there are still problems if input contains CR. Also, there is a problem with calling while using delayed expansion if input contains ""L.
Ok, I was a bit imprecise. The ""L problem can be solved by the correct order of the commands.
And the support for | < > ( ) is equal to &.
But I didn't look at this way anymore, as I think it comes to a dead end.
dbenham wrote:So it looks like the functions can support CR or LF, but not both.
I had already given up on calling functions while delayed expansion is enabled. But it looks like you have discovered a "universal" technique that works as long as returned value does not contain CR or LF! Awsome!
It was a bit tricky, but I created a slightly other solution.
It can handle CR and LF and all the other characters
And it's shorter and doesn't need to handle the & | < > characters.
Code: Select all
@echo off
setlocal EnableDelayedExpansion
cls
for /f %%a in ('copy /Z "%~dpf0" nul') do set "CR=%%a"
set LF=^
rem TWO Empty lines are neccessary
set "original=zero*? %%~A%%~B%%~C%%~L!LF!one&line!LF!two with exclam^! !LF!three with "quotes^&"&"!LF!four with ^^^^ ^| ^< ^> ( ) ^& ^^^! ^"!LF!xxxxxwith CR!CR!five !LF!six with ^"^"Q ^"^"L still six "
setlocal DisableDelayedExpansion
call :lfTest result original
setlocal EnableDelayedExpansion
echo The result with disabled delayed expansion is:
if !original! == !result! (echo OK) ELSE echo !result!
call :lfTest result original
echo The result with enabled delayed expansion is:
if !original! == !result! (echo OK) ELSE echo !result!
echo ------------------
echo !original!
goto :eof
::::::::::::::::::::
:lfTest
setlocal
set "NotDelayedFlag=!"
echo(
if defined NotDelayedFlag (echo lfTest was called with Delayed Expansion DISABLED) else echo lfTest was called with Delayed Expansion ENABLED
setlocal EnableDelayedExpansion
set "var=!%~2!"
rem echo the input is:
rem echo !var!
echo(
rem ** Prepare for return
set "var=!var:%%=%%~A!"
set "var=!var:"=%%~B!"
for %%a in ("!LF!") do set "var=!var:%%~a=%%~L!"
for %%a in ("!CR!") do set "var=!var:%%~a=%%~C!"
rem ** It is neccessary to use two IF's else the %var% expansion doesn't work as expected
if not defined NotDelayedFlag set "var=!var:^=^^^^!"
if not defined NotDelayedFlag set "var=%var:!=^^^!%" !
set "replace=%% """ !CR!!CR!"
for %%L in ("!LF!") do (
for /F "tokens=1,2,3" %%A in ("!replace!") DO (
ENDLOCAL
ENDLOCAL
set "%~1=%var%" !
@echo off
goto :eof
)
)
It replaces
% with %~A
" with %~B
<CR> with %~C
<LF> with %~L
And in the return statement it conterts them back to the original values.
I need two FOR statements, as the FOR /F can't assign <LF> to a parameter, but a simple FOR %%a.
The % substituition to %~A is neccessary, else there could be content with something like "100%C" which would expand the wrong way.
And in the enabled delayed expansion mode, there are two more substituitions
^ with ^^
! with ^!
The caret doubling is neccessary, as in a line is one or more exclamation marks the ^ works a second time as an escape character, but now in and outside of quotes.
To assure that the doubled carets will always reduced to a single caret, I append an exlamation mark after the last quote in the set statements.
Hope this is the end ...
jeb
Re: new functions: :chr, :asc, :asciiMap
jeb wrote:To assure that the doubled carets will always reduced to a single caret, I append an exlamation mark after the last quote in the set statements.
That's not clear to me. How can an exclamation mark behind the last quote influence the variable content?
Regards
aGerman
Re: new functions: :chr, :asc, :asciiMap
The simple rule for delayed expansion is.
For each character in the line do:
- If it is a caret (^) the next character has no special meaning, the caret itself is removed
- If it is an exclamation mark, search for the next exclamation mark (carets are not observed here), then expands to the content of the variable
- If no exclamation mark is found in this phase, the result is discarded, the result of the phase before is used instead (important for the carets)
So, at this point the difference should be clear, the carets are removed even if the exclamation mark have no other effect in a line.
A sample
But these was only the simple rule.
There is a bit more to know about it.
- The line is split into two parts, the command-token and the rest of the line, for each part this phase is executed separatly
Sample for the "two part" rule
This uses a trick to append something to the first/commando token, but after the parsing phases, the echo splits and prints it.
jeb
For each character in the line do:
- If it is a caret (^) the next character has no special meaning, the caret itself is removed
- If it is an exclamation mark, search for the next exclamation mark (carets are not observed here), then expands to the content of the variable
- If no exclamation mark is found in this phase, the result is discarded, the result of the phase before is used instead (important for the carets)
So, at this point the difference should be clear, the carets are removed even if the exclamation mark have no other effect in a line.
A sample
Code: Select all
@echo off
setlocal EnableDelayedExpansion
echo one caret^^
echo none caret^^ !
set "var1=one caret^"
set "var2=none caret^" !
echo !var1!
echo !var2!
----- OUTPUT ----
one caret^
none caret
one caret^
one caret
But these was only the simple rule.
There is a bit more to know about it.
- The line is split into two parts, the command-token and the rest of the line, for each part this phase is executed separatly
Sample for the "two part" rule
Code: Select all
setlocal EnableDelayedExpansion
set "x=######"
echo:^^^^"^^^^" ^^^^"^^^^"
echo:^^^^"^^^^" ^^^^"^^^^"!x!
echo:^^^^"^^^^"!x! ^^^^"^^^^"
echo:^^^^"^^^^"!x! ^^^^"^^^^"!x!
--- Output ---
^^"^^^^" ^^"^^^^"
^^"^^^^" ^"^^"######
^"^^"###### ^^"^^^^"
^"^^"###### ^"^^"######This uses a trick to append something to the first/commando token, but after the parsing phases, the echo splits and prints it.
jeb
Re: new functions: :chr, :asc, :asciiMap
Thanks for your explanation. I always appreciate your comments because there's so much stuff I never saw before.
Regards
aGerman
Regards
aGerman
Re: new functions: :chr, :asc, :asciiMap
Oh my goodness Jeb! 
Your solution is insanely clever. I'm able to follow the substitutions, but I'll have to spend more time looking at your explanation to fully understand the games you play with !
Thankfully I don't have to fully understand it to use it.
I've finished incorporating your improved map lookup for asc and str2hex. It doesn't make much difference for :asc, but str2hex is now nearly 7 times faster! I solved the case issue by doubling each character after # and testing the 1st remaining character after substitution against the target character. If no match then I just perform one more substitution to get the correct case.
I've eliminated the clunky /X option and now create the special characters only as needed.
For :hex2str I added options to perform substitutions for NUL, CR, LF, and Errors. By default NUL is represented as <NUL>, LF as <LF>, invalid hex as <ERR>, and CR as itself. But the options allow setting each representation to any string you want.
The only thing that was missing was support for a true LF in :hex2str. I was all ready to post what I had but now you've provided the last missing piece.
I'll incorporate your solution and post the final result in the next few days.
Dave
Your solution is insanely clever. I'm able to follow the substitutions, but I'll have to spend more time looking at your explanation to fully understand the games you play with !
Thankfully I don't have to fully understand it to use it.
I've finished incorporating your improved map lookup for asc and str2hex. It doesn't make much difference for :asc, but str2hex is now nearly 7 times faster! I solved the case issue by doubling each character after # and testing the 1st remaining character after substitution against the target character. If no match then I just perform one more substitution to get the correct case.
I've eliminated the clunky /X option and now create the special characters only as needed.
For :hex2str I added options to perform substitutions for NUL, CR, LF, and Errors. By default NUL is represented as <NUL>, LF as <LF>, invalid hex as <ERR>, and CR as itself. But the options allow setting each representation to any string you want.
The only thing that was missing was support for a true LF in :hex2str. I was all ready to post what I had but now you've provided the last missing piece.
I'll incorporate your solution and post the final result in the next few days.
Dave
Re: new functions: :chr, :asc, :asciiMap
I think this project is now complete! (with the exception of bugs which are bound to crop up 
).
The library now supports 255 ASCII characters (only 00 NUL not supported). I've fully incorporated Jeb's new found technique allowing a function to return any string value, regardless whether the function was called with delayed expansion enabled or disabled.
I've turned the library into a complete, self-contained, callable library, complete with built-in help functionality. There is no need to copy the routines into your own batch file - you can simply call the library directly. Of course there is nothing stopping you from copying the routines if that is your preference.
To avoid corruption of the code that I experienced earlier, I've made the file available on a free Google web site. Download the CharLib_bat.txt file found here: https://sites.google.com/site/dbenhamfiles/home/CharLib_bat.txt
and rename it to CharLib.bat. Ideally the file should be in a directory that is in your path.
Run CharLib without any options to get basic syntax and other general info.
CharLib help will provide a list of available functions.
CharLib test will run a test suite that exercises most of the functionality.
One more time I have to thank Jeb for all his work and advice that enabled me to make these routines fully functional.
I hope others find this library useful. Please post a bug report here if you find problems. However I am not interested in adding additional functionality (except for maybe some form of versioning info in the case of bug fix updates)
Dave Benham
(with the exception of bugs which are bound to crop up ).The library now supports 255 ASCII characters (only 00 NUL not supported). I've fully incorporated Jeb's new found technique allowing a function to return any string value, regardless whether the function was called with delayed expansion enabled or disabled.
I've turned the library into a complete, self-contained, callable library, complete with built-in help functionality. There is no need to copy the routines into your own batch file - you can simply call the library directly. Of course there is nothing stopping you from copying the routines if that is your preference.
To avoid corruption of the code that I experienced earlier, I've made the file available on a free Google web site. Download the CharLib_bat.txt file found here: https://sites.google.com/site/dbenhamfiles/home/CharLib_bat.txt
and rename it to CharLib.bat. Ideally the file should be in a directory that is in your path.
Run CharLib without any options to get basic syntax and other general info.
CharLib help will provide a list of available functions.
CharLib test will run a test suite that exercises most of the functionality.
One more time I have to thank Jeb for all his work and advice that enabled me to make these routines fully functional.
I hope others find this library useful. Please post a bug report here if you find problems. However I am not interested in adding additional functionality (except for maybe some form of versioning info in the case of bug fix updates)
Dave Benham
Re: new functions: :chr, :asc, :asciiMap
GBK test:
Code: Select all
亗儎厗噲墛媽崕彁憭摂晼棙櫄洔潪煚Re: new functions: :chr, :asc, :asciiMap
Hi plp626. I'm assuming the functions failed miserably for you. There is an interesting discussion at "universal" %DATE% parser concerning multi-national byte representation of DOS strings. I think I have a handle on how to work with single byte character sets. But I have no idea if functions like :asc, :chr, :parseDate etc. can be made to work with multibyte character sets like GBK.
I would like to see if the functions can be made to work for you, but I will need your help. I don't have a machine on which I can test GBK. One quick question - how does DOS substring work with Chinese characters like your test string? In other words, if variable str contains your string, would %str:~0,1% return the 1st Chinese character consisting of two bytes?
Dave Benham
I would like to see if the functions can be made to work for you, but I will need your help. I don't have a machine on which I can test GBK. One quick question - how does DOS substring work with Chinese characters like your test string? In other words, if variable str contains your string, would %str:~0,1% return the 1st Chinese character consisting of two bytes?
Dave Benham