How to handle the exceptions to replace any character with any other character ?
Posted: 25 May 2024 21:19
Hi,
In the split function and getsubstringindex functions that I am writing
I need the fastest way to check if a character is present within a variable
It looks like this
Unfortunately this fails with some special characters
I think this explains why I couldn't find an all purpose pure batch split function in the wild :\
So I made a function to test all poison character substitution in the source or substitution string, to find which substitution don't work and how to create a workaround for all cases.
I identified 5 major ways this failes
without delayed expansion
When you try to replace something WITH a percentsign (works with enableelayedexpansion)
When you try to replace a percentsign with something else (works with enableelayedexpansion)
with delayed expansion
When you try to replace something WITH an exclamation (works with disabledelayedexpansion)
When you try to replace an exclamation with something else (works with disabledelayedexpansion)
and if you try to replace a doublequote with &<>^| then you must escape them (works with disabledelayedexpansion)
Problematic lines (test result and test code below)
Not working without delayed expansion
::replacing with a percentsign
::replaceing a percentsign
Not working with delayed expansion
:: replacing with !
:: replacing a !
set "_substvar11=!_testvar:!=!!"
set "_substvar12=!_testvar:!="!"
set "_substvar13=!_testvar:!=%%!"
set "_substvar14=!_testvar:!=&!"
set "_substvar15=!_testvar:!=(!"
set "_substvar16=!_testvar:!=)!"
set "_substvar17=!_testvar:!=<!"
set "_substvar18=!_testvar:!=>!"
set "_substvar19=!_testvar:!=^!"
set "_substvar20=!_testvar:!=|!"
replacing a doublequote
set "_substvar11=!_testvar:"=!!" <- doesn't work as usual
set "_substvar12=!_testvar:"="!" <- works fine
set "_substvar13=!_testvar:"=%%!" <- works fine
set "_substvar14=!_testvar:"=&!" --> escape with carret set "_substvar14=!_testvar:"=^&!"
set "_substvar15=!_testvar:"=(!" <- works fine
set "_substvar16=!_testvar:"=)!" <- works fine
set "_substvar17=!_testvar:"=<!" --> escape with carret set "_substvar17=!_testvar:"=^<!"
set "_substvar18=!_testvar:"=>!" --> escape with carret set "_substvar18=!_testvar:"=^>!"
set "_substvar19=!_testvar:"=^!" --> escape with carret set "_substvar19=!_testvar:"=^^!"
set "_substvar20=!_testvar:"=|!" --> escape with carret set "_substvar20=!_testvar:"=^|!"
Here is the output of the test script
In the split function and getsubstringindex functions that I am writing
I need the fastest way to check if a character is present within a variable
It looks like this
Code: Select all
set _SPLT_CurrentWindow=!%_SPLT_Input_Pointer%:~%_SPLT_Index%,%_SPLT_Actual_Search_Window_len%!
set _SPLT_CurrentWindowResult=!_SPLT_CurrentWindow:%_SPLT_Search%=!
if %_SPLT_CaseSensitivity% !_SPLT_CurrentWindow! NEQ !_SPLT_CurrentWindowResult!
I think this explains why I couldn't find an all purpose pure batch split function in the wild :\
So I made a function to test all poison character substitution in the source or substitution string, to find which substitution don't work and how to create a workaround for all cases.
I identified 5 major ways this failes
without delayed expansion
When you try to replace something WITH a percentsign (works with enableelayedexpansion)
When you try to replace a percentsign with something else (works with enableelayedexpansion)
with delayed expansion
When you try to replace something WITH an exclamation (works with disabledelayedexpansion)
When you try to replace an exclamation with something else (works with disabledelayedexpansion)
and if you try to replace a doublequote with &<>^| then you must escape them (works with disabledelayedexpansion)
Problematic lines (test result and test code below)
Not working without delayed expansion
::replacing with a percentsign
Code: Select all
set "_substvar13=%_testvar:s=%%%"
set "_substvar13=%_testvar:!=%%%"
set "_substvar13=%_testvar:"=%%%"
set "_substvar13=%_testvar:&=%%%"
set "_substvar13=%_testvar:(=%%%"
set "_substvar13=%_testvar:)=%%%"
set "_substvar13=%_testvar:<=%%%"
set "_substvar13=%_testvar:>=%%%"
set "_substvar13=%_testvar:|=%%%"
Code: Select all
set "_substvar11=%_testvar:%%=!%"
set "_substvar12=%_testvar:%%="%"
set "_substvar13=%_testvar:%%=%%%"
set "_substvar14=%_testvar:%%=&%"
set "_substvar15=%_testvar:%%=(%"
set "_substvar16=%_testvar:%%=)%"
set "_substvar17=%_testvar:%%=<%"
set "_substvar18=%_testvar:%%=>%"
set "_substvar19=%_testvar:%%=^%"
set "_substvar20=%_testvar:%%=|%"
:: replacing with !
Code: Select all
set "_substvar11=!_testvar:s=!!"
set "_substvar11=!_testvar:"=!!"
set "_substvar11=!_testvar:%%=!!"
set "_substvar11=!_testvar:&=!!"
set "_substvar11=!_testvar:(=!!"
set "_substvar11=!_testvar:)=!!"
set "_substvar11=%_testvar:<=!%"
set "_substvar11=%_testvar:>=!%"
set "_substvar11=%_testvar:^=!%"
set "_substvar11=%_testvar:|=!%"
set "_substvar11=!_testvar:!=!!"
set "_substvar12=!_testvar:!="!"
set "_substvar13=!_testvar:!=%%!"
set "_substvar14=!_testvar:!=&!"
set "_substvar15=!_testvar:!=(!"
set "_substvar16=!_testvar:!=)!"
set "_substvar17=!_testvar:!=<!"
set "_substvar18=!_testvar:!=>!"
set "_substvar19=!_testvar:!=^!"
set "_substvar20=!_testvar:!=|!"
replacing a doublequote
set "_substvar11=!_testvar:"=!!" <- doesn't work as usual
set "_substvar12=!_testvar:"="!" <- works fine
set "_substvar13=!_testvar:"=%%!" <- works fine
set "_substvar14=!_testvar:"=&!" --> escape with carret set "_substvar14=!_testvar:"=^&!"
set "_substvar15=!_testvar:"=(!" <- works fine
set "_substvar16=!_testvar:"=)!" <- works fine
set "_substvar17=!_testvar:"=<!" --> escape with carret set "_substvar17=!_testvar:"=^<!"
set "_substvar18=!_testvar:"=>!" --> escape with carret set "_substvar18=!_testvar:"=^>!"
set "_substvar19=!_testvar:"=^!" --> escape with carret set "_substvar19=!_testvar:"=^^!"
set "_substvar20=!_testvar:"=|!" --> escape with carret set "_substvar20=!_testvar:"=^|!"
Here is the output of the test script
Code: Select all
substitution with poison characters, without delayed expansion
_testvar=This is a test
_substvar11=Thi! i! a te!t
_substvar12=Thi" i" a te"t
_substvar13=Thi i a tet%
_substvar14=Thi& i& a te&t
_substvar15=Thi( i( a te(t
_substvar16=Thi) i) a te)t
_substvar17=Thi< i< a te<t
_substvar18=Thi> i> a te>t
_substvar19=Thi^ i^ a te^t
_substvar20=Thi| i| a te|t
substitution with poison characters, with delayed expansion
_testvar=This is a test
_substvar11=Thi i a tet
_substvar12=Thi" i" a te"t
_substvar13=Thi% i% a te%t
_substvar14=Thi& i& a te&t
_substvar15=Thi( i( a te(t
_substvar16=Thi) i) a te)t
_substvar17=Thi< i< a te<t
_substvar18=Thi> i> a te>t
_substvar19=Thi^ i^ a te^t
_substvar20=Thi| i| a te|t
substitution of ! with poison characters, without delayed expansion
_testvar=This is ! test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of with poison characters, with delayed expansion
_testvar=This is ! test
_substvar11==
_substvar12=="
_substvar13==%
_substvar14==&
_substvar15==(
_substvar16==)
_substvar17==<
_substvar18==>
_substvar19==!
_substvar20==|
substitution of " with poison characters, without delayed expansion
_testvar=This is " test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of " with poison characters, with delayed expansion
_testvar=This is " test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of % with poison characters, without delayed expansion"
_testvar=This is % test
_substvar13=%
"substitution of % with poison characters, with delayed expansion"
_testvar=This is % test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of & with poison characters, without delayed expansion"
_testvar=This is & test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of & with poison characters, with delayed expansion"
_testvar=This is & test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ( with poison characters, without delayed expansion
_testvar=This is ( test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ( with poison characters, with delayed expansion
_testvar=This is ( test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ) with poison characters, without delayed expansion
_testvar=This is ) test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ) with poison characters, with delayed expansion
_testvar=This is ) test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of < with poison characters, without delayed expansion"
_testvar=This is < test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of < with poison characters, with delayed expansion"
_testvar=This is < test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of > with poison characters, without delayed expansion"
_testvar=This is > test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of > with poison characters, with delayed expansion"
_testvar=This is > test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of ^ with poison characters, without delayed expansion"
_testvar=This is ^ test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of ^ with poison characters, with delayed expansion"
_testvar=This is ^ test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of | with poison characters, without delayed expansion"
_testvar=This is | test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of | with poison characters, with delayed expansion"
_testvar=This is | test
_substvar11=This is test
_substvar12=This is " test
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
----------------------------------------------------------------------
Retrying everything with the trailing doublequotes removed
substitution with poison characters, without delayed expansion
_testvar=This is a test
_substvar11=Thi! i! a te!t
_substvar12=Thi" i" a te
_substvar13=Thi i a tet%
_substvar14=Thi& i& a te&t
_substvar15=Thi( i( a te(t
_substvar16=Thi) i) a te)t
_substvar17=Thi< i< a te<t
_substvar18=Thi> i> a te>t
_substvar19=Thi^ i^ a te^t
_substvar20=Thi| i| a te|t
substitution with poison characters, with delayed expansion
_testvar=This is a test
_substvar11=Thi i a tet
_substvar12=Thi" i" a te
_substvar13=Thi% i% a te%t
_substvar14=Thi& i& a te&t
_substvar15=Thi( i( a te(t
_substvar16=Thi) i) a te)t
_substvar17=Thi< i< a te<t
_substvar18=Thi> i> a te>t
_substvar19=Thi^ i^ a te^t
_substvar20=Thi| i| a te|t
substitution of ! with poison characters, without delayed expansion
_testvar=This is ! test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of with poison characters, with delayed expansion
_testvar=This is ! test
_substvar11==
_substvar12==
_substvar13==%
_substvar14==&
_substvar15==(
_substvar16==)
_substvar17==<
_substvar18==>
_substvar19==!
_substvar20==|
substitution of " with poison characters, without delayed expansion
_testvar=This is " test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of " with poison characters, with delayed expansion
_testvar=This is " test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of % with poison characters, without delayed expansion"
_testvar=This is % test
_substvar13=%
"substitution of % with poison characters, with delayed expansion"
_testvar=This is % test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of & with poison characters, without delayed expansion"
_testvar=This is & test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of & with poison characters, with delayed expansion"
_testvar=This is & test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ( with poison characters, without delayed expansion
_testvar=This is ( test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ( with poison characters, with delayed expansion
_testvar=This is ( test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ) with poison characters, without delayed expansion
_testvar=This is ) test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
substitution of ) with poison characters, with delayed expansion
_testvar=This is ) test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of < with poison characters, without delayed expansion"
_testvar=This is < test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of < with poison characters, with delayed expansion"
_testvar=This is < test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of > with poison characters, without delayed expansion"
_testvar=This is > test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of > with poison characters, with delayed expansion"
_testvar=This is > test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of ^ with poison characters, without delayed expansion"
_testvar=This is ^ test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of ^ with poison characters, with delayed expansion"
_testvar=This is ^ test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of | with poison characters, without delayed expansion"
_testvar=This is | test
_substvar11=This is ! test
_substvar12=This is
_substvar13=This is test%
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
"substitution of | with poison characters, with delayed expansion"
_testvar=This is | test
_substvar11=This is test
_substvar12=This is
_substvar13=This is % test
_substvar14=This is & test
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is < test
_substvar18=This is > test
_substvar19=This is ^ test
_substvar20=This is | test
----------------------------------------------------------------------
Retrying everything with the leading doublequotes also removed
substitution with poison characters, without delayed expansion
_testvar=This is a test
_substvar11=Thi! i! a te!t
_substvar12=Thi" i" a te"t
_substvar13=Thi i a tet%
_substvar14=Thi& i& a te&t
_substvar15=Thi( i( a te(t
_substvar16=Thi) i) a te)t
_substvar17=Thi< i< a te<t
_substvar18=Thi> i> a te>t
_substvar19=Thi^ i^ a te^t
_substvar20=Thi| i| a te|t
substitution with poison characters, with delayed expansion
_testvar=This is a test
_substvar11=Thi i a tet
_substvar12=Thi" i" a te"t
_substvar13=Thi% i% a te%t
_substvar14=Thi& i& a te&t
_substvar15=Thi( i( a te(t
_substvar16=Thi) i) a te)t
_substvar17=Thi< i< a te<t
_substvar18=Thi> i> a te>t
_substvar19=Thi^ i^ a te^t
_substvar20=Thi| i| a te|t
substitution of ! with poison characters, without delayed expansion
_testvar=This is ! test
_substvar11=This is ! test
_substvar12=This is " test
_substvar13=This is test%
_substvar14=This is
_substvar15=This is ( test
_substvar16=This is ) test
_substvar17=This is
_substvar18=This is
_substvar19=This is test
substitution of with poison characters, with delayed expansion
The system cannot find the path specified.
The system cannot find the path specified.
'' is not recognized as an internal or external command,
operable program or batch file.
C:\Users\user\BatchfileFrameWork\lib\experimental>