how to replace all occurrences of ;;;; with ; in a string

[timbertuck]

i have a string like so: ,,,value1,value2 opt,,,value3,,value4 opt,,,value5,,,,,

and i want to replace all of the multiple comma's with a single comma like so:
value 1,value2 opt,value3,value4,value5

so i need to suppress all leading and trailing comma's plus substitute all multiple commas with a single comma.

i know that i can probably do this in a for loop (but the values go up to 21 so the loop could be quite large) so what would be an elegant way of doing this without having to count the comma's?

note the max of value is 21, but not all values would normally be shown, sometimes only one or two values are shown, the rest are all comma's

[dbenham]

You can repeatedly replace ",," with ",". Each iteration will increase the maximum number of handled consecutive commas by a factor of 2.

1 iteration handles 2
2 handles 4
3 handles 8
4 handles 16
5 handles 32 -> more than enough for up to 21 values.

Whenever performing search/replace operations on a variable, you should make sure the variable is defined, otherwise you get unwanted results.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "str=,,,value1,value2 opt,,,value3,,value4 opt,,,value5,,,,,"
if defined str (
  for /l %%N in (1 1 5) do set "str=!str:,,=,!"
  if "!str:~0,1!"=="," set "str=!str:~1!"
  if defined str if "!str:~-1!"=="," set "str=!str:~0,-1!"
)
set str

EDIT - I believe jeb has the optimum solution on page 3 of this thread: viewtopic.php?p=16801#p16801
Below his post I have a slightly modified version that addresses some missing functionality


Dave Benham

[Squashman]

Dave's code certainly looks more efficient. But will post my 2 cents and realized I should check to see if the leading and trailing characters are commas.

Code: Select all

@echo off
setlocal enabledelayedexpansion

set string=,,,value1,value2 opt,,,value3,,value4 opt,,,value5,,,,,
set commas=,,,,,,,,,,,,,,,,,,,,,

for /L %%G in (5,-1,1) do (
   set commas=!commas:~0,%%G!
   call :_remove "!commas!"
)
if "!string:~0,1!"=="," set "string=!string:~1!"
if "!string:~-1!"=="," set "string=!string:~0,-1!"
echo %string%

goto :EOF

:_remove
set string=!string:%~1=,!

EDIT: I guess technically I could make the start value of the FOR loop to 5 because if there were 21 consecutive commas it would knock it down to 5 the first time through the loop and then down to 2 the 2nd time thru the loop. But, for the life of me I couldn't figure out how to do the comma replacement inside the for loop. If anyone has an answer to that let me know. Must be something I am not understanding about using delayed expansion.

[dbenham]

@Squashman - You can eliminate the subroutine call by transferring the value of !commas! to a FOR variable

I haven't figured out the math, but empirical testing shows your technique only requires 8 iterations to handle the maximum number of consecutive commas possible (~8k).

My binary technique requires 13 iterations.

Here is a modified version of your technique that should handle up to ~8k consecutive commas. I added additional IF statements to guard against string starting out undefined, or becoming undefined after the leading comma is stripped.

Code: Select all

@echo off
setlocal enableDelayedExpansion
set "string=,,,value1,value2 opt,,,value3,,value4 opt,,,value5,,,,,"
if defined string (
  set "commas=,,,,,,,,"
  for /L %%G in (8,-1,1) do (
    for /f %%S in ("!commas:~0,%%G!") do set "string=!string:%%S=,!"
  )
  if "!string:~0,1!"=="," set "string=!string:~1!"
  if defined string if "!string:~-1!"=="," set "string=!string:~0,-1!"
)
set string

Dave Benham

[Squashman]

Thanks Dave. That was bugging me. Not sure why I didn't think of that. I think I was stuck on the fact that I should have been able to do it with one FOR command. Thought I could use the CALL SET trick but that must not work with delayed expansion.

Still like your 1st solution better.

[foxidrive]

Sometimes the long way is the best. No hassle with ! and easy to read?

Code: Select all

@echo off
set "string=,,,value1,value2 opt,,,value3,,value4 opt,,,value5,,,,,"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string set "string=%string:,,=,%"
if defined string if "%string:~0,1%"=="," set "string=%string:~1%"
if defined string if "%string:~-1%"=="," set "string=%string:~0,-1%"
echo %string%
pause

[timbertuck]

many thanks to DaveBenham, squashman, and Foxidrive for responding with solutions, im working on implementing them now.

turns out i had an additional 7 values for a total of 28, so dave's first solution seems to be the winner as it can handle up to 32.

for sq, yes there can be (and usually is) multiple comma's in both the front and end, i just never know when though.

and a nod to foxi, cause when you have to get it done.....sometimes brute force wins on the first go around with coding. so i ended up with something like this (before i got solutions that is, realize that i post code only to show my ignorance )

Code: Select all

set osql_opt=%osopt26% %osopt25% %osopt24% %osopt23% %osopt22% %osopt21% %osopt20% %osopt19% %osopt18% %osopt17% %osopt16% %osopt15% %osopt14% %osopt13% %osopt12% %osopt11% %osopt10% %osopt9% %osopt8% %osopt7% %osopt6% %osopt5% %osopt4% %osopt3% %osopt2% %osopt1%
set swapspace=%osql_opt: =;%
for /f "tokens=1-26* delims=;" %%a in ("%swapspace%") do (
   echo Options are: %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p %%q %%s %%t %%u %%v %%w %%x %%y %%z
)

so certainly not elegant (and only handles 26) but it got me down the road a little. now that i have 28 and not 21 vars to code for, i can use one of the three solutions posted above so that it can handle vars 27 and 28 and any others.

thanks again to all

[foxidrive]

many thanks to DaveBenham, squashman, and Foxidrive for responding with solutions, im working on implementing them now.

turns out i had an additional 7 values for a total of 28, so dave's first solution seems to be the winner as it can handle up to 32.

You may not realise but the solution I posted will handle 256 consecutive commas i think.

Each time this line runs
if defined string set "string=%string:,,=,%"
it will halve the number of doubled commas.

[timbertuck]

hmmm, it appears that you are correct
but a question on the logic:
the max char's for a variable is 8192 bytes, so theoretically one could add 5 add'l "if defined string set..." to reach that limit, is there a way to put the "if defined" in a for /l loop (to make the code more concise)? maybe like so:

Code: Select all

if defined string (
for /l %%a in (1,1,14) do (
   set "string=!string:,,=,!" 
)

and the last two "if defined" lines are meant to remove leading and trailing comma's? i like it

Sometimes the long way is the best. No hassle with ! and easy to read?

Code: Select all

@echo off
set "string=,,,value1,value2 opt,,,value3,,value4 opt,,,value5,,,,,"  [256]
if defined string set "string=%string:,,=,%"    [128]
if defined string set "string=%string:,,=,%"    [64]
if defined string set "string=%string:,,=,%"    [32]
if defined string set "string=%string:,,=,%"    [16]
if defined string set "string=%string:,,=,%"    [8]
if defined string set "string=%string:,,=,%"    [4]
if defined string set "string=%string:,,=,%"    [2]
if defined string set "string=%string:,,=,%"    [1]
if defined string if "%string:~0,1%"=="," set "string=%string:~1%"
if defined string if "%string:~-1%"=="," set "string=%string:~0,-1%"
echo %string%
pause

[dbenham]

the max char's for a variable is 8192 bytes, so theoretically one could add 5 add'l "if defined string set..." to reach that limit, is there a way to put the "if defined" in a for /l loop (to make the code more concise)?

Of course there is - just take my original code, and change the 5 to a 13 in the FOR loop. The code to strip leading and trailing commas is already there, as are all the necessary IF DEFINED statements.

And the code I already posted using Squasman's technique handles the max size of 8192 using 8 instead of 13 iterations.


Dave Benham

[Squashman]

I am sometimes amazed that you can do something in batch a couple of different ways and still get the same result.

And I guess technically the FOR /L loop could just stop at 2. No need to replace 1 comma with 1 comma

Code: Select all

for /L %%G in (8,-1,2) do

[dbenham]

And I guess technically the FOR /L loop could just stop at 2. No need to replace 1 comma with 1 comma

Code: Select all

for /L %%G in (8,-1,2) do

Face palm - I don't know how I missed that optimization

So make that 7 iterations to handle the maximum size of 8192.


Dave Benham

[timbertuck]

this is fascinating stuff, i just love the way that the DOS command prompt has so much power. viva la batch!

thanks again to dave, squashman and foxidrive for all of their input. i learn something new everyday that im on this site.

one question to dave/squashman: on the set comma's command, does the amount of comma's equal the first value in the for (start,#,#) loop?

[dbenham]

one question to dave/squashman: on the set comma's command, does the amount of comma's equal the first value in the for (start,#,#) loop?

Yes.

[Squashman]

many thanks to DaveBenham, squashman, and Foxidrive for responding with solutions, im working on implementing them now.

turns out i had an additional 7 values for a total of 28, so dave's first solution seems to be the winner as it can handle up to 32.

for sq, yes there can be (and usually is) multiple comma's in both the front and end, i just never know when though.

and a nod to foxi, cause when you have to get it done.....sometimes brute force wins on the first go around with coding. so i ended up with something like this (before i got solutions that is, realize that i post code only to show my ignorance )

Code: Select all

set osql_opt=%osopt26% %osopt25% %osopt24% %osopt23% %osopt22% %osopt21% %osopt20% %osopt19% %osopt18% %osopt17% %osopt16% %osopt15% %osopt14% %osopt13% %osopt12% %osopt11% %osopt10% %osopt9% %osopt8% %osopt7% %osopt6% %osopt5% %osopt4% %osopt3% %osopt2% %osopt1%
set swapspace=%osql_opt: =;%
for /f "tokens=1-26* delims=;" %%a in ("%swapspace%") do (
   echo Options are: %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p %%q %%s %%t %%u %%v %%w %%x %%y %%z
)

so certainly not elegant (and only handles 26) but it got me down the road a little. now that i have 28 and not 21 vars to code for, i can use one of the three solutions posted above so that it can handle vars 27 and 28 and any others.

thanks again to all

And Technically you can have 31 Tokens in a single FOR LOOP and a total of 65 Unique variables with nested For Loops. It is undocumented but it works apparently.
http://www.robvanderwoude.com/clevertricks.php