DosTips.com

Posted: **13 Jun 2011 11:06**

I apologize if this isn't the right place to ask these types of questions, but I was looking at the example here:

http://www.dostips.com/DtTipsStringMani ... .MapLookup

Code: Select all

REM ---- Example 1: Translate name of month into two digit number ----
SET v=Mai

SET map=Jan-01;Feb-02;Mar-03;Apr-04;Mai-05;Jun-06;Jul-07;Aug-08;Sep-09;Oct-10;Nov-11;Dec-12
CALL SET v=%%map:*%v%-=%%


ECHO.%v%
SET v=%v:;=&rem.%

And I have a few questions about this part:

Code: Select all

CALL SET v=%%map:*%v%-=%%
SET v=%v:;=&rem.%

Q1: Why is set "Called" ?

Q2: What exactly is happening on these lines? What is the %% before map used for? I know that the : is used to modify the string being assigned, but what does the *%v%-=%% actually doing?

Q3: These questions are very acccurate examples of the holes in my batch scripting knowledge, can anyone point me towards a reference for more advanced techniques?

These questions came about because I am trying to understand the macro discussion and no matter how well documented, I seem to be missing some basic concepts. I examined some of the functions converted, and found my first problem, I didn't understand what was going on with the functions either. So I checked out the articles on this site, answered a few questions, but got stuck on the one above...

Posted: **13 Jun 2011 11:49**

CALL SET v=%%map:*%v%-=%%
Q1: Why is set "Called" ?

this is simple, it's like a sort of eval ( x ).
we can do this set v=%x%, would expand to contents of x, but what if we wanted to do this:
set 1.array=value
set x=1
set v=%%x%.array% ← this is a problem, but call causes values to be evaluated twice so
call set v=%%%x%.array%% works.

Q2: What exactly is happening on these lines? What is the %% before map used for? I know that the : is used to modify the string being assigned, but what does the *%v%-=%% actually doing?

CALL SET v=%%map:*%v%-=%% becomes
set v=%map:*thevalueofv-=%
it will replace *thevalueofv- inside variable map with nothing and store this inside v
Enter set /? in the console, you'll understand.

Q3: These questions are very acccurate examples of the holes in my batch scripting knowledge, can anyone point me towards a reference for more advanced techniques?

I find this one of the best sites:
MS-DOS/MSDOS Batch File Command Reference (writing it like this because of 2 link per post limit

)

www.
allenware.com/icsw/icswref.htm

I am trying to understand the macro discussion

I hope this will help you understand better:
[SOLVED] SET /a -- Random Number?
http://www.dostips.com/forum/viewtopic.php?f=3&t=1817
And my projects at scriptingpros.
http://www.scriptingpros.com/viewforum.php?f=118

Posted: **13 Jun 2011 12:39**

CALL SET v=%%map:*%v%-=%% becomes
set v=%map:*thevalueofv-=%
If you enter set /? in the console, you'll understand.
it will replace *thevalueofv- inside variable map with nothing and store this inside v

I got it, I use the substring functionality of SET all the time, but completely missed the paragraph above it explaining string replacement. So I apologize for the RTFM question.

Unfortunately, this might be another (Characters are sooo hard to use as search criteria...)

I completely understand what that's doing now..

The *%v%-=% sets everything up to mai- to nothing. leaving 05 as the start of the string.

What I would do from that point though is Set %v=:~1,2%
but mostly because I already know that works.

So what about the second line, I don't understand what the &rem is doing. I mean I see that it looks like it's delimiting the string removing all of the ; semicolons.

Is the & treated as a line feed causing the rest of the value to be considered a remark? If that's what is happening, I've got a handle on this, if not, then I'm still lost at that point...

Posted: **13 Jun 2011 15:17**

Acy Forsythe wrote:Is the & treated as a line feed causing the rest of the value to be considered a remark? If that's what is happening, I've got a handle on this, if not, then I'm still lost at that point...

You nearly got it - though & is not a line feed. It is a command delimiter allowing multiple commands to be concatenated on one line. You got the rest right, the remainder of the string is is treated as a remark, which effectively truncates the string. Kudos to you for figuring out the logic as well as you did. It took me a while to catch on.

Just for fun, take a look at the :format function. It uses a similar string substitution trick. Instead of injecting a REM command, it injects an ECHO command. Try running the :format command with ECHO ON to see how it works! It blew my mind when I first saw it.

Dave Benham

Posted: **13 Jun 2011 15:37**

That's awsome guys. Sorry for being such a noob, but it's amazing how one small unknown detail can make everything seem so complicated.

Dave, the format function definitely sheds some more light on string manipulation. I've been using substrings, temp files and FOR loops and really over-complicating simple tasks.

I can't beleive I missed that...

Ed,

Thanks for that link, I've been to the site before, but it seemed very basic and retro, and I've found a lot of old DOS stuff that just doesn't work anymore is different now, so I skipped over it and moved on. I am now revisiting the site and going through it in it's entirety just to make sure I have all the basics down.

Posted: **14 Jun 2011 17:20**

So I've been seriously messing with the Format function like you advised, and I know what the command is doing, I just don't quite understand what call is doing to make it work.

For example, let's say later down the road, I run into a similar need for something like this, what is going to tell me: "Hey Call Call Call Set would work for this!"

Anyway here is what I did. First I echo'd everything everywhere in that function to see what it actually did. And I understand, it's taking parameter i (which in the examples below is 2) and it's adding space in the middle. Why it's adding parameter 2 on the end I don't know, because then it truncates down to %%b in length essentially cutting off the other half, but I assume that the second %2 is required to make the call call set work.

Code: Select all

OK so:
%2=String1
i=2
subst=

call call set "subst=%%%%~%%i%%%spac%%%%%~%%i%%"

result:
subst="string1                                                     string1"

Then I knocked out one of the calls:

Code: Select all

call set "subst=%%%%~%%i%%%spac%%%%%~%%i%%"

result:
subst="%~2                                                     %~2"

Then I knocked out the next call

Code: Select all


set "subst=%%%%~%%i%%%spac%%%%%~%%i%%"

result:
subst="%%~%i%                                                     %%~%i%"

Posted: **15 Jun 2011 07:51**

Just to clarify the question...

I understand that adding the second call expands the %~2 into the parameter passed to it.

And I can *guess* that the first call is expanding %%~%i% into %~2, but I guess what I'm not understanding is all of the %'s and what they are for.

Code: Select all

"subst=%%%%~%%i%%%spac%%%%%~%%i%%"

Why all the percents? And what are the "" for?

That's the key, if someone could explain that to me, I'd have this down.

Posted: **15 Jun 2011 11:52**

Acy Forsythe wrote:And what are the "" for?

Quotes are often used to enable support for special characters without the need to escape them with a ^. Special characters include & ^ < > |. The set "var=value" syntax eliminates the need to escape special characters, yet does not introduce the "" into the value. The "" don't do much good in this function however because the final echo.%line% will fail and/or give unwanted results if %line% contains special characters.

Acy Forsythe wrote:I understand that adding the second call expands the %~2 into the parameter passed to it.

And I can *guess* that the first call is expanding %%~%i% into %~2, but I guess what I'm not understanding is all of the %'s and what they are for.

Your "guess" is spot on. Why the many %'s are needed has to do with the peculiarities of the batch parser and the decision to write the function without using delayed expansion. Read the discussion of delayed expansion within the FOR help for an explanation of the issue accessing a variable that has been set within a FOR loop. People sometimes use CALL along with doubled up %% to avoid the use of delayed expansion. In this case we have quadrupled %%%% because we need two passes, one to access the iteration count, the 2nd to access the function parameter.

See CALL me, or better avoid call for a discussion of the performance penalty of the CALL %%var%% technique.

Here is the function rewritten to use delayed expansion.

Code: Select all

:Format fmt str1 str2 ... -- outputs columns of strings right or left aligned
::                        -- fmt [in] - format string specifying column width and alignment, i.e. "[-10][10][10]"
:$created 20060101 :$changed 20091130 :$categories Echo
:$source http://www.dostips.com
:: rewritten to use delayed expansion
setlocal enableDelayedExpansion
set "fmt=%~1"
set "line="
set "spac=                                                     "
set "i=1"
for /f "tokens=1,2 eol=[ delims=[" %%a in ('"echo.%fmt:]=&echo.%"') do (
  set /a i+=1
  call set "subst=%%~!i!!spac!%%~!i!"
  if %%b0 GEQ 0 (set "subst=!subst:~0,%%b!") ELSE (set "subst=!subst:~%%b!")
  set "line=!line!%%a!subst!"
)
echo.!line!
exit /b

On my machine this function is nearly 3 times faster than the original.

We can't completely elliminate the CALL within the loop because we need to access a different function parameter on each pass. There is no delayed expansion option for function parameters.

Besides using delayed expansion, I added eol=[ to the FOR /F options, thereby eliminating the need to prepend strings with a . that later needed to be stripped again. Without either technique the function will have problems with strings that begin with a semicolon (the default EOL character). It is only a recent discovery that EOL can be disabled by setting it to match one of the delimiters. (See the 3rd, 5th and 9th posts in this thread)

One other benefit of the rewrite is it now allows special characters in the replacement strings because the final ECHO is using delayed expansion. However the function still fails if there are special characters in the format string.

One detractor of this modified version is it now fails if there is a ^ or ! in either the format or replacement string because of added escape requirements needed while delayed expansion is enabled.

Dave Benham

Posted: **15 Jun 2011 15:33**

dbenham wrote:However the function still fails if there are special characters in the format string.

With a small change this can be solved, and it should be faster too.
I change the replace statement in the FOR-statement from

Code: Select all

'"echo.%fmt:]=&echo.%"'

to

Code: Select all

!fmt:]=<LF>!

It creates the same but without the overhead of calling ECHO, btw. echo. is slower than echo(

Code: Select all

:Format fmt str1 str2 ... -- outputs columns of strings right or left aligned
::                        -- fmt [in] - format string specifying column width and alignment, i.e. "[-10][10][10]"
setlocal enableDelayedExpansion
set "fmt=%~1"
set "line="
set "spac=                                                     "
set "i=1"
for /f "tokens=1,2 delims=[" %%a in (^"!fmt:]^=^

!^") do (
  set /a i+=1
  call set "subst=%%~!i!!spac!%%~!i!"
  if %%b0 GEQ 0 (set "subst=!subst:~0,%%b!") ELSE (set "subst=!subst:~%%b!")
  set "line=!line!%%a!subst!"
)
echo(!line!
endlocal
goto :eof

jeb

Posted: **15 Jun 2011 16:39**

Hi jeb. I was just getting ready to post the same <LF> solution when I saw you beat me to it! You win again and order is restored to the universe! :lol:

jeb wrote:btw. echo. is slower than echo(

Interesting.

I tried doing some timing comparing ECHO. vs. ECHO: vs. ECHO(
My initial test harness used a FOR loop and the ECHO( failed while the others did not.
I then did a simple test removing the loop and confirmed your speed info. I saw about a 25% speed improvement with ECHO( over the other two.

I think given that there are situations that ECHO( can fail, and I haven't run into a problem with ECHO: yet, I'll probably stick with the latter. If you know of a test case where ECHO: fails, I'd be interested in seeing it. I think I saw a claim that there was a case, but I haven't seen the actual code.

Dave Benham

Posted: **15 Jun 2011 17:04**

Dave what are you using to test? Have you tried this on the other variations (echo\ echo/ etc..etc..)

Posted: **15 Jun 2011 17:15**

Acy Forsythe

My old thread here may be easier to understand:

viewtopic.php?t=1486

Posted: **15 Jun 2011 19:28**

Dave, Orange, Jeb and Ed, you guys are awsome, thank you so much for the detailed explainations! I finally understand what's going on with all those special characters!

Posted: **15 Jun 2011 21:58**

dbenham wrote:I tried doing some timing comparing ECHO. vs. ECHO: vs. ECHO(
My initial test harness used a FOR loop and the ECHO( failed while the others did not.

I did some more testing and identified the source of my ECHO( failure - it stems from trying to execute the command via a variable (simple macro). It fails if the "macro" is expanded after phase 1.

test code:

Code: Select all

setlocal enableDelayedExpansion
set "cmd=echo("
%cmd%Hello
(
  %cmd%Hello
  REM
)
!cmd!
for /f %%c in ("!cmd!") do %%cHello

Results:

Code: Select all

>setlocal enableDelayedExpansion

>set "cmd=echo("

>echo(Hello
Hello

>(
echo(Hello
 REM
)
Hello

>!cmd!
'echo(' is not recognized as an internal or external command,
operable program or batch file.

>for /F %c in ("!cmd!") do %cHello

>echo(Hello
'echo(Hello' is not recognized as an internal or external command,
operable program or batch file.

If I change the definition of cmd to ECHO: or ECHO. then all four tests succeed.

Dave Benham

Posted: **15 Jun 2011 23:33**

dbenham wrote:I think given that there are situations that ECHO( can fail, and I haven't run into a problem with ECHO: yet, I'll probably stick with the latter. If you know of a test case where ECHO: fails, I'd be interested in seeing it. I think I saw a claim that there was a case, but I haven't seen the actual code.

Here is a sample "echo" VS "echo." VS "echo:"
both echo. and echo: fail

jeb

DosTips.com

String Manipulation examples...

String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: Still stuck on Call

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...

Re: String Manipulation examples...