For Loop to Parse Delimited String Variable--how?

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Message
Author
Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

For Loop to Parse Delimited String Variable--how?

#1 Post by Jer » 22 May 2015 23:01

I need help with this error: "tokens=1 delims=; " was unexpected at this time.
Here's the code:

Code: Select all

@Echo Off
setlocal enabledelayedexpansion

Set target=my home ; sweet home

For /L "tokens=1 delims=; " %%a In (%target%) Do (
   Echo %%a
)

endlocal


(Yes, I've looked at For /?)

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#2 Post by foxidrive » 23 May 2015 00:12

Try this again and look at the /L syntax.

for /?

ShadowThief
Expert
Posts: 1166
Joined: 06 Sep 2013 21:28
Location: Virginia, United States

Re: For Loop to Parse Delimited String Variable--how?

#3 Post by ShadowThief » 23 May 2015 00:42

Yeah... you absolutely do NOT want to be using /L for that loop.

Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

Re: For Loop to Parse Delimited String Variable--how?

#4 Post by Jer » 23 May 2015 10:02

Thanks for the replies. I should have used /f in the For loop.
I found that this is what works:

Code: Select all

@Echo Off
setlocal enabledelayedexpansion

Set target=my home;sweet home

For /f "tokens=1,2* delims=;" %%a In ("%target%") Do (
   Set tkns1= %%a
   Set tkns2= %%b
)

Echo #1 tokens: %tkns1%
Echo #2 tokens: %tkns2%

endlocal


Checking For/? again, the closest instruction that I see is
FOR /F ["options"] %variable IN ("string") DO command [command-parameters]

Per my solution above, this would be more applicable:
FOR /F ["options"] %variable IN ("%variable%") DO command [command-parameters]

I would have included that in For /?, but maybe its not correct. Any opinions?

I am developing, in a batch file, a word-search utility for my song file library (18,300 and counting),
so when I enter: glide;strut at a prompt, all song file names that contain either of those words in the
song title appear on the console, with pauses if needed. Using FINDSTR, one file is searched that has the file names
in whole words like this: _Wedding_Glide_.mid and all line numbers with the whole word 'glide' are put in a file followed
by all line numbers where 'strut' is found. In the file to be searched, all punctuation has been removed from the name.

My prompted entry 'glide' becomes '_glide_' (without the quotes), or if the target was 'come home' the words are
converted to '_come_home_' to locate these whole contiguous words in the file.

The line numbers are aligned in a file, sorted, and deduped (duplicates removed). Knowing the line numbers,
the actual song file names are retrieved, in a For loop with Skip, from a second file which is in the same order as
the search file. Besides the two-word option (word1;word2) this scheme can also take single words or single
phrases as targets.
Each of the individual steps work and I am in the process of joining each process in one application.

Thanks again for helping me to get further along on this batch coding journey.
Jerry

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#5 Post by foxidrive » 23 May 2015 19:10

Is there a reason why the punctuation is removed from your filenames?

I use a batch script to search my large text file, with a few free utilities, but it will also give me
some false positives - do you need 100% positive matches?

In my case, all I need is a list of filenames that I can view and check manually, with eyeballs V1.0

Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

Re: For Loop to Parse Delimited String Variable--how?

#6 Post by Jer » 23 May 2015 22:00

All punctuation except for the apostrophe mark are edited out of the search file so that only whole
words can be searched for. A search on the word who's would be okay. The actual search would be for
_who's_, then each line number with _who's_ is saved and used to retrieve the actual name.
If I am searching for the word 'them' I don't want to get 'theme' or 'anthem', and that is why I am using
this whole-word searching method.
If the search text file included all punctuation, FINDSTR would not know that 'theme,' was not 'them,'

My music file collection has been organized over many years, and I've renamed many of the files to accommodate
lot's of duplicates, for example Alexander's Ragtime Band (Arr 1).mid, ...(Arr 2).mid, etc. because there are
different arrangements, some piano only, some with multiple instruments, and various styles of music.

When MIDI files are played from the computer through a keyboard with a MIDI port, its close to owning a player piano.

My adding word searching via code in a batch file, will allow me to locate a song title by searching for words that
are after the first word of the title. The search utility will be added as an option to my homemade vbScript music
playing application.

I assembled the batch code today and testing with 18,000+ lines return a list in 1 second.
The give-me-word1-or-word2 option takes about 2 seconds to display the list.
Thanks for your interest.

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#7 Post by foxidrive » 24 May 2015 04:46

ok - the MIDI via keyboard sounds neat.

Findstr can give you a line with them in it and ignore the others - test this:
It will also give you the line number.


Code: Select all

@echo off
(
echo A them 1
echo A theme 1
echo A them 2
echo A anthem 1
)>file.txt
Findstr /irn /c:".* them .*" "file.txt"
pause


This should give you two words on the command line:

Code: Select all

@echo off
Findstr /irn /c:".* %~1 .*" "file.txt" |Findstr /ir /c:".* %~2 .*"
pause


If you wish to use _ in place of the spaces then change tham.

Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

Re: For Loop to Parse Delimited String Variable--how?

#8 Post by Jer » 25 May 2015 10:34

foxdrive, your first example picks up every 'them' but would ignore 'them,' . That's why
I've removed punctuation and converted all spaces to underscore plus the first
word has a leading _ and the last word ends with _.

I am using Findstr line numbers to create the display list after parsing out the numbers.

Your second example does not display anything when I try it.
For learning's sake could you see if I'm doing something wrong.
Here's the batch file I tested, and with file.txt in the same path.
My command is: test1 them anthem

Code: Select all

@Echo Off
Echo entered on command line: %1 AND %2
Echo.& Echo file contents:& Type file.txt

Echo.&Echo running Findstr...
Findstr /irn /c:".* %~1 .*" "file.txt" |Findstr /ir /c:".* %~2 .*"


And finally, you taught me something: using parenthesis around a
block of echo-to-file commands.

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#9 Post by foxidrive » 26 May 2015 04:11

Jer wrote:foxdrive, your first example picks up every 'them' but would ignore 'them,' . That's why
I've removed punctuation and converted all spaces to underscore plus the first
word has a leading _ and the last word ends with _.
Your second example does not display anything when I try it.


The punctuation is a problem there, I can see that, but this code works below for me.


Code: Select all

d:\abc>A.BAT "them" "bill"
1:A them bill 1
3:A them and bill 2

d:\abc>type a.bat
@echo off
(
echo A them bill 1
echo A bill theme 1
echo A them and bill 2
echo A anthem 1
)>file.txt
Findstr /irn /c:".* %~1 .*" "file.txt" |Findstr /ir /c:".* %~2 .*"



For learning's sake could you see if I'm doing something wrong.

And finally, you taught me something: using parenthesis around a
block of echo-to-file commands.


I can't see your input and output from your command, but perhaps you can try it again, and post both the in-text and result, to see which part trips it up.

Findstr has a couple of dozen foibles and fails which Dave Benham has documented on Stack Overflow.
findrepl.bat and jrepl.bat are both useful tools, using native Windows features, to do things that plain batch needs crutches to do.

The parentheses method is good, as long as closing brackets are escaped etc.

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#10 Post by foxidrive » 26 May 2015 04:38

I may have misunderstood what you were after, and this should find terms with them OR anthem in them.

Jer wrote:My command is: test1 them anthem


Code: Select all

@Echo Off
Echo entered on command line: %1 AND %2
Echo.& Echo file contents:& Type file.txt

Echo.&Echo running Findstr...
Findstr /irn /c:".* %~1 .*" /c:".* %~2 .*" "file.txt"

Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

Re: For Loop to Parse Delimited String Variable--how?

#11 Post by Jer » 26 May 2015 09:26

I get the same output that you do with both those last examples. It
ignores words joined to punctuation.

I needed a way to pick up 'them, Bill and ...', so I use FINDSTR to search a file that
has no commas or other punctuation; it gets the line numbers. Then text is retrieved
on the same line numbers from a second file that has music file names, for display
in a list sorted alphabetically. My batch file code is working and this forum has helped.

Thanks for your code and comments.

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#12 Post by foxidrive » 26 May 2015 16:37

If you had supplied some sample input then I could test this - but see what it does on your end.

Code: Select all

@Echo Off
set "punc=[,.;:'?{}+=-_`~@#$&() ]"
Echo entered on command line: "%~1" AND "%~2"
Echo.& Echo file contents:& Type file.txt

Echo.&Echo running Findstr...
Findstr /irn /c:".*%punc%%~1%punc%.*" /c:".*%punc%%~2%punc%.*" "file.txt"
pause

Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

Re: For Loop to Parse Delimited String Variable--how?

#13 Post by Jer » 27 May 2015 10:09

Here is what seems to work with my file of file names that includes all punctuation.
It required that I add a leading space to each line (my text editor is KEDIT for windows),
otherwise the first word was not found, and allow for just one word entry or one quoted set of words.

Code: Select all

@Echo Off
set "punc=[,.;:'?{}+=-_`~@#$&() ]"
Echo entered on command line: "%~1" AND "%~2"
Echo.& Echo file contents:& Type file.txt

Echo.&Echo running Findstr...
If "%~2"=="" (
    Findstr /irn /c:".*%punc%%~1%punc%.*" "file.txt"
) Else (
    Findstr /irn /c:".*%punc%%~1%punc%.*" /c:".*%punc%%~2%punc%.*" "file.txt"
)


C:\Temp>
entered on command line: "street" AND "st."

file contents:
11th Street Rag.mid
38th St. Rag.mid
A Bag Of Rags.mid
A Line Of Rags.mid
Beale Street Blues (1917).mid
Blue (1922).mid
Blue Ribbon Rag (1910).mid

running Findstr..
1: 11th Street Rag.mid
2: 38th St. Rag.mid
5: Beale Street Blues (1917).mid

C:\Temp>

foxidrive
Expert
Posts: 6031
Joined: 10 Feb 2012 02:20

Re: For Loop to Parse Delimited String Variable--how?

#14 Post by foxidrive » 28 May 2015 02:23

The code I provided gives the same output as your code with the samples you provided.

I'm not sure which part is the problem....

Jer
Posts: 177
Joined: 23 Nov 2014 17:13
Location: California USA

Re: For Loop to Parse Delimited String Variable--how?

#15 Post by Jer » 28 May 2015 11:42

When I run the code and the text file lines have no leading space, FINDSTR
does not locate the line if you are searching for first words. Add
a leading space in the text file and the first words can be found.

If you get different results, that would be very curious. My O/S is Windows 7 32-bit.

This line added to the beginning of the batch file makes it cleaner, since I see that
in this batch file FINDSTR locates lines when there are no command line args.

Code: Select all

If "%1"=="" Echo No args entered& Exit /b

Post Reply