Page 1 of 1
Newline character as delimiter in for-loop
Posted: 24 May 2015 07:18
by Reino
I have the following situation:
Code: Select all
youtube-dl.exe -gf 135+141 https://www.youtube.com/watch?v=KpvLMyC4a8o
https://r3---sn-mn4vg5aa-5hns.googlevideo.com/...longurl...
https://r3---sn-mn4vg5aa-5hns.googlevideo.com/...longurl...
When you want youtube-dl to return the videolink of 2 formats of a certain video, it will return the 2nd videolink on a new line.
I have the following test-script:
Code: Select all
@ECHO off
SET youtube-dl="youtube-dl.exe"
SET "url=https://www.youtube.com/watch?v=KpvLMyC4a8o"
SET format=135+141
set LF=^
FOR /F "tokens=1,2" delims=%LF%" %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%"^"') DO (
SET "_url1=%%A"
SET "_url2=%%B"
)
ECHO "%_url1%"
ECHO.
ECHO "%_url2%"
PAUSE
I want to put each videolink in a variable, but the linefeed is posing a problem.
Some weeks ago while searching for another issue I stumbled upon a website which showed that with a trick the linefeed character could be used as a delimiter in a for-loop. The problem is, that despite lots of Google-queries, I just can't find that website anymore.
What I do still remember from that trick was that the double quotes surrounding "tokens= delims=" were replaced by lots of carets.
Does anyone here know how to use a newline character as a delimiter in a for-loop?
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 10:05
by Ed Dyreen
Setting delims to a linefeed has no advantage over simply disabling eol and delims because 'For' can't handle linefeeds.
Code: Select all
@echo off &setlocal enableDelayedExpansion
for /F delims^=^ eol^= %%r in ('ping') DO set /A line[0] += 1 &set "line[!line[0]!]=%%r"
set line
pause
exit
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 14:27
by Reino
This isn't about ignoring lines with eol=x, but the other way around, so I can't use your code. I want to create a singleline of youtube-dl's multiline output, by using CRLF as a delimiter, so I can "tokenize" it into 2 variables. You're saying I can't?
I've tried...
Code: Select all
FOR /F "tokens=1,2 delims=ΒΆ" %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%"^"') DO (
and...
Code: Select all
set LF=^
FOR /F tokens^=1^,2^ delims^=^%LF%^ %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%"^"') DO (
but both to no avail.
In the meantime the following temporary solution would have to do:
Code: Select all
@ECHO off
SET youtube-dl="youtube-dl.exe"
SET jq="jq.exe"
SET "url=https://www.youtube.com/watch?v=KpvLMyC4a8o"
SET format=135+141
FOR /F %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%" ^| %jq% -R -s .^"') DO SET "_url=%%A"
FOR /F "tokens=1,2" %%A IN (%_url:\n= %) DO SET "_url1=%%A" & SET "_url2=%%B"
ECHO "%_url1%"
ECHO.
ECHO "%_url2%"
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 15:08
by ShadowThief
Why can't you send the output of the command to a temp file and then process the output from there?
Code: Select all
%youtube-dl% --nowarnings -gf %format "%url%" >output.txt
(
set /p ignore_line=
set /p _url1=
set /p _url2=
)<output.txt
del output.txt
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 15:42
by Reino
Sorry, but I refuse to use temp files.
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 15:59
by Ed Dyreen
This isn't about ignoring lines with eol=x, but the other way around, so I can't use your code. I want to create a singleline of youtube-dl's multiline output, by using CRLF as a delimiter, so I can "tokenize" it into 2 variables. You're saying I can't?
My code does not ignore any lines with eol=x, in fact both eol and delims are disabled. For always splits when it encounters a linefeed, therefore the effect is exactly the same as what you get when you set 'delims=^%$lf%%$lf%'. And because there is no difference, setting delims to a lineFeed is depriciated. Because for can't handle lineFeeds, your attempt to tokenize multiLine output inevitably fails.
Code: Select all
@echo off &setlocal enableDelayedExpansion &set $lf=^
::
set "$=echo.https://longurlA...&echo.https://longurlB..."
echo. &echo.with eol and delims disabled;
for /F delims^=^ eol^= %%r in ( '!$!' ) DO set /A url[0] += 1 &set "url[!url[0]!]=%%r"
for /L %%i in ( 1, 1, !url[0]! ) do echo.url[%%i]=!url[%%i]!_ &set "url[%%i]="
set /A url[0] = 0
echo. &echo.with eol disabled and delims=lineFeed;
for /F delims^=^%$lf%%$lf%^ eol^= %%r in ( '!$!' ) DO set /A url[0] += 1 &set "url[!url[0]!]=%%r"
for /L %%i in ( 1, 1, !url[0]! ) do echo.url[%%i]=!url[%%i]!_ &set "url[%%i]="
set /A url[0] = 0
pause
exit
Each line is added to an array and you can create a single variable out of it by iterating the array.
Code: Select all
with eol and delims disabled;
url[1]=https://longurlA..._
url[2]=https://longurlB..._
with eol disabled and delims=lineFeed;
url[1]=https://longurlA..._
url[2]=https://longurlB..._
Druk op een toets om door te gaan. . .
you can also use a temp file as ShadowThief suggest.
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 16:28
by Aacini
Try this:
Code: Select all
SET "_url1="
FOR /F "delims=" %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%"^"') DO (
IF NOT DEFINED _url1 (
SET "_url1=%%A"
) ELSE (
SET "_url2=%%A"
)
)
Antonio
Re: Newline character as delimiter in for-loop
Posted: 24 May 2015 17:43
by Reino
I'm sorry, Ed Dyreen. I didn't understand your code at first.
Code: Select all
@ECHO off
SET youtube-dl="youtube-dl.exe"
SET "url=https://www.youtube.com/watch?v=KpvLMyC4a8o"
SET format=135+141
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%"^"') DO (
SET /A _url[0] += 1
SET "_url[!_url[0]!]=%%A"
)
ECHO "%_url[1]%"
IF DEFINED _url[2] (
ECHO.
ECHO "%_url[2]%"
)
ENDLOCAL
This works. BUT, I'm going for Antonio's more straight forward code:
Code: Select all
@ECHO off
SET youtube-dl="youtube-dl.exe"
SET "url=https://www.youtube.com/watch?v=KpvLMyC4a8o"
SET format=135+141
SET "_url1="
FOR /F %%A IN ('^"%youtube-dl% --no-warnings -gf %format% "%url%"^"') DO (
IF NOT DEFINED _url1 (
SET "_url1=%%A"
) ELSE (
SET "_url2=%%A"
)
)
ECHO "%_url1%"
IF DEFINED _url2 (
ECHO.
ECHO "%_url2%"
)
It makes perfect use of the for-loop principle; During the first iteration the first line of youtube-dl's output is processed and assigned to _url1. During the second iteration the second line is processed, and assigned to _url2, because _url1 is already defined. Code I can better understand.
Thanks a lot for your help you two.
Re: Newline character as delimiter in for-loop
Posted: 25 May 2015 01:44
by foxidrive
CoRoNe wrote:Sorry, but I refuse to use temp files.
Back in the dark ages of batch files temp files were used so very often, because they were absolutely necessary.
Many people say "no temp files" these days but they seem to miss the point that Windows uses
temporary files all day long.
I think a thank you to ShadowThief should be in your post, also.
Re: Newline character as delimiter in for-loop
Posted: 25 May 2015 02:21
by Reino
Perhaps that was a little bit harsh, yes.
ShadowThief, thanks for your help too, but I still won't be using temp files.
Re: Newline character as delimiter in for-loop
Posted: 25 May 2015 06:39
by foxidrive
CoRoNe wrote:ShadowThief, thanks for your help too, but I still won't be using temp files.
One point wrt temp files is that when they are in an inner loop and speed is paramount, and it is badly affected by temp files,
then there is a case for writing extra code to get around it.
However there are cases where a temp file will speed up the batch file enormously - in particular a bug with
for /f where a huge
number of files being parsed,or with long names, can lead to 30 or more minutes before it starts to do anything.
Re: Newline character as delimiter in for-loop
Posted: 25 May 2015 09:04
by dbenham
CoRoNe wrote:Sorry, but I refuse to use temp files.
That position can lead to a world of hurt. There are multiple scenarios where the use of one or more temp files is MUCH faster than any other batch alternative. One example is processing a command that produces lots of output. It is much faster to redirect the output to a temp file and process the file with FOR /F. Processing the command output directly with FOR /F is painfully slow if the output is really large.
Dave Benham