Hello DOStips,
I am looking for some advice on how to manage wget.
We are currently downloading with wget from a list of urls, called urls.txt
The log shows downloads that succeeded and also failed
I made a working batch to list the fails:
findstr /i "ERROR:" c:\urlstrip\test.txt >c:\urlstrip\certificate_errors.txt
ERROR: cannot verify fas.org's certificate, issued by `/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2':
ERROR: cannot verify www.berghahnjournals.com's certificate, issued by `/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2':
Where I am struggling, is to make a new list of ONLY the domains that failed (not the extra commentary), in a file (failed_domains.txt) so:
fas.org
www.berghahnjournals.com
...hopefully we can then use the failed_domains.txt and create a new file, with only urls from the urls.txt file that were also on the failed_domains.txt list
https://fas.org/irp/agency/dod/ig020907.pdf
https://fas.org/irp/agency/dod/ig020907.pdf
https://www.berghahnjournals.com/downlo ... 480111.pdf
etc.
Any help, even on part of it would be really appreciated.
List lines from partial match in file
Moderator: DosItHelp
Re: List lines from partial match in file
Simple FOR /F command. It is the 4th delimited token based on spaces but you will also need to use the apostrophe as a delimiter as well to remove everything after the 's.
Re: List lines from partial match in file
Squashman,
Thank you for the tip - I have never really understood the FOR /F command, so your suggestion made me pursue it a bit more diligently (thank you).
So these are baby steps for me, but when I run it from the command line it performs pretty well
FOR /F "tokens=1 delims='" %G IN (errors.txt) DO @echo %G
ERROR: cannot verify fas.org
ERROR: cannot verify www.berghahnjournals.com
BUT when I make it for a batch:
FOR /F "tokens=1 delims='" %%G IN (errors.txt) DO @echo %%G>clean.txt
...then it ignores the first line:
ERROR: cannot verify www.berghahnjournals.com
I am so new to FOR /F that I do not know why the batch versus command line would give different results - any ideas?
I imagine I will be able to FART.exe away the "ERROR: cannot verify" just fine.
Aisha
Thank you for the tip - I have never really understood the FOR /F command, so your suggestion made me pursue it a bit more diligently (thank you).
So these are baby steps for me, but when I run it from the command line it performs pretty well
FOR /F "tokens=1 delims='" %G IN (errors.txt) DO @echo %G
ERROR: cannot verify fas.org
ERROR: cannot verify www.berghahnjournals.com
BUT when I make it for a batch:
FOR /F "tokens=1 delims='" %%G IN (errors.txt) DO @echo %%G>clean.txt
...then it ignores the first line:
ERROR: cannot verify www.berghahnjournals.com
I am so new to FOR /F that I do not know why the batch versus command line would give different results - any ideas?
I imagine I will be able to FART.exe away the "ERROR: cannot verify" just fine.
Aisha
Re: List lines from partial match in file
You have a different perception of new. Myself and a few other people have given you several answers using FOR /F over the past 17 months you have been a member on this forum.
Read this page on redirection and you will see why your code is not working.