Recursive Searching Across Directories & Date Sorting

[atfon]

Hello Folks,

I apologize if this has already been answered, but I have yet to find it. I am attempting to generate a batch file that will search for files of a type recursively and generate a list sorted by date regardless of directory. I know I can do something simple like this:

for /f "delims=*" %%g in ('dir *.txt /s /b o:d') do echo %%~tg %%g >> %userprofile%\Desktop\Text_Files.txt

However, this only sorts within the individual directories. I am looking to have the entire list sorted regardless of directory. I know this can be achieved within the Windows GUI as well as in a UNIX/LINUX environment. I would like to accomplish this in batch. For what it's worth, I have also explored forfiles and for/r loops, not to mention PowerShell and VBS.

I know I can easily sort the exported file in another application like Excel, but I would like to know if there is a way to do this in batch.

Thank you for your time.

[Squashman]

What is the date format of your output?

Ultimately what you can do is reformat the date of the file and then pipe everything to the SORT command.

[atfon]

With the example I provided in my initial post, the output to file looks something like the following (with hundreds of files):

09/05/2017 10:57 AM C:\list\file.txt
09/05/2017 01:01 PM C:\list\sub\file2.txt
03/05/2014 03:11 AM C:\mylist\new_file.txt
08/26/2015 04:25 PM C:\mylist\new_file2.txt

With the sort pipe, I have only been able to sort alphabetically. That doesn't seem to help with the date sort I need.

[Squashman]

With the sort pipe, I have only been able to sort alphabetically. That doesn't seem to help with the date sort I need.

Correct. Which is why I said in my previous post that you need to reformat the Date output. The Windows SORT command, sorts character by character. It has no idea that the first 10 characters are a date. So if you reformat the date to YYYYMMDD, and then pipe it to the SORT command it will sort correctly. You will also need to reformat the time as well if you need that included in the sort.

[atfon]

Thanks for the tip, Squashman. I will see what I can come up with that will re-format the date in yyyymmdd that will work regardless of regional settings.

[dbenham]

Have a look at viewtopic.php?t=3250#p15680

I think that is what you are looking for.


Dave Benham

[atfon]

I found this posting, but it seems to be primarily focused on getting the current date and timestamp in the yyyymmdd format:

viewtopic.php?f=3&t=4555&hilit=how+to+get+data%2Ftime+independent+from+localization

There are some good ideas here, but it doesn't work with the "%%~ti" output for existing files. The suggestion was to use 'dir /a-d /tc,' but you then lose the full path information.

[dbenham]

Did you check out the link I gave you? It has two solutions. The first uses WMIC and gives microsecond precision, and is locale agnostic. But it is a bit slow.

The 2nd solution is faster, but only gives minute precision, and is locale dependent.

Both solutions give the most recent file, but you can eliminate the last step in each process to end up with a sorted list of all files.

[atfon]

Thanks, dbenham. I will attempt to incorporate elements of your script. I intend to recurse the entire local hard drive, but just for files of a type, like *.txt.

[dbenham]

Be careful scanning an entire drive.

Some drives have logical folder entries (I forget what they are called) that can lead to endless loops when trying to recursively scan all folders.

[atfon]

Generally speaking, I've done this using a command such as the following without error in test environments:

FOR /R C:\ %%G in (*.txt) ECHO %%~TG %%G

As an addendum to my original posting, I am able to achieve what I need with robocopy, but it is incredibly slow as I'm sure you all know. As the txt files can be in any folder on the system, the recursion needs to start from the root directory.

[Compo]

Here's my suggestion using RoboCopy, (It certainly isn't slow!):

Code: Select all

@For /F "Tokens=*" %A In ('RoboCopy . . *.txt /L /NOCOPY /S /IS /TS /FP /NC /NP /NS /NDL /NJH /NJS^|Sort') Do @Echo(%A

Double up the % characters, (both instances of %A become %%A), if running it within a batch file.

What may be very slow is output due to access denied messages and retries if you are running it from the root of your system drive, (not recommended).

Run this from anywhere within the system drive to see what I mean; you'll need to Ctrl-C it!

Code: Select all

RoboCopy \ . *.txt /L /NOCOPY /S /IS /TS /FP /NC /NP /NS /NDL /NJH /NJS

Here is a possible alternative if you're searching the system drive:

Code: Select all

@Echo Off
SetLocal EnableDelayedExpansion
(For /F "Tokens=*" %%A In ('Where/R \ *.txt'
   ) Do For /F "Tokens=1-5* Delims=/ " %%B In ("%%~ftA"
   ) Do For /F "Tokens=1-2 Delims=:" %%H In ("%%E") Do (
      If /I "%%F"=="PM" (Set/A hh=1%%H+12) Else Set/A hh=1%%H
      Echo=%%D/%%B/%%C !hh:~-2!:%%I %%G))>"tmp.log"
Sort<"tmp.log">"TxtsByDate.txt" && Del "tmp.log"

…this is based upon your previously provided output and not guaranteed to work on any system other than yours due to localisation and PC settings differences.

Of course this would probably be a much easier task utilising another scripting language.

[atfon]

Thanks, Compo

Here's my suggestion using RoboCopy

I appreciate the suggestion, but I was already testing with RobobCopy like this:

for /f "tokens=*" %%a in ('robocopy "\." "\." *.txt /l /s /is /ts /ndl /njh /njs /nc /ns ^| sort /r') do echo %%a >>%userprofile%\Desktop\Text_Files.txt

However, I do need it to recurse from the root of the drive. Unless I can suppress error messages or ignore locations that may produce such errors in some way to speed up this process, it does not appear to be a viable option. The .txt files can be anywhere on the drive and not just on under the local user account.

this is based upon your previously provided output and not guaranteed to work on any system other than yours due to localisation and PC settings differences.

I know how to convert the date format to work in my own localization, but I need this to be localization independent.

Of course this would probably be a much easier task utilising another scripting language.

I agree this would be easier in another scripting language. However, which native language would you recommend that would also be backwards compatible to XP?

[atfon]

I have seen some users mention the parameters /R:0 /W:0 with robocopy can be used to suppress access denied messages. I will try this on some test environments tomorrow.

[Compo]

If you don't mind ignoring the last line of output would this do?

Code: Select all

@Echo Off
WMIC /OUTPUT:"ListIn.tmp" DataFile Where (Drive='%SystemDrive%' And^
 Extension='txt' And Hidden='FALSE' And System='FALSE' And Writeable='TRUE')^
 Get LastModified, Name
Sort<"ListIn.tmp">"ListOut.txt"
Del "ListIn.tmp"