Search large folders, copy found files, paste to new folder?

[abc0502]

@Bobbo Jones,
Foxidrive code is working, you must missed something,
first make a list of the files names with it's extension and put them in a txt file with the name "list.txt"
now take the batch and the list.txt and put them in the main folder where your files is, then run the batch.

and as foxidrive explained, it will create a list of all your files that exist under tha main folder then search for the names of the files in your list and when find them create a new list with there location, and finally copy then to the folder on your desktop, and any file that it couldn't find, will exist in the NOTFOUND.txt file in the same folder woth the batch.

This way is alot faster than mine where it keep going over and over on the same files every time it search for a file.

[foxidrive]

The NOTFOUND.TXT seems to have simply copied all the files in my List.txt file
The copied files folder remains empty.

Well, it would seem that there are no matches for the filenames in Z:\2011-MASTERS and below.

Or you did not put list.txt and the batch file in Z:\2011-MASTERS

Is that the case?

[foxidrive]

I'm curious. Did you get this to work?

[Bobbo Jones]

OK, yes it does seem to be working, but its going to take a long time. the folder has 1.2 terrabytes in it.

I had been trying the script in a different folder, and it didnt work there. the folder was only 125 gb large. I did not change anything, all I did was drag the list.txt and the batch file in. The list contained files that I knew were in that folder. Which is what I have done with the list.txt which is currently running. except I have added one file which does not exist.

[Bobbo Jones]

Actually no. Its doing the exact same thing, its just taking a lot longer at the start due to the folder size.

SO just now, all it did was think for a long time, then it said 'finding' with each file in the 'list.txt' after it per line,
then it said 'Check NOTFOUND.TXT for filenames that were missing
and
done.

but the NOTFOUND.TXT has returned every file the was in the list.txt, and I KNOW that most if not all of them are actually in the folder.

[Bobbo Jones]

oh and it also created a folder called 'copied files' on the desktop which remained empty

[foxidrive]

Show me a sample of what you are entering in the list.txt please.

[Bobbo Jones]

AH I know what ive done wrong, its working perfectly now!

In the first script you sent me, there was this line ':: source = Path to the file that contain the names list "names without the extension"'

I interpreted this to mean remove the extension, which I had continued to do with all my lists. woops!

SO its working, Its 11.30 am here in melbourne, ill use it for the day and let you know how I go.

[Bobbo Jones]

Its working very well. So fast!

one potential loose end, what if there are duplicates of a found file? OR a different file with the exact same file name?

[Bobbo Jones]

Yep I just had a file appear twice. This can only be fixed manually by listening to both files and ensuring they are the same and deleting one,
it would be perfect if it warned you if more than one file of the same name is found though.

[foxidrive]

This will alert you if a filename is repeated and provide a list of repeated filenames, and a list of all the files with paths so you can identify the places they came from.

Tested but not thoroughly.

Be aware that filenames/paths containing ^ % ! characters may result in issues.

Code: Select all

@echo off
set "list=list.txt"

setlocal EnableExtensions EnableDelayedExpansion

dir /a:-d /o:n /b /s >filelist.tmp

del list2.tmp 2>nul
del notfound.txt 2>nul
del samefile.txt 2>nul
set c=0

for /f "delims=" %%a in ('type "%list%"') do echo finding "%%a"& findstr /r /i /c:"\\\%%a$" filelist.tmp >>list2.tmp || >>notfound.txt echo %%a

md "%userprofile%\desktop\copied files" 2>nul

set "name="

for /f "delims=" %%a in ('type "list2.tmp"') do (
if /i "%%~nxa"=="!name!" >>"samefile.txt" echo Same filename - "!name!"
set /a c=c+1
set num=000!c!
set num=!num:~-3!
echo copying "!num! %%~nxa"
copy /b "%%a" "%userprofile%\desktop\copied files\!num! %%~nxa" >nul
set "name=%%~nxa"
)

del filelist.tmp 2>nul
if not exist "samefile.txt" del list2.tmp 2>nul
echo.
if exist notfound.txt echo Check NOTFOUND.TXT for filenames that were missing
if exist "samefile.txt" (
ren "list2.tmp" "samefile.list.txt"
echo Check "samefile.txt" and "samefile.list.txt" for the list of repeated files
)
echo.
echo done
echo.
pause

[Bobbo Jones]

This is great. Thanks mate!

[Bobbo Jones]

Hmm how would one change this script to delete duplicates automatically?

[Bobbo Jones]

To organise my files best, I have been running some searches with lists of about 1000 filenames, and each seems to have at least one duplicate. Its creating folders too large, ie bad use of disk space. So an option to simply delete duplicates might actually be most viable.

Once I have the folder with 1000 files, I can then use that as the root folder in the future when searching for 20 or so files, and have a fast system. Its worth the work to create these large folders with the large lists, just the duplicates are the problem

Thanks for your time
Evan

[Bobbo Jones]

Ah.. I have just realised the error of my plan. using the search tool twice will not work as the tool renames the files with 3 digits at the start.
OK.
So I need a version that does not rename the files, and deletes duplicates,
and I can use the original version which DOES rename the files and inform of duplicates.