Search large folders, copy found files, paste to new folder?

[Bobbo Jones]

HI all! This question has come up before here : viewtopic.php?f=3&t=3285

I need to search a folder of 10,000 plus .wav files for 20 different files, 10 times a day. It gets tedious to say the least! If I could simply make a .txt file with the needed filenames, and run a .bat file to do the finding for me in the background, I could focus on more important elements of my task, like actually listening to the .wavs!

my file names will in 99.9% of cases be correct. the thread which already exists on this topic seems to have 10% incorrectly named file names, so has a way of discerning this built into the script. I do not need this element.

On this forum, and 1 other, I have found scripts which are designed to solve my problem.

Code: Select all

@echo off
setlocal EnableDelayedExpansion
cd /D "%~DP1"
set newFolder=%~N1
if not exist "%newFolder%" ECHO md "%newFolder%"
if exist "%newFolder%.log" del "%newFolder%.log"
set cutNumber=
set /P firstLine=< "%~NX1"
if "%firstLine:~0,1%" equ "1" (
   if "%firstLine:~1,1%" equ "." (
      set "cutNumber=. "
   ) else (
      set "cutNumber=- "
   )
)
for /F "usebackq delims=" %%a in ("%~NX1") do (
   set "fileName=%%~Na"
   if defined cutNumber (
      set "fileName=!fileName:*%cutNumber%=!"
   )
   if exist "!fileName!.doc" (
      ECHO copy "!fileName!.doc" "%newFolder%"
      echo File "!fileName!.doc" copied
   ) else (
      echo/
      echo File not found: "!fileName!.doc"
      call :getFileList "!fileName!"
      if !n! gtr 0 (
         if !n! equ 1 (
            set i=1
         ) else (
            set i=0
            set /P "i=Enter the desired file number: "
         )
         if !i! gtr 0 (
            call :setElem name=name[!i!]
            ECHO copy "!name!" "%newFolder%"
            echo File "!name!" copied
         ) else (
            echo FILE NOT COPIED
            echo File not found: "!fileName!.doc">> "%newFolder%.log"
         )
      ) else (
         echo I can't find any similar file...
      )
   )
)
if exist "%newFolder%.log" (
   echo/
   echo/
   echo THESE FILES WERE NOT COPIED:
   type "%newFolder%.log"
)
PAUSE
goto :EOF

:getFileList
set n=0
set "name=%~N1"
:cutName
   set "name=%name:~0,-1%"
   if not defined name exit /B
   if not exist "%name%*.doc" goto cutName
for %%a in ("%name%*.doc") do (
   set /A n+=1
   set "name[!n!]=%%a"
   echo     !n!- %%a
)
exit /B

:setElem
set %1=!%2!
exit /B

this was made my Aacini on this forum.

elsewhere, I found

Code: Select all

@echo off
rem Text file with filenames
rem To be prompted for this change to  'set /p LIST=Path to text file: '
rem Or 'set LIST=%1' to run from cmd line like, filename.bat "D:\List.txt"
set LIST="D:\List.txt"

rem Path to the files to copy
rem Can be change the same as LIST
rem Or 'set FILESPATH=%2' to run like , filename.bat "D:\List.txt" "D:\Test"
set FILESPATH="D:\Test"

rem Destination path can be added here. eg set DESTPATH=C:\New Folder
rem If set change xcopy cmd to "%DESTPATH%\%DEST%\*"
rem Do not use quotes when setting DESTPATH

rem Set Destination folder based on LIST name
for %%i in (%LIST%) do set DEST=%%~ni

rem Read LIST and call :COPY_FILES for each line passing the line of text
for /F "usebackq delims==" %%i in (%LIST%) do (call :COPY_FILES "%%i")

exit

:COPY_FILES
rem Destination folder created in same location the batch file is run from
xcopy /qv %FILESPATH%\%1 .\%DEST%\*

created by 'Duzzy'
and again,

Code: Select all

@echo off

set LISTFOLDER=D:\List Files
set FILESPATH=D:\Test
set DESTPATH=D:\New Folder

for /f "tokens=*" %%i in ('dir /b ^"%LISTFOLDER%\*.txt^"') do (call :COPY_FILES "%LISTFOLDER%\%%i")

pause
exit

:COPY_FILES
for %%i in (%1) do set DEST=%%~ni
for /f "usebackq delims==" %%i in (%1) do xcopy /qv "%FILESPATH%\%%i" "%DESTPATH%\%DEST%\*"

also by duzzy.

I only learnt that this coding existed a few days ago. I have spent 8 hours watching tutorials trying to understand it, and am only finding basic ones, or little games.
What I need to learn is how to apply one of these scripts to MY context.
lets call my .txt file with the 20 .wavs 'breaks filenames world.txt'
the folder to search in is Z:\2011-MASTERS
The folder I would want them to go in, can be a new folder created by the script, that appears on the desktop

Hopefully my question is clear, if there is any info I have not provided that I need to please let me know! thanks in advance for any help! I would also be interested in some good tutorials which teach how I could create a script for this myself.
cheers all
Evan

[abc0502]

Forget about bill

You first have to set the four variables at the beginning of the batch then run it,

The list will have to contain the names of the files only without the extension, and each file name in separate line.

The batch will take one name at the time and search in the folder you set and all sub-folders for files with the "same" name, if it exist before, it will create the destination folder if not exist and before copying it, will check to see if the file exist before, if so will append a random number to the end of the file name, and all files that copied in the same session will have the same random number.

Code: Select all

@Echo Off & Mode 50,13 & Color 0E

:: source = Path to the file that contain the names list "names without the extension"
:: type = the type of the Files that is being searched for "without the dot . befor it"
:: loc = the location to search in
:: destination = the path to the distenation folder even if it is not exist

set "source=%userprofile%\desktop\list.txt"
set "type=py"
set "loc=D:\Test"
set "destination =%userprofile%\desktop\Auto-Search"


For /F "tokens=* delims=" %%A in ('Type "%source%"') Do (
   For /F "tokens=* delims=" %%B in ('DIR /B /S /A:-D "%loc%\*.%type%"') Do (
      IF "%%~nxB" EQU "%%A.%type%" (
         IF not Exist "%destination%" MD "% destination%" >nul
         IF not Exist "%destination%\%%~nxB" ( Copy /b "%%B" "%destination%\%%~nxB" >nul
            ) Else ( Copy /b "%%B" "%destination%\%%~nB_%RANDOM%.%type%" >nul )
      )
   )
)
Echo.&Echo.&Echo.&Echo.&Echo.&Echo              All Done !
pause >nul

[foxidrive]

Don't forget to use /b in the copy command for a binary copy. Windows is cleverer now and will use binary mode for some files but I don't think you can be certain and you should always specify it for a binary file.

Also, the %random% number will always be the same number, as delayed expansion isn't being used.

[abc0502]

@ Foxidrive, i enabled the delayed expansion and tested the !RANDOM!, in the first run when there is no old files. it work find and generate random numbers.

but when running the second time the files are multiplied, if there was one file and the total list has three names, it create three copies and same in the 3rd time you run the batch.

I added the /b switch up, but the repeated files name problem was't fixed yet.

[Bobbo Jones]

Brilliant, thanks!
I will check this first thing at work tomorrow. Im not sure i understand completely how to make it work, but it looks like your instructions are clear. Will let you know how I go!
thanks again!
evan

[Bobbo Jones]

OK so when I run it, it is showing the message

"Environment variable wav not defined
File Not Found
File Not Found
File Not Found
File Not Found


All Done !"

here is exactly what I entered.

Code: Select all

@Echo Off & Mode 50,13 & Color 0E

:: source = Path to the file that contain the names list "names without the extension"
:: type = the type of the Files that is being searched for "without the dot . befor it"
:: loc = the location to search in
:: destination = the path to the distenation folder even if it is not exist

set "source=C:\Users\Evan\Desktop\breaks filenames.txt"
set "wav"
set "loc=C:\Users\Evan\Desktop\location folder"
set "destination =C:\Users\Evan\Desktop\Collating folder"


For /F "tokens=* delims=" %%A in ('Type "%source%"') Do (
   For /F "tokens=* delims=" %%B in ('DIR /B /S /A:-D "%loc%\*.%type%"') Do (
      IF "%%~nxB" EQU "%%A.%type%" (
         IF not Exist "%destination%" MD "% destination%" >nul
         IF not Exist "%destination%\%%~nxB" ( Copy /b "%%B" "%destination%\%%~nxB" >nul
            ) Else ( Copy /b "%%B" "%destination%\%%~nB_%RANDOM%.%type%" >nul )
      )
   )
)
Echo.&Echo.&Echo.&Echo.&Echo.&Echo              All Done !
pause >nul

[Bobbo Jones]

woops I hadn't done this correctly

:: type = the type of the Files that is being searched for "without the dot . befor it"

I have corrected this, and now it simple says

" the system cannot find the file specified
All Done !"

[Bobbo Jones]

WOOO I got it working... just by removing a spacebar!

thanks!!

[Bobbo Jones]

Hmm so its slow. It is searching a 125 gb folder with 220 sub folders for 19 files. It did it correctly, but took 21 minutes.

the thing is, all of the files were actually in 2 of the sub folders. I wonder if this could be a piece of info that could help speed it up?

It still worked perfectly though! this is great

[Bobbo Jones]

OK I found an error.

I created a text file with 13 files, it returned only 12 and gave the message All Done !
Ideally it would finish and say All Done, except for missing files "insert file name"

[Bobbo Jones]

I will need to summarise all this!

I have tried to run two batch files at once and it is a faster way of doing it. Another way, is to put all of my days requirements in the one .txt, and then sort out the files I need. I am currently trying this with a .txt with 150 files, and some dummy files named 'breaker 1' 'breaker 2' etc sitting in between the files i need to keep separate. A problem here will be if the folder automatically sorts them into alphabetical order, instead of the order they were copied in.

in this case, another addition that would help, would be to add a number at the start of the file name? is that possible? So if I add 100 filenames in the .txt doc, I can run a longer search, but when it copy/pasted the files into the new folder, it would add a 01/02/03 etc at the start of each, meaning I can grab them in order and sort them out to where they need to go.

[foxidrive]

Try this: list.txt in the 2nd line contains your filenames, one per line.

It creates a folder on the desktop called "copied files" and puts the files inside.

Run it in the root folder where you want to search from.

Code: Select all

@echo off
set "list=list.txt"

setlocal EnableExtensions EnableDelayedExpansion

dir /a:-d /o:n /b /s >filelist.tmp

del list2.tmp 2>nul
del notfound.txt 2>nul
set c=0

for /f "delims=" %%a in ('type "%list%"') do echo finding "%%a"& findstr /r /i /c:"\\\%%a$" filelist.tmp >>list2.tmp || >>notfound.txt echo %%a

md "%userprofile%\desktop\copied files" 2>nul

for /f "delims=" %%a in ('type "list2.tmp"') do (
set /a c=c+1
set num=000!c!
set num=!num:~-3!
echo copying "!num! %%~nxa"
copy /b "%%a" "%userprofile%\desktop\copied files\!num! %%~nxa" >nul
)

del filelist.tmp 2>nul
del list2.tmp 2>nul
echo.
if exist notfound.txt echo Check NOTFOUND.TXT for filenames that were missing
echo.
echo done
echo.
pause

[Bobbo Jones]

Hmm im not sure i understand.

I ran this, and it created a .txt of all the files in my list. I seem to have applied it incorrectly

[foxidrive]

Place the batch file and the list.txt in Z:\2011-MASTERS

inside list.txt you would have files like this:

matrix.reloaded abc.srt
document xyz.doc
setup irfanview.exe

Then run the batch file and check your desktop for "copied files" folder.
Watch to see what it writes to the screen.


BTW, the first thing it does is create a list of the files in Z:\2011-MASTERS and below. If you have a million files then expect it to take a little while to create the list.

[Bobbo Jones]

Ok,, so ive tried this,

It runs through all the files in the dos window, and then says
"check NOTFOUND.TXT for filenames that were missing
done
press any key to continue"

it then creates a 'copied files' folder, and quits.

The NOTFOUND.TXT seems to have simply copied all the files in my List.txt file
The copied files folder remains empty.

This all takes 5 seconds.