Move from FTP and count in Target folder automated

[abc0502]

Also, The error is one of these , missing/mis-located FILE.lst1 file, Or Missing/mis-located server_list.lst,
Or problem happen when writing the files names in the commands_2.ftp.

Can you describe the locations of the files, like where you put the star.bat, code.bat, file.lst1, ftp.set .. etc.

[abc0502]

I re-read your old posts again, it was downloading ok, when this code exist:

Code: Select all

:: get the all file names that dosn't exist in the local folder and put then in ftp command file 2 to be downloaded
For /F "tokens=* delims=" %%a in ('Type "%temp%\%~7\server_list.lst"') Do (
   IF not exist "%~5\%%a" echo get %%a >>"%temp%\%~7\commands_2.ftp"
   )

after changeing it to:

Code: Select all

For /F "delims=" %%a in ('Type "%~1"') Do (
   For /F "delims=" %%b in ('Type "%temp%\%~8\server_list.lst"') Do (
   Set "sf=%%b"
   setlocal enabledelayedexpansion
      IF "%%a"=="!sf:~1,-1!" (
         IF NOT EXIST "%~6\%%b" echo get %%b>>"%temp%\%~8\commands_2.ftp"
      )
   endlocal
   )
)

to compare file names the errors starts, but the batch working for me as it is now, so here is what i'm doing, compare to what you do and tell me the differences.

Start.bat
Code.bat
FTP.set
FILE.lst1

The are all in a folder on my desktop "FTP project", & the local folder is on my desktop too "1234".
I'm testing on a drivehq ftp account, and these is the temp files:
commands_1.ftp

open ftp.drivehq.com
username
password
ls
disconnect
Bye

dump.log

ftp> User (ftp.drivehq.com:(none)): open ftp.drivehq.com
ftp>

1.7z

2.7z

test.txt

ftp> ls
ftp> disconnect
Bye

server_list.lst

"1.7z"
"2.7z"
"test.txt"

commands_2.ftp

open ftp.drivehq.com
username
password
lcd "C:\Documents and Settings\admin\desktop\1234"
get "1.7z"
get "2.7z"
disconnect
Bye

FILE.lst1

1.7z
2.7z

compare this to your files.

[foxidrive]

I hope you aren't too frustrated abc0502. I think you deserve several hundred dollars for all your efforts

Just to comment here on a possible issue - I noticed that aaksar has very poor skills when editing files - which you can see in a previous thread where he edited two filenames but added enters where they should plainly not be and broke the script. It's possible that there is some finger trouble causing issues in this task.

[aaksar]

here is the output:
command1.ftp

Code: Select all

open ftp.us.abc.com
Ftpabc0013
^abcs3
cd "2 abc Tables"
ls
disconnect
Bye

command2.ftp

Code: Select all

open ftp.us.abc.com
Ftpabc0013
^abcs3
lcd "D:\folder\test"
cd "2 abc Tables"
disconnect
Bye

dump.log

Code: Select all

ftp> User (ftp.us.abc.com:(none)): open ftp.us.abc.com
ftp>

ftp> cd "2 abc Tables"
NO_ABC_abc.TXT
NO_XYZ_xyz.TXT
ftp> ls
ftp> disconnect
Bye

server_list.lst

Code: Select all

"NO_ABC_abc.TXT"
"NO_XYZ_xyz.TXT"

there is no space after "(double quotes ) in server_list.lst file

File.lst1

Code: Select all

NO_ABC_abc.TXT
115A_asdf
302A_12345

[aaksar]

all my files in "D:\folder"
and the names are
code.bat
start.bat
File.lst1
ftp.set

in start.bat file, i have given my folder location like

Code: Select all

('Type "D:\folder\FTP.set"')

Code: Select all

start "%%B" D:\folder\Code.bat

after matching ur files, i hv found command2.ftp contents are not matching.

[aaksar]

one more thing in start.bat file i have given location of the files(ftp.set and code.bat, but for file.lst1 i haven't given any location where i need to give the location for file.lst1, as without giving the path it was not working previously in start.bat
here is my code.bat

Code: Select all

@ECHO OFF

:: Don't Forget to Specify the Zipped file type in line 18

::"file_list" --------------------> %~1
::"remote_server" ----------------> %~2
::"remote_server_username" -------> %~3
::"remote_server_password" -------> %~4
::"total_number" -----------------> %~5
::"local_folder" -----------------> %~6
::"remote_folder"  ---------------> %~7
::"FTP servers temp folder" ------> %~8

::Create Local Folder if not exist
Echo Creating LOCAL FOLDER
IF not Exist "%~6" MD "%~6" >nul

:loop
set "zip_program=%programfiles%\7-Zip\7z.exe"
set "zipped=zip"

:: FTP commands to get list of files from the server
Echo Creating Commands_1.ftp file
IF "%~7"=="" (
   (
   Echo open %~2
   Echo %~3
   Echo %~4
   Echo ls
   Echo disconnect
   Echo Bye
   )>"%temp%\%~8\commands_1.ftp"
) Else (
   (
   Echo open %~2
   Echo %~3
   Echo %~4
   Echo cd "%~7"
   Echo ls
   Echo disconnect
   Echo Bye
   )>"%temp%\%~8\commands_1.ftp"
   )

:: start FTP session and log output
Echo Connicting to FTP server to get files list
Ftp -v -i -s:"%temp%\%~8\commands_1.ftp" >"%temp%\%~8\dump.log"

:: extract file name from the ftp dump file
Echo Processing dump.log File and creating server_list.lst
For /F "delims=" %%a in ('Type "%temp%\%~8\dump.log"') Do (
   Echo "%%a"|findstr /I /V /R ^ftp^>|findstr /I /V /R ^bye>>"%temp%\%~8\server_list.lst"
   )

:: 2nd FTP commands to download only new files that dosn't exist locally [Part1]
Echo Creating commands_2.ftp File (part1)
IF "%~7"=="" (
   (
   Echo open %~2
   Echo %~3
   Echo %~4
   Echo lcd "%~6"
   )>>"%temp%\%~8\commands_2.ftp"
) Else (
   (
   Echo open %~2
   Echo %~3
   Echo %~4
   Echo lcd "%~6"
   Echo cd "%~7"
   )>>"%temp%\%~8\commands_2.ftp"
   )

:: compare files that will be downloaded with the file.list and if there is a match
:: it see if this match isn't in the local folder so it can download that file. [Part2]
Echo Creating commands_2.ftp File (part2)
For /F "delims=" %%a in ('Type "%~1"') Do (
   For /F "delims=" %%b in (%temp%\%~8\server_list.lst) Do (
   Set "sf=%%b"
   setlocal enabledelayedexpansion
      IF "%%a"=="!sf:~1,-1!" (
         IF NOT EXIST "%~6\%%b" echo get %%b>>"%temp%\%~8\commands_2.ftp"
      )
   endlocal
   )
)

:: Add disconnect & bye commands to 2nd FTP command file [Part3]
Echo Creating commands_2.ftp File (part3)
(
Echo disconnect
Echo Bye
)>>"%temp%\%~8\commands_2.ftp"

:: start FTP session and download new files
Echo Connecting to FTP server to download files
Ftp -v -i -s:"%temp%\%~8\commands_2.ftp"

:: Check files count
Echo Counting files in local folder
set Fcount=0
For /F "delims=" %%a in ('Dir /B "%~6\*.%zipped%"') Do set /a Fcount += 1
echo %Fcount%
pause
:: Check Condition to Exit or Continue Looping
Echo Check count number
Setlocal EnableDelayedExpansion
IF "%Fcount%" EQU "%~5" (
   Echo Unzipping
   "%zip_program%" x -y "%~6\*.%zipped%" -o"%~6" >nul
   Echo Deleting zip files
   Del /F /Q "%~6\*.%zipped%" >nul
   Echo Removing temp folders
   RMDIR /S /Q "%temp%\%~8" >nul
   Goto :EOF
) Else (
   Echo deleting server_list.lst file
   Del /F /Q "%temp%\%~8\server_list.lst" >nul
   :: Wait 5 min
   Ping Localhost -l 1 -n 300 >nul
   Goto :loop )
pause

[aaksar]

start.bat

@Echo OFF
Title FTP Main Download Script

:: To prevent errors of existing directory when a previous run of this batch
:: wasn't completed successfully, so create a new one instead
set C1=0
set C2=%RANDOM%
setlocal enabledelayedexpansion
For /F "skip=1 tokens=1-7 delims=;" %%A in ('Type "D:\folder\FTP.set"') Do (

set /a C1 += 1
set "file_list=%%A
set "remote_server=%%B"
set "remote_server_username=%%C"
set "remote_server_password=%%D"
set "total_number=%%E"
set "local_folder=%%F"
set "remote_folder=%%G"

IF exist "%temp%\!C2!" (
Set /a C2 += 51
MD "%temp%\!C2!" >nul
) Else ( MD "%temp%\!C2!" >nul )

start "%%B" D:\folder\Code.bat "!file_list!" "!remote_server!" "!remote_server_username!" "!remote_server_password!" "!total_number!" "!local_folder!" "!remote_folder!" "!C2!"
)

[abc0502]

@Foxidrive, Thanks for your replay

Just to comment here on a possible issue - I noticed that aaksar has very poor skills when editing files - which you can see in a previous thread where he edited two filenames but added enters where they should plainly not be and broke the script. It's possible that there is some finger trouble causing issues in this task.

That gave me an idea, i will just upload my files and pm aaskar with the link, then he will just have to replace the FILE.lst1 and the FTP.set with his cause as you said it seems there is something wrong happen here, the batch work with me, but it give him errors.


@aaksar, i will pm you with the link, just unzip the folder FTP Project , put it any where and then modify the FILE.lst1 and the FTP.set to match your settings then save and test.

[aaksar]

now i am able to copy files from FTP to my local. the issue was full file path for file.lst1 file. i dont understand if all the files in same folder then why it asks for full path.

Code: Select all

:: Check Condition to Exit or Continue Looping
Echo Check count number
Setlocal EnableDelayedExpansion
IF "%Fcount%" EQU "%~5" (
   "%zip_program%" x -y "%~6\*.%zipped%" -o"%~6" >nul
   Del /F /Q "%~6\*.%zipped%" >nul
   RMDIR /S /Q "%temp%\%~8" >nul
   Goto :EOF

now i have tested with 6 files and all the files copied to specified location, but it still looping. its not coming out from script. seems like problem in above code its counting the zip file, and when i m giving the zip file it count and out of the loop

[abc0502]

that's ok, change this line :

Code: Select all

For /F "delims=" %%a in ('Dir /B "%~6\*.%zipped%"') Do set /a Fcount += 1

with this:

Code: Select all

For /F "delims=" %%a in ('Dir /B "%~6\*.*"') Do set /a Fcount += 1

BTW, can you test this for me, instead of giving the full location of the FILE.lst1 and FTP.set file and Code.bat try adding this
"%CD%\FTP.set"
"%CD%\FILE.ls1"
"%CD%\FTP.set"
and see if this work, if so, i will post modify the batch files so it just take the file names without the full locations so it work in general without the need of modifying the locations every time you change the location of that files

[aaksar]

Will check tmrw
*.* will work for the files which has no extension...?

[abc0502]

yes, the mistake was just counting the zipped files only without any other files, as i was just testing on 7z file only
IF %CD% works, you will only have to provide the names of the files without the need to provide the full location

[aaksar]

%CD% is not working, it still ask for full path,

and 1 more question, suppose i have given file count 3 is ftp.set file
but in File.lst file i have mentioned only 2 files name,
in that case i m seeing it goes into the loop and download the 2 files everytime.

is it correct?

coz command2.ftp running with get command and everytime it runs it fetch the previously copied file also .

pls correct me if i m wrong.

and *.* is working fine.

[abc0502]

you are right, the file that is in the FILE.lst is the one that it will be downloading from that server, so it keep looping till it finish the download of all of them.

i m seeing it goes into the loop and download the 2 files everytime.

it doesn't download them every time, it just skip them if they are already downloaded

[aaksar]

Ok, so seems like script is done.
If we can put if condition when we r deleting/ unzip the zip files,
So it wont give error zip file is not there,

Otherwise script looks gud to me,
Thanks buddy for your great help