Monday 28 February 2011

MGET with delete for FTP

A common problem for FTP users is how to download all the available files from an FTP server and delete them once they are downloaded. You can not simply perform mget *.* followed by mdelete *.* because there may be new files arrived onto the FTP server since your started and you will delete them before they are downloaded.
i.e. bad - potentially deletes files that we havent retrieved yet.
mget *.*
mdelete *.*

There are a number of approaches to solving this problem for example WINSCP grabs a file then deletes it one at a time. I wanted to use the inbuilt Microsoft FTP command in a batch to download the files to our local machine then delete the files that have successfully arrived one at a time from the FTP server but not delete files that I have not retrieved that may have arrived since we started work.
This is how it is done.
1) Create a batch file that gets the files that you want using an ftp script and then call two batch files which create a delete script and then call the delete script, which we will do in steps 3 and 4 below

FTP -i -s:C:\TWC\GetFiles_FTP.txt ftp.XXX.XXX
call C:\Data\createDelete.bat
call C:\Data\Del_FTP.bat


2) The GetFiles_FTP.txt looks like this, a simple mget with your particular login details and path details

username
password
cd remotefolder
bin
mget *.* "C:\DATA\FILES\"
bye

3) This is the magic step in createDelete.bat. The trick here is that we generate an ftp script file on the fly by echoing out the ftp login details to a file along with the output of the Forfiles command. This command loops though a path "-p" (note: no trailing back slash on the path) and for a file mask "-m" it performs a command "-c". I am echoing "del filename" into the script file so we get a list of del commands for each file that we have successfully retrieved onto our local machine. This means we wont delete files that we haven't retrieved.

echo username> C:\Data\deleteFiles.txt
echo password>> C:\Data\deleteFiles.txt
echo cd /remotefolder>> C:\Data\deleteFiles.txt
echo bin>> C:\Data\deleteFiles.txt
Forfiles -p "C:\Data\Files" -m *.* -c "cmd /c echo del @file>> C:\Data\deleteFiles.txt"
echo bye>> C:\Data\deleteFiles.txt

This creates a file similar to this
username
password
cd /remotefolder
bin
del "filename1"
del "filename2" etc
bye
4) Final step is to create Del_FTP.bat which simply calls the FTP command with our newly created script file.
cd C:\Data\
FTP -i -s:C:\Data\deleteFiles.txt ftp.xxx.xxx



Now of course this is all a lot easier with PowerShell, if you are so inclined.