Fastest copy for multiple gigs?

Discussion forum for all Windows batch related topics.

Moderator: DosItHelp

Post Reply
Message
Author
SIMMS7400
Posts: 546
Joined: 07 Jan 2016 07:47

Fastest copy for multiple gigs?

#1 Post by SIMMS7400 » 04 Mar 2018 05:43

.................................................................
Last edited by SIMMS7400 on 29 Sep 2018 07:40, edited 1 time in total.

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Fastest copy for multiple gigs?

#2 Post by aGerman » 04 Mar 2018 06:45

ROBOCOPY should be absolutely okay. Depends on what your settings are, the limits of your network and hardware, what your backup strategy is. Since you didn't show us your ROBOCOPY command we can only guess.

- ROBOCOPY already works multithreaded (8 threads by default). You may consider to change the number of threads using /MT
- Of course multithreading doesn't help much if you have slow HDDs or a slow network in place. If that is your bottleneck then also other programs can't be more quick.
- ROBOCOPY always tries to copy files multiple times if it was not able to copy them in the first place. By default it tries a million times and waits 30s in between. Have a look at your /R and /W settings.
- Consider to copy new files and files that have been changed only. ROBOCOPY will do that by default, provided you copy to the same destination. It may be sufficient to save a full backup once a week or once a month beside of that.

Steffen

elzooilogico
Posts: 128
Joined: 23 May 2016 15:39
Location: Spain

Re: Fastest copy for multiple gigs?

#3 Post by elzooilogico » 04 Mar 2018 07:45

Are you running it from task sheduler?

If the answer is no, skip this post. If yes, I've faced the same copying up a database of 3Gb from NAS Storage to an alternative server to perform a RESTORE over it.

Both systems are Windows Server 2008R2. Either XCOPY or ROBOCOPY take more than an hour to copy the database backup file, so as it grows, I'd to adjust the restore task run time to wait until copy task ends. When I run copy task interactive, the copy is performed in more or less 12 minutes.

By default tasks are created with a priority level of 7. Thats is BELOW_NORMAL_PRIORITY_CLASS, whereas 4 is NORMAL_PRIORITY_CLASS. Aha!!!
So I went to System administration, task Sheduler, and export the task to an XML file. Above the Action section, I found <Priority>7</Priority>. I changed it to <Priority>4</Priority>,delete the task and import the task XML file, Voila!!. Now copy task runs at the same speed as interactive.

see TaskSettings.Priority property

Hope it helps.

SIMMS7400
Posts: 546
Joined: 07 Jan 2016 07:47

Re: Fastest copy for multiple gigs?

#4 Post by SIMMS7400 » 04 Mar 2018 15:12

............................................
Last edited by SIMMS7400 on 29 Sep 2018 07:39, edited 1 time in total.

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Fastest copy for multiple gigs?

#5 Post by aGerman » 05 Mar 2018 05:40

I already gave you some hints but of course we don't work with the same environment as you. You should study the options that ROBOCOPY provides and then do some tests in order to find the settings that fit best for your tasks.

Steffen

SIMMS7400
Posts: 546
Joined: 07 Jan 2016 07:47

Re: Fastest copy for multiple gigs?

#6 Post by SIMMS7400 » 05 Mar 2018 05:54

....................
Last edited by SIMMS7400 on 29 Sep 2018 07:40, edited 1 time in total.

aGerman
Expert
Posts: 4678
Joined: 22 Jan 2010 18:01
Location: Germany

Re: Fastest copy for multiple gigs?

#7 Post by aGerman » 05 Mar 2018 06:34

SIMMS7400 wrote:
05 Mar 2018 05:54
Would it be best ot replace all instances with ROBOCOPY?
Yes most likely. Not only that XCOPY doesn't work with multi-threading to increase the performance. It also doesn't retry if files were not copied correctly.

Steffen

Post Reply