Robocopy Sync Folders

Posted onby

On Windows 10, when you need to copy files to another drive, you typically use the standard select, copy, and paste process. Although this process works perfectly fine, speed becomes a bottleneck as trying to transfer a lot of files can take a very long time.

  1. Powershell Robocopy Sync Folders
  2. Robocopy Sync Folders Download

As an alternative, many technical users use Robocopy (Robust File Copy), a command-line tool built into Windows 10 that provides more features to move data to a different location more quickly.

One particular feature that makes Robocopy special (and often overlooked) is its multi-threaded feature that allows you to simultaneously copy multiple files. Instead of one file at a time using the copy feature built into File Explorer.

In this guide, you will learn to use the multi-threaded copies feature on Robocopy to speed up the transfer process of files and folders to another drive on Windows 10.

SyncSync

@echo off echo To begin backing up data: pause robocopy 'D:' 'K:' /L /v echo. Pause exit And this is what shows up when I double-click on the backup.bat. As you can see, source is defined as d: temp. This is where the batch file is located, but in the batch file I defined it as D. For some reason, the destination is defined correctly. From Wikipedia: Robocopy, or 'Robust File Copy', is a command-line directory and/or file replication command. Robocopy functionally replaces Xcopy, with more options. It has been available as part.

How to use multi-threaded feature with Robocopy

To use the Robocopy multithreaded feature to copy files and folders to another drive faster, use these steps:

  1. Open Start on Windows 10.

  2. Search for Command Prompt, right-click the result, and select the Run as administrator option.

  3. Type the following command to copy the files and folders to another drive and press Enter:

    In the command, make sure to update the source and destination paths and the options. For example, this command copies data from the drive “C” to “D” and uses the “32” threads for copying:

Powershell Robocopy Sync Folders

Robocopy command breakdown

Robocopy has many features, and in the command shown in this guide, we’re using the following switches to make copy reliable and fast.

  • /S — Copy subdirectories, but not empty ones.
  • /E — Copy Subdirectories, including empty ones.
  • /Z — Copy files in restartable mode.
  • /ZB — Uses restartable mode. If access is denied, use backup mode.
  • /R:5 — Retry 5 times (you can specify a different number, default is 1 million).
  • /W:5 — Wait 5 seconds before retrying (you can specify a different number, the default is 30 seconds).
  • /TBD — Wait for sharenames To Be Defined (retry error 67).
  • /NP — No Progress – don’t display percentage copied.
  • /V — Produce verbose output, showing skipped files.
  • /MT:32 — Do multi-threaded copies with n threads (default is 8).

The most important switch to focus on in the above command is /MT, which is the switch that enables Robocopy to copy files in multi-threaded mode. If you do not set a number next to the /MT switch, then the default number will be 8, which means that Robocopy will try to simultaneously copy eight files. However, Robocopy supports 1 to 128 threads.

In this command, we are using 32, but you can set it to a higher number. The only caveat is that the higher the number, the more resources and bandwidth will be used. If you have an older processor, using a high number will affect performance. As a result, make sure to test before executing the command with a high number of threads.

Once you complete the steps, you will notice that copying files and folders will take significantly less time.

You are not limited to copying files and folders to an external or internal drive. This also works to migrate files over the network.

If you need additional help getting Robocopy to work, submit your questions in the Pureinfotech forums.

2009

On numerous occations I’ve had a need for synchronizing directories of files & subdirectories. I’ve used it for synchronizing work files from my stationary PC to my laptop in the pre-always-on era (today I use SVN for almost all files that needs to be in synch). Recently I needed to implement a very KISS backup solution that simply synchronized two directories once a week for offsite storing of the backup data.

While seemingly simple the only hitch was that there would be several thousands of files in binary format, so compression was out of the question. All changes would be additions and deletions - that is, no incremental updates. Finally I’d only have access to the files through a share so it wouldn’t be possible to deploy a local backup solution that monitored for changes and only sent diffs.

I started out using ViceVersa PRO with the VVEngine addon for scheduling. It worked great for some time although it wasn’t the most speedy solution given my scenario. Some time later ViceVersa stopped working. Apparently it was simply unable to handle more than a million files (give or take a bit). Their forums are filled with people asking for solutions, though the only suggestion they have is to split the backup job into several smaller jobs. Hence I started taking backups of MyShare/A, MyShare/B, etc. This quickly grew out of hand as the number of files increased.

Sometime later I was introduced to Robocopy.aspx). Robocopy ships with Vista, Win7 and Server 2008. For server 2003 and XP it can be downloaded as part of theWindows Server 2003 Resource Kit Tools.

Robocopy (Robust File Copy) does one job extremely well - it copies files. It’s run from the command line, though there has been made a wrapper GUI for it. The basic syntax for calling robocopy is:

You give it a source and a destination address and it’ll make sure all files & directories from the source are copied to the destination. If an error occurs it’ll wait for 30 seconds (configurable) before retrying, and it’ll continue doing this a million times (configurable). That means robocopy will survive a network error and just resume the copying process once the network is back up again.

What makes robocopy really shine is not it’s ability to copy files, but it’s ability to mirror one directory into another. That means it’ll not just copy files, it’ll also delete any extra files in the destination directory. In comparison to ViceVersa, robocopy goes through the directory structure in a linear fashion and in doing so doesn’t have any major requirements of memory. ViceVersa would initially scan the source and destination and then perform the comparison. As source and destination became larger and larger, more memory was required to perform the initial comparison - until a certain point where it’d just give up.

I ended up with the following command for mirroring the directories using robocopy:

Robocopy Sync Folders Download

  • /MIR specifies that robocopy should mirror the source directory and the destination directory. Beware that this may delete files at the destination.
  • /FFT uses fat file timing instead of NTFS. This means the granularity is a bit less precise. For across-network share operations this seems to be much more reliable - just don’t rely on the file timings to be completely precise to the second.
  • /Z ensures robocopy can resume the transfer of a large file in mid-file instead of restarting.
  • /XA:H makes robocopy ignore hidden files, usually these will be system files that we’re not interested in.
  • /W:5 reduces the wait time between failures to 5 seconds instead of the 30 second default.

The robocopy script can be setup as a simply Scheduled Task that runs daily, hourly, weekly etc. Note that robocopy also contains a switch that’ll make robocopy monitor the source for changes and invoke the synchronization script each time a configurable number of changes has been made. This may work in your scenario, but be aware that robocopy will not just copy the changes, it will scan the complete directory structure just like a normal mirroring procedure. If there’s a lot of files & directories, this may hamper performance.

I'm the CTO at iPaper where I cuddle with databases, mold code and maintain the overall technical & team responsibility. I'm an avid speaker at user groups & conferences. I love life, motorcycles, photography and all things technical. Say hi on Twitter, write me an email or look me up on LinkedIn.