In most cases where I need to compare two folders recursively on a Windows system I use my go-to tool Beyond Compare. It is an excellent utility, and one that I think should be among the first utilities any developer should install on a new machine.
However, today I was doing a reconciliation as part of a very large file migration project that required comparing two folders that each contained hundreds of millions of files spread across thousands of sub-folders. BC was having a lot of trouble and choked on many of my comparisons. It just wasn’t the tool for today’s job. I needed another solution.
Necessity mothered some invention and I found an inventive way to use a combination of command switches on RoboCopy to perform the comparison. If you are not familiar with RoboCopy, and you do a lot of mass copying of files, you need to stop what you are doing and learn about it pronto. It is a supercharged version of XCopy that has been included with Windows since Vista. It has a ton of great features such as multi-threaded file copying, selectively copying changed files, and resumable copies that make it a must especially for big file copy jobs over flaky network connections.
Diff Command Using RoboCopy
So here’s the command to perform a basic comparison of two folders and write a log file listing the differences.
ROBOCOPY “\\FileShare\SourceFolder” “\\FileShare\ComparisonFolder” /e /l /ns /njs /njh /ndl /fp /log:reconcile.txt
Explanation of the command switches used above:
/e Recurse through sub-directories (including empty ones)
/l Don’t modify or copy files, log differences only
/fp Include the full path of files in log (only necessary if you omit /ndl)
/ns Don’t include file sizes in log
/ndl Don’t include folders in log
/njs Don’t include Job Summary
/njh Don’t include Job Header
/log:reconcile.txt Write log to reconcile.txt (Recreate if exists)
/log+: reconcile.txt (Optional variant) Write log to reconcile.txt (Append if exists)
Usage Notes and Warnings Regarding the /NDL Option
The /NDL option is a handy way to suppress the inclusion of every folder checked (regardless of whether it contains differences) in the log, but there because of the way it works it is not a good idea in all circumstances. Consider the following before you use /NDL.
- Folders that exist only on source or destination are not logged unless at least one mismatched file is present or a source file is missing on destination.
- Folders that exist only on the destination are not logged at all regardless of contents.
If you omit the /NDL option, it is necessary to include the /FP option if you want full paths listed for each file.
Example Output
(with /NDL option)
*EXTRA File c:\dest\log.txt
New File c:\source\newfolder\Blah.txt
Newer c:\source\Files\CONCORD.DAT
New File c:\source\Files\COWCO.DAT
(without NDL Option)
c:\work\test\source\ (extraneous folder listing)
*EXTRA Dir c:\dest\newfolderdest\
*EXTRA Dir c:\dest\newfolderrestempty\
*EXTRA File c:\dest\log.txt
New Dir c:\source\newfolder\
New File c:\source\newfolder\Blah.txt
New Dir c:\source\newfolderempty\
c:\source\Files\ (extraneous folder listing)
Newer c:\source\Files\CONCORD.DAT
New File c:\source\Files\COWCO.DAT
c:\test\source\FilesSame\ (Included despite no diffs)
Filed under: How To..., Technology Tips | Tagged: Compare, DOS, RoboCopy, tips, Windows | 13 Comments »
Improving Software 