User Tools

(aka 'the best point military institute')

Site Tools

tools:rsync

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
tools:rsync [2025/10/12 04:46] – [rsync] Humphrey Boa-Garttools:rsync [2025/10/12 07:40] (current) – [rsync] Humphrey Boa-Gart
Line 1: Line 1:
 ====== rsync ====== ====== rsync ======
  
-{{wst>expand}}+**rsync** //(or **remote sync**)// is a [[tools:bash|command line]] utility for transferring and synchronizing files between any given //"source"// and //"destination"// pair. It is extremely popular for use in backup utilities, at it can synchronize backups between not only local filesystems, but also over the internet via [[tools:ssh|SSH]].
  
 +While you can technically use the ''cp -R'' command to make quick and dirty backups, ''rsync'' offers far more flexibility for many common real-world scenarios.
 +
 +If you are already familiar with using [[tools:wget|Wget]] to make mirrors of entire websites, then use of ''rsync'' should come to you pretty naturally - it just operates over local filesystems & SSH instead of HTTP/HTTPS/FTP.
  
 ===== Useful Commands ===== ===== Useful Commands =====
Line 11: Line 14:
  
 ''rsync --h'' and ''man grep'' will tell you all the flags which are available to use. We will only detail some of the most important ones below: ''rsync --h'' and ''man grep'' will tell you all the flags which are available to use. We will only detail some of the most important ones below:
 +
 +
 +==== Local Backups ====
 +
 +To have ''rsync'' copy all files from one directory to another, use this command:
 +
 +  $ rsync -av /path/to/source/dir/ /path/to/destination/dir/
 +
 +The ''-a'' flag tells rsync to not only recursively go into subdirectories, but it also ensures all attributes like timestamps & permissions are preserved. ''-v'' is for verbose output, so you can see what ''rsync'' is doing.
 +==== Remote & Network Backups ====
 +
 +''rsync'' supports the use of [[tools:ssh|SSH]] for transferring backups over network connections, using the ''-e'' flag.
 +
 +To back up files from your local shell to a remote machine via SSH:
 +
 +  $ rsync -avz -e ssh /source/directory/ user@remote:/destination/directory/
 +
 +Here we are also using the ''-z'' flag, which compresses the data to speed up transfer.
 +
 +To back up files from a remote machine to your local shell via SSH:
 +
 +  $ rsync -avz -e ssh user@remote:/source/directory/ /destination/directory/
 +
 +In the case of both of these commands, //"local shell"// just means whatever terminal you are running ''rsync'' from. And no, you cannot use ''rsync'' between a remote source and remote destination at the same time:
 +
 +  $ rsync -avz -e ssh user@remote1:/source/directory/ user@remote2:/destination/directory/
 +  The source and destination cannot both be remote.
 +
 +So if you want to sync between two remote machines, you will have to first SSH into one of them and run ''rsync'' from there "locally" with the other machine as the remote.
 +
 +
 +==== Incremental Backups ====
 +
 +If you are regularly backing up the same folders, drive, server, etc, then ''rsync'' needs to be tweaked further with two more flags, or over time your backups are not going to look like the folders you were backing up:
 +
 +  $ rsync -avu --delete /path/to/source/dir/ /path/to/destination/dir/
 +
 +With the ''-u'' //(or ''--update'')// flag, we can tell ''rsync'' to only copy files from the source that do not exist in the destination, or are newer than the same files in the destination, and skip all the others. With the ''--delete'' flag, we can tell ''rsync'' to delete files in the destination that no longer exist in the source.
 +
 +This is the form of ''rsync'' that you want to use in automated backup scripts.
 +
 +If you are looking for an even more failsafe way to ensure the quality of backups, use the ''-c'' flag to have ''rsync'' compare checksums of every conflicting file transfer instead of modification times. This method is a lot slower, but far more accurate when left on autopilot.
 +
 +==== Interrupted Backups ====
 +
 +When ''rsync'' has one of its backup runs interrupted, it just stops. Any files that were finished, you get to keep. Whatever file it was on last, is lost. Just run ''rsync'' again in incremental mode //(see above)// and it will start over on whatever file it was processing when it was interrupted.
 +
 +However, if you're dealing with very large files, you do not want to have to start over. Use the ''-P'' flag if you want to keep partially-downloaded files, so you can resume transfers mid-file instead of between files.
 +
 +
 +==== Excluding Things From Backups ====
 +
 +If there are files/folders that you wish to have ''rsync'' skip over, use the ''--exclude'' flag like this:
 +
 +  $ rsync -av --exclude 'filename' --exclude 'path/' /path/to/source/dir/ /path/to/destination/dir/
 +
 +If you have a lot of exclusions, you can list them all out one-per-line in a text file, and pass it to ''rsync'' with ''--exclude-from'' like this:
 +
 +  $ rsync -av --exclude-from 'exclusions.txt' /path/to/source/dir/ /path/to/destination/dir/
 +
 +You can even exclude things like specific filetypes:
 +
 +  $ rsync -av --exclude='*.ext' /path/to/source/dir/ /path/to/destination/dir/
 +
 +
 +==== More Command Flags ====
 +
 +There are more flags you can combine with these basic rsync commands to fine-tune how your backups are executed. Play around with them on some [[diy:testing-environment|demo directories]] to see how they affect your files!
 +
 +  * Use the ''-n'' or ''--dry-run'' flag in combination with ''-v'' to perform a simulated transfer.
 +  * Use ''--progress'' to see real-time transfer status.
 +  * Use ''--log-file=transfer.log'' to generate a log file of the transfer.
 +  * Use ''--remove-source-files'' to delete the source directory once the transfer is complete.
 +
 +
 +
 +===== What About Automatic Backups? =====
 +
 +''rsync'' does not have a built-in scheduling service. The easiest way to set your rsync-powered backups on autopilot, is with the use of [[tools:cron|crontab]].
  
    
-{{tag>Tools Stubs}}+{{tag>Tools}}
tools/rsync.1760244408.txt.gz · Last modified: by Humphrey Boa-Gart

Find this page online at: https://bestpoint.institute/tools/rsync