top of page

What is Rsync?

Oct 24
13 min read
What is Rsync?

 

Rsync is a file-copying utility that can update files and directory trees locally or between systems. It can skip files that appear unchanged and, when its delta-transfer algorithm is used, reconstruct an updated file by reusing matching data already at the destination. That can reduce the data needed for repeated transfers.

 

Two decisions are involved: which files need updating, and how to transfer the files selected. Rsync's default size-and-modification-time check handles the first decision. Delta transfer handles the second. Confusing them leads to mistakes such as assuming that a successful run proves every destination file was checked byte for byte.

 

Rsync is normally used for one-way copying, mirroring and backup workflows. It does not automatically resolve edits made independently on both sides, preserve recovery history or encrypt every connection. Those outcomes depend on the command, transport and wider system around it.

 

Where rsync fits

 

Common uses include:

 

  • Copying a project directory to another machine.

  • Maintaining a published download mirror.

  • Refreshing a second copy of a NAS dataset.

  • Sending a consistent backup dataset off site.

 

A first transfer may have little existing data to reuse. Later runs can benefit when much of the source still matches the destination.

 

Rsync can operate entirely between local paths. For a remote-shell transfer, it starts an rsync process at the other end, commonly through SSH. A direct daemon connection instead accesses an rsync service and its configured modules. The two forms have different authentication, encryption and access arrangements.

 

Transfer form

What is involved?

Important requirement

Local paths

Rsync copies within the local system's filesystem view

Correct paths, permissions and destination capacity

Remote shell, commonly SSH

Rsync processes communicate through the selected remote shell

Rsync available at both ends and permission to run the remote process

Direct rsync daemon

A client connects to a configured module

Appropriate module access and a separate decision about transport protection

 

Using SSH provides a protected transport when the server identity and authentication are correctly handled. Rsync over SSH is still rsync. It does not become an SFTP transfer simply because both can use SSH.

 

An SFTP account may permit only file operations through the SFTP subsystem. That does not necessarily permit starting the remote rsync process. Check the actual service instead of assuming that any account accepting an SSH connection can run rsync.

 

How rsync decides what to copy

 

First decision: does the file need an update?

 

By default, rsync uses a quick check based on file size and last-modified time. If those values agree, it normally skips transferring that file's contents. Requested metadata changes can still be applied separately.

 

This avoids reading every unchanged file in full, but matching size and time are not proof of matching bytes. A program can alter data without changing its length, and a timestamp can be preserved or restored. Our test deliberately creates that situation below.

 

The --checksum option, also written -c, changes the pre-transfer selection check. For files of matching size, it compares content checksums rather than relying on modification time. This requires reading file data at both ends and can add substantial disk work even when nothing needs transferring.

 

The official rsync manual distinguishes that selection check from the whole-file checksum normally used to verify reconstruction of a file that is actually transferred. Adding -c does not simply “turn on delta transfer” or enable a previously absent verification of transferred files.

 

The checksum guide explains why the algorithm, scope and reference matter. Rsync's transfer checks are not a substitute for authenticating a remote server, establishing a file publisher's identity or checking that you selected the intended version.

 

Second decision: send the whole file or reuse existing data?

 

When delta transfer is used, the receiving side describes blocks in its existing basis file using checksums. The sender searches its source for matching material. A rolling checksum makes it efficient to test candidate matches at successive byte offsets, with a stronger comparison used to confirm candidates.

 

The sender then supplies literal data for unmatched sections and references to reusable basis blocks. The receiver uses that information to reconstruct the updated file. Matching data may have moved to a different offset after an insertion, so reuse is not limited to blocks that remain at exactly the same position.

 

This principle is described in Tridgell and Mackerras' original rsync algorithm report. It is historical algorithm documentation, not a recommendation to configure modern software with its old checksum choices. Current capabilities and negotiation depend on the builds at both ends.

 

Rsync normally uses whole-file updating when both paths are local. The delta algorithm can be forced with --no-whole-file, which is useful for testing but is not an automatic performance improvement. Reading the existing destination to find reusable data can cost more than sending the whole file over a fast path.

 

Delta transfer also needs something useful to match. A new file, a substantially rewritten file or an archive whose internal representation changes widely may provide little reusable material. Compression or encryption of a file before rsync sees it can alter how well its contents match.

 

Original test: file selection and delta transfer are separate

 

On 24 October 2026, we tested the installed rsync 3.2.7, protocol 31, using Ubuntu package 3.2.7-1ubuntu1.5. The test used only disposable local directories and a deterministic synthetic 4 MiB file, exactly 4,194,304 bytes. Files were compared directly and with independent SHA-256 hashes.

 

This is an observation of that installed build, not a recommendation to deploy that upstream version. Current releases and security maintenance are discussed later. No SSH, daemon, internet connection or broadband-speed test was involved.

 

After the initial copy, we inverted 4,096 bytes at an offset of 1 MiB in the source. We deliberately preserved its original size and modification time. The default archive-mode repeat skipped the file, leaving the old destination contents.

 

We then enabled content-based selection with --checksum and forced delta transfer using a controlled 1,024-byte block size. A separate comparison used whole-file transfer against an identical old basis.

 

Test

Literal data reported

Matched data reported

Observed file result

First copy into an empty destination

4,194,304 bytes

0 bytes

Exact copy

Unchanged repeat

0 bytes

0 bytes

No file-data update needed

Same-size edit with preserved modification time, default check

0 bytes

0 bytes

Old destination incorrectly remained for this deliberately unusual input

Content check plus forced delta, 4 KiB edit

4,096 bytes

4,190,208 bytes

Exact updated file

Content check plus whole-file transfer, same edit

4,194,304 bytes

0 bytes

Exact updated file

Separate 37-byte insertion with forced delta

1,061 bytes

4,193,280 bytes

Exact 4,194,341-byte result

 

The insertion test put 37 new bytes at offset 12,345 in the original data. Most of the later source data shifted position, yet rsync reused it. The 1,061 literal bytes also show why “a 37-byte edit means exactly 37 bytes transferred” would be wrong: block boundaries and the chosen update representation matter.

 

For the controlled 4 KiB edit, literal file-update data fell by 99.90234375% compared with the whole-file case: 1 minus 4,096 divided by 4,194,304. That is a calculation about this test's literal data, not a universal traffic saving or speed increase.

 

Read the statistics correctly

 

In the forced-delta repair, rsync still reported a total transferred file size of 4,194,304 bytes. That counter describes the complete sizes of the files selected for transfer, rather than the amount of unmatched data sent.

 

Literal data describes unmatched file-update data; matched data describes material reused by the receiver. Their sum in this case was the reconstructed file's size. Rsync also exchanges file lists, checksums, block references and other protocol information.

 

The same run reported 20,602 bytes sent and 24,611 bytes received. Those process-level counters demonstrate why 4,096 literal bytes are not the entire exchange. We did not capture network packets, and a real SSH/TCP connection would introduce its own transport overheads.

 

The wider experiment also checked source-directory slashes, destination-only files, deletion previews, actual deletion of disposable test data, exclusion protection and a missing-source error. All 13 cases produced their expected outcomes. Temporary files were removed and subprocesses finished.

 

Rsync infographic explaining file selection, comparison with existing data, reusable blocks, new data and block references, destination reconstruction and separate recovery points

 

A practical first rsync command

 

Start with a small source directory and an intentionally chosen destination. The following examples assume that source/ contains your test files and destination/ is the directory you intend to update. Check your working directory before using relative paths.

 

Preview the operation first:

 

rsync -an --itemize-changes --stats source/ destination/

 

Here, -a requests archive mode and -n requests a dry run. The itemised output describes proposed changes. The statistics help explain the operation, but a dry run does not transfer the actual file contents and cannot measure real transfer volume or throughput.

 

After reviewing the paths and changes, the corresponding real copy is:

 

rsync -a --itemize-changes --stats source/ destination/

 

This can replace destination files that rsync selects for updating. It does not request deletion of unrelated destination files. The local experiment executed these operation types against synthetic data; use a separate test destination until the paths and replacement behaviour are clear.

 

The source's trailing slash changes the layout

 

For the examples below, destination/ already exists and the source directory is called source. We tested both forms.

 

Source argument

Meaning in this example

Destination layout

source/

Copy the directory's contents

destination/file.txt

source

Copy the directory by name into the existing destination

destination/source/file.txt

 

This is about the source argument, not a universal rule that every slash anywhere is interchangeable. Existing paths, file-versus-directory arguments and single-item copies can affect the result. Prefer an explicit directory layout over relying on a path that may or may not exist.

 

Options worth understanding before combining them

 

Option

Purpose

What it does not establish

-a

Archive mode: recursion and a set of common metadata-preservation options

A complete backup or preservation of every metadata type

-n

Preview without applying the requested transfer changes

Future success, actual data volume or an unchanged source after preview

--itemize-changes

Explain proposed or applied changes

That each file was content-checked

--checksum

Use content checksums when selecting matching-size files for update

That delta mode was selected, or that every source is trustworthy

--stats

Report transfer and reconstruction counters

A packet capture or a broadband-line measurement

--delete

Remove eligible destination-only entries within synchronised directories

Recovery history or conflict resolution

--partial-dir=DIR

Retain partial transfer material separately for possible reuse

An application-consistent snapshot or a universal safe-resume guarantee

 

Archive mode is equivalent to -rlptgoD in the usual case. It includes recursion, symbolic links, permissions, modification times, group, owner and device/special-file handling. Preserving particular attributes still depends on permissions and filesystem support. Hard links, ACLs and extended attributes require separate consideration, including -H, -A and -X respectively; they are not all implied by -a.

 

Mirroring, deletion and backup recovery

 

Deletion is an explicit policy decision

 

Ordinary rsync copying can leave destination-only files in place. Adding --delete changes the intended relationship: eligible entries that are absent from the source can be removed from the receiving directories being synchronised.

 

Preview that change separately:

 

rsync -an --delete --itemize-changes source/ destination/

 

In our test, the preview announced the deletion of a disposable extra.txt while leaving its bytes untouched. A subsequent real run removed that test file. An ordinary exclude rule protecting keep.txt left that excluded destination file in place, even with --delete.

 

Filters and deletion interact. Options such as --delete-excluded, or rules applying only to the sending side, can change protection. A shell wildcard can also turn a request to synchronise a directory into a list of individual source arguments with different scope. Check the exact command instead of assuming the word “mirror” defines it.

 

A dry run is a preview of the state encountered then. Files can change afterwards, a mount can disappear and permissions can differ during the real run. Verify the source mount and intended dataset before propagating deletion, especially in scheduled jobs. An unexpectedly empty mounted directory is not necessarily an intentionally empty source.

 

A current mirror may contain the same mistake

 

If the source contains an accidental deletion, damaged content or an unwanted replacement, faithfully updating the destination can propagate that state. A second current copy is useful, but recovery may require an older independent state.

 

Design retention and restore testing alongside the copy. Depending on the system, that may involve protected snapshots, separate versioned destinations or an appropriate backup tool. Rsync offers building blocks such as backup directories and link-based reuse, but a collection of options is not proof of an effective recovery policy.

 

Running rsync in both directions is also not a general conflict-resolution system. If users edit both copies, you need a policy for competing changes and deletions. Do not assume that a second run will discover which version is semantically correct.

 

Copy a consistent source

 

Ordinary file copying does not coordinate all files of a running database or a changing multi-file application. A transfer can finish successfully while combining states from different moments. Use an application-supported export, backup or appropriately coordinated snapshot where consistency matters.

 

The normal update approach uses temporary destination files rather than modifying every final file directly, but this is not an atomic transaction for the entire directory tree. --inplace changes the existing destination file during transfer, which can expose intermediate or interrupted contents and affect shared hard-linked data. It should be a deliberate workload-specific choice.

 

Retaining partial material can reduce repeated work after interruption. That is different from assuming an arbitrary shorter file is a valid prefix of a growing source. Options such as --append have specific assumptions; avoid treating them as generic repair switches. We did not simulate interrupted-transfer recovery in this experiment.

 

Using rsync securely between machines

 

For a remote-shell transfer, rsync must be available and permitted at both ends. Verify the remote host identity and use an account limited to the required paths and operations. Encryption protects the connection; it does not make a compromised endpoint or an overly powerful account safe.

 

This is a documentation-checked remote preview example, using a reserved example hostname. It was not executed against a public server:

 

rsync -an --itemize-changes -e ssh source/ backupuser@backup.example.com:/srv/backups/source/

 

The single-colon remote-path form uses the selected remote shell. A double-colon module form or rsync URL normally addresses daemon mode. A direct rsync daemon connection does not itself encrypt file data; password authentication should not be mistaken for transport encryption. Protected daemon arrangements require an appropriate SSH tunnel or verified TLS setup.

 

Keep both ends maintained

 

As checked on 24 October 2026, the upstream project advertised rsync 3.5.1, released on 21 September 2026. Its security advisories describe important fixes and the conditions affecting particular deployments. The installed 3.2.7 build used for our isolated local experiment is not the current upstream release.

 

Operating-system vendors can backport fixes, so the upstream version string alone does not establish a package's patch status. Check the vendor's security information and the installed package revision as well as the upstream project. Maintain the receiving side and the sending side, including clients that pull from other systems.

 

What rsync changes for UK broadband workloads

 

Less update data can help, but the first copy still matters

 

For an off-site NAS copy sent from home, available upload speed matters. A large initial dataset may need a substantial first transfer. Later runs can require less data if the existing destination is useful and the changed files have reusable contents.

 

That saving is workload-dependent. A few edited documents, a repeatedly rebuilt compressed archive and a virtual-machine image can behave differently. Literal-byte savings do not guarantee an equal reduction in wall-clock time because directory scans, checksum reads, storage writes, protocol work and remote limits can dominate.

 

When the home system initiates an outbound SSH connection, CGNAT does not automatically prevent access to a reachable remote server. Hosting an endpoint that another system must reach from outside is a different problem, requiring an appropriate addressing and firewall arrangement. No particular UK provider offering is assumed here.

 

Find the actual bottleneck

 

Large numbers of small files can make metadata scanning and per-file operations expensive. A slower disk or NAS CPU can limit progress even on fast full fibre. Enabling content checks across a large mostly unchanged dataset may spend more time reading storage than moving new data.

 

Delta transfer trades extra comparison and reconstruction work for reduced update data. On a fast local path, whole-file transfer can be the better trade. Compression can help some data, but already-compressed media and archives may gain little while consuming CPU. Check the actual builds and workload instead of adding -z or forcing delta mode by habit.

 

If a transfer competes with calls or other interactive traffic, scheduling or a suitable rate limit can help reserve capacity. Rsync's --bwlimit is expressed in byte-based units; an unsuffixed value uses units of 1,024 bytes per second, not megabits per second. It is not a router-wide latency guarantee, and it does not control every other application sharing the connection.

 

A reliable rsync workflow

 

  1. Define which side is authoritative and whether the job is a copy, a mirror or part of a versioned backup. Establish how you will recover from a wrong source or destination.

  2. Confirm the exact paths, directory-slash behaviour, mounted filesystems and account permissions. For remote use, confirm both rsync installations, transport and server identity.

  3. Make the source consistent for the workload and choose the metadata and exclusions that must be preserved. Check destination capacity and filesystem compatibility.

  4. Preview the exact command with --dry-run and --itemize-changes. Review unexpected replacements, omitted files, directory levels and every proposed deletion.

  5. Run the reviewed command and capture its exit status and useful logs. Treat a nonzero result as something to investigate rather than a successful job with a completed progress bar.

  6. Check the intended result, including contents where required. Compare a trusted manifest or independent hashes for important datasets, and confirm that the application can use the copy.

  7. Test recovery from the retained state. Monitor recurring jobs for failures, unexpected empty sources, unusual deletion counts and changes to credentials or paths.

 

Troubleshooting rsync

 

A file was skipped or keeps transferring

 

Check the selection rules first. Size and modification time, --checksum, --size-only, exclusions and options that skip existing files can produce different decisions. Our preserved-timestamp example shows why a default skip is not a byte comparison.

 

For repeated transfers, inspect timestamp precision, metadata support and permissions on both filesystems. A destination unable to preserve requested attributes can create errors or repeated differences. Case sensitivity, filename handling and symlink behaviour also matter when crossing platforms.

 

A remote connection or update fails

 

Separate connection reachability, SSH host verification, account authentication, remote rsync availability and filesystem access. A protocol-stream error can also come from unexpected output generated by a remote startup script; it does not uniquely identify packet loss.

 

Exit code 23 means a partial transfer due to error; it does not identify one universal cause. Our missing-source test produced 23. Exit code 24 indicates a partial transfer due to vanished source files. Read the actual error messages and establish whether a changing source is expected before deciding how the scheduled job should respond.

 

The statistics look implausible

 

Confirm whether it was a dry run, whether the paths selected whole-file or delta updating, and which counter you are reading. Total transferred file size, literal data, matched data and bytes sent describe different things.

 

Do not read rsync's reported speedup as an ISP-speed measurement. Compare the same workload and outcome, check scanning and disk activity, and use a suitable separate network measurement if the question is the available path capacity.

 

Key takeaway

 

Rsync can make repeated file copying efficient by skipping apparently unchanged files and reusing matching destination data. Those are separate mechanisms. Choose the direction, paths, transport, comparison rules and deletion policy deliberately, then verify the result and retain the recovery history the workload needs.

 

For related networking and storage definitions, use the Techie Broadband terminology guide.

 

Frequently asked questions

 

What is rsync used for?

 

Rsync copies and updates files and directory trees locally or between systems. It is often used for one-way synchronisation, mirrors and backup workflows, but recovery history and restore testing must be designed separately.

 

Does rsync copy only changed files?

 

By default, rsync selects files using size and modification time. When delta transfer is used, it can also reuse matching data within selected files. Local copies normally use whole-file updating unless configured otherwise.

 

Does --checksum enable delta transfer?

 

No. --checksum changes the pre-transfer decision about which files need updating. Delta transfer is a separate mechanism. Rsync normally verifies reconstruction of files it transfers without requiring --checksum for that purpose.

 

Does rsync delete files at the destination?

 

An ordinary copy can leave destination-only files in place. --delete requests removal of eligible destination-only entries within the synchronised directories. Exclusions and other deletion options affect the scope, so preview the exact command first.

 

Is rsync encrypted?

 

Rsync over a correctly authenticated SSH connection uses SSH's protection. A direct rsync daemon connection does not itself encrypt file data. Check the transport and server identity rather than assuming every rsync command is protected.

 

Is rsync a backup?

 

Rsync can be part of a backup system, but an updated mirror alone may propagate deletions or damaged content. A useful backup needs appropriate recovery points, consistent source data, retention and tested restores.

 
 
UK Techie Broadband Providers
bottom of page