Are you still using scp to copy files? Use rsync instead! It can resume an interrupted transfer!

We all have used scp to transfer files. We all have cursed at the moment that the transfer got interrupted in the middle of the transfer or even worse at 99% (my heart pains every time I remember a 99% interrupted transfer); let’s see how we can use rsync in place of scp and avoid such misfortunes.

What is rsync?

Rsync is a fast and extraordinarily versatile file-copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers many options that control every aspect of its behavior and permit very flexible specifications of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a lqquick checkrq algorithm (by default) that looks for files that have changed in size or in the last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file’s data does not need to be updated.

Platforms that rsync is available

Rsync is available to any UNIX-like platforms such as Linux and MacOS, maybe there are some forks of rsync for Windows as well, but i haven’t worked with any, so i don’t know if all features like the one we discuss now exist!

How to install rsync

In ubuntu and debian derivatives use apt

apt-get install rsync

In RPM systems like RedHat and Centos use yum

yum install rsync

In Mac, use brew

brew install rsync

You can verify that rsync is installed by checking its version

rsync -version

How to use rsync?

To copy a file from localhost to a remote host is similar to scp but with more parameters (bash aliases are your friend if you cannot memorize parameters)

rsync -partial -z -e 'ssh -p 22' data.txt <user>@<host>:/destination_path

• data.txt is the source file .
• <user>@<host>:/destination_path is the remote host, the path, and the user that will be used to connect.
• -e 'ssh -p' We say to rsync to use ssh in port 22
• -z to compressed data transfer to speed up things
• -partial this is the “golden” parameter that allows us to resume interrupted transfers. By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances, it is more desirable to keep partially transferred files. Using the — partial option tells rsync to keep the partial file which should make a subsequent transfer of the rest of the file much faster.

How i resume an interrupted transfer

If you started the transfer with the -partial parameter, as I mentioned in the previous paragraph, and the transfer was interrupted for some reason, you can re-enter the command, and boom! It’s like magic! the transfer starts from the point it stopped! no more cursing in front of a screen!

Conclusion

Rsync is powerful, it has many more parameters, like bandwidth limiting, etc., but the most crucial parameter to have peace of mind is to resume a transfer. comment me with your experiences with scp and rsync! did you find my article helpful?

Join Medium with my referral link - Konstantinos Patronas

As a Medium member, a portion of your membership fee goes to writers you read, and you get full access to every story…