rsync examples: how to resume your interrupted copy and how to copy over a third host
How to copy a file/directory from a host to another who can communicate only by a 3rd host (Jump Host)
How to copy a file/directory from a host to another who can communicate only by a 3rd host (Jump Host)
Using — append and — partial allows us to resume rsync in case that the rsync is interrupted
rsync — bwlimit=20000 — progress — append — partial -vz -e ‘ssh -J <USER>@<JUMP_HOST> -p 22’ <USER>@<SOURCE_HOST>:/SOURCE/PATH /LOCAL/PATH
Parameters explaination
- bwlimit: if omitted uses all bandwidth else limit I/O bandwidth; KBytes per second
- progress: show progress during transfer
- append: append data onto shorter files
- partial: keep partially transferred files
- vz: verbose output and zip file data during the transfer
- -e ‘ssh -J <USER>@<JUMP_HOST> -p 22’: rsync over a jump host
- <USER>@<SOURCE_HOST>:/SOURCE/PATH
- /LOCAL/PATH
Example:
rsync — bwlimit=20000 — progress — append — partial -vz -e ‘ssh -J root@10.0.0.1 -p 22’ root@10.1.0.1:/APATH/AND_A_FILE.data /LOCAL_PATH/DESTINATION
How to copy a file/directory from a host to another directly
rsync — bwlimit=20000 — progress — append — partial -vz -e ‘ssh -p 22’ <USER>@<SOURCE_HOST>:/SOURCE/PATH /LOCAL/PATH
Parameters explaination
- bwlimit: if omitted uses all bandwidth else limit I/O bandwidth; KBytes per second
- progress: show progress during transfer
- append: append data onto shorter files
- partial: keep partially transferred files
- vz: verbose output and zip file data during the transfer
- -e ‘ssh -p 22’: rsync over ssh
- <USER>@<SOURCE_HOST>:/SOURCE/PATH
- /LOCAL/PATH
Example:
rsync — progress — append — partial -vz -e ‘ssh -p 22’ user@source:~/test .
Note: in case that the host you want to fetch the files is behind an SSH proxy you need either to use the Jump Host syntax or modify your ssh_config file to automate the jump.
Note: append does not verify data checksum, after a restart of rsync, to verify use — append-verify BUT is very slow on large files.
