Rsync with tricks!
If you often need to copy or sync data from a server to another you will probably miss some of the following features
If you often need to copy or sync data from a server to another you will probably miss some of the following features
- Parallel rsync jobs
- Limit the bandwidth per job
- Limit the number of parallel jobs to a desired number
- In case of rsync failure resume from where you left
- Append only new files on every run
- Ensure that no rsync job will start until the current has been completed (a very common problem in cron jobs)
Of course there are plenty examples on the internet how to achieve all those but wouldn’t be nice that a single tool could do all this in a very easy manner?
This tool exists and is called syncmaster
https://github.com/kpatronas/syncmaster
Examples on how to use the tool
Usage:
- -t: transfers space separated each transfer is delimited by ";" first column: source second column: destination third column: bandwidth
- -p: concurrent parallel transfers
Example: local dir sync
syncmaster.sh -t "/home/user/src/*;/home/user/dst/;10000" -p 3
Example: Remote dir to local sync
syncmaster.sh -t "user@server1:/home/user/src/*;/home/user/dst/;10000" -p 3
Example: Local dir to remote sync
syncmaster.sh -t "/home/user/src/*;user@server1:/home/user/dst/;10000" -p 3
Example: Local dir to many remote servers sync
syncmaster.sh -t "/home/user/src/*;user@server1:/home/user/dst/;10000 /home/user/src/*;user@server2:/home/user/dst/;10000 /home/user/src/*;user@server3:/home/user/dst/;10000" -p 3
I hope you will find this tool useful!
