Rsync to transfer heavy load

Hello Community,

I'd like to bring to the table this topic for discussion Rsync for transferring heavy load of data/files between two systems across two data centers.

As we all know, "rsync" is native and versatile in nature to accommodate this capability for simple/local files transfer/sync-up which may go up to 10GB (in my case, I've never used rsync to transfer file/files which are more than this capacity). I've never tested/used rsync to copy/move files of heavy size which is more than 10GB.

So, I'm in the opinion that rsync can't the right choice when there is a demand to move/copy/transfer files of heavy storage capacity (simply keep it to 10TB). Though it can do this task(if we can relax and give all the time for it to complete), however, there are other factors which makes this as un-reliable, and those points are:

• If network connection between hosts drops or SSH connections becomes weak then there is no reliable or guaranteed method which can assure that the disconnected or partially transferred data is patched up properly after connection resumes.

• There is no guarantee that the destination copy/file is as same as my original file in terms of size, attributes set/defined, and other characters. Since I presume that rsync can't look back and verify all this.

Note: I'm keeping the other factors which contributes to the overall data transfer speed such as network devices such as the routers/gateways/firewalls etc, away from the discussion for a while. I know that they are the main contributing factors but lets keep them aside.

So, considering the above factors:

• Is rsync a good tool/method to move/transfer files of such heavy load between two points?
• Do we really have any such use case of rsync?
• Even if we have the enough network bandwidth between two points, should we still recommend rsync or better recommend some third party tool or opensource tools which can do better than rsync considering the points discussed above?
• If someone has used other opensource tools which gets the best compared to rsync? If so, please recommend.

Let me know your opinion/view on this.

Thanks,
Sadashiva Murthy M