AWS Service - DataSync

DataSync:

• Move large amount of data to and from

  • On-premises / other cloud to AWS (NFS, SMB, HDFS, S3 API) - needs agent

  • AWS to AWS (different storage services) - no agent needed

• Can synchronize to

  • Amazon S3 (any storage classes - including Glacier)

  • Amazon EFS

  • Amazon FSx (Windows, Lustre, NetApp, OpenZFS)

• Replication tasks can be scheduled hourly, daily, weekly

• File permissions and metadata are preserved (NFS POSIX, SMB)

• One agent task can use 10 Gbps, can setup a bandwidth limit

• Transferring between on-premises storage and AWS

• Transferring between AWS storage services

• Transferring between cloud storage systems and AWS storage services

Relevant Doc:

https://docs.aws.amazon.com/datasync/latest/userguide/how-datasync-transfer-works.html

Disclaimer: This is a personal blog that might come in handy when I suffer from Dementia in future.