Simple Storage Service (S3) is a cloud-based storage service from Amazon Web Services (AWS). You can upload and download data of any size from anywhere. AWS provides a web-based dashboard to access this data from browsers. For command line users and automated scripts, there is a tool called AWSCLI. This CLI tool offers various options to manage your data, including synchronizing files between your local computer and S3 buckets.
What is AWS S3 Sync Command?
The aws s3 sync command is an option provided by AWS CLI tool. It helps you to keep the files synchronized between your computer storage and the files in an S3 bucket. This means if you add, delete, or change a file on your computer, you can use the sync command to make the same changes in the S3 bucket.
Imagine you have a folder on your computer with some important documents. You want to back up these documents to the cloud so that you don’t lose them if something happens to your computer. AWS S3 sync makes this easy. With one command, you can copy all the files in your folder to the S3 bucket.
Using AWS S3 Sync Command
Here’s a simple example of how it works. Let’s say you have a folder on your computer called “my-documents” and you want to sync it with a bucket called “my-s3-bucket”. You would open your command line tool and type:
aws s3 sync my-documents s3://my-s3-bucket
When you run this command, AWS will compare the files in “my-documents” with the files in “my-s3-bucket.” If there are new or updated files in “my-documents,” they will be copied to “my-s3-bucket”.
Sometimes, you may need to delete unused files from your local system, but they are still available in S3 buckets. The default aws s3 sync command only syncs new and updated files. In this case, you can use the --delete option to also remove files from the S3 bucket.
AWS S3 Sync Command Options
There are various command line options available to change the default behavior of the aws s3 sync command. Here are some useful options listed below. You can use these options as per your requirements.
| Option | Description |
|---|---|
--delete | Delete files from the destination that are not present in the source. |
--exact-timestamps | Use exact timestamps when comparing files. |
--exclude | Exclude specific files from being synced (e.g., –exclude “*.txt”). |
--include | Include specific files in the sync (used with –exclude). |
--dryrun | Show what files would be synced without actually performing the sync. |
--acl | Set the Access Control List (ACL) for synced files (e.g., –acl public-read). |
--storage-class | Set the storage class for synced files (e.g., –storage-class STANDARD_IA). |
--sse | Enable server-side encryption for synced files. |
--follow-symlinks | Follow symbolic links when syncing files. |
--no-follow-symlinks | Do not follow symbolic links when syncing files. |
The aws s3 sync command is very powerful and can save you a lot of time, especially when working with large amounts of data. It ensures that your local files and your S3 files are always the same.