Using AWS CLI , You can easily manage S3 Buckets and the objects in the Buckets.
Before proceeding with the below tutorial , Make sure you have full S3 permissions.
1.Create S3 Bucket
We have to use s3 mb option with aws command to create a new S3 bucket.
The name for the s3 bucket should be unique globally.
mb stands for make bucket. And the bucket will be created as per the region specified during the aws cli configuration. You can check the same under User’s Home directory (~/.aws/config)
Bucket name can contain lowercase letters , hyphens , numbers and periods. But It should not end with hyphens and hyphens.
aws s3 mb s3://bucketname
If the bucket name already exists and owned by some other users , You will get the below error.
make_bucket failed: s3://testbucket An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
To Create S3 bucket in a specific AWS region , Other than the region specified in AWS CLI , Use the –region option.
Using the s3 ls command we can list the buckets and the Objects in the S3 bucket.
The Below command will list all the buckets.
aws s3 ls
To lists all the objects in a particular bucket , You can use the below command.
aws s3 ls s3://bucket-name
Lets say , You want to list the objects in the folder within the S3 bucket.Use the below command.
In the terms of S3 , Folder is referred as prefix.
aws s3 ls s3://bucket-name/foldername/
3.Deleting S3 Buckets and Objects
If you want to delete a S3 bucket which is empty , You can use the below command.
You should use s3 rb with the aws command. rb stands for Remove bucket.
aws s3 rb s3://bucket-name
If you try to delete an S3 bucket with objects , You will get an error.
remove_bucket failed: s3://testbucket0072543 An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty
if you want to delete the S3 bucket with the objects inside it. You should use
aws s3 cp s3://testbucketmumforawscli/sample.txt s3://testbucketmumforawscli2/
copy: s3://testbucketmumforawscli/sample.txt to s3://testbucketmumforawscli2/sample.txt
In the above example We have copied files in the / (root level) of the S3 bucket.
Let’s say you have a file inside a folder , For example s3://bucketname/foldername/filename , In this case If you copy the file from Source to destination bucket ,
The folder won’t be created in the destination bucket , Instead the file will be copied to the / root level of the S3 bucket as shown below.
aws s3 cp s3://testbucketmumforawscli/samplefolder/sample.txt s3://testbucketmumforawscli2/ copy: s3://testbucketmumforawscli/samplefolder/sample.txt to s3://testbucketmumforawscli2/sample.txt
If you want to copy files along with the folder , You should use the below command.
While Copying, If the folder is not present in the local system , It will automatically create during the backup process and backup all the files over there.
Conclusion
In this article , We have learnt few commands to manage S3 buckets and Objects in the S3 buckets.And also copying files between Local and S3 bucket.
Hope you found it helpful. Thanks for reading this article.