AWS CLI Quick Reference for S3

Reference

June 28, 2020

I finally got myself up and running with the AWS CLI! For easy reference here are the commands I think I will use:

For My Machine

Dry run everything. For all of the commands below, add a --dryrun flag to view what will happen before actually doing it.

Sync current directory to specific place in S3 bucket. This will be recursive and directories will be created where they don’t exist. Existing files won’t be deleted unless you add the --delete flag.

aws s3 sync . s3://nine-elms/photos-raw/ --exclude ".DS_Store" --storage-class STANDARD_IA

Copy a specific local path to a specific remote path. Copy commands must include the --recursive flag to transfer all files.

aws s3 cp /Users/me/Desktop/txfr/2020-01-10 s3://nine-elms/photos-raw/ --recursive --storage-class STANDARD_IA

For Raspberry Pi

Copy all photos for the designated months to the remote path. Set --exclude to everything, and only transfer the desired directories. Probably ok to use glacier instead of infrequent access for this.

aws s3 sync /home/pi/Desktop/raw-photos s3://nine-elms/photos-raw/ --exclude "*" --include "*2020-01-*/*" --include "*2020-02-*/*" --include "*2020-03-*/*" --storage-class GLACIER --dryrun

To be bold about it, use mv instead of sync. It avoids the step of manually deleting, but doesn’t give the opportunity to verify the transfer before deleting.

aws s3 mv /home/pi/Desktop/raw-photos s3://nine-elms/photos-raw/ --exclude "*" --include "*2020-01-*/*" --include "*2020-02-*/*" --include "*2020-03-*/*" --recursive --storage-class GLACIER --dryrun

Note: This Doesn’t Work

This is why AWS CLI needs to be directly installed onto Pi.

$ aws s3 cp pi@192.168.0.97:/home/pi/Desktop/raw-photos/2020-01-20 s3://nine-elms/photos-raw/ --recursive --storage-class STANDARD_IA
The user-provided path pi@192.168.0.97:/home/pi/Desktop/raw-photos/2020-01-20 does not exist.

Useful References

Storage Class Options & EU Ireland Pricing

I think this pricing applies to N. Virginia too. Here’s the official pricing page.

  • STANDARD (default) — $0.023 per GB/month
  • STANDARD_IA — $0.0125 per GB/month
  • GLACIER — $0.004 per GB/month
  • ONEZONE_IA — $0.01 per GB/month
  • DEEP_ARCHIVE — $0.00099 per GB/month
  • REDUCED_REDUNDANCY — $0.024 per GB/month
  • INTELLIGENT_TIERING — $0.023 per GB/month
Tags icon Stacked price tag looking things Tags: