luzifer/s3sync
1
0
Fork 0
mirror of https://github.com/Luzifer/s3sync.git synced 2024-10-18 06:24:20 +00:00
Code Issues Releases Wiki Activity
Small utility to sync local directories from/to Amazon S3 without installing any dependencies
12 commits 1 branch 13 tags 7.4 MiB
Go 98.8%
Makefile 1.2%
Find a file
Knut Ahlers 3237e3fd8d Fix: Made channels bigger for S3 processing 
Buckets with many folders were able to freeze s3sync when all threads were trying to write into the prefix channel but it was at its CAP.
2015-07-26 18:39:42 +02:00
Godeps Initial version 2015-07-26 16:06:24 +02:00
.gitignore Initial version 2015-07-26 16:06:24 +02:00
.gobuilder.yml Initial version 2015-07-26 16:06:24 +02:00
History.md Prepared release 1.1.0 2015-07-26 18:02:04 +02:00
LICENSE Initial version 2015-07-26 16:06:24 +02:00
local.go Initial version 2015-07-26 16:06:24 +02:00
main.go Do file sync with more than 1 thread in parallel 2015-07-26 17:21:41 +02:00
README.md Updated README 2015-07-26 18:02:42 +02:00
s3.go Fix: Made channels bigger for S3 processing 2015-07-26 18:39:42 +02:00

README.md

Luzifer / s3sync

License: Apache v2.0 Gobuild Download

s3sync is a small utility to sync local directories from/to Amazon S3 without installing any dependencies. Just put the binary somewhere into your path and set three ENV variables and your're ready to sync.

Features

  • Static binary, no dependencies required
  • Sync files only if required (judged by MD5 checksum)
  • Optionally delete files at target
  • Optionally make files public on sync (only if file needs sync)
  • Sync local-to-s3, s3-to-local, local-to-local or s3-to-s3

Usage

  1. Set AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY and AWS_REGION
  2. Execute your sync
# s3sync --help
Sync files from <from> to <to>

Usage:
  s3sync <from> <to> [flags]

Flags:
  -d, --delete=false: Delete files on remote not existing on local
  -h, --help=false: help for s3sync
      --max-threads=10: Use max N parallel threads for file sync
  -P, --public=false: Make files public when syncing to S3
  -v, --version=false: Print version and quit

# s3sync -d 1/ s3://knut-test-s3sync/
(1 / 3) 05/11/pwd_luzifer_io.png OK
(2 / 3) 07/26/bkm.png OK
(3 / 3) 07/26/luzifer_io.png OK

# s3sync -d 1/ s3://knut-test-s3sync/
(1 / 3) 05/11/pwd_luzifer_io.png Skip
(2 / 3) 07/26/bkm.png Skip
(3 / 3) 07/26/luzifer_io.png Skip

# rm -rf 1/05
# s3sync -d s3://knut-test-s3sync/ 1/
(1 / 3) 05/11/pwd_luzifer_io.png OK
(2 / 3) 07/26/bkm.png Skip
(3 / 3) 07/26/luzifer_io.png Skip