Skip to content
/ s3scan Public

S3 can! - Provides native Node.js streams for S3 operations.

License

Notifications You must be signed in to change notification settings

mapbox/s3scan

Repository files navigation

Build Status

s3scan

Provides native Node.js streams for S3 operations such as

  • listing all the keys in a bucket/prefix
  • retrieving all the objects under a given bucket/prefix
  • deleting all the children of a given bucket/prefix

Also provides CLI tools:

s3keys: prints a line-delimited list of keys to stdout

$ s3keys s3://my-bucket/some-prefix

s3purge: deletes all objects beneath the provided prefix

$ s3purge s3://my-bucket/all-finished-with-these

s3scan: GET and print a prefix-worth of objects to stdout

$ s3scan s3://my-bucket/some-prefix
$ s3scan s3://my-bucket/some-prefix --gunzip
$ s3scan s3://my-bucket/some-prefix --gunzip --concurrency=1

Running tests

You can run tests against your own bucket/prefix by specifying them as environment variables:

TestBucket=my-bucket TestPrefix=my-prefix npm test

Otherwise these values default to buckets owned by Mapbox which require appropriate authentication.

About

S3 can! - Provides native Node.js streams for S3 operations.

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Packages

No packages published