All posts

s3verify — A simple tool to verify AWS S3 API compatibility

s3verify performs a series of API calls against an object storage server and checks the responses for AWS S3 signature version 4 compatibility. s3verify is licensed under Apache V2.0 license.

Why we built s3verify

Amazon Web Service’s (AWS) Simple Storage Service (S3) has emerged as the goto cloud storage, so any new competing service has a strong incentive to comply with their APIs.

Version 4 of the AWS S3 API has matured and become quite extensive. AWS now supports over 20 different APIs and countless variations on those APIs. It is no longer just simple get/put requests.

Without a tool like s3verify, making frequent high quality releases is nearly impossible. New feature additions and Pull Requests can easily cause regressions and break existing compatibility.

Dependency on other SDKs or libraries could mask bugs. Instead s3verify creates its own handcrafted HTTP requests and sends them to the user specified server.

s3verify’s primary goal is to verify the correctness of the implementation. For each possible valid input the correct response must be returned. It also tests with invalid inputs and expects failures in response. Possible invalid inputs are far too many; however the current invalid inputs are a good start.

Who can use s3verify

s3verify is intended to be used by both application developers and object storage vendors. Applications that are dependent on the AWS S3 API may use s3verify to eliminate bugs introduced by incompatible changes by the cloud storage provider. Object storage vendors are strongly advised to integrate s3verify into their CI to stop incompatible changes from creeping in. s3verify is designed to be easy to be integrated as a part of CI builds.

Install s3verify

Assuming that you have a working Go environment copy/paste the following command to your terminal. If you do not yet have a Go environment follow the instructions here: Install Go.

$ go get -u https://github.com/minio/mint

Conclusion

s3verify is meant to be full and robust. Please feel free to fork our project and contribute any new test or feature. We’d love to see your Pull Request.

Please note that s3verify has been replaced by Mint: https://github.com/minio/mint. Mint is a testing framework for Minio object server, available as a docker image. It runs correctness, benchmarking and stress tests.

For any questions and nice comments join our Slack.

Previous Post Next Post