alt Hacker NewsApart from Minio, we tried Garage and Ceph. I think there's definitely a need for something that interfaces using S3 API but is just a simple file system underneath, for local, testing and small scale deployments. Not sure that exists? Of course a lot of stuff is being bolted onto S3 and it's not as simple as it initially claimed to be.
Replies
Minio started like that but they migrated away from it. It's just hard to keep it up once you start implementing advanced S3 features (versioning/legal hold, metadata etc.) and storage features (replication/erasure coding)
> for local, testing and small scale deployments
Yes I'm looking for exactly that and unfortunately haven't found a solution.
Tried garage, but they require running a proxy for CORS, which makes signed browser uploads a practical impossibility for the development machine. I had no idea that such a simple popular scenario is in fact too exotic.
What about s3 stored in SQLite? https://github.com/seddonm1/s3ite
This was written to store many thousands of images for machine learning
From what I can gather, S3Proxy[1] can do this, but relies on a Java library that's no longer maintained[2], so not really much better.
I too think it would be great with a simple project that can serve S3 from filesystem, for local deployments that doesn't need balls to the walls performance.
I've been looking into rclone which can serve s3 in a basic way https://rclone.org/commands/rclone_serve_s3/
Try versitygw.
I’m considering it for a production deployment, too. There’s much to be said for a system that doesn’t lock your data in to its custom storage format.
The problem with that approach is that S3 object names are not compatible with POSIX file names. They can contain characters that are not valid on a filesystem, or have special meaning (like "/")
A simple litmus test I like to do with S3 storages is to create two objects, one called "foo" and one called "foo/bar". If the S3 uses a filesystem as backend, only the first of those can be created
For testing, consider https://github.com/localstack/localstack
SeaweedFS's new `weed mini` command[0] does a great job at that. Previously our most flakey tests in CI were due to MinIO sometimes not starting up properly, but with `weed mini` that was completely resolved.
[0]: https://github.com/seaweedfs/seaweedfs/wiki/Quick-Start-with...