S3
Ozone provides S3 compatible REST interface to use the object store data with any S3 compatible tools.
Getting started
S3 Gateway is a separated component which provides the S3 compatible. It should be started additional to the regular Ozone components.
You can start a docker based cluster, including the S3 gateway from the release package.
Go to the compose/ozones3 directory, and start the server:
docker-compose up -dYou can access the S3 gateway at http://localhost:9878
URL Schema
Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. http://bucketname.host:9878) and the path-style addresses (eg. http://host:9878/bucketname)
By default it uses the path-style addressing. To use virtual host style URLs set your main domain name in your ozone-site.xml:
<property>
<name>ozone.s3g.domain.name</name>
<value>s3g.internal</value>
</property>Bucket browser
Bucket’s could be browsed from the browser with adding ?browser=true to the bucket URL.
For example the content of the ‘testbucket’ could be checked from the browser using the URL http://localhost:9878/testbucket?browser=true
Implemented REST endpoints
Operations on S3Gateway service:
| Endpoint | Status |
|---|---|
| GET service | implemented |
Operations on Bucket:
| Endpoint | Status | Notes |
|---|---|---|
| GET Bucket (List Objects) Version 2 | implemented | |
| HEAD Bucket | implemented | |
| DELETE Bucket | implemented | |
| PUT Bucket (Create bucket) | implemented | |
| Delete Multiple Objects (POST) | implemented |
Operation on Objects:
| Endpoint | Status | Notes |
|---|---|---|
| PUT Object | implemented | |
| GET Object | implemented | Range headers are not supported |
| Multipart Uplad | not implemented | |
| DELETE Object | implemented | |
| HEAD Object | implemented |
Security
Security is not yet implemented, you can use any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
Note: Ozone has a notion for ‘volumes’ which is missing from the S3 Rest endpoint. Under the hood S3 bucket names are maped to Ozone ‘volume/bucket’ locations (depending from the given authentication information).
To show the storage location of a S3 bucket, use the ozone sh bucket path <bucketname> command.
aws s3api --endpoint-url http://localhost:9878 create-bucket --bucket=bucket1
ozone sh bucket path bucket1
Volume name for S3Bucket is : s3thisisakey
Ozone FileSystem Uri is : o3fs://bucket1.s3thisisakey
Clients
AWS Cli
aws CLI could be used with specifying the custom REST endpoint.
aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest
Or
aws s3 ls --endpoint http://localhost:9878 s3://buckettest
S3 Fuse driver (goofys)
Goofys is a S3 FUSE driver. It could be used to any mount any Ozone bucket as posix file system:
goofys --endpoint http://localhost:9878 bucket1 /mount/bucket1