Backing up and versioning your data is a task often underestimated by those operating their own server. Loss of data can have a tremendous negative impact. At nine.ch, we offer a central S3-compatible backup cluster for our Root Server and Cloudserver customers. Data is saved with multiple levels of redundancy and can also be uploaded in encrypted form.
What Is S3?
S3 (Simple Storage Service) is a storage system developed by Amazon that can be accessed and managed via an HTTP(S) REST-API. S3 is an object-based storage system (see below for explanation), making it highly flexible and easily scalable. Amazon has so far not published the source code for S3. This means that it is not possible to offer a system that is identical. It is for this reason that nine.ch is offering S3-compatible storage rather than an actual S3 system. Amazon S3 offers a REST-API, upon which our S3-compatible storage solutions have been modelled.
Examples of services which offer the same (or a similar) API:
- Ceph with RADOS Gateway
- Openstack Swift
- Apache CloudStack
- …
nine.ch has developed a solution that uses an underlying Ceph cluster and a RADOS gateway. This provides us with easily scalable storage as a base upon which the RADOS gateway provides the S3 compatible features.
Object-Based Storage
The storage of data as objects is the core of an S3 compatible storage system. Unlike a file system where data is stored in a file hierarchy, S3-compatible storage systems store data together with meta-data in objects. Each object has its own ID. This has the advantage that individual objects can, for example, be distributed across different hard drives or different servers. In addition, data stored in objects can be easily duplicated and stored on different media regardless of its content or origin.
Based on this technology, highly available and scalable storage solutions for unstructured data can be developed accordingly. This means that even huge volumes of data can be stored on relatively inexpensive hardware. Facebook, for example, uses object storage for images and videos.
We use the open source solution Ceph. Ceph combines all of these benefits and thus offers a reliable basis for backing up your data.
Backups Using S3-Compatible Storage
To backup your data to our storage solutions, you will receive an “access key” that is used to login to the storage. A new key can be generated at any time in the Cockpit under Products / Backupspace.
As previously mentioned, this storage is compatible with Amazon S3 and therefore many of the tools available for Amazon S3 can also be used for creating a backup on our storage system. We have had the best experiences with Duplicity.
For more information on creating a backup on our storage system, please visit our support area.
Managing Buckets
S3 uses so-called “buckets” for data storage. These are used to store related data - comparable to a folder where you can drop files. These buckets can also be used to organise your backups from different systems. Users can create any needed buckets themselves. Please note that users are limited to creating a maximum of 100 buckets.
For security reasons, buckets cannot be managed directly via the REST-API. This means that buckets can be created either via the Cockpit or by writing data into a bucket which does not yet exist. However, these cannot be deleted. Data within a bucket is not affected by this issue and can be deleted as necessary.
Entire buckets can only be deleted via our Cockpit. When a bucket is deleted, all backups in the bucket are also deleted and cannot be restored.
Martin Wittwer works as a Linux system engineer at nine.ch and, together with his System Team colleagues, is responsible for the development of new services at nine.ch.
