Close menu

Introduction to Carbonio Storages

Introduction to Carbonio Storages

One primary volume and an unknown number of subsidiary volumes make up each Carbonio installation. The secondary volumes are managed and transferred between using the Carbonio Storages module.

Using Hierarchical Storage Management (Hierarchical Storage Management), a policy-based method, objects may be relocated in accordance with: One of the most practical is, for instance, to set aside the fastest storage for intense I/O operations and for often accessed data, while managing older data with the slower storage.

The remaining paragraphs of this part go with policies, HSM, other advanced approaches, and volumes and their management.

The Foundations of Carbonio Stores: Store Types and Their Functions

Carbonio permits two distinct kinds of stores:

Catalogue Store

    a repository that houses your data’s metadata and is utilised by Apache Lucene to do indexing and search.

Data Centre

    a location where all of your Carbonio data is stored and is arranged in a MySql database.

However, only one Index Store, one Primary Data Store, and one Secondary Data Store may be designated as Current, meaning that these are the ones that Carbonio is presently using. You can have more than one store of each kind.

Secondary and Primary Data Stores

Carbonio’s data stores come in two flavours: primary data stores and secondary data stores.

Volumes
    Carbonio distinguishes three categories of volumes:
 
Principal Current
    a storage device where new data is immediately written.
 
Primary Current
    a volume to which data are written once an HSM policy has been applied.
 
Not Current
    Volumes are those that are not marked as Current and where data may only be written via particular manual procedures.
 
Items are usually put in the destination server’s Primary Current volume by default.
Centralised Storage
Instead of using “independent” volumes, which are isolated and whose directory structure is only related to the server and volume itself, the  feature enables the use of an S3 bucket to host data from multiple servers simultaneously sharing the same directory structure.
 
This dramatically increases mailbox move speed and enables better data management in big multistore systems.
 
Before using centralised storage, the following two crucial factors need to be considered.
 
  1. Deduplication of the item fails.
  2. The only storage type that may be utilised centrally is S3 buckets.
Configuring Centralised Storage
There are a few procedures involved in setting up a bucket for centralised storage. The steps are outlined below and will walk you through creating the bucket and connecting the new Storage to various AppServers.
 
All supported bucket types follow the same process, albeit the syntax or some of the parameters may vary significantly based on the bucket type. For instance, the URL argument must be stated in a language that the object storage itself can comprehend because it specifies the API endpoint of the object storage.
 
We set up an S3 bucket in our example; to set up a different sort of bucket, just use the corresponding command.
 

1. Use the command doCreateBucket to create an S3 bucket.

zextras$ carbonio core doCreateBucket S3 BucketName X58Y54E5687R543 abCderT577eDfjhf My_New_Bucket
We employ the following values in this example:
  • S3 as the bucket type
  • The bucket name, BucketName, must match the name on the remote provider exactly for the command to succeed.
  • The remote username is X58Y54E5687R543.
  • The remote password is abCderT577eDfjhf.
  • The bucket is given the label My_New_Bucket.
  • The bucket connection endpoint for Carbonio Storages is https://example_bucket_provider.com.

Note

Pay attention to the format of the URL, because some provider might require that also the port be specified as part of the URL or that the IP address be used instead of the URL.

If the command is successful, it prints the bucket’s unique identification number (UUID), which looks like this: 60b8139c-d56f-4012-a928-4b6182756301. Make a note of it because the rest of the operation will require it.

2. Test the connection using the bucket ID (60b8139c-d56f-4012-a928-4b6182756301) that was received in the previous step:

zextras$ carbonio core testS3Connection 60b8139c-d56f-4012-a928-4b6182756301

You will see that the message connection ok is fine if the command is successful.

3. Make a volume on the first AppServer connected to the bucket:

zextras$ carbonio powerstore doCreateVolume S3 Store_01 secondary  \
60b8139c-d56f-4012-a928-4b6182756301 volume_prefix Main_Volume Centralized true
These values are utilised in this illustration:
 
  • The type of bucket, S3
  • The volume name defined on the server where the command is run is Store_01. 
  • secondary: the volume’s kind
  • The bucket ID as received in step 1 is 60b8139c-d56f-4012-a928-4b6182756301.
  • volume_prefix A designation given to the volume, such as main_vol, is utilised for rapid searches.
  • True: The volume is centralised and accessible to several AppServers.

4. Use the following command to set the volume to current so that it may receive data right away:

zextras$ carbonio powerstore doUpdateVolume S3 Store_01 secondary current_volume true
These values are utilised in this illustration:
 
  • The type of bucket, S3
  • The volume name defined on the server where the command is run is Store_01.
  • secondary: the volume’s kind

5. After the Centralised Volume has been built, it must be added to the volume list and its configuration copied from the first server to all mailbox servers. Run the following commands on every other AppServer to do this:

zextras$ carbonio powerstore doCreateVolume Centralized mailbox_01.example.com Store_01
These values are utilised in this illustration:
  • The type of bucket, S3
  • The volume name defined on the server where the command is run is Store_01 
  • The _servername_ of the server on which the volume was specified and created is mailbox_01.example.com.
The command reported in the preceding step is the second one that must be executed:
zextras$ carbonio powerstore doUpdateVolume S3 Store_01 secondary current_volume true
Centralised Storage Structure
Storage Structure –  A centralised volume simply stores data by having a directory for each mailbox that is saved in it at the same level and a single empty directory for each server that is connected to the volume.
 
The servers 595a4409-6aa1-413f-9f45-3ef0f1e560f5 and 3aa2d376-1c59-4b5a-94f6-101602fa69c6 are both linked to the same centrally located volume, which has three mailboxes. As you can see, the storage has no bearing on the actual server where the mails are hosted: 
_
|- 3aa2d376-1c59-4b5a-94f6-101602fa69c6/
|- 595a4409-6aa1-413f-9f45-3ef0f1e560f5/
|- ff46e039-28e3-4343-9d66-92adc60e60c9/
\
 |-- 357-104.msg
 |-- 368-115.msg
 |-- 369-116.msg
 |-- 373-120.msg
 |-- 374-121.msg
 |-- 375-122.msg
 |-- 376-123.msg
 |-- 383-130.msg
|- 4c022592-f67d-439c-9ff9-e3d48a8c801b/
\
 |-- 315-63.msg
 |-- 339-87.msg
 |-- 857-607.msg
 |-- 858-608.msg
 |-- 859-609.msg
 |-- 861-611.msg
 |-- 862-612.msg
 |-- 863-613.msg
 |-- 864-614.msg
 |-- 865-615.msg
 |-- 866-616.msg
 |-- 867-617.msg
 |-- 868-618.msg
|- dafd5569-4114-4268-9201-14f4a895a3d5/
\
 |-- 357-104.msg
 |-- 368-115.msg
 |-- 369-116.msg
 |-- 373-120.msg
 |-- 374-121.msg
 |-- 375-122.msg
 |-- 376-123.msg
 |-- 383-130.msg
 |-- 384-131.msg