Unrecoverable Items
Items are referred to be unrestorable when an automated restoration during the recovery process was not feasible. The most frequent causes of this might be many, but they include:
Read Error
Either the raw item or the metadata file is not readable due to an I/O exception or a permission issue.
Broken item
Both the the raw item or the metadata file are readable by Carbonio Backup but their content is broken or corrupted.
Invalid item
Both the the raw item or the metadata file are readable and the content is correct, but there is some other issue during the restore.
Search for irrepairable items
Following the completion of the recovery procedure, the administrators will receive a thorough message of Operation Completed that includes a list of missed items for each account, as demonstrated by the following excerpt:
[...] - stats - Restored Items: 15233 Skipped Items: 125 Unrestored Items: 10 - unrestored items - account: account1@example.com unrestored items: 1255,1369 account: account2@example.com unrestored items: 49965 account: account14@example.com unrestored items: 856,13339,45200, 45655 [...]
In the passage above, we indicate:
omitted elements
Everything that had previously been restored, either during this restoration or a prior one. Consequently, this communication serves just as information.
unrestored objects
a thing that wasn’t restored because the restore procedure ran into trouble. If you want to try to reclaim these things, make a note of all their IDs. They will be mentioned in the section’s reminder.
Note
Recall that an Item can be an e-mail, a file, a contact, or any other object within an account.
Recognise the unrestored items
There are two ways to identify unrestored items: using the CLI and through the Carbonio Admin Panel. The backup/import path may be searched for the item using the first method, and the Web interface can be used to examine the item using the second method.
Using the Carbonio Admin Panel
The comma separated list of unrestored items displayed in the Operation Complete notification can be used as a search argument in the Carbonio Admin Panel to perform an item search.
To do so:
-
Log into the Carbonio Admin Panel of the source server.
-
Use the View Mail feature to access the account containing the unrestored items
-
In the search box, enter item: followed by the comma separated list of itemIDs, for example:
item: 856,13339,45200,45655
Warning
Remember that any search is executed only within the current tab, so if you are running the search from the Email tab and get no results try to run the same search in the other tabs, e.g., Address Book, Calendar, Tasks.
Using the CLI
The backup getItem CLI command can display an item and the related metadata, extracting all information from a backup path/external backup.
zextras$ carbonio backup getItem {account} {item} [attr1 value1 [attr2 value2...
For example
zextras$ carbonio backup getItem account2@example.com 49965 dump blob true
Extract the raw data and metadata information of the item whose itemID is 49965 belonging to account2@example.com ,also including the full dump of the item’s BLOB
Restore Unrestored Items:
When an item cannot be restored, there is most likely a problem with the item itself or with your current Carbonio configuration. In some circumstances, even if an item cannot be restored on the first try, there are still strong odds that it can be recovered.
You can find a number of useful hints and pointers in the paragraphs that follow for handling various sorts of irreparable objects.
Items Not Restored Because of Read Errors
Read errors that can lead to items not to be restored are of two types:
- Hard errors
Hardware failures and all other destructive errors that cause an unrecoverable data loss.
- Soft errors
non-destructive errors, including for example wrong permissions, filesystem errors, RAID issues (e.g.: broken RAID1 mirroring), and so on.
While there is nothing much to do about hard errors, you can prevent or mitigate soft errors by following these guidelines:
Run a filesystem check
If using a RAID disk setup, check the array for possible issues (depending on RAID level)
Make sure that the
zextrasuser has r/w access to the backup/import path, all its subfolders and all thereby contained filesCarefully check the link quality of network-shared filesystems. If link quality is poor, consider transferring the data with rsync
If using SSHfs to remotely mount the backup/import path, make sure to run the mount command as root using the
-o allow_otheroption
Items Not Restored Because Identified as Invalid Items
An item is identified as Invalid when, albeit being formally correct, is discarded by the LMTP Validator upon injection.
If you experienced a lot of unrestored items during an import, it might be a good idea to momentarily disable the LMTP validator and repeat the import:
To disable the LMTP Validator, run the following command as the
zextrasuser.zextras$ zmlocalconfig -e zimbra_lmtp_validate_messages=falseOnce the import is completed, you can enable the LMTP validator by running
zextras$ zmlocalconfig -e zimbra_lmtp_validate_messages=true
Warning
This is a dirty workaround, as items deemed invalid by the LMTP validator might cause display or mobile synchronisation errors. Use at your own risk.
Items Not Restored Because Identified as Broken Items
Unfortunately, this is the worst category of unrestored items, and their recovery may be difficult when not impossible, depending on the degree of corruption of the item. However, it might be possible to recover either a previous state of the item or, in case of e-mails, the raw object. To identify the degree of corruption, use the backup getItem CLI command.
zextras$ carbonio backup getItem {account} {item} [attr1 value1 [attr2 value2...To search for a broken item, setting the backup_path parameter to the import path and the date parameter to all, will display all valid states for the item:
zextras$ carbonio backup getItem admin@example.com 24700 backup path /mnt/import/ date all
itemStates
start date 12/07/2013 16:35:44
type message
deleted true
blob path /mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ=
start date 12/07/2013 17:04:33
type message
deleted true
blob path /mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ=
start date 15/07/2013 10:03:26
type message
deleted true
blob path /mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ=If the item is an email, you will be able to recover a standard .eml file through the following steps:
Identify the latest valid state
From the above snippet, consider:
/mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ= start_date 15/07/2013 10:03:26 type message deleted true blob path /mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ=
Identify the
blob pathTake the blob path from the previous step:
blob path /mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ=
Use gzip to uncompress the BLOB file into an
.emlfile# gunzip -c /mnt/import/items/c0/c0,gUlvzQfE21z6YRXJnNkKL85PrRHw0KMQUqo,pMmQ= > /tmp/restored.eml # cat /tmp/restored.eml Return-Path: carbonio@test.example.com Received: from test.example.com (LHLO test.example.com) (192.168.1.123) by test.example.com with LMTP; Fri, 12 Jul 2013 16:35:43 +0200 (CEST) Received: by test.example.com (Postfix, from userid 1001) id 4F34A120CC4; Fri, 12 Jul 2013 16:35:43 +0200 (CEST) To: admin@example.com From: admin@example.com Subject: Service mailboxd started on test.example.com Message-Id: <20130712143543.4F34A120CC4@test.example.com> Date: Fri, 12 Jul 2013 16:35:43 +0200 (CEST) Jul 12 16:35:42 test zmconfigd[14198]: Service status change: test.example.com mailboxd changed from stopped to running
Done! You can now import the
.emlfile into the appropriate mailbox using your favorite client.