ASIGRA Solution: Autonomic Healing
Autonomic Healing is similar to DS-System Admin: it works at the file level for the online storage, REGARDLESS OF THE BACKUP SET TYPE OR APPLICATION.
- The Autonomic Healing Module is enabled on DS-System. It is designed to automatically find and fix errors in a much shorter period than DS-System Admin, and without considerably slowing down other activities, thereby maximizing DS-System resources.
- Autonomic Healing is designed to run in the background to check the DS-System Online Storage. It features a speed throttle that enables the Module to slow down when the DS-System is experiencing a high load of activities. Unlike the NTFS CheckDisk, Autonomic Healing will never require all activities to stop.
- Autonomic Healing can be started and stopped on-demand, using DS-Operator. It can also be configured to start when DS-System service starts.
- In very rare situations, there may be some special corruptions (e.g. some corruptions that result in "Digital signature does not match"). In order to detect such data corruptions, all online data of the file must be processed with the encryption keys. Since Autonomic Healing is designed to continuously monitor the online storage and find / fix corruptions / inconsistencies as soon as possible without knowing the encryption keys, it might not be able to detect such corruptions. Instead, ASIGRA implemented another process called Validation (in the core) that is triggered on DS-Client side. Validation confirms the restorability of files in online storage and moves any corrupted files that are found. Running Autonomic Healing constantly will improve confidence in the online storage restorability, but not up to the 100% certainty that the Validation process does. This does not mean that continuously running Validation is enough to remove corruptions / inconsistencies. Compared to Autonomic Healing, Validation does not have as powerful fixing capabilities and many other features (speed throttling, space optimizing, continuously running in background, etc.). Validation is also much slower.
The following outlines the features of the Autonomic Healing Module and how it can help with managing the DS-System:
1. The Autonomic Healing Module features more advanced detection and fixing capabilities than the DS-System Admin (Core) and will record any anomalies found (caused by 3rd-Parties) so that manual fixing is possible for problems that cannot be fixed automatically.
2. The Autonomic Healing Module is designed to continuously monitor the DS-System Online Storage for any data corruptions. It can also be configured with a variety of options (optimize space, process the selected sets repeatedly / only once for troubleshooting, speed control options, simultaneous processes, etc.).
3. In case corruptions are found, Autonomic Healing will notify the DS-System Operator. For each backup set, one notification will be sent when the first corruption is found.
4. The Autonomic Healing Module has some fixing capabilities, and as long as an error can be corrected, it will do so by either fixing the problem or moving the file to the trash location. If DS-System is part of a replication group, Autonomic Healing will try to retrieve a correct version of the file from one of the DS-Systems in the group. If retrieval is successful, then DS-Client is no longer involved. If the retrieval is not successful, the backup set will be marked as "out-of-sync" and DS-Client will be forced to synchronize the backup set and to re-send a replacement for the corrupted file during next backup session.
5. Real-Time features of the Autonomic Healing Module:
- The minimum check interval is greatly reduced to one hour.
- Checking Speed is automatically adjusted based on DS-System Load (I/O, CPU, activities, etc.)
- The Process can be manually started/stopped.
- The priorities of backup sets are automatically adjusted based on the healing result.
- Enhanced healing status monitoring capabilities.
- Configurable number of simultaneously running Autonomic Healing processes.
- Options to enable and disable speed control.
- Balanced backup set level task assigned between all Autonomic Healing processes.
6. Autonomic Healing Module Checking Capabilities - Autonomic Healing verifies the following categories to identify any data corruption, damage, or inconsistency:
- Online File Headers (header ID, version, compression and encryption types, invalid library links, invalid file names, header size)
- Directory Streams (header ID, version, invalid names, header size)
- Library links
- Delta Consistency
- File naming consistencies across generations
- File and Directory consistencies (name and ID)
- Directory location consistencies
- Session consistencies across generations
- Verify data integrity and restorability
7. Fixing Capabilities of the Autonomic Healing Module
In case any data corruption, damage, or inconsistency is identified, Autonomic Healing will try to fix it if possible. However, if the file is totally damaged, or a fix is not possible, Autonomic Healing will report the problem and move the file to a special directory for later analysis (manual).
The following data corruption/damage/inconsistency scenarios can be fixed by Autonomic Healing:
- Totally corrupted (delete it so that next backup will send a fresh new version)
- Inconsistent Files / Directories IDs
- Inconsistent Directory Location
- Inconsistent file name within directories
- Inconsistent filename across generations
- Delta linking/reconstruction inconsistencies
- Inconsistent library link
- Inconsistent sessions across generations
A regular monitor like the one for backup activities monitors the Autonomic Healing progress.
A separate "Autonomic Healing Manager" GUI monitors and manages the Autonomic Healing process:
- Overview of all backup sets' healing status
- Healing progress and statistics (checked/total: files, physical size, logical size, etc.)
- Problems found and fixed so far
- When Autonomic Healing started and completed/was interrupted for every backup set
- Check/set priority at the backup set level
- Storage space optimization settings
- Start / Stop Autonomic Healing process
- Backup set level event log browser
- Filter by Customer, DS-Client, Error Status, etc.
- Estimate processing sequence
- Automatically adjust backup sets' priority according to the healing result
- Options / settings for Autonomic Healing process (Process the specified sets, process repeatedly, process only once, no speed control etc.)
- Control multiple Autonomic Healing processes
Enhanced healing history management functionalities are also available (mainly integrated in the "Autonomic Healing Manager"):
- Overview of all backup sets' historical healing status
- Healing historical statistics (checked: files, physical size, logical size, etc.)
- Problems found and fixed in each historical healing period
- Historical records of when Autonomic Healing started and completed/was interrupted for every healing period of each backup set
- Event log browser for each backup set's historical healing period
- Filter by Customer, DS-Client, Backup set, Error Status, From / To etc.
- Clear old healing history
![]() ![]() |