7 In-File Delta Technology

The chapter describes what in-file delta technology is and how in-file delta can be used to backup large database files (e.g. a 10GB Outlook.pst file) without uploading the whole database file everyday.

7.1 Overview

In-file delta technology is an advanced data block matching algorithm which has the intelligence to pick up changes (delta) of file content between two files when one of the files is not accessible and use the delta information between two files to rebuild one file from the other. Using this algorithm, daily backing up of large file (e.g. a 10GB Outlook.pst file) over low-speed internet connection is made possible because it requires only the changes of information (should be marginal) since last backup to be sent over a low-speed internet connection to complete the backup of a large file (here we assume that the full backup of the file has been saved on the backup server already).

This is what would happen to the backup of a 10GB Outlook.pst file when it is backed up by backup 4 business OBM with in-file delta technology.

  1. The whole files (10GB), along with its checksum (128-bit) file, are backed up to the backup server. This can be done directly through the internet or indirectly using the seed loading utility on a removable hard disk.
  2. When backup is run again later (normally the next day), backup 4 business OBM will download a checksum listing of all data blocks of the full backup file from the backup server and use it to pick up all changes that have been made to the current Outlook.pst file from the first full backup.
  3. Changes detected are then saved in a delta file which is uploaded to the backup server. (This delta file is assumed to be small because the content of all PST files doesn’t change lot of even after it has been updated)
  4. Subsequent backups of this 10GB Outlook.pst file will go through step ii and step iii again. As explained, only a small delta file will be uploaded to the backup server.
  5. With in-file delta technology, daily backing up of large file over low-speed internet connection is now possible

All delta files are generated with respect to changes made since the last full backup file (i.e. differential backup). This means that only last full backup file and the last delta file are required to restore the latest snapshot of a backup file. This means that other intermediate delta files are only required if you want to restore other snapshots of a backup file.

The full backup file, its checksum file and the last delta file uploaded (if more than one delta files have been uploaded to the backup server) are always stored in the data area. This means that these files are not affected by the setting of the retention policy and will always be kept on the backup server. This is done this way because all these files are required to get the latest snapshot of the backup file and they should not be removed from the backup server by the retention area cleanup routine. All other intermediate delta files are stored in the retention area.

7.2 Block Size

The block size defines the size of data block being used to detect changes between last full backup file and the file sitting on the local computer right now. In general, the smaller the block size, the more likely a matched data block can be found between the last full backup file and the file on local computer. It, therefore, produces in a smaller delta file but it would require more processing power to detect these changes. On the other hand, in-file delta backup running with larger block size will run faster but this will generally produce a larger delta file.

In most case, the default setting [Auto] will choose the optimal block size for each file (depending on the size of the file) for you.

7.3 Maximum Number of Delta

The [Maximum number of delta] setting defines the maximum number of delta files from the same full backup file to be generated and backed up to the backup server before a full backup (the whole file) of this file is uploaded to the backup server instead.

For example, if you have created 100 delta files from the full backup file already and the [Maximum number of delta] setting is 100, the next backup will upload a full backup file (the whole file) instead of just the delta file. However, if the [Maximum number of delta] setting is unlimited, it will keep generating a delta file and uploading the delta file to the backup server until any of the other delta setting conditions force a full backup (e.g. delta ratio is exceeded). This setting is here to make sure that there will always be a full backup file after a certain number of delta files have been generated.

7.4 Minimum File Size

The [Minimum File Size] setting defines the smallest file size a file must have before the use and application of in-file delta backup technology.

If the size of a file that is being backed up is smaller than the [Minimum File Size] setting, in-file delta backup technology won’t be applied to this file and the whole file, instead of just the delta file, will be uploaded to the backup server. It is not necessary to perform in-file delta backup on small files because backing up the whole file doesn’t take too long anyway. Backing up the whole file instead reduces the time required to restore a backup file.

7.5 Delta Ratio

The [Delta Ratio] setting is defined to be the size of a delta file divided by the size of its full backup file (i.e. the percentage of changes detected between the last full backup file and the current file).

If delta ratio calculated from the size of the generated delta file and the size of the full backup is greater than the [Delta Ratio] setting, the whole file, instead of just the delta file, will be backed up to the backup server. It is done this way because the difference between backing up the whole file and the delta file is not significant and it is better to upload the whole file instead to reduce the time required to restore the file.

The default setting of [Delta Ratio] is 50%. This means that if more than 50% changes have been detected, the whole file, instead of just the delta file, will be backed up and uploaded to the backup server.

7.6 Temporary Directory

The [Temporary Directory] setting defines a temporary directory where backup 4 business OBM can be used to save its temporary files during the delta file generation operation. Please set this directory to a partition with lots of free space.

Top / Previous / Next / Table of Contents