snapraid - Man Page

To periodically check the data and parity for errors, you can run the "scrub" command.

This command verifies the data in your array comparing it with the hash computed in the "sync" command.

Every run of the command checks about the 8% of the array, but not data already scrubbed in the previous 10 days. You can use the -p, --plan option to specify a different amount, and the -o, --older-than option to specify a different age in days. For example, to check 5% of the array older than 20 days use:

If during the process, silent or input/output errors are found, the corresponding blocks are marked as bad in the "content" file, and listed in the "status" command.

To fix them, you can use the "fix" command filtering for bad blocks with the -e, --filter-error options:

At the next "scrub" the errors will disappear from the "status" report if really fixed. To make it fast, you can use -p bad to scrub only blocks marked as bad.

Take care that running "scrub" on a not synced array may result in errors caused by removed or modified files. These errors are reported in the "scrub" result, but related blocks are not marked as bad.

To have all the files in your array shown in the same directory tree, you can enable the "pooling" feature. It consists in creating a read-only virtual view of all the files in your array using symbolic links.

You can configure the "pooling" directory in the configuration file with:

or, if you are in Windows, with:

and then run the "pool" command to create or update the virtual view.

If you are using a Unix platform and you want to share such directory in the network to either Windows or Unix machines, you should add to your /etc/samba/smb.conf the following options:

In Windows the same sharing operation is not so straightforward, because Windows shares the symbolic links as they are, and that requires the network clients to resolve them remotely.

To make it working, besides sharing in the network the pool directory, you must also share all the disks independently, using as share points the disk names as defined in the configuration file. You must also specify in the "share" option of the configure file, the Windows UNC path that remote clients needs to use to access such shared disks.

For example, operating from a server named "darkstar", you can use the options:

and share the following dirs in the network:

to allow remote clients to access all the files at \\darkstar\\pool.

You may also need to configure remote clients enabling access at remote symlinks with the command:

SnapRAID is more like a backup program than a RAID system, and it can be used to restore or undelete files to their previous state using the -f, --filter option :

or for a directory:

You can also use it to recover only accidentally deleted files inside a directory using the -m, --filter-missing option, that restores only missing files, leaving untouched all the others.

Or to recover all the deleted files in all the drives with:

The worst happened, and you lost one or more disks!

DO NOT PANIC! You will be able to recover them!

The first thing you have to do is to avoid further changes at your disk array. Disable any remote connection to it, any scheduled process, including any scheduled SnapRAID nightly sync or scrub.

Then proceed with the following steps.

You need some space to recover, even better if you already have additional spare disks, but in case, also an external USB or remote disk is enough.

Change the SnapRAID configuration file to make the "data" or "parity" option of the failed disk to point to the place where you have enough empty space to recover the files.

For example, if you have that disk "d1" failed, you can change from:

to:

data d1 /mnt/new_spare_disk/

If the disk to recover is a parity disk, change the appropriate "parity" option. If you have more broken disks, change all their configuration options.

Run the fix command, storing the log in an external file with:

Where NAME is the name of the disk, like "d1" as in our previous example. In case the disk to recover is a parity disk, use the "parity", "2-parity" names. If you have more broken disks, use multiple -d options to specify all of them.

This command will take a long time.

Take care that you need also few gigabytes free to store the fix.log file. Run it from a disk with some free space.

Now you have recovered all the recoverable. If some file is partially or totally unrecoverable, it will be renamed adding the ".unrecoverable" extension.

You can get a detailed list of all the unrecoverable blocks in the fix.log file checking all the lines starting with "unrecoverable:"

If you are not satisfied of the recovering, you can retry it as many time you wish.

For example, if you have removed files from the array after the last "sync", this may result in some other files not recovered. In this case, you can retry the "fix" using the -i, --import option, specifying where these files are now, to include them again in the recovering process.

If you are satisfied of the recovering, you can now proceed further, but take care that after syncing you cannot retry the "fix" command anymore!

As paranoid check, you can now run a "check" command to ensure that everything is OK on the recovered disk.

Where NAME is the name of the disk, like "d1" as in our previous example.

The options -d and -a tell SnapRAID to check only the specified disk, and ignore all the parity data.

This command will take a long time, but if you are not paranoid, you can skip it.

Run the "sync" command to re-synchronize the array with the new disk.

If everything is recovered, this command is immediate.

Prints a summary of the state of the disk array.

It includes information about the parity fragmentation, how old are the blocks without checking, and all the recorded silent errors encountered while scrubbing.

Note that the information presented refers at the latest time you run "sync". Later modifications are not taken into account.

If bad blocks were detected, their block numbers are listed. To fix them, you can use the "fix -e" command.

It also shows a graph representing the last time each block was scrubbed or synced. Scrubbed blocks are shown with ´*´, blocks synced but not yet scrubbed with ´o´.

Nothing is modified.

Prints a SMART report of all the disks of the array.

It includes an estimation of the probability of failure in the next year allowing to plan maintenance replacements of the disks that show suspicious attributes.

This probability estimation obtained correlating the SMART attributes of the disks, with the Backblaze data available at:

If SMART reports that a disk is failing, "FAIL" or "PREFAIL" is printed for that disk, and SnapRAID returns with an error. In this case an immediate replacement of the disk is highly recommended.

Other possible strings are:

If the -v, --verbose option is specified a deeper statistical analysis is provided. This analysis can help you to decide if you need more or less parity.

This command uses the "smartctl" tool, and it´s equivalent to run "smartctl -a" on all the devices.

If your devices are not auto-detected correctly, you can configure a custom command using the "smartctl" option in the configuration file.

Nothing is modified.

Spins up all the disks of the array.

You can spin-up only some specific disks using the -d, --filter-disk option.

Take care that spinning-up all the disks at the same time needs a lot of power. Ensure that your power-supply can sustain that.

Nothing is modified.

Spins down all the disks of the array.

This command uses the "smartctl" tool, and it´s equivalent to run "smartctl -s standby,now" on all the devices.

You can spin-down only some specific disks using the -d, --filter-disk option.

Nothing is modified.

Lists all the files modified from the last "sync" that need to have their parity data recomputed.

This command doesn´t check the file data, but only the file time-stamp size and inode.

At the end of the command, you´ll get a summary of the file changes grouped by:

If a "sync" is required, the process return code is 2, instead of the default 0. The return code 1 is instead for a generic error condition.

Nothing is modified.

Updates the parity information. All the modified files in the disk array are read, and the corresponding parity data is updated.

You can stop this process at any time pressing Ctrl+C, without losing the work already done. At the next run the "sync" process will start where interrupted.

If during the process, silent or input/output errors are found, the corresponding blocks are marked as bad.

Files are identified by path and/or inode and checked by size and time-stamp. If the file size or time-stamp are different, the parity data is recomputed for the whole file. If the file is moved or renamed in the same disk, keeping the same inode, the parity is not recomputed. If the file is moved to another disk, the parity is recomputed, but the previously computed hash information is kept.

The "content" and "parity" files are modified if necessary. The files in the array are NOT modified.

Scrubs the array, checking for silent or input/output errors in data and parity disks.

For each command invocation, about the 8% of the array is checked, but nothing that was already scrubbed in the last 10 days. This means that scrubbing once a week, every bit of data is checked at least one time every three months.

You can define a different scrub plan or amount using the -p, --plan option that takes as argument: bad - Scrub blocks marked bad. new - Scrub just synced blocks not yet scrubbed. full - Scrub everything. 0-100 - Scrub the exact percentage of blocks.

If you specify a percentage amount, you can also use the -o, --older-than option to define how old the block should be. The oldest blocks are scrubbed first ensuring an optimal check. If instead you want to scrub the just synced blocks, not yet scrubbed, you should use the "-p new" option.

To get the details of the scrub status use the "status" command.

For any silent or input/output error found the corresponding blocks are marked as bad in the "content" file. These bad blocks are listed in "status", and can be fixed with "fix -e". After the fix, at the next scrub they will be rechecked, and if found corrected, the bad mark will be removed. To scrub only the bad blocks, you can use the "scrub -p bad" command.

It´s recommended to run "scrub" only on a synced array, to avoid to have reported error caused by unsynced data. These errors are recognized as not being silent errors, and the blocks are not marked as bad, but such errors are reported in the output of the command.

Files are identified only by path, and not by inode.

The "content" file is modified to update the time of the last check of each block, and to mark bad blocks. The "parity" files are NOT modified. The files in the array are NOT modified.

Fix all the files and the parity data.

All the files and the parity data are compared with the snapshot state saved in the last "sync". If a difference is found, it´s reverted to the stored snapshot.

The "fix" command doesn´t differentiate between errors and intentional modifications. It unconditionally reverts the file state at the last "sync".

If no other option is specified the full array is processed. Use the filter options to select a subset of files or disks to operate on.

To only fix the blocks marked bad during "sync" and "scrub", use the -e, --filter-error option. As difference from other filter options, with this one the fixes are applied only to files that are not modified from the latest "sync".

All the files that cannot be fixed are renamed adding the ".unrecoverable" extension.

Before fixing, the full array is scanned to find any moved file, after the last "sync" operation. These files are identified by their time-stamp, ignoring their name and directory, and are used in the recovering process if necessary. If you moved some of them outside the array, you can use the -i, --import option to specify additional directories to scan.

Files are identified only by path, and not by inode.

The "content" file is NOT modified. The "parity" files are modified if necessary. The files in the array are modified if necessary.

Verify all the files and the parity data.

It works like "fix", but it only simulates a recovery and no change is written in the array.

This command is mostly intended for manual verification, like after a recovery process or in other special conditions. For periodic and scheduled checks uses "scrub".

If you use the -a, --audit-only option, only the file data is checked, and the parity data is ignored for a faster run.

Files are identified only by path, and not by inode.

Nothing is modified.

Lists all the files contained in the array at the time of the last "sync".

Nothing is modified.

Lists all the duplicate files. Two files are assumed equal if their hashes are matching. The file data is not read, but only the pre-computed hashes are used.

Nothing is modified.

Creates or updates in the "pooling" directory a virtual view of all the files of your disk array.

The files are not really copied here, but just linked using symbolic links.

When updating, all the present symbolic links and empty sub-directories are deleted and replaced with the new view of the array. Any other regular file is left in place.

Nothing is modified outside the pool directory.

Prints the low level devices used by the array.

This command prints the devices associations in place in the array, and it´s mainly intended as a script interface.

The first two columns are the low level device id and path. The next two columns are the high level device id and path. The latest column if the disk name in the array.

In most cases you have one low level device for each disk in the array, but in some more complex configurations, you may have multiple low level devices used by a single disk in the array.

Nothing is modified.

Sets arbitrarily the sub-second time-stamp of all the files that have it at zero.

This improves the SnapRAID capability to recognize moved and copied files as it makes the time-stamp almost unique, removing possible duplicates.

More specifically, if the sub-second time-stamp is not zero, a moved or copied file is identified as such if it matches the name, size and time-stamp. If instead the sub-second time-stamp is zero, it´s considered a copy only if it matches the full path, size and time-stamp.

Note that the second precision time-stamp is not modified, and all the dates and times of your files will be maintained.

Schedules a rehash of the whole array.

This command changes the hash kind used, typically when upgrading from a 32 bits system to a 64 bits one, to switch from MurmurHash3 to the faster SpookyHash.

If you are already using the optimal hash, this command does nothing and tells you that nothing has to be done.

The rehash isn´t done immediately, but it takes place progressively during "sync" and "scrub".

You can get the rehash state using "status".

During the rehash, SnapRAID maintains full functionality, with the only exception of "dup" not able to detect duplicated files using a different hash.

Defines the files to use to store the parity information. The parity enables the protection from a single disk failure, like RAID5.

You can specify multiples files that should be in different disks. When a file cannot grow anymore, the next one is used. The total space available must be as big as the biggest data disk in the array.

You can add additional parity files at later time, but you cannot reorder or remove them.

Leaving the parity disks reserved for parity ensures that it doesn´t get fragmented, improving the performance.

In Windows 256 MB are left unused in each disk to avoid the warning about full disks.

This option is mandatory and it can be used only one time.

Defines the files to use to store extra parity information.

For each parity specified, one additional level of protection is enabled:

• 2-parity enables RAID6 dual parity.
• 3-parity enables triple parity
• 4-parity enables quad (four) parity
• 5-parity enables penta (five) parity
• 6-parity enables hexa (six) parity

Each parity level requires the presence of all the previous parity levels.

The same considerations of the ´parity´ option apply.

These options are optional and they can be used only one time.

Defines an alternate file and format to store the triple parity.

This option is an alternative at ´3-parity´ mainly intended for low-end CPUs like ARM or AMD Phenom, Athlon and Opteron that don´t support the SSSE3 instructions set. In such cases it provides a better performance.

This format is similar, but faster, at the one used by the ZFS RAIDZ3. Like ZFS, it doesn´t work beyond triple parity.

When using ´3-parity´ you will be warned if it´s recommended to use the ´z-parity´ format for a performance improvement.

It´s possible to convert from one format to another, adjusting the configuration file with the wanted z-parity or 3-parity file, and using ´fix´ to recreate it.

Defines the file to use to store the list and check-sums of all the files present in your disk array.

It can be placed in the disk used to store data, parity, or any other disk available. If you use a data disk, this file is automatically excluded from the "sync" process.

This option is mandatory and it can be used more times to save more copies of the same files.

You have to store at least one copy for each parity disk used plus one. Using some more doesn´t hurt.

Defines the name and the mount point of the data disks of the array. NAME is used to identify the disk, and it must be unique. DIR is the mount point of the disk in the file-system.

You can change the mount point as you like, as long you keep the NAME fixed.

You should use one option for each data disk of the array.

You can rename later a disk, changing the NAME directly in the configuration file, and then run a ´sync´ command. In the rename case, the association is done using the stored UUID of the disks.

Excludes all the hidden files and directory. In Unix hidden files are the ones starting with ".". In Windows they are the ones with the hidden attribute.

Defines the file or directory patterns to exclude and include in the sync process. All the patterns are processed in the specified order.

If the first pattern that matches is an "exclude" one, the file is excluded. If it´s an "include" one, the file is included. If no pattern matches, the file is excluded if the last pattern specified is an "include", or included if the last pattern specified is an "exclude".

See the Pattern section for more details in the pattern specifications.

This option can be used many times.

Defines the basic block size in kibi bytes of the parity. One kibi bytes is 1024 bytes.

The default blocksize is 256 and it should work for most cases.

WARNING! This option is for experts only, and it´s highly recommended to not change it. To change again this value in future you´ll have to recreate the whole parity!

A reason to use a different blocksize is if you have a lot of small files. In the order of many millions.

For each file, even of few bytes, a whole block of parity is allocated, and with many files this may result in a lot of unused parity space. And when you completely fill the parity disk, you are not allowed to add more files in the data disks. Anyway, the wasted parity doesn´t sum between data disks. Wasted space resulting from a high number of files in a data disk, limits only the amount of data in such data disk and not in others.

As approximation, you can assume that half of the block size is wasted for each file. For example, with 100000 files and a 256 KiB block size, you are going to waste 13 GB of parity, that may result in 13 GB less space available in the data disk.

You can get the amount of wasted space in each disk using "status". This is the amount of space that you must leave free in the data disks, or use for files not included in the array. If this value is negative, it means that your are near to fill the parity, and it represents the space you can still waste.

To avoid the problem, you can use a bigger partition for parity. For example, if you have the parity partition bigger than 13 GB than data disks, you have enough extra space to handle up to 100000 files in each data disk.

A trick to get a bigger parity partition in Linux, is to format it with the command:

This results in about 1.5% of extra space. Meaning about 60 GB for a 4 TB disk, that allows about 460000 files in each data disk without any wasted space.

Defines the hash size in bytes of the saved blocks.

The default hashsize is 16 bytes (128 bits), and it should work for most cases.

WARNING! This option is for experts only, and it´s highly recommended to not change it. To change again this value in future you´ll have to recreate the whole parity!

A reason to use a different hashsize is if your system has small memory. As a rule of thumb SnapRAID usually requires 1 GiB of RAM memory for each 16 TB of data in the array.

Specifically, to store the hashes of the data, SnapRAID requires about TS*(1+HS)/BS bytes of RAM memory. Where TS is the total size in bytes of your disk array, BS is the block size in bytes, and HS is the hash size in bytes.

For example with 8 disks of 4 TB and a block size of 256 KiB (1 KiB = 1024 bytes), and an hash size of 16, you get:

RAM = (8 * 4 * 10^12) * (1+16) / (256 * 2^10) = 1.93 GiB

Switching to a hash size of 8, you get:

RAM = (8 * 4 * 10^12) * (1+8) / (256 * 2^10) = 1.02 GiB

Switching to a block size of 512, you get:

RAM = (8 * 4 * 10^12) * (1+16) / (512 * 2^10) = 0.96 GiB

Switching to both a hash size of 8, and a block size of 512 you get:

RAM = (8 * 4 * 10^12) * (1+8) / (512 * 2^10) = 0.51 GiB

Automatically save the state when syncing or scrubbing after the specified amount of GB processed. This option is useful to avoid to restart from scratch long "sync" commands interrupted by a machine crash, or any other event that may interrupt SnapRAID.

Defines the pooling directory where the virtual view of the disk array is created using the "pool" command.

The directory must already exist.

Defines the Windows UNC path required to access the disks remotely.

If this option is specified, the symbolic links created in the pool directory use this UNC path to access the disks. Without this option the symbolic links generated use only local paths, not allowing to share the pool directory in the network.

The symbolic links are formed using the specified UNC path, adding the disk name as specified in the "disk" option, and finally adding the file dir and name.

This option is only required for Windows.

Defines a custom smartctl command to obtain the SMART attributes for each disk. This may be required for RAID controllers and for some USB disk that cannot be auto-detected.

DISK is the same disk name specified in the "disk" option. Parity is one of the parity name as "parity,(1,2,3,4,5,6,z)-parity".

In the specified Options, the "%s" string is replaced by the device name. Note that in case of RAID controllers the device is likely fixed, and you don´t have to use "%s".

Refers at the smartmontools documentation about the possible options:

An example of a typical configuration for Unix is:

An example of a typical configuration for Windows is: