||En/Decryption plugin for dd_rescue
ddr_crypt - Data de/encryption plugin for dd_rescue
This plugin allows to en/decrypt data on the fly as it is written out by
dd_rescue. It supports a variance of AES ciphers and uses the hardware
acceleration on x86 (AESNI) and ARMv8 -- if available -- to provide good
This plugin has been written for dd_rescue and uses the
plugin interface from it. See the dd_rescue(1) man page for more
information on dd_rescue.
Options are passed using dd_rescue option passing syntax: The name of the
plugin (crypt) is followed by an equal sign (=) and options are separated by a
colon (:). the crypt plugin also allows for some options to be
abbreviated. At least encryption or decryption and a way to determine the key
(and IV) needs to be passed.
Pass help to have ddr_crypt output a short list of options.
The crypt dd_rescue plugin (subsequently referred to as just ddr_crypt which
reflects the variable parts of the file name libddr_crypt.so) need to be told
whether to encrypt or decrypt. This is done by specifying enc[rypt] or
dec[rypt] parameters on the command line.
The crypt plugin supports a number of the de/encryption variants of AES. You can
specify which one you want to use by passing
There are several implementations of the AES algorithms that this plugin can
- where xxx can be 128, 192, 256, 128+, 192+, 256+, 128x2, 192x2, or 256x2,
and yyy can be ECB, CBC, or CTR. Pass algo=help to get a list of
available algorithms. See section ALGORITHMS AND STREAM MODES for a
discussion of the options. Note that decrypting a file can only be
successful if the exact same algorithms is chosen as was used for
encryption. There is no way to tell from an encrypted file which algorithm
has been chosen.
The default (AES192-CTR) is a good choice if you can ensure that the key/IV
combination is NEVER reused.
- This will use an own AES implementation using the AESNI instructions of
the intel x86 core CPUs since Westmere. If possible, this should be used,
as it provides both superior performance and avoids some of the cache
timing attacks that the lookup tables used in discrete implementations may
be prone to. This engine supports all algorithms.
- This will use an own AES implementation using the AESv8 crypto extensions
of the ARMv8 (AArch64 = ARM 64bit) architecture (available on CPUs/SOCs
based on ARM's Cortex A53,A57,A72 or Qualcomm's Kryo or Apple's A7x or
newer designs). If possible, this should be used, as it provides both
superior performance and avoids some of the cache timing attacks that the
lookup tables used in discrete implementations may be prone to. This
engine supports all algorithms.
- This uses an AES implementation in C which is based on the rijndael
reference code. It is available on all CPUs and supports all algorithms.
Some prefetching is done on the tables to make cache timing attacks a bit
harder, but this is most likely insufficient to thwart sophisticated
- This uses the openssl library to implement the AES routines, which should
take advantage of hardware acceleration where available and have received
some hardening against attacks. Note that the openssl implementation does
not support the + variants (increased number of rounds) that the other
The engines are used in this order if available, which means that
aesni will be used by default on x86 if supported, aesarm64
will be used on ARMv8 and aes_c by default on all other
As AES is a block cipher (with a block size of 16 bytes), files which are not a
multiple of the block size need padding at the end to have a full block.
- The last block is filled up with zeroes and then encrypted. Note that on
decryption, these zeroes can't be automatically stripped, as the decryptor
has no way to tell whether there are true trailing zeros or whether those
had been added by padding. This option is thus not recommended for copying
files of arbitrary length.
- This uses the PKCS7 scheme used by openssl, appending one to sixteen bytes
with the number of bytes to discard when decrypting. This is always safe,
but always makes the output file larger than the input file. Use this when
copying files. This is the default.
- This is a hybrid between zero and always. This does PKCS7 padding when the
file size does not fill a complete block, but does no padding when it
does. This mechanism has in the worst case (no pad bytes) a chance of
1/255 to produce an incorrect decryption result by wrong unpadding (which
means that it has an overall chance of misrepresenting the final file size
of 1/4080 for arbitrary file sizes). Don't use this unless you can deal
with such errors (or exclude them)!
Note that the CTR stream mode does NOT require padding and indeed
ddr_crypt does not apply any padding regardless of the pad option when it is
The debug parameter makes the ddr_crypt module output some debugging
information. With outkeyiv the key and IV will be output. The
benchmark parameter will result in reporting the en/decryption speed.
There are many ways to set the key and IV (for -CBC and -CTR stream modes).
They can be set directly using keyhex= and ivhex= on the command
line. The argument is interpreted as a hex ASCII representation of the key/IV
(without leading 0x). This way to specify keys/IVs should only be used for
testing - the key will be visible to all local users by looking at the command
line (unless specific measures are taken to lock down access to /proc). Not
normally a good idea on a multi-user system. ddr_crypt does overwrite the
sensitive data with X to make the attack a bit harder, but this still leaks
the length and -- more importantly -- there is still a time window where the
sensitive data is visible.
It's better to pass in the key and IV via a file descriptor via keyfd=
and ivfd= . If the integer is preceded by an x, the key/IV will be read
as hex string, otherwise binary data will be used. Optionally, @OFF@LEN can be
appended, designating the offset and length (in bytes) to be read in the file
passed via the file descriptor. Note that a file descriptor of 0 without shell
redirection will result in an interactive prompt to the user and the answer
won't be echoed to the screen of course.
This is useful mainly when dd_rescue is called from another program.
Alternatively, with keyfile= and ivfile= a file name
to be opened and read from can be specified. The syntax does support the
same optional @OFF@LEN designation, but the key and IV will always be read
in binary form. (See below, Index files for a way to read in hex form.)
Currently (unlike with salt) there is no way to use ddr_crypt to write out
binary key and IV data with these options.
The Operating System's random number generator can be used to generate key and
IV on the fly; if your system offers good random numbers, this is the most
secure way to specify and encryption key. The options to specify are
keygen and ivgen . You need to save the key/IV somehow,
otherwise you can not decrypt again later. (The program will warn you!) Best
way is to use the next options.
Keys and IVs can be stored as hex strings in index files; the file format is the
same as the one used in MD5SUMS: The hex representation of the key/IV is
followed by the file name. Obviously, appropriate care needs to be taken to
keep those files confidential.
If the ddr_crypt plugin gets the option keysfile and
ivsfile it will store already created keys/IVs (from the other
options) to files names KEYS.algname and IVS.algname in the MD5SUMS format.
(The files will be created or updated accordingly.) If key/IV have not been
created yet, ddr_crypt will try to retrieve the key/IV from those files and
error out upon failure.
Similar to index files, keys and IVs can also be stored in and retrieved from
the encrypted file's extended attributes. This can be achieved using the
options keyxattr and ivxattr . Please review the comments in the
main dd_rescue (1) man page for general considerations about using extended
These options combine well with keygen and ivgen on encryption (and should be
used alone on decryption).
Note that storing the key in the xattr is normally not a good idea. A
user who can access the encrypted file can (locally) also read the xattrs --
so the secrecy normally achieved by encryption is defeated this way. (There
may be valid scenarios, though, e.g. when the file tself is only accessible
via a remote protocol that does not expose the xattrs, such as http or DAV or
You can specify kxfallb[ack] and ixfallb[ack] in addition if you
want ddr_crypt to try using xattrs and falling back to keysfile and ivsfile in
case the file system does not support the extended attributes.
Using the same key/IV for many files harms security severely (see below in
ALGORITHMS). So using a directly specified (non-generated) key is not a good
idea. However, if you prefer to have something memorable rather than stored,
you can use a password and salt to generate many keys from one password.
The key and IV are derived from an expensive to compute (and even
more expensive to revert) function of password and salt. By default,
ddr_crypt uses 17000 rounds of pbkdf2() for the key (and a third for the
IV), although a more compute intense function (like scrypt) is planned for
the future. The expensiveness of this function is a protection against brute
forcing passwords. To use pbkdf2, you need to specify pbkdf2 or
pbkdf2=rounds . The latter format allows overriding the number of
iterations for key generation. (IV generation will be done with a third
For compatibility with openssl, key and IV can also be derived
using an openssl compatible key derivation function with opbkdf .
Note that this is not recommended; only one round of md5 hashing is used
which makes brute-forcing very effective. Using this option also has the
side-effect of writing (encryption) or parsing (decryption) an openSSL style
Salted__ header. Note the openssl version 1.1 started to default to one
round of sha256 hashing instead which can be forced on older openssl
versions with -md sha256 and overriden by specifying -md md5 on the openssl
command line. You can instruct dd_rescue to use an openssl compatible KDF
with sha256 by specifying opbkdf11 . One round of sha256 can of
course still be very efficiently brute-forced, so use high-entropy passwords
if you really need to use this.
The salt can be derived automatically from the name (and length)
of the encrypted file; this allows to work with just one password to be
memorized. However, be aware that file size or name changes will result in a
different salt and thus different key/IV which render your encrypted file
undecryptable. If there is a risk of this to happen, rather memorize one
salt per file (or better save key and IV using keysfile and ivsfile options
or save the salt using saltsfile or saltxattr, see below). Remember that
file names are case sensitive (as always with Un*x). Of course the keysfile
needs to be well protected from being read by unauthorized persons.
Password and salt can be specified with a string pass= and
salt= or using the passfd= passfile= and
salthex= saltfd= saltfile= options with the same
possible parameters as above for direct specification of key and IV. (Note
that the salt is hashed, like when derived from file name and length.) The
password/passphrase is treated as a string, null-terminated and with a
trailing CRLF stripped off.
The warnings about passing confidential data (here: pass, salt, salthex) on
the command line apply -- only do it for testing or in a single-user
If the file name based automatic salt derivation is used, the
assumed file length for salt generation can be overridden by saltlen=
Alternatively, the salts can also be stored and retrieved from an
MD5SUMS style index file (like with keysfile and ivsfile) by specifying the
option saltsfile . When saltsfile is used to store salts, using
random salts on encryption becomes a good idea. This can be achieved by
specifying the saltgen option.
Instead of a salt index file (saltsfile), the salt can also be
stored in (and retrieved from) an extended attribute. This can be done using
the saltxattr[=xattr_name] option. The attribute name is optional and
defaults to user.salt.ALGNAME (with ALGNAME replaced by the algorithm).
Since ddr_crypt 1.99, the password-based key derivation function (and the
number of iterations) is also stored and retrieved in the xattr user.pbkdf
with this option.
The AES (Rijndael) family of algorithms is considered cryptographically safe at
the time of writing, as no practicable attacks have been published against it.
It is up to the reader to judge whether (s)he believes that the worst
criminals or intelligence agencies are significantly ahead of common
(published) knowledge. In reality, it is typically easier to use social
engineering or flaws in key handling and random number generation to carry out
Given that the best known attacks are against AES versions with a reduced number
of rounds with only small round number reductions, it appears that increasing
the number of rounds would seem a reasonable countermeasure against
cryptographic attacks. (This has been inspired by a comment from Bruce
Schneier who the author of this document has very high respect for.)
It's also possible to try using the xattr feature and fall back to using the
index file (saltsfile) if your file system does not support extended
attributes. Use the sxfallback option to tell ddr_crypt to do this.
Note that the pbkdf can not be stored (or retrieved) if the fallback
actually takes place.
See the main dd_rescue (1) man page for a discussion of advantages and
disadvantages of using extended attributes.
The C and AESNI implementations support AES128,192,256 modes with
2,3,4 additional rounds respectively, resulting in 12, 15, 18 rounds. These
modes are named AES128+, AES192+, and AES256+ (plus modes) respectively.
They do offer a computationally relatively cheap way to enhance security.
The author of this document e.g. would chose AES192+ over AES256. While the
author of this document would never judge himself as a cryptography expert
strong enough to create new algorithms or even devise significant changes to
existing ones, he considers this variation a choice that is more secure than
the original. Please note however, that these custom algorithms result in
files that can not be decrypted using any other tools. Also, the openSSL
engine does not support the plus modes.
A computationally more expensive method to enhance security is doubling the
number of rounds. This is equivalent to encrypting twice (where the second key
is a simple derivation of the first). These methods are supported by all
engines and are named AES128x2, AES192x2, and AES256x2.
The AES algorithm is a block cipher -- it transforms 16 byte blocks. The trivial
application to a file of arbitrary size is to apply this to every block in the
file. This is called ECB (electronic codebook) mode. This is very insecure ...
the same input will always result in the same output. Patterns can be easily
recognized and known plain text attacks are trivial.
It's better to make the transformation dependent on the previous
content of the file or the position within it. This is what the CBC (chained
block cipher) and CTR modes do.
The CBC mode has several disadvantages: It can't be parallelized
(every block depends on all previous blocks for encryption; things are
better for decryption) and random access is impossible.
The CTR mode has many desirable properties. It is basically a
stream of (reproducible) pseudo random numbers that are XORed with the input
for encryption. Decryption is just another XOR of course. It's a one time
pad -- which has been proven to be secure, if the pad is unknown to an
attacker and only used once.
The latter can't be stressed enough: Don't ever use the same key/IV
combination for two files. Mathematically spoken: c1 = r1 XOR p1 and c2 = r2
XOR p2 (c = ciphertext, r = AES random numbers, p = plaintext). With r == r1
== r2, it can be trivially seen that the attacker can calculate c1 XOR c2 =
r XOR p1 XOR r XOR p2 = p1 XOR p2. If the plaintext of one of the files is
partially known, so is the other.
The CTR mode has more nice properties: It allows random access as
the AES random numbers (belonging to a key/IV combination) with a certain
offset can be directly calculated and the last block does not require
padding, as partial blocks can be processed.
The author of this documents prefers CTR stream mode and ensures
that keys/IVs are not reused.
With CTR mode, you can do partial writes to encrypted files and the result will
still be a consistent file (of course assuming that the used key and IV are
the same). Same with appending (-x) or reverse direction copies.
With ECB mode, this will only work, if file size and offsets are all block
(16byte) aligned. With CBC, none of this is possible.
The ddr_crypt plugin has no specific support for encoding holes;
if however previous correctly encrypted content is present in a hole, the
support for partial writes in CTR and ECB mode will result in a meaningful
output. If no previous content is in holes, then the result of decrypting
zeros will result upon decryption.
You can pass the option skiphole to make ddr_crypt leave 512byte blocks
of zeros untouched. This will reveal blocks of zeros and may thus disclose
valuable information to an attacker, so use with care. Also note that you
need to use this with en- and decryption and with the same alignment (mod
512) for encryption to be reversible. You have been warned. (You don't need
to be worried about misdetecting zeros on decrypting -- the chances of
non-zero plaintext resulting in an aligned 512byte block of zeros is smaller
than 2^-4096. So this option is safe on decrypting -- if some of the
ciphertext has been overwritten with blocks of zeros, you might even prefer
to have zeros in the decrypted file rather than random gibberish.)
Note that you can compress and encode holes with ddr_lzo and then
pass to ddr_crypt to encrypt and pass through ddr_crypt to decrypt and
ddr_lzo to uncompress and extract holes again. This only works with CTR
The option weakrnd is provided for testing in environments,
where strong random numbers are not available. It will cause weaker random
numbers to be used for key generation. Don't use it if you want
Files that are encrypted with openssl enc where you specify the key (with -K)
and the IV (with -iv) result in the same output that ddr_crypt generates for
-ECB and -CBC modes. ddr_crypt uses a 64bit counter in -CTR modes.
With the option opbkdf ddr_crypt also reads/writes the
openSSL Salted__ header to be compatible with openssl. This function needs
more testing and better error handling though.
The plugin is new as of dd_rescue 1.98. Do not yet rely on data saved with
ddr_crypt as the only backup for valuable data. Also expect some changes to
ddr_crypt in the not too distant future.
Due to an issue in ddr_crypt's initialization of the IV for CTR
mode, the last 32bits would always be zeroed out prior to adding the
counter. This has been fixed in 1.99. It order to be compatible with 1.98,
the option ctrbug198 can be specified on the command line.
While care has been applied to check the result of memory allocations ..., the
code has not been audited and only limited fuzzing has been applied to ensure
it's not vulnerable to malicious data -- be careful when you process data from
Key handling is a tricky business -- the author may have screwed up resulting in
some ways to use this program to encrypt data may not result in the level of
secrecy that is desired.
The crypt plugin does not yet have the same test coverage as the other plugins,
which means it has not been tested as intensively as the others.
Except for more testing and auditing a few more features are envisioned for this
Support for other (non-AES) algorithms such as twofish (and possibly also
Stronger function to derive keys/IVs from passwords than pbkdf2.
Support for other streaming modes (XTS, GCM, ...)
- dd_rescue -ptAL crypt=algo=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile infile outfile
- encrypts data from infile with AES256 in CTR mode using a generated
(random) key and IV and writes the result to outfile It adds a line
to KEYS.AES256-CTR and to IVS.AES256-CTR where the used key and IV are
written to respectively. (Please ensure that this file is not accessible
by any unauthorized person!) Decryption can be performed by
dd_rescue -ptAL crypt=algo=AES256-CTR:dec:keysfile:ivsfile outfile infile
Kurt Garloff <email@example.com>
The x86 AESNI optimized AES implementation has been inspired by an intel
whitepaper from 2009:
- dd_rescue -AL crypt=AES192+-CTR:enc:saltgen:saltxattr:sxfallback:passfd=0:pbkdf2 infile outfile
- will ask for a password, generate a random salt and store it in the
extended attribute of outfile (and fallback to SALTS.AES192+-CTR
index file) and uses pbkdf2 function to produce a key and IV for
encrypting the data. For decrypting, just omit the saltgen
- dd_rescue -ptL lzo=compr,crypt=AES256-CTR:enc:keygen:ivgen:keysfile:ivsfile infile outfile
- will compress the data (using lzo) and then encrypt. Use the reverse order
and omit keygen and ivgen to decrypt and uncompress. Compression has the
nice side effect of dealing with holes, which otherwise get compressed to
non-zero values (unless you specify skiphole). Feel free to add the hash
plugin at the beginning and/or the end to produce cryptographic checksums
for both the original file and the end result.
The ARMv8 AES support has been inspired by studying openSSL assembly as well as
Linaro's in-kernel implementation.
This plugin is under the same license as dd_rescue: The GNU General Public
License (GPL) v2 or v3 - at your option.
ddr_crypt plugin was first introduced with dd_rescue 1.98 (May 2015).
Version 1.99 brought support for ARMv8 crypto acceleration and support for
openssl style key derivation and Salted__ headers. It also added storing pbkdf
related infos in xattrs and added support for storing and retrieving keys (not
recommended!) and IVs in/from xattrs. A bug with CTR initialization was
resolved (see ctrbug198 option).
Some additional information can be found on
Visit the GSP FreeBSD Man Page Interface.
Output converted with ManDoc.