4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R

Review Request #223 - Created June 7, 2015 and submitted

Information
Matthew Ahrens
illumos-gate
4185
222
013cc43...
Reviewers
openzfs
skiselkov
4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R

Note that this allocates new checksum algorithm values.  If you have developed new checksum algorithms on a private branch, now would be a good time to speak up.

enum zio_checksum {
	...
	ZIO_CHECKSUM_NOPARITY, // == 10, last one in illumos
	ZIO_CHECKSUM_SHA512,   // == 11, proposed addition
	ZIO_CHECKSUM_SKEIN,    // == 12, proposed addition
	ZIO_CHECKSUM_EDONR,    // == 13, proposed addition
}

Note that this doesn't change any defaults, e.g. "zfs set dedup=on" still changes the checksum algorithm to sha256.  You need to explicitly opt in to use the new checksum algorithms.  We'll probably eventually change that default for dedup=on to sha512.  (This is the same strategy we used with checksum=on and lz4.)

Summary from the zpool-features manpage:

       sha512

           This feature enables the use of the SHA-512/256 truncated hash
           algorithm (FIPS 180-4) for checksum and dedup. The native 64-bit
           arithemtic of SHA-512 provides an approximate 50% performance boost
           over SHA-256 on 64-bit hardware and is thus a good minimum-change
           replacement candidate for systems where hash performance is
           important, but these systems cannot for whatever reason utilize the
           faster skein and edonr algorithms.

           Booting off of pools utilizing SHA-512/256 is supported (provided
           that the updated GRUB stage2 module is installed).

       skein

           This feature enables the use of the Skein hash algorithm for
           checksum and dedup. Skein is a high-performance secure hash
           algorithm that was a finalist in the NIST SHA-3 competition. It
           provides a very high security margin and high performance on 64-bit
           hardware (80% faster than SHA-256). This implementation also
           utilizes the new salted checksumming functionality in ZFS, which
           means that the checksum is pre-seeded with a secret 256-bit random
           key (stored on the pool) before being fed the data block to be
           checksummed. Thus the produced checksums are unique to a given
           pool, preventing hash collision attacks on systems with dedup.

       edonr

           This feature enables the use of the Edon-R hash algorithm for
           checksum, including for nopwrite (if compression is also enabled,
           an overwrite of a block whose checksum matches the data being
           written will be ignored).  In an abundance of caution, Edon-R can
           not be used with dedup (without verification).

           Edon-R is a very high-performance hash algorithm that was part of
           the NIST SHA-3 competition. It provides extremely high hash
           performance (over 350% faster than SHA-256), but was not selected
           because of its unsuitability as a general purpose secure hash
           algorithm.  This implementation utilizes the new salted
           checksumming functionality in ZFS, which means that the checksum is
           pre-seeded with a secret 256-bit random key (stored on the pool)
           before being fed the data block to be checksummed. Thus the
           produced checksums are unique to a given pool, blocking hash
           collision attacks on systems with dedup.

Original author: Matthew Ahrens & Saso Kiselkov

ztest, zfs test suite
(note that ztest uses the new algorithms)

http://jenkins.delphix.com/job/zfs-precommit/2447/

Issues

  • 2
  • 7
  • 0
  • 9
Description From Last Updated
Perhaps this should be "ZIO_CHECKSUM_SHA512_256" just to be clear to the uninitiated? I know that there's only space to store ... Justin Gibbs Justin Gibbs
Same chance for confusion here, but probably worse since this is visible from userspace utilities. Justin Gibbs Justin Gibbs
Rich Lowe
Josef 'Jeff' Sipek
Saso Kiselkov
Rich Lowe
Saso Kiselkov
Justin Gibbs
Matthew Ahrens
Rich Lowe
Matthew Ahrens
Review request changed

Status: Closed (submitted)

Loading...