File encryption techniques

We implement a file encryption scheme where you encrypt a file (AES-256 CBC PKCS#7) and sign it (AES-256 CMAC) by using unique keys per file. The keys, along with information about the encrypted file contents, are stored in a header at the beginning of the file. The file is encrypted and signed in 48-kilobyte chunks that are chained together by using SHA-512 hashes. Each chunk contains a header with a unique initialization vector.

Currently, we support two versions of the file encryption algorithm. The main difference between the versions is how the keys that encrypt and sign the file are protected and used.

Version 1 uses keys that the HSM protects directly as X9.24 keys (referred to as HSM Trusted keys in the 6.3.1.x nomenclature). To decrypt and verify the file, you must stream the whole file to the HSM because the value of these keys is only accessible on the HSM. Version 6.1.4.x handles all keys in this way.

Version 2 uses keys that the HSM data encryption keys protect. The data encryption keys are X9.24 keys, but you can decrypt the keys they protect into clear values outside of the HSM (known as HSM Protected keys in 6.3.1.x nomenclature). This enables you to perform encryption and decryption without streaming the file to the HSM. These files are commonly encrypted or decrypted on the client side for performance. You must have appropriate access to the HSM to retrieve the clear values of the keys from the encrypted file header, and any client should destroy the clear key values immediately after the encryption or decryption finishes.

The version 1 encrypted file uses two AES-256 HSM Trusted (X9.24) keys protected as an AES-KWP keyblock by an AES-256 HSM Trusted key stored on the application server. One key is tagged with decryption usage, and the other key is tagged with verification usage. The HSM can use these keys only for those specific usages. The key stored on the application server is protected as an AES-KWP keyblock by the AES-256 Platform Master Key (PMK) tagged with key wrapping and unwrapping usage.

The HSM generates two new random AES-256 keys protected by the PMK. One key has encrypt and decrypt usage, and the other key has signing and verification usage. The HSM translates both keys to be protected by the key wrapping key associated with the file encryption profile. After the translation, the new key blocks contain only the decrypt or verify usage. These translated keys are put into the header along with additional information such as the file name, size, hash, and the application-identity role performing the encryption operation. The PMK-wrapped version of these keys encrypts and signs the file and then discards it. Translating the versions from the file header back to PMK keys and using them to decrypt and verify the file requires proper authorization on the application server.

The version 2 encrypted file uses two AES-256 HSM Protected keys protected by an AES-256 HSM Trusted key stored on the application server. The key block format for the keys in the header, a proprietary format (fxgcm), uses AES-GCM. One key is tagged with decryption usage, and the other key is tagged with verification usage. The application server and client libraries respect these usages, but they are meaningless otherwise. The AES-256 Platform Master Key (PMK) tagged with data encryption and decryption usage protects the key stored on the application server as an AES-KWP key block.

The HSM generates 64 bytes of random data. 32 bytes of this data serve as the AES-256 file encryption key, and 32 bytes of this data serve as the AES-256 file signing key. The data is encrypted with the HSM Trusted data encryption key associated with the file encryption profile by using the -proprietary AES-GCM keyblock format, fxgcm. We call this key format as HSM Protected keys.

These key blocks are put into the header along with additional information like the file name, size, hash, and the application identity role performing the encryption operation. The system communicates the plaintext version of these keys with whichever entity performs the file encryption and signing. You should destroy the plaintext version of these keys immediately after the file has been protected. Retrieval of the plaintext keys for decryption and verification requires proper authorization on the application server.

The system encodes the header in Excrypt format and computes a 16-byte CMAC value over the header, appending it in Excrypt format. For each 48-kilobyte chunk of the file, a new 16-byte initialization vector is randomly generated for the CBC encryption. The block number (monotonically incremented starting at zero), the SHA-512 hash of the previous header concatenated with CMAC value, the size of the chunk in bytes, as well as the new IV are encoded in Excrypt format as the new block header. The next 48 kilobytes of the file data (or however much data remains) is then encrypted with CBC using IV and PKCS #7 padding. A 16-byte CMAC value is generated over the previous header concatenated with the cleartext value and appended to the end of the block ciphertext. If more data exists, then the next block header is generated and the same encryption and signing scheme repeats until no more data remains.