Data-centric Security: The Importance and Advantages of Format Preserving Data Protection

Data-centric Security: The Importance and Advantages of Format Preserving Data Protection

The emergence of strict new data privacy regulations, such as the GDPR and CCPA, is driving the need for CISOs to more effectively address data protection and governance in complex and geographically diverse hybrid IT ecosystems. The terms pseudonymization and anonymization are now common in the context of these privacy regulations when it comes to data privacy and protection. While pseudonymization of data still allows for some form of re-identification (even indirect and remote), anonymization of data means it cannot be re-identified. CISOs look to the vendor community for data security solutions to address these privacy requirements but may struggle with the confusing array of security models and services.

Enterprises must choose a solution that offers a variety of data protection formats that not only allow pseudonymization and anonymization of sensitive data, but also enable business processes, applications, and analytics workloads to operate on the data in its protected state. The ability to protect high value data, with data utility and usability, is critical to achieving cyber resilience, the ability to evolve and adapt to rapidly changing threats and regulatory mandates. Format-Preserving Encryption (FPE) is a powerful data protection technology, and is currently becoming the de facto standard across the industry. FPE warrants a deeper examination, and the following section expands on FPE and its importance.

Format-Preserving Encryption

Format-Preserving Encryption refers to encrypting data in such a way that the output (the ciphertext) is in the same format as the input (the plaintext). “Format-preserving” implies that encrypting a 16-digit credit card number produces a ciphertext which is another 16-digit number; encrypting an English word produces a ciphertext comprising the same number of English characters; and so forth. These properties have several benefits and simplify data protection, especially for legacy applications, where it avoids major redesign and refactoring of applications and business processes:

1. Minimal or zero database schema impact – FPE facilitates retrofitting encryption technology to existing devices or software where conventional encryption modes would not be feasible. In particular, database applications may not support changes to data length or format.
2. Minimal or zero data storage impact – Since length preservation is an inherent property of FPE, enterprises do not have to worry about additional storage usage, unlike conventional (non‑format-preserving) encryption methods, which typically expand data.

Note: Some exceptions do apply where the length of the output with some variants of FPE can be slightly longer than that of the input data.

1. Analytics on protected data – Format-preserved protected data elements such as credit card numbers, SSNs, etc., can still be used as index keys to facilitate statistical research, even across databases. With FPE, the same inputs to the algorithm will create the same ciphertext. This deterministic encryption preserves the referential integrity of the data and thereby the ability to glean valuable information from the protected dataset. Other crucial benefits of Secure Analytics enabled by the use of FPE is expanding the access to data across a broader set of analysts, and potential monetization of data sets, without compromising on security and privacy.
2. Cross-application dataflow preservation – FPE lets protected data flow across applications without requiring changes to those applications to accept the protected data, an infeasible approach with conventional encryption methods, since applications require data of specific lengths and formats.
3. Using protected data without requiring decryption – FPE can allow protection of only specified key portions of data elements, enabling use of the data in its protected state. For example, the “first six” digits of credit card numbers are used for charge routing, and the “last four” of SSNs is used for customer verification. If these are left in the clear, many applications in the data flow will not need access to the entire data element, and can perform required business functions without requiring any change to the applications, and not requiring to perform any decryption. Such partial encryption can facilitate functions such as sort and certain search use cases, such as “Starts with”, “Ends with”, etc., without requiring any decryption of the encrypted data.
4. Test data management – FPE can also be used, especially in the form of its irreversible variant, to obfuscate/scrub production data to populate test databases, enabling realistic test conditions based on production volume, variability, etc.

NIST Special Publication 800-38G, Recommendation for Block Cipher Modes of Operation: Methods for Format-Preserving Encryption, specifies two AES modes, FF1 and FF3/FF3-1, for format-preserving encryption.

We’d love to hear your thoughts on this blog. Comment below.