Skip to main content

SecureField Key Management System (KMS)

The SecureField Key Management System (KMS) is a critical security component of ValkyrAI that manages encryption keys for protecting sensitive data. This system ensures that data is securely encrypted at rest and in transit, providing robust protection against unauthorized access.

Overview​

ValkyrAI includes a built-in SecureField key management system that automatically manages the keys associated with each row of data in your database. This system ensures that even if your default encryption key changes, previously encrypted records will still be decryptable.

The SecureField KMS provides:

  • Automatic key management for encrypted data
  • Key rotation capabilities
  • Support for multiple encryption keys
  • Secure storage of keys in the database

How SecureField Works​

SecureField is a powerful encryption mechanism provided by ThorAPI that ensures sensitive data is protected both at rest and in memory. By intercepting and wrapping field access, SecureField offers the following benefits:

  • Data Encryption: Automatically encrypts data when storing it in the database, safeguarding against unauthorized access.
  • On-Demand Decryption: Decrypts data transparently when it is accessed within your Java code, providing seamless integration.
  • Enhanced Security: Adds an additional layer of protection against data breaches, including raw data dumps or memory inspection.

SecureField also supports password hashing and plays a crucial role in ThorAPI's user authentication and authorization functionality. It delivers a standards-based, secure implementation for permissions and Access Control Lists (ACLs), enabling robust security management within your applications.

Key Management​

Default Operation​

By default, the SecureField KMS operates automatically, managing the key associated with each row of data in your database. The system uses the THORAPI_SECRET_KEY environment variable as the default encryption key.

Key Storage​

Encryption keys are stored in the SecureField database table. This table maintains a record of all keys used to encrypt data, ensuring that data can be decrypted even if the default key changes.

Key Rotation​

The SecureField KMS supports key rotation, allowing you to change your encryption keys periodically for enhanced security. When you change the default key (set with the THORAPI_SECRET_KEY environment variable), the KMS ensures that previously encrypted records remain decryptable.

Implementation Details​

Column Size for Encrypted Data​

One important factor to consider when using SecureField is that encrypted values are typically much larger (often 2x or more) than their plaintext equivalents. ThorAPI generates column sizes to accommodate these encrypted values and includes validations that account for the size of the plaintext.

This results in validations on field size that are sized differently to accommodate the larger size of encrypted data.

SecureField Annotations​

In your OpenAPI specifications, you can use the x-thorapi-secure-field annotation to indicate that a field should be encrypted. For example:

properties:
  creditCardNumber:
    type: string
    description: "The credit card number"
    x-thorapi-secure-field: true

This annotation tells ThorAPI to generate the necessary code to encrypt and decrypt this field automatically.

Setting Up SecureField KMS​

Generating a Secret Key​

A new THORAPI_SECRET_KEY can be generated with the generatekey command:

java -jar lib/generator-<valkyrai_version>-exec.jar generatekey

After generating the key, set it as an environment variable:

export THORAPI_SECRET_KEY=generatedPrivateKey

Important Security Considerations​

WARNING: YOUR DATA WILL BE PERMANENTLY LOST IF YOU LOSE OR OTHERWISE BUNGLE THE THORAPI_SECRET_KEY

It is critical to securely manage your encryption keys. If you lose access to your encryption keys, you will not be able to decrypt your data. Consider implementing a secure backup strategy for your encryption keys.

Best Practices​

  • Secure Key Storage: Store your THORAPI_SECRET_KEY in a secure location, such as a secrets manager or encrypted environment variable.
  • Regular Key Rotation: Rotate your encryption keys periodically to enhance security.
  • Backup Keys: Maintain secure backups of your encryption keys to prevent data loss.
  • Monitor Key Usage: Regularly audit key usage to detect potential security issues.
  • Limit Access: Restrict access to encryption keys to only those who absolutely need it.

Troubleshooting​

Encryption Failures​

If there is a problem with key management, or a key is corrupted or otherwise made unusable, the SecureField system will not be able to decrypt previously encrypted values and will return a standard error message such as:

{'error':'decryption failure: Tag mismatch'}

Since there are innumerable ways in which key data might be compromised or made unavailable, it is important to manage your keys carefully.

NOTE: As long as you have the valid key associated with a piece of encrypted data, you will always be able to decrypt it using the open source ThorAPI decryption utilities.

Future Enhancements​

The roadmap for SecureField KMS includes:

  • Pluggable KMS providers to support external key management systems
  • Enhanced key rotation policies
  • Improved key backup and recovery mechanisms
  • Integration with cloud-based key management services