Secret Management Pattern

Overview

The Secret Management pattern is a fundamental security mechanism in enterprise integration architectures that handles the secure storage, access, rotation, and lifecycle management of sensitive information such as passwords, API keys, certificates, and cryptographic keys. Like a high-security vault system that not only stores valuable items but also controls access, tracks usage, and regularly updates security measures, secret management ensures that sensitive configuration data and credentials are protected from unauthorized access while remaining accessible to authorized applications and services.

Core Principles

1. Centralized Secret Storage

Consolidating secret management into secure, dedicated systems: - Dedicated secret stores - using specialized systems designed for secret storage (HashiCorp Vault, AWS Secrets Manager) - Encryption at rest - ensuring all stored secrets are encrypted using strong cryptographic algorithms - Access control - implementing fine-grained permissions for secret access - Audit logging - tracking all secret access and management operations

2. Secret Lifecycle Management

Systematic control of secrets throughout their existence: - Secret creation - generating cryptographically strong secrets with appropriate entropy - Secret rotation - regularly updating secrets to maintain security - Secret versioning - maintaining multiple versions during rotation periods - Secret revocation - immediately invalidating compromised or obsolete secrets

3. Runtime Secret Injection

Securely delivering secrets to applications without persistent storage: - Just-in-time access - retrieving secrets only when needed - Memory-only storage - avoiding persistent storage of secrets in application memory - Secure transmission - encrypting secret delivery to applications - Temporary access - using time-limited secret access tokens

Implementation Patterns

Secret Management Service

@Service
public class SecretManagementService {

    @Autowired
    private VaultTemplate vaultTemplate;

    @Autowired
    private SecretEncryptionService encryptionService;

    @Autowired
    private SecretAuditService auditService;

    public void storeSecret(String secretPath, String secretName, String secretValue) {
        try {
            // Encrypt secret before storage
            EncryptedSecret encryptedSecret = encryptionService.encrypt(secretValue);

            // Store in vault
            Map<String, Object> secretData = Map.of(
                "value", encryptedSecret.getEncryptedValue(),
                "keyId", encryptedSecret.getKeyId(),
                "createdAt", Instant.now().toString(),
                "version", 1
            );

            vaultTemplate.write(secretPath + "/" + secretName, secretData);

            // Audit secret storage
            auditService.recordSecretStored(secretPath, secretName);

            log.info("Secret stored successfully: {}/{}", secretPath, secretName);

        } catch (Exception e) {
            log.error("Failed to store secret: {}/{}", secretPath, secretName, e);
            throw new SecretManagementException("Secret storage failed", e);
        }
    }

    public String retrieveSecret(String secretPath, String secretName) {
        try {
            // Read from vault
            VaultResponse response = vaultTemplate.read(secretPath + "/" + secretName);

            if (response == null || response.getData() == null) {
                throw new SecretNotFoundException("Secret not found: " + secretPath + "/" + secretName);
            }

            Map<String, Object> data = response.getData();
            String encryptedValue = (String) data.get("value");
            String keyId = (String) data.get("keyId");

            // Decrypt secret
            String secretValue = encryptionService.decrypt(encryptedValue, keyId);

            // Audit secret access
            auditService.recordSecretAccessed(secretPath, secretName);

            return secretValue;

        } catch (Exception e) {
            log.error("Failed to retrieve secret: {}/{}", secretPath, secretName, e);
            throw new SecretManagementException("Secret retrieval failed", e);
        }
    }

    public void rotateSecret(String secretPath, String secretName, String newSecretValue) {
        try {
            // Get current secret version
            VaultResponse currentResponse = vaultTemplate.read(secretPath + "/" + secretName);
            int currentVersion = currentResponse != null ? 
                (Integer) currentResponse.getData().getOrDefault("version", 0) : 0;

            // Encrypt new secret
            EncryptedSecret encryptedSecret = encryptionService.encrypt(newSecretValue);

            // Store new version
            Map<String, Object> newSecretData = Map.of(
                "value", encryptedSecret.getEncryptedValue(),
                "keyId", encryptedSecret.getKeyId(),
                "createdAt", Instant.now().toString(),
                "version", currentVersion + 1,
                "rotatedAt", Instant.now().toString()
            );

            vaultTemplate.write(secretPath + "/" + secretName, newSecretData);

            // Keep previous version for rollback
            if (currentResponse != null) {
                vaultTemplate.write(
                    secretPath + "/" + secretName + "/v" + currentVersion, 
                    currentResponse.getData()
                );
            }

            // Audit secret rotation
            auditService.recordSecretRotated(secretPath, secretName, currentVersion + 1);

            log.info("Secret rotated successfully: {}/{} to version {}", 
                    secretPath, secretName, currentVersion + 1);

        } catch (Exception e) {
            log.error("Failed to rotate secret: {}/{}", secretPath, secretName, e);
            throw new SecretManagementException("Secret rotation failed", e);
        }
    }
}

Secret Injection Service

@Service
public class SecretInjectionService {

    @Autowired
    private SecretManagementService secretManagementService;

    @Autowired
    private TemporarySecretCache temporarySecretCache;

    public Map<String, String> injectSecrets(Map<String, String> configuration) {
        Map<String, String> resolvedConfig = new HashMap<>(configuration);

        // Find secret references in configuration
        for (Map.Entry<String, String> entry : configuration.entrySet()) {
            String value = entry.getValue();

            if (isSecretReference(value)) {
                String secretValue = resolveSecretReference(value);
                resolvedConfig.put(entry.getKey(), secretValue);
            }
        }

        return resolvedConfig;
    }

    private boolean isSecretReference(String value) {
        return value != null && value.startsWith("${secret:") && value.endsWith("}");
    }

    private String resolveSecretReference(String secretReference) {
        // Parse secret reference: ${secret:path/secretName}
        String secretPath = secretReference.substring(9, secretReference.length() - 1);
        String[] parts = secretPath.split("/");

        if (parts.length < 2) {
            throw new IllegalArgumentException("Invalid secret reference: " + secretReference);
        }

        String path = String.join("/", Arrays.copyOf(parts, parts.length - 1));
        String name = parts[parts.length - 1];

        // Check temporary cache first
        String cachedSecret = temporarySecretCache.get(secretPath);
        if (cachedSecret != null) {
            return cachedSecret;
        }

        // Retrieve from secret store
        String secretValue = secretManagementService.retrieveSecret(path, name);

        // Cache temporarily (with short TTL)
        temporarySecretCache.put(secretPath, secretValue, Duration.ofMinutes(5));

        return secretValue;
    }
}

Apache Camel Implementation

@Component
public class SecretManagementRoute extends RouteBuilder {

    @Override
    public void configure() throws Exception {
        // Secret injection route for application startup
        from("direct:injectSecrets")
            .routeId("secret-injection-route")
            .process(exchange -> {
                @SuppressWarnings("unchecked")
                Map<String, String> configuration = exchange.getIn().getBody(Map.class);

                // Inject secrets into configuration
                Map<String, String> resolvedConfig = secretInjectionService.injectSecrets(configuration);

                exchange.getIn().setBody(resolvedConfig);
                exchange.getIn().setHeader("secrets-injected", true);
            })
            .log("Secrets injected successfully into configuration");

        // Scheduled secret rotation route
        from("quartz://secretRotation?cron=0+0+2+*+*+?") // Daily at 2 AM
            .routeId("secret-rotation-route")
            .process(exchange -> {
                // Get list of secrets that need rotation
                List<SecretMetadata> secretsToRotate = secretMetadataService.getSecretsNeedingRotation();

                for (SecretMetadata secret : secretsToRotate) {
                    try {
                        // Generate new secret value
                        String newSecretValue = secretGeneratorService.generateSecret(
                            secret.getType(),
                            secret.getComplexity()
                        );

                        // Rotate secret
                        secretManagementService.rotateSecret(
                            secret.getPath(),
                            secret.getName(),
                            newSecretValue
                        );

                        // Notify dependent services
                        notificationService.notifySecretRotated(secret);

                    } catch (Exception e) {
                        log.error("Failed to rotate secret: {}", secret.getFullPath(), e);
                        alertService.sendAlert("Secret rotation failed", secret.getFullPath());
                    }
                }
            })
            .log("Secret rotation process completed");
    }
}

Best Practices

1. Secret Storage Security

Use dedicated secret management systems rather than storing secrets in configuration files
Implement strong encryption for secrets at rest using industry-standard algorithms
Use hardware security modules (HSMs) for protecting encryption keys
Implement proper access controls with principle of least privilege
Enable comprehensive audit logging for all secret operations

2. Secret Lifecycle Management

Implement regular secret rotation schedules based on security policies
Use automated secret generation with sufficient entropy and complexity
Maintain secret versioning to support rollback scenarios
Implement proper secret retirement and cleanup procedures
Monitor secret usage patterns and detect anomalies

3. Runtime Security

Avoid storing secrets in application configuration files or environment variables
Use just-in-time secret retrieval rather than loading secrets at startup
Implement secure secret caching with appropriate TTL values
Clear secrets from memory immediately after use
Use secure channels for secret transmission to applications

4. Operational Excellence

Implement automated secret rotation and renewal processes
Provide secret management APIs for application integration
Create runbooks for secret emergency procedures and incident response
Implement monitoring and alerting for secret management operations
Maintain documentation of secret dependencies and impact analysis

5. Compliance and Governance

Implement secret classification and handling procedures
Maintain inventory of all secrets and their business criticality
Implement approval workflows for sensitive secret operations
Provide audit reports for compliance and security reviews
Ensure secret management practices meet regulatory requirements

The Secret Management pattern is essential for maintaining the security and integrity of enterprise integration systems by ensuring that sensitive configuration data and credentials are properly protected, managed, and controlled throughout their lifecycle.

← Back to All Patterns