Integrating GCP Secret Manager with NestJS Config

When building applications, managing sensitive information like API keys, database credentials, and other secrets is a critical security concern. Hardcoding these values or storing them in version control is a significant risk. Google Cloud Platform (GCP) offers a robust solution called Secret Manager for securely storing and managing such secrets. This guide will walk you through integrating GCP Secret Manager with a NestJS application, leveraging the @nestjs/config module for a seamless and secure configuration setup.

Why Use GCP Secret Manager?

• Centralized Secret Management: Provides a single place to manage secrets across your projects and applications.
• Strong Security: Secrets are encrypted at rest and in transit.
• Fine-grained Access Control: Uses IAM (Identity and Access Management) to control who and what can access secrets.
• Auditing: Provides audit logs for tracking access and changes to secrets.
• Versioning: Supports multiple versions of a secret, allowing you to roll back if needed.

Prerequisites

Before we begin, ensure you have the following:

• A GCP project with the Secret Manager API enabled.
• The gcloud CLI installed and authenticated.
• A NestJS project set up.
• Basic understanding of NestJS and its module system.

Step 1: Create a Secret in GCP Secret Manager

First, let's create a secret in GCP. We will store a simple JSON object containing our application's configuration.

1. Create a JSON file with your secrets: secrets.json

   {
     "DATABASE_USER": "my-db-user",
     "DATABASE_PASSWORD": "super-secret-password",
     "API_KEY": "a1b2c3d4e5f6"
   }
   

2. Create the secret in GCP using the gcloud CLI:

   gcloud secrets create my-app-secrets --data-file=secrets.json
   

   This command creates a new secret named my-app-secrets with the content of your secrets.json file.

Step 2: Set Up Service Account and Permissions

Your NestJS application will need credentials to authenticate with the GCP Secret Manager API. The best practice is to use a service account.

1. Create a Service Account:

   gcloud iam service-accounts create nestjs-secret-accessor \
     --description="Service account for NestJS to access Secret Manager" \
     --display-name="NestJS Secret Accessor"
   

2. Grant the Service Account Access to the Secret: You need to give the service account the "Secret Manager Secret Accessor" role for the specific secret you created.

   gcloud secrets add-iam-policy-binding my-app-secrets \
     --member="serviceAccount:nestjs-secret-accessor@<YOUR_GCP_PROJECT_ID>.iam.gserviceaccount.com" \
     --role="roles/secretmanager.secretAccessor"
   

   Replace <YOUR_GCP_PROJECT_ID> with your actual GCP project ID.

3. Create and Download a Key for the Service Account: For local development, you can use a service account key file. In a production environment (like Cloud Run or GKE), you should use the environment's built-in service account.

   gcloud iam service-accounts keys create sa-key.json \
     --iam-account="nestjs-secret-accessor@<YOUR_GCP_PROJECT_ID>.iam.gserviceaccount.com"
   

   This command creates a sa-key.json file. Treat this file as highly sensitive and do not commit it to version control! Add it to your .gitignore file.

Step 3: Install Required Packages

Now, let's add the necessary libraries to our NestJS project.

npm install @nestjs/config @google-cloud/secret-manager

• @nestjs/config: The official configuration module for NestJS.
• @google-cloud/secret-manager: The official Google Cloud client library for Secret Manager.

Step 4: Create a Custom Configuration Loader

The power of @nestjs/config lies in its ability to use custom configuration loaders. We will create a function that fetches our secret from GCP and parses it.

Create a new file, for example src/config/gcp.config.ts.

src/config/gcp.config.ts

import { SecretManagerServiceClient } from '@google-cloud/secret-manager';

const client = new SecretManagerServiceClient();

export const loadGcpSecrets = async () => {
  const secretName = 'projects/<YOUR_GCP_PROJECT_ID>/secrets/my-app-secrets/versions/latest';

  try {
    const [version] = await client.accessSecretVersion({
      name: secretName,
    });

    const payload = version.payload.data.toString('utf8');
    const secrets = JSON.parse(payload);

    // Merge secrets into process.env
    // This makes them available via process.env and configService.get()
    Object.assign(process.env, secrets);

    console.log('Successfully loaded secrets from GCP Secret Manager.');
    return secrets;
  } catch (error) {
    console.error('Failed to load secrets from GCP Secret Manager:', error);
    // In a real application, you might want to throw the error
    // to prevent the application from starting without its configuration.
    throw error;
  }
};

Important:

• Replace <YOUR_GCP_PROJECT_ID> with your project ID.
• The SecretManagerServiceClient will automatically use the credentials from the environment. For local development, it will look for the GOOGLE_APPLICATION_CREDENTIALS environment variable, which we will set to point to our sa-key.json.

Step 5: Integrate with AppModule

Now, we integrate our custom loader into the main application module using ConfigModule.

src/app.module.ts

import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { loadGcpSecrets } from './config/gcp.config';

@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true, // Make the config module available globally
      load: [loadGcpSecrets], // Use our custom loader
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

By using ConfigModule.forRoot, NestJS will execute our loadGcpSecrets function during application startup. The isGlobal: true option makes the ConfigService available for injection in any other module without needing to import ConfigModule again.

Step 6: Using the Secrets in Your Application

With the setup complete, you can now access your secrets in any service or controller by injecting the ConfigService.

src/app.service.ts

import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';

@Injectable()
export class AppService {
  constructor(private readonly configService: ConfigService) {}

  getHello(): string {
    // Access secrets using the ConfigService
    const dbUser = this.configService.get<string>('DATABASE_USER');
    const apiKey = this.configService.get<string>('API_KEY');

    console.log(`Database User: ${dbUser}`);
    console.log(`API Key: ${apiKey}`);

    return 'Hello World with secrets!';
  }
}

Step 7: Running the Application Locally

To run the application locally, you need to tell the GCP client library where to find your service account key.

Set the GOOGLE_APPLICATION_CREDENTIALS environment variable.

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/sa-key.json"
npm run start:dev

When your application starts, you should see the "Successfully loaded secrets..." message in your console, followed by the logs from AppService showing the retrieved secret values.

Production Considerations

• IAM Roles: In a production environment on GCP (like App Engine, Cloud Run, or GKE), you should not use a service account key file. Instead, assign the "Secret Manager Secret Accessor" role directly to the service account used by your compute resource. The client library will automatically pick up these credentials without any extra configuration.
• Error Handling: Our loadGcpSecrets function currently throws an error if it can't fetch the secrets, which will stop the application from starting. This is generally the desired behavior, as the application likely can't function correctly without its configuration. Ensure your deployment process can handle such startup failures.
• Caching: For performance and cost reasons, you might consider caching the configuration. However, ConfigModule loads the configuration only once on startup, so it's not a concern for repeated requests. If you need to refresh secrets, you'll need a more advanced strategy.

By integrating GCP Secret Manager with @nestjs/config, you create a robust, secure, and scalable way to manage your application's configuration, following best practices for modern cloud-native development.