Backstage App-Config Structure and Plugin Configuration Patterns Analysis

Analysis Date: 2025-11-19 Project: repo-devex-backstage Analyzed by: Hive Mind Config Analysis Agent

---

Executive Summary

This analysis reviews the Backstage configuration structure and existing plugin patterns to ensure consistency for the Terraform State Backend plugin configuration.

Key Findings

1. Multi-tier Configuration Architecture: Base config (app-config.yaml) + environment-specific overlays
2. Environment Variable Pattern: ${VAR_NAME} with optional defaults ${VAR_NAME:-default}
3. Nested Provider Configuration: catalog.providers.<pluginName> for entity providers
4. Vault Integration Pattern: Plugins store sensitive tokens in Vault, not config files
5. Schedule Configuration: Consistent structure with frequency, timeout, initialDelay

---

Configuration File Structure

1. Configuration Files Hierarchy

backstage/
├── app-config.yaml                              # Base configuration (minimal, shared)
├── app-config.local.yaml                        # Local development (with secrets)
├── app-config.production.yaml                   # Production overrides (env vars only)
└── plugins/terraform-state-backend/dev/
    └── app-config.local.yaml                    # Plugin-specific local dev config

Pattern: Base config is minimal, environment-specific configs override and extend.

2. Configuration Sections

# Standard Backstage sections
app:                    # App metadata (title, baseUrl)
organization:           # Organization name
backend:                # Backend server config (baseUrl, database, auth)
auth:                   # Authentication providers
integrations:           # External integrations (GitHub, etc.)
catalog:                # Catalog configuration
  import:              # Import settings
  rules:               # Entity type rules
  locations:           # Static catalog locations
  providers:           # Dynamic entity providers ← PLUGIN CONFIGS HERE
permission:             # Permission framework
scaffolder:             # Software templates
techdocs:              # Documentation
kubernetes:            # Kubernetes integration (optional)

# Custom plugin sections
vault:                  # Vault secrets backend config
claude-flow:           # Claude Flow plugin config

---

Environment Variable Patterns

Convention Analysis

Format: ${ENVIRONMENT_VARIABLE_NAME} or ${VAR_NAME:-default_value}

Common Prefixes:

• AUTH_* - Authentication credentials
• POSTGRES_* - Database configuration
• BACKEND_* - Backend server settings
• APP_* - Frontend application settings
• VAULT_* - Vault configuration
• TECHDOCS_* - TechDocs configuration
• GCP_* - Google Cloud Platform settings

Examples from Production Config:

backend:
  baseUrl: ${BACKEND_BASE_URL}
  database:
    connection:
      host: ${POSTGRES_HOST}
      port: ${POSTGRES_PORT}
      user: ${POSTGRES_USER}
      password: ${POSTGRES_PASSWORD}

auth:
  providers:
    github:
      production:
        clientId: ${AUTH_GITHUB_CLIENT_ID}
        clientSecret: ${AUTH_GITHUB_CLIENT_SECRET}

Examples with Defaults (Local Config):

backend:
  auth:
    keys:
      - secret: ${BACKEND_SECRET:-h6ldBXk8U4lCCxkgmccBVZrhYJC/o9xx}

auth:
  providers:
    google:
      development:
        clientId: ${AUTH_GOOGLE_CLIENT_ID:-placeholder-client-id}

---

Catalog Provider Configuration Pattern

Standard Structure

All entity providers follow this pattern under catalog.providers.<pluginName>:

catalog:
  providers:
    <providerName>:
      <providerId>:                    # Unique identifier for this provider instance
        # Provider-specific configuration
        # ...

        # Standard fields across all providers
        schedule:
          frequency: { minutes: N }
          timeout: { minutes: N }
          initialDelay: { seconds: N }

Existing Examples

1. GitHub Organization Provider

catalog:
  providers:
    githubOrg:
      id: production
      githubUrl: https://github.com
      orgs: ['badal-io']
      schedule:
        frequency: { minutes: 10 }
        timeout: { minutes: 3 }
        initialDelay: { seconds: 30 }

2. GCP Projects Provider

catalog:
  providers:
    gcp:
      gcpProjects:
        organization: 'foundations.badal.io'
        schedule:
          frequency: { minutes: 15 }
          timeout: { minutes: 5 }
          initialDelay: { seconds: 60 }

3. GitHub Catalog Provider

catalog:
  providers:
    github:
      providerId:
        catalogPath: /catalog-info.yaml
        organization: badal-io
        schedule:
          frequency: { minutes: 5 }
          initialDelay: { seconds: 15 }
          timeout: { minutes: 3 }

---

Terraform State Plugin Configuration Analysis

Current Plugin Implementation

Plugin ID: terraform-state Provider Types:

• GcsStateProvider - Google Cloud Storage
• TfcStateProvider - Terraform Cloud

Config Path Expected: catalog.providers.terraformState

Provider Configuration Schema

GCS Provider Config Schema

interface GcsStateSourceConfig {
  id: string;                           // Provider instance ID
  bucketName: string;                   // GCS bucket name
  projectId?: string;                   // GCP project ID (optional, for ADC)
  prefix?: string;                      // Bucket prefix filter
  environment: 'non-production' | 'production';
  credentialsFile?: string;             // Path to credentials.json (dev only)
  schedule?: {
    frequency: { minutes: number };
    timeout: { minutes: number };
  };
}

TFC Provider Config Schema

interface TfcStateSourceConfig {
  id: string;                           // Provider instance ID
  organization: string;                 // Terraform Cloud organization
  workspacePrefix?: string;             // Workspace filter
  environment: 'non-production' | 'production';
  token: {
    value?: string;                     // Direct token (dev only)
    vaultPath?: string;                 // Vault path for token
  };
  schedule?: {
    frequency: { minutes: number };
    timeout: { minutes: number };
  };
  rateLimit?: {
    maxConcurrent: number;
    retryAfterMs: number;
    maxRetries: number;
  };
}

Recommended Configuration Structure

Based on existing patterns, the plugin should use this structure:

catalog:
  providers:
    terraformState:
      # GCS provider instances
      gcs-nonprod:
        type: gcs                       # Provider type discriminator
        bucketName: terraform-state-nonprod
        projectId: ${GCP_PROJECT_ID}    # Environment variable pattern
        prefix: 'backstage/'
        environment: non-production
        schedule:
          frequency: { minutes: 30 }
          timeout: { minutes: 5 }
          initialDelay: { seconds: 60 }
        sensitivityLevel: moderate

      gcs-prod:
        type: gcs
        bucketName: terraform-state-prod
        projectId: ${GCP_PROJECT_ID}
        prefix: 'backstage/'
        environment: production
        schedule:
          frequency: { minutes: 60 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 120 }
        sensitivityLevel: strict

      # Terraform Cloud provider instances
      tfc-nonprod:
        type: tfc                       # Provider type discriminator
        organization: Badal_devex
        workspacePrefix: 'wrkspc-np-'
        environment: non-production
        vaultPath: /backstage/backend/data/terraform-state-plugin/tfc-token#Badal_devex
        schedule:
          frequency: { minutes: 30 }
          timeout: { minutes: 5 }
          initialDelay: { seconds: 60 }
        sensitivityLevel: moderate
        rateLimit:
          maxConcurrent: 25
          retryAfterMs: 2000
          maxRetries: 3

---

Vault Integration Pattern

How Other Plugins Use Vault

1. Vault Configuration Section

vault:
  address: https://localhost:8200      # Vault server URL
  skipTLS: true                        # Skip TLS verification (dev only)
  paths:                               # Standardized secret paths
    userSecrets: /backstage/users
    groupSecrets: /backstage/groups
    backendSecrets: /backstage/backend

2. Vault Secrets Backend Plugin Pattern

The vault-secrets-backend plugin:

• Reads config from vault.address, vault.skipTLS, vault.paths.*
• Authenticates using environment variables or ADC
• Stores user/group/backend secrets in hierarchical paths

3. Terraform Cloud Provider Vault Pattern

The TFC provider expects:

• Vault Path: /backstage/backend/data/terraform-state-plugin/tfc-token#<organization>
• Token Retrieval: Via VaultTokenService class
• Fallback: Direct token value in config (dev only)

// From TfcStateProvider.ts
if (config.token.vaultPath || !config.token.value) {
  this.vaultTokenService = new VaultTokenService(backstageConfig, logger);
}

this.tfcClient = new TfcClient(
  {
    organization: config.organization,
    token: {
      value: config.token.value,      // Direct value (dev)
      vaultPath: config.token.vaultPath,  // Vault path (prod)
    },
  },
  this.vaultTokenService,             // Token provider
);

Vault Path Convention: /backstage/backend/data/<plugin-id>/<secret-type>#<identifier>

Example:

• Plugin ID: terraform-state-plugin
• Secret Type: tfc-token
• Organization: Badal_devex
• Full Path: /backstage/backend/data/terraform-state-plugin/tfc-token#Badal_devex

---

Security Patterns

1. Secrets Management

NEVER in Config Files (production):

• API tokens
• Passwords
• Private keys
• Service account credentials

Environment Variables (with defaults for dev):

clientSecret: ${AUTH_GOOGLE_CLIENT_SECRET:-placeholder-client-secret}

Vault (production):

vaultPath: /backstage/backend/data/plugin-name/secret-type#identifier

Credential Files (local dev only):

credentialsFile: /path/to/service-account.json  # Local dev only

2. GitHub Integration Pattern

GitHub secrets in local config:

integrations:
  github:
    - host: github.com
      apps:
        - appId: 1644052
          clientId: Iv23liJiVwdeMmoRoLXM
          clientSecret: 7b2d3d540c1111928c48f70b274c9073898e0005
          privateKey: |
            -----BEGIN RSA PRIVATE KEY-----
            [REDACTED - DEMO KEY ONLY]
            -----END RSA PRIVATE KEY-----

Production: Environment variables only:

auth:
  providers:
    github:
      production:
        clientId: ${AUTH_GITHUB_CLIENT_ID}
        clientSecret: ${AUTH_GITHUB_CLIENT_SECRET}

---

Plugin Registration Pattern

Backend Plugin Registration

File: backstage/packages/backend/src/index.ts

Pattern:

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

// Plugin registration
backend.add(import('@internal/plugin-terraform-state-backend'));

All Registered Plugins:

// Core Backstage
backend.add(import('@backstage/plugin-app-backend'));
backend.add(import('@backstage/plugin-catalog-backend'));
backend.add(import('@backstage/plugin-scaffolder-backend'));
backend.add(import('@backstage/plugin-auth-backend'));
backend.add(import('@backstage/plugin-permission-backend'));
backend.add(import('@backstage/plugin-search-backend'));

// Entity providers (catalog modules)
backend.add(import('@backstage/plugin-catalog-backend-module-github-org'));

// Custom internal plugins
backend.add(import('@internal/plugin-terraform-cloud-backend'));
backend.add(import('@internal/backstage-plugin-vault-secrets-backend'));
backend.add(import('@internal/plugin-claude-flow-backend'));
backend.add(import('@internal/plugin-terraform-state-backend'));

Entity Provider Registration: Providers are automatically discovered via fromConfig() static factory method.

Plugin Structure Pattern

Standard Backend Plugin:

import { createBackendPlugin, coreServices } from '@backstage/backend-plugin-api';

export const myPlugin = createBackendPlugin({
  pluginId: 'my-plugin-id',
  register(env) {
    env.registerInit({
      deps: {
        logger: coreServices.logger,
        config: coreServices.rootConfig,
        httpRouter: coreServices.httpRouter,
        // ... other services
      },
      async init({ logger, config, httpRouter }) {
        // Plugin initialization
        logger.info('My plugin loaded');

        // Optional: Register HTTP router
        httpRouter.use(await createRouter({ config, logger }));
      },
    });
  },
});

Entity Provider Plugin (like Terraform State):

export const terraformStatePlugin = createBackendPlugin({
  pluginId: 'terraform-state',
  register(env) {
    env.registerInit({
      deps: {
        logger: coreServices.logger,
      },
      async init({ logger }) {
        logger.info('Terraform State plugin loaded');
        logger.info('Entity providers must be registered manually in backend/src/index.ts');
      },
    });
  },
});

Note: Entity providers are NOT registered in the plugin itself. They must be manually registered in backend/src/index.ts using the catalog extension point.

---

Recommended Configuration Approach

1. Configuration Naming Conventions

Environment Variables:

• TERRAFORM_STATE_GCS_BUCKET_<ENV> - GCS bucket names
• TERRAFORM_STATE_GCP_PROJECT - GCP project ID
• VAULT_TERRAFORM_CLOUD_TOKEN_PATH - Vault path for TFC tokens

Config Keys:

• catalog.providers.terraformState - Root config path
• Type discriminator: type: gcs or type: tfc
• Instance ID: descriptive name (e.g., gcs-nonprod, tfc-prod)

2. Multi-Environment Pattern

app-config.yaml (base):

# Minimal - no provider configs (too environment-specific)
catalog:
  rules:
    - allow: [Component, System, API, Resource, Location]

app-config.local.yaml (development):

catalog:
  providers:
    terraformState:
      gcs-local-emulator:
        type: gcs
        bucketName: terraform-state-local
        apiEndpoint: http://localhost:4443  # GCS emulator
        projectId: local-dev-project
        prefix: 'test/'
        environment: non-production
        schedule:
          frequency: { minutes: 5 }
          timeout: { minutes: 2 }

      tfc-dev:
        type: tfc
        organization: Badal_devex
        workspacePrefix: 'test-'
        environment: non-production
        token:
          value: ${TERRAFORM_CLOUD_TOKEN:-placeholder}  # Fallback for dev
        schedule:
          frequency: { minutes: 10 }
          timeout: { minutes: 2 }

app-config.production.yaml (production):

catalog:
  providers:
    terraformState:
      gcs-prod:
        type: gcs
        bucketName: ${TERRAFORM_STATE_GCS_BUCKET_PROD}
        projectId: ${GCP_PROJECT_ID}
        prefix: 'production/'
        environment: production
        schedule:
          frequency: { minutes: 60 }
          timeout: { minutes: 10 }
          initialDelay: { seconds: 120 }
        sensitivityLevel: strict

      tfc-prod:
        type: tfc
        organization: ${TERRAFORM_CLOUD_ORGANIZATION}
        workspacePrefix: ${TERRAFORM_CLOUD_WORKSPACE_PREFIX}
        environment: production
        vaultPath: /backstage/backend/data/terraform-state-plugin/tfc-token#${TERRAFORM_CLOUD_ORGANIZATION}
        schedule:
          frequency: { minutes: 30 }
          timeout: { minutes: 5 }
          initialDelay: { seconds: 60 }
        sensitivityLevel: strict

3. Security Best Practices

Development:

• Use emulators (GCS emulator, local Vault)
• Hardcode test credentials in app-config.local.yaml
• Add app-config.local.yaml to .gitignore

Production:

• ALL credentials via environment variables
• Sensitive tokens in Vault
• Use ${VAR_NAME} without defaults for required secrets
• Use Application Default Credentials (ADC) for GCP

---

Configuration Validation

Type Safety

The provider config is validated by TypeScript interfaces:

// GCS validation
const providerConfig: GcsStateSourceConfig = {
  id: `gcs-${id}`,
  bucketName: sourceConfig.bucketName,        // Required
  projectId: sourceConfig.projectId,          // Optional
  prefix: sourceConfig.prefix,                // Optional
  environment: sourceConfig.environment || 'non-production',
  credentialsFile: sourceConfig.credentialsFile,
  schedule: sourceConfig.schedule,
};

// TFC validation
const providerConfig: TfcStateSourceConfig = {
  id: `tfc-${id}`,
  organization: sourceConfig.organization,    // Required
  workspacePrefix: sourceConfig.workspacePrefix,
  environment: sourceConfig.environment || 'non-production',
  token: {
    value: sourceConfig.token?.value,
    vaultPath: sourceConfig.token?.vaultPath,
  },
  schedule: sourceConfig.schedule,
  rateLimit: sourceConfig.rateLimit,
};

Runtime Validation

Provider factory methods validate config:

static fromConfig(config: Config, options): Provider[] {
  const configs = config.getOptionalConfig('catalog.providers.terraformState');

  if (!configs) {
    return [];  // No providers configured
  }

  for (const [id, sourceConfig] of Object.entries(configs.get())) {
    // Validate required fields
    if (!sourceConfig.bucketName) {
      throw new Error(`bucketName required for GCS provider ${id}`);
    }

    providers.push(new GcsStateProvider(providerConfig, logger, schedule));
  }

  return providers;
}

---

Recommendations

1. Configuration Structure

Use nested provider structure:

catalog:
  providers:
    terraformState:
      <provider-id>:
        type: gcs | tfc
        # ... provider-specific config

Provider ID naming: <type>-<environment> (e.g., gcs-nonprod, tfc-prod)

2. Environment Variables

Define standard variables:

• GCP_PROJECT_ID - GCP project for GCS buckets
• TERRAFORM_CLOUD_ORGANIZATION - TFC organization name
• TERRAFORM_CLOUD_WORKSPACE_PREFIX - Workspace filter

Vault paths:

• /backstage/backend/data/terraform-state-plugin/tfc-token#<organization>

3. Schedule Configuration

Follow existing pattern:

schedule:
  frequency: { minutes: N }
  timeout: { minutes: N }
  initialDelay: { seconds: N }

Recommended intervals:

• Development: 5-10 minutes
• Non-production: 15-30 minutes
• Production: 30-60 minutes

4. Multi-Instance Support

Support multiple provider instances for:

• Different environments (nonprod/prod)
• Different organizations (multiple TFC orgs)
• Different buckets (regional separation)

Example:

catalog:
  providers:
    terraformState:
      gcs-us-nonprod:
        type: gcs
        bucketName: terraform-state-us-nonprod
        # ...

      gcs-eu-nonprod:
        type: gcs
        bucketName: terraform-state-eu-nonprod
        # ...

      tfc-devex:
        type: tfc
        organization: Badal_devex
        # ...

      tfc-platform:
        type: tfc
        organization: Badal_platform
        # ...

5. Type Discriminator

Add type field to distinguish GCS vs TFC providers:

terraformState:
  instance-id:
    type: gcs  # or 'tfc'

This allows parsing both provider types from the same config section.

---

Summary

Naming Conventions

Pattern	Example
Environment variables	${POSTGRES_HOST}, ${AUTH_GITHUB_CLIENT_ID}
Config paths	catalog.providers.terraformState
Provider IDs	gcs-nonprod, tfc-prod
Vault paths	/backstage/backend/data/<plugin>/<secret>#<id>

Configuration Hierarchy

1. app-config.yaml - Base, minimal shared config
2. app-config.local.yaml - Local dev with hardcoded secrets
3. app-config.production.yaml - Production with env vars only

Security Patterns

• Local dev: Hardcoded credentials in app-config.local.yaml (gitignored)
• Production: Environment variables + Vault for tokens
• GCP Auth: Application Default Credentials (ADC)
• TFC Auth: Vault token storage

Provider Registration

• Backend plugin registered in backend/src/index.ts via backend.add()
• Entity providers auto-discovered via fromConfig() static factory
• Config path: catalog.providers.<pluginId>

---

Next Steps for Coder Agent

1. Update Provider Factory: Add type discriminator handling
2. Environment Variables: Document required env vars in README
3. Vault Integration: Test VaultTokenService for TFC tokens
4. Multi-Instance: Support multiple GCS/TFC provider instances
5. Config Validation: Add runtime validation for required fields
6. Documentation: Create config examples for each environment

---

Analysis Complete - Ready for implementation phase.