Trigger Mechanisms

Overview

The onboarding system supports four trigger mechanisms, each with specific use cases, advantages, and trade-offs.

1. GitHub Webhooks (Recommended Primary)

Configuration

# GitHub App or Webhook configuration
events:
  - repository.created
  - repository.edited
  - repository.renamed
  - push  # For .backstage/config.yaml changes

Event Flow

GitHub → Webhook Endpoint → Event Router → Onboarding Workflow

Implementation

// src/onboarding/triggers/github-webhook.ts
import { WebhookEvent } from '@octokit/webhooks-types';
import { EventRouter } from '../workflows/event-router';
import { TenantResolver } from '../validation/tenant-resolver';

export class GitHubWebhookHandler {
  constructor(
    private eventRouter: EventRouter,
    private tenantResolver: TenantResolver
  ) {}

  async handleRepositoryCreated(event: WebhookEvent<'repository.created'>) {
    const { repository, organization } = event.payload;

    // Resolve tenant from GitHub organization
    const tenantId = await this.tenantResolver.fromGitHubOrg(organization.login);

    // Quick pre-validation
    if (!this.isBusinessUnitRepository(repository)) {
      console.log(`Skipping non-BU repository: ${repository.name}`);
      return;
    }

    // Route to onboarding workflow
    await this.eventRouter.routeEvent({
      type: 'repository.created',
      tenantId,
      source: 'github',
      payload: {
        repoName: repository.name,
        repoUrl: repository.html_url,
        defaultBranch: repository.default_branch,
        owner: organization.login,
        createdAt: repository.created_at,
        topics: repository.topics,
        description: repository.description,
      },
      metadata: {
        triggeredBy: event.sender.login,
        correlationId: this.generateCorrelationId(),
        timestamp: new Date().toISOString(),
      }
    });
  }

  private isBusinessUnitRepository(repo: any): boolean {
    // Quick heuristic checks
    return (
      repo.name.startsWith('bu-') ||
      repo.topics?.includes('business-unit') ||
      repo.description?.includes('[BU]')
    );
  }

  private generateCorrelationId(): string {
    return `gh-${Date.now()}-${Math.random().toString(36).substring(7)}`;
  }
}

Pros

⚡ Real-time (< 1 second latency)
🎯 Event-driven, no polling overhead
📊 Rich event metadata
🔄 Automatic retry by GitHub (exponential backoff)

Cons

🌐 Requires public endpoint (or GitHub Enterprise connectivity)
🔒 Webhook secret management
🚨 Must handle duplicate events (at-least-once delivery)
⏰ Potential delays during GitHub incidents

Best Practices

Idempotency: Always check if onboarding already completed
Async Processing: Acknowledge webhook immediately, process async
Rate Limiting: Implement token bucket for burst protection
Validation: Verify webhook signatures
Correlation IDs: Track events end-to-end

2. Terraform Cloud Webhooks

Configuration

# Terraform Cloud workspace notification configuration
resource "tfe_notification_configuration" "onboarding" {
  name             = "backstage-onboarding"
  enabled          = true
  destination_type = "generic"
  url              = "https://backstage.example.com/api/onboarding/tfc-webhook"

  triggers = [
    "run:created",
    "run:completed",
    "workspace:created"
  ]
}

Event Flow

TFC → Webhook Endpoint → Validate State → Onboarding Workflow

Implementation

// src/onboarding/triggers/tfc-webhook.ts
import { TerraformCloudWebhook } from '../types/tfc';
import { EventRouter } from '../workflows/event-router';

export class TerraformCloudWebhookHandler {
  constructor(
    private eventRouter: EventRouter,
    private tfcClient: TerraformCloudClient
  ) {}

  async handleWorkspaceCreated(webhook: TerraformCloudWebhook) {
    const { workspace_id, workspace_name, organization_name } = webhook.payload;

    // Resolve tenant from TFC organization
    const tenantId = await this.tenantResolver.fromTfcOrg(organization_name);

    // Check if this is a business unit workspace
    const workspaceInfo = await this.tfcClient.getWorkspace(workspace_id);

    if (!this.isBusinessUnitWorkspace(workspaceInfo)) {
      console.log(`Skipping non-BU workspace: ${workspace_name}`);
      return;
    }

    // Wait for first successful run (workspace must have resources)
    if (workspaceInfo.run_count === 0) {
      console.log(`Workspace ${workspace_name} has no runs yet, deferring onboarding`);
      return;
    }

    await this.eventRouter.routeEvent({
      type: 'workspace.created',
      tenantId,
      source: 'terraform-cloud',
      payload: {
        workspaceId: workspace_id,
        workspaceName: workspace_name,
        organization: organization_name,
        tags: workspaceInfo.tags,
        vcsRepo: workspaceInfo.vcs_repo,
        createdAt: workspaceInfo.created_at,
      },
      metadata: {
        correlationId: this.generateCorrelationId(),
        timestamp: new Date().toISOString(),
      }
    });
  }

  async handleRunCompleted(webhook: TerraformCloudWebhook) {
    const { run_id, workspace_id, run_status } = webhook.payload;

    if (run_status !== 'applied') {
      return; // Only process successful applies
    }

    // Check if this is the first successful run (triggers onboarding)
    const workspace = await this.tfcClient.getWorkspace(workspace_id);

    if (workspace.run_count === 1) {
      // First successful run - trigger onboarding
      await this.handleWorkspaceCreated(webhook);
    } else {
      // Existing workspace - trigger synchronization instead
      await this.eventRouter.routeEvent({
        type: 'workspace.state_updated',
        tenantId: await this.tenantResolver.fromTfcOrg(workspace.organization),
        source: 'terraform-cloud',
        payload: {
          workspaceId: workspace_id,
          runId: run_id,
        }
      });
    }
  }

  private isBusinessUnitWorkspace(workspace: any): boolean {
    return (
      workspace.name.match(/^bu-[a-z0-9-]+-infrastructure$/) ||
      workspace.tags?.some((tag: string) => tag.includes('business-unit'))
    );
  }
}

Pros

🎯 Captures infrastructure state changes
📊 Rich Terraform metadata (resources, outputs)
🔄 Guaranteed delivery by TFC
🏗️ Directly tied to infrastructure lifecycle

Cons

⏰ Delayed until first successful run
🔗 Requires TFC webhook configuration (manual or via Terraform)
🚨 May fire before GitHub repository is ready
📝 State sanitization required (secrets removal)

3. Polling-Based Discovery (Fallback)

Configuration

# config/onboarding/polling-config.yaml
polling:
  enabled: true
  interval: "5m"
  sources:
    - type: github
      orgs: ["acme-corp", "acme-labs"]
      filters:
        prefix: "bu-"
        topics: ["business-unit"]
    - type: terraform-cloud
      orgs: ["acme-corp-tfc"]
      filters:
        name_pattern: "^bu-.*-infrastructure$"

Implementation

// src/onboarding/triggers/polling-discovery.ts
import { CronJob } from 'cron';
import { Octokit } from '@octokit/rest';
import { TerraformCloudClient } from '../clients/tfc';

export class PollingDiscovery {
  private job: CronJob;

  constructor(
    private githubClient: Octokit,
    private tfcClient: TerraformCloudClient,
    private eventRouter: EventRouter,
    private onboardingState: OnboardingStateStore
  ) {
    this.job = new CronJob('*/5 * * * *', () => this.poll());
  }

  async poll() {
    console.log('Starting discovery polling...');

    await Promise.all([
      this.discoverGitHubRepositories(),
      this.discoverTfcWorkspaces(),
    ]);
  }

  async discoverGitHubRepositories() {
    const orgs = ['acme-corp', 'acme-labs'];

    for (const org of orgs) {
      const repos = await this.githubClient.repos.listForOrg({
        org,
        type: 'all',
        sort: 'created',
        direction: 'desc',
        per_page: 100,
      });

      for (const repo of repos.data) {
        // Skip if already onboarded
        const existing = await this.onboardingState.findByRepoUrl(repo.html_url);
        if (existing?.status === 'completed') {
          continue;
        }

        // Check if matches BU criteria
        if (this.isBusinessUnitRepository(repo)) {
          console.log(`Discovered new BU repository: ${repo.name}`);

          await this.eventRouter.routeEvent({
            type: 'repository.discovered',
            tenantId: await this.tenantResolver.fromGitHubOrg(org),
            source: 'polling',
            payload: {
              repoName: repo.name,
              repoUrl: repo.html_url,
              owner: org,
              createdAt: repo.created_at,
            },
            metadata: {
              discoveryMethod: 'polling',
              correlationId: this.generateCorrelationId(),
            }
          });
        }
      }
    }
  }

  async discoverTfcWorkspaces() {
    const orgs = ['acme-corp-tfc'];

    for (const org of orgs) {
      const workspaces = await this.tfcClient.listWorkspaces(org);

      for (const workspace of workspaces) {
        // Skip if already onboarded
        const existing = await this.onboardingState.findByWorkspaceId(workspace.id);
        if (existing?.status === 'completed') {
          continue;
        }

        // Check if matches BU criteria AND has successful run
        if (this.isBusinessUnitWorkspace(workspace) && workspace.run_count > 0) {
          console.log(`Discovered new BU workspace: ${workspace.name}`);

          await this.eventRouter.routeEvent({
            type: 'workspace.discovered',
            tenantId: await this.tenantResolver.fromTfcOrg(org),
            source: 'polling',
            payload: {
              workspaceId: workspace.id,
              workspaceName: workspace.name,
              organization: org,
            }
          });
        }
      }
    }
  }

  start() {
    this.job.start();
    console.log('Polling discovery started');
  }

  stop() {
    this.job.stop();
    console.log('Polling discovery stopped');
  }
}

Pros

🛡️ Catches missed webhook events
🔄 Self-healing (discovers manually created resources)
🧪 Easier to test (no webhook setup needed)
📊 Can backfill historical resources

Cons

⏰ Higher latency (up to polling interval)
🔋 Higher API usage (constant polling)
💰 Rate limiting concerns
🐌 Not real-time

4. Manual Trigger (UI/API)

API Endpoint

// src/onboarding/triggers/manual-trigger.ts
import { Request, Response } from 'express';
import { EventRouter } from '../workflows/event-router';
import { Authentication } from '../auth/authentication';

export class ManualTriggerAPI {
  constructor(
    private eventRouter: EventRouter,
    private auth: Authentication
  ) {}

  /**
   * POST /api/onboarding/trigger
   * Body: { repoUrl: string, workspaceName?: string, force?: boolean }
   */
  async triggerOnboarding(req: Request, res: Response) {
    // Authenticate request
    const user = await this.auth.authenticate(req);
    if (!user) {
      return res.status(401).json({ error: 'Unauthorized' });
    }

    const { repoUrl, workspaceName, force } = req.body;

    if (!repoUrl) {
      return res.status(400).json({ error: 'repoUrl is required' });
    }

    // Resolve tenant from authenticated user
    const tenantId = user.tenantId;

    // Check if already onboarded (unless force=true)
    if (!force) {
      const existing = await this.onboardingState.findByRepoUrl(repoUrl);
      if (existing?.status === 'completed') {
        return res.status(409).json({
          error: 'Repository already onboarded',
          onboardingId: existing.id,
          entities: existing.generatedEntities
        });
      }
    }

    // Trigger onboarding
    const onboardingId = await this.eventRouter.routeEvent({
      type: 'repository.manual_trigger',
      tenantId,
      source: 'manual',
      payload: {
        repoUrl,
        workspaceName,
        force,
      },
      metadata: {
        triggeredBy: user.email,
        correlationId: this.generateCorrelationId(),
      }
    });

    res.status(202).json({
      message: 'Onboarding initiated',
      onboardingId,
      statusUrl: `/api/onboarding/status/${onboardingId}`
    });
  }

  /**
   * GET /api/onboarding/status/:id
   */
  async getOnboardingStatus(req: Request, res: Response) {
    const { id } = req.params;
    const status = await this.onboardingState.getStatus(id);

    if (!status) {
      return res.status(404).json({ error: 'Onboarding not found' });
    }

    res.json(status);
  }
}

UI Component (Backstage Plugin)

// plugins/onboarding/src/components/ManualOnboardingButton.tsx
import React, { useState } from 'react';
import { Button, Progress, Alert } from '@backstage/core-components';
import { useApi } from '@backstage/core-plugin-api';
import { onboardingApiRef } from '../api';

export const ManualOnboardingButton = ({ repoUrl }: { repoUrl: string }) => {
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [success, setSuccess] = useState(false);
  const onboardingApi = useApi(onboardingApiRef);

  const handleOnboard = async () => {
    setLoading(true);
    setError(null);

    try {
      const result = await onboardingApi.triggerOnboarding({ repoUrl });
      setSuccess(true);

      // Poll for status
      const statusInterval = setInterval(async () => {
        const status = await onboardingApi.getStatus(result.onboardingId);

        if (status.status === 'completed') {
          clearInterval(statusInterval);
          setLoading(false);
        } else if (status.status === 'failed') {
          clearInterval(statusInterval);
          setError(status.error);
          setLoading(false);
        }
      }, 2000);
    } catch (err) {
      setError(err.message);
      setLoading(false);
    }
  };

  return (
    <div>
      <Button onClick={handleOnboard} disabled={loading || success}>
        {loading ? 'Onboarding...' : success ? 'Onboarded!' : 'Onboard Business Unit'}
      </Button>

      {loading && <Progress />}
      {error && <Alert severity="error">{error}</Alert>}
      {success && <Alert severity="success">Business unit onboarded successfully!</Alert>}
    </div>
  );
};

Pros

🎛️ Full user control
🔍 Easy debugging (known trigger point)
🛠️ Override automatic detection
🔄 Force re-onboarding

Cons

👤 Requires manual intervention
🔒 Requires authentication
📝 No automatic discovery

Recommended Strategy

Hybrid Approach (Best of All Worlds)

Primary:   GitHub Webhooks (real-time onboarding)
Secondary: TFC Webhooks (infrastructure validation)
Fallback:  Polling (5-minute interval, catch missed events)
Manual:    UI trigger (admin override, debugging)

Event Priority

class EventRouter {
  private async deduplicateEvent(event: OnboardingEvent): Promise<boolean> {
    const fingerprint = this.generateFingerprint(event);

    // Check if already processed in last 1 hour
    const existing = await this.cache.get(`event:${fingerprint}`);
    if (existing) {
      console.log(`Duplicate event detected: ${fingerprint}`);
      return true; // Is duplicate
    }

    // Mark as processed
    await this.cache.set(`event:${fingerprint}`, event, { ttl: 3600 });
    return false; // Not duplicate
  }

  private generateFingerprint(event: OnboardingEvent): string {
    // Fingerprint based on tenant + repo/workspace
    return `${event.tenantId}:${event.payload.repoUrl || event.payload.workspaceId}`;
  }
}

Timing Coordination

class EventRouter {
  async routeEvent(event: OnboardingEvent): Promise<string> {
    // Deduplicate
    if (await this.deduplicateEvent(event)) {
      return null; // Already processing
    }

    // For GitHub events, wait for TFC workspace (up to 2 minutes)
    if (event.source === 'github') {
      await this.waitForTfcWorkspace(event, { timeout: 120000 });
    }

    // Route to workflow engine
    return await this.workflowEngine.start(event);
  }

  private async waitForTfcWorkspace(
    event: OnboardingEvent,
    options: { timeout: number }
  ): Promise<void> {
    const startTime = Date.now();

    while (Date.now() - startTime < options.timeout) {
      // Try to find matching TFC workspace
      const workspace = await this.tfcClient.findWorkspaceByRepoUrl(
        event.payload.repoUrl
      );

      if (workspace && workspace.run_count > 0) {
        console.log(`Found TFC workspace: ${workspace.name}`);
        event.payload.workspaceName = workspace.name;
        event.payload.workspaceId = workspace.id;
        return;
      }

      // Wait 5 seconds before retry
      await new Promise(resolve => setTimeout(resolve, 5000));
    }

    console.warn('TFC workspace not found within timeout, proceeding anyway');
  }
}

Summary Table

Mechanism	Latency	Reliability	Complexity	Use Case
GitHub Webhooks	< 1s	High	Medium	Primary trigger
TFC Webhooks	< 1s	High	Medium	Infrastructure validation
Polling	1-5min	Medium	Low	Fallback & backfill
Manual	Immediate	High	Low	Admin override

Next Steps

See 03-workflow-state-machine.md for the onboarding workflow design.

Overview​

1. GitHub Webhooks (Recommended Primary)​

Configuration​

Event Flow​

Implementation​

Pros​

Cons​

Best Practices​

2. Terraform Cloud Webhooks​

Configuration​

Event Flow​

Implementation​

Pros​

Cons​

3. Polling-Based Discovery (Fallback)​

Configuration​

Implementation​

Pros​

Cons​

4. Manual Trigger (UI/API)​

API Endpoint​

UI Component (Backstage Plugin)​

Pros​

Cons​

Recommended Strategy​

Hybrid Approach (Best of All Worlds)​

Event Priority​

Timing Coordination​

Summary Table​

Next Steps​

Overview

1. GitHub Webhooks (Recommended Primary)

Configuration

Event Flow

Implementation

Pros

Cons

Best Practices

2. Terraform Cloud Webhooks

Configuration

Event Flow

Implementation

Pros

Cons

3. Polling-Based Discovery (Fallback)

Configuration

Implementation

Pros

Cons

4. Manual Trigger (UI/API)

API Endpoint

UI Component (Backstage Plugin)

Pros

Cons

Recommended Strategy

Hybrid Approach (Best of All Worlds)

Event Priority

Timing Coordination

Summary Table

Next Steps