Required Kubernetes Secrets for SyRF Services

Overview

This document lists all Kubernetes secrets required for SyRF services to run in the cluster. These secrets must be created in the syrf-staging namespace before services can start.

GCP Configuration Reference

Important: All GCP resources for SyRF are in project camarades-net (region europe-west2).

For complete infrastructure details — project/region values, DNS zones, GCP Secret Manager and the External Secrets Operator gcpsm-secret-store ClusterSecretStore — see SyRF Cloud Infrastructure Reference. GKE cluster configuration itself is documented in the camarades-infrastructure repository and published at docs.syrf.org.uk/infrastructure/cluster-config/.

External Services Behind These Secrets

All credentials are stored in GCP Secret Manager and synced into Kubernetes by the External Secrets Operator. This table maps each external service to the Kubernetes secret that carries its credentials; the per-secret YAML shapes follow below.

Service	Details	K8s Secret
Auth0	Domain: syrf.eu.auth0.com, Tenant: syrf	auth0
MongoDB Atlas	Cluster0 (M20, production), Preview (staging + PRs), Region: AWS eu-west-1	mongo-db (API, PM, Quartz)
MongoDB Atlas (Identity)	Non-Development Identity reads its own database via mongodb-identity and needs a separate read-only mongodb-project-management secret; its chart does not consume mongo-db	mongodb-identity, mongodb-project-management
Elasticsearch	Elastic Cloud - search indexing + APM	elastic-db
S3	Bucket: syrfapp-uploads (user file uploads)	aws-s3
SES	Email from noreply@syrf.org.uk	aws-ses (API, PM, Quartz); Identity uses ses-credentials (keyId, accessKey, fromAddress)
Google OAuth (Identity)	Google sign-in for the Identity service (clientId, clientSecret; reused Auth0-era client)	google-oauth
Sentry	Error tracking and performance	sentry
Elastic APM	Application Performance Monitoring	elastic-apm
ROB AI	Risk of Bias analysis via MapsGroupHttpClient	rob-api-credentials
Google Sheets	Data export (service account auth)	google-sheets
RabbitMQ	In-cluster message broker, receives S3 Lambda events	rabbit-mq

Critical Note on Secret Management

DO NOT commit secrets to Git. These secrets should be managed via:

1. External Secrets Operator (recommended) - syncs from Google Secret Manager
2. Manual kubectl creation (temporary/development only)
3. Sealed Secrets (alternative GitOps-friendly approach)

API Service Secrets

The API service requires 14 secrets:

Authentication & Authorization

1. auth0

apiVersion: v1
kind: Secret
metadata:
  name: auth0
  namespace: syrf-staging
type: Opaque
stringData:
  clientSecret: "<AUTH0_CLIENT_SECRET>"

2. swagger-auth

apiVersion: v1
kind: Secret
metadata:
  name: swagger-auth
  namespace: syrf-staging
type: Opaque
stringData:
  clientSecret: "<SWAGGER_CLIENT_SECRET>"

3. public-api

apiVersion: v1
kind: Secret
metadata:
  name: public-api
  namespace: syrf-staging
type: Opaque
stringData:
  apiKey: "<PUBLIC_API_KEY>"

Databases

4. mongo-db

apiVersion: v1
kind: Secret
metadata:
  name: mongo-db
  namespace: syrf-staging
type: Opaque
stringData:
  username: "<MONGODB_USERNAME>"
  password: "<MONGODB_PASSWORD>"

5. elastic-db

apiVersion: v1
kind: Secret
metadata:
  name: elastic-db
  namespace: syrf-staging
type: Opaque
stringData:
  username: "<ELASTICSEARCH_USERNAME>"
  password: "<ELASTICSEARCH_PASSWORD>"

6. dev-postgres-credentials

apiVersion: v1
kind: Secret
metadata:
  name: dev-postgres-credentials
  namespace: syrf-staging
type: Opaque
stringData:
  postgresql-password: "<POSTGRES_PASSWORD>"

Message Queue

7. rabbit-mq

apiVersion: v1
kind: Secret
metadata:
  name: rabbit-mq
  namespace: syrf-staging
type: Opaque
stringData:
  password: "<RABBITMQ_PASSWORD>"

AWS Services

8. aws-s3

apiVersion: v1
kind: Secret
metadata:
  name: aws-s3
  namespace: syrf-staging
type: Opaque
stringData:
  keyId: "<AWS_ACCESS_KEY_ID>"
  accessKey: "<AWS_SECRET_ACCESS_KEY>"

9. aws-ses

apiVersion: v1
kind: Secret
metadata:
  name: aws-ses
  namespace: syrf-staging
type: Opaque
stringData:
  keyId: "<AWS_ACCESS_KEY_ID>"
  accessKey: "<AWS_SECRET_ACCESS_KEY>"

External Services

10. google-sheets

apiVersion: v1
kind: Secret
metadata:
  name: google-sheets
  namespace: syrf-staging
type: Opaque
stringData:
  serviceAccountEmail: "<SERVICE_ACCOUNT_EMAIL>"
  password: "<SERVICE_ACCOUNT_PASSWORD>"
  key-cert.json: |
    <GOOGLE_SERVICE_ACCOUNT_JSON_KEY>

11. rob-api-credentials

apiVersion: v1
kind: Secret
metadata:
  name: rob-api-credentials
  namespace: syrf-staging
type: Opaque
stringData:
  open-ai-api-key: "<OPENAI_API_KEY>"
  maps-api-key: "<MAPS_API_KEY>"

Monitoring & Error Tracking

12. elastic-apm

apiVersion: v1
kind: Secret
metadata:
  name: elastic-apm
  namespace: syrf-staging
type: Opaque
stringData:
  secretToken: "<ELASTIC_APM_SECRET_TOKEN>"
  serverUrl: "<ELASTIC_APM_SERVER_URL>"

13. sentry

apiVersion: v1
kind: Secret
metadata:
  name: sentry
  namespace: syrf-staging
type: Opaque
stringData:
  dsnUrl: "<SENTRY_DSN_URL>"

Project Management Service Secrets

PM service requires the same secrets as API (shared configuration).

Quartz Service Secrets

Quartz requires a subset of the API secrets:

• mongo-db
• rabbit-mq
• elastic-apm (optional)
• sentry (optional)

Web Service Secrets

Web service (Angular) typically doesn't require secrets for startup, but may need:

• Configuration for API endpoints (via ConfigMap, not Secret)

Quick Deploy All Secrets (Development Only)

WARNING: Only use this for local development. For production, use External Secrets Operator.

# Create all secrets from a secrets.env file
kubectl create secret generic auth0 \
  --from-literal=clientSecret="${AUTH0_CLIENT_SECRET}" \
  -n syrf-staging

kubectl create secret generic mongo-db \
  --from-literal=username="${MONGODB_USERNAME}" \
  --from-literal=password="${MONGODB_PASSWORD}" \
  -n syrf-staging

# ... repeat for all secrets

Using External Secrets Operator (Recommended)

Prerequisites

1. External Secrets Operator installed in cluster
2. Google Secret Manager configured
3. Workload Identity set up for ESO

Example ExternalSecret

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: mongo-db
  namespace: syrf-staging
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: gcpsm-secret-store
    kind: ClusterSecretStore
  target:
    name: mongo-db
    creationPolicy: Owner
  data:
  - secretKey: username
    remoteRef:
      key: syrf-staging-mongodb-username
  - secretKey: password
    remoteRef:
      key: syrf-staging-mongodb-password

Verification

Check if all required secrets exist:

# List all secrets in staging namespace
kubectl get secrets -n syrf-staging

# Expected secrets for API service:
# - auth0
# - swagger-auth
# - public-api
# - mongo-db
# - elastic-db
# - dev-postgres-credentials
# - rabbit-mq
# - aws-s3
# - aws-ses
# - google-sheets
# - rob-api-credentials
# - elastic-apm
# - sentry

Troubleshooting

Pod stuck in "Pending" state

Symptom: Pod shows FailedMount events

Check:

kubectl describe pod <pod-name> -n syrf-staging | grep -A 5 "Events:"

Common errors:

• secret "google-sheets" not found - Create the missing secret
• secret "mongo-db" not found - Create the missing secret

Service fails to start after secrets are created

Check pod logs:

kubectl logs <pod-name> -n syrf-staging

Common issues:

• Incorrect secret key names (must match what service expects)
• Invalid credentials (wrong password, expired tokens)
• Missing permissions (IAM, database access)

Security Best Practices

1. Never commit secrets to Git
2. Use External Secrets Operator for production
3. Rotate secrets regularly (at least every 90 days)
4. Use RBAC to limit access to secrets
5. Encrypt secrets at rest (enabled by default in GKE)
6. Audit secret access via GCP Cloud Audit Logs

Next Steps

1. For Production: Set up External Secrets Operator

2. See: External Secrets Operator documentation

3. For Development: Create secrets manually using kubectl create secret

4. Migration from Jenkins X: Extract secrets from old cluster using kubectl get secret -o yaml

GitHub Actions Repository Secrets

The following secrets must be configured in the GitHub repository settings (Settings > Secrets and variables > Actions) for CI/CD workflows to function correctly.

PR Preview Workflow Secrets

The PR preview workflow (.github/workflows/pr-preview.yml) requires these secrets for resource cleanup and MongoDB Atlas integration:

GCP Workload Identity (Required for cleanup jobs)

Secret Name	Description	Example
GCP_WORKLOAD_IDENTITY_PROVIDER	Workload Identity provider URI	projects/123456789/locations/global/workloadIdentityPools/github/providers/github
GCP_SERVICE_ACCOUNT	GCP service account email	github-actions@camarades-net.iam.gserviceaccount.com

These secrets enable the cleanup job to:

• Authenticate to GKE cluster
• Delete RabbitMQ vhosts for closed PRs
• Manage preview environment resources

MongoDB Atlas (Required for PR previews)

Secret Name	Description	Example
ATLAS_PROJECT_ID	MongoDB Atlas project ID	abc123def456

This secret is required for generating AtlasDatabaseUser manifests for PR preview databases.

Setting Up Workload Identity Federation

1. Create Workload Identity Pool (if not exists):

gcloud iam workload-identity-pools create "github" \
  --project="camarades-net" \
  --location="global" \
  --display-name="GitHub Actions Pool"

1. Create Provider:

gcloud iam workload-identity-pools providers create-oidc "github" \
  --project="camarades-net" \
  --location="global" \
  --workload-identity-pool="github" \
  --display-name="GitHub" \
  --attribute-mapping="google.subject=assertion.sub,attribute.repository=assertion.repository" \
  --issuer-uri="https://token.actions.githubusercontent.com"

1. Grant Access to Service Account:

gcloud iam service-accounts add-iam-policy-binding \
  "github-actions@camarades-net.iam.gserviceaccount.com" \
  --project="camarades-net" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/github/attribute.repository/camaradesuk/syrf"

1. Get Provider URI:

gcloud iam workload-identity-pools providers describe "github" \
  --project="camarades-net" \
  --location="global" \
  --workload-identity-pool="github" \
  --format="value(name)"

1. Add secrets to GitHub:
2. Navigate to Settings > Secrets and variables > Actions
3. Add GCP_WORKLOAD_IDENTITY_PROVIDER with the provider URI
4. Add GCP_SERVICE_ACCOUNT with the service account email

Related Documentation

• GitOps Deployment Guide
• External Secrets Operator
• GCP Workload Identity Federation