tercul-backend/.github/workflows

GitHub Actions CI/CD Documentation

Overview

This document describes the GitHub Actions CI/CD pipeline for the Tercul backend project, updated to 2025 best practices. The pipeline ensures code quality, security, and reliable deployments through automated testing, linting, security scanning, and containerized deployments.

Quick Reference

Workflow	File	Purpose	Triggers
Lint	lint.yml	Code quality & style	Push/PR to main, develop
Test	test.yml	Unit tests & compatibility	Push/PR to main, develop
Build	build.yml	Binary compilation	Push/PR to main, develop
Security	security.yml	CodeQL scanning	Push/PR to main + Weekly
Docker Build	docker-build.yml	Container images	Push to main, Tags, PRs
Deploy	deploy.yml	Production deployment	Tags (v*), Manual

Architecture

The CI/CD pipeline follows the Single Responsibility Principle with focused workflows:

1. Lint (lint.yml) - Code quality and style enforcement
2. Test (test.yml) - Unit tests and compatibility matrix
3. Build (build.yml) - Binary compilation and verification
4. Security (security.yml) - CodeQL security scanning
5. Docker Build (docker-build.yml) - Container image building and publishing
6. Deploy (deploy.yml) - Production deployment orchestration

Workflows

Lint Workflow (lint.yml)

Purpose: Ensures code quality and consistent style across the codebase.

Triggers:

• Push to main and develop branches
• Pull requests targeting main and develop branches

Jobs:

• golangci-lint: Go linting with golangci-lint
  • Checkout code
  • Setup Go 1.25 with caching
  • Install dependencies
  • Tidy modules (ensures go.mod/go.sum are clean)
  • Run linter with 5-minute timeout

Configuration:

• Timeout: 5 minutes
• Target: All Go files (./...)
• Cache: Enabled for faster runs

Test Workflow (test.yml)

Purpose: Validates code functionality through comprehensive testing.

Triggers:

• Push to main and develop branches
• Pull requests targeting main and develop branches

Jobs:

Unit Tests

• Environment: Ubuntu with PostgreSQL 15 and Redis 7

• Features:

  • Race detection enabled
  • Code coverage reporting (atomic mode)
  • HTML coverage report generation
  • Test result summaries in GitHub UI
  • 30-day artifact retention

Services:

• PostgreSQL 15 with health checks
• Redis 7-alpine with health checks

Compatibility Matrix

• Trigger: Push to main branch only
• Strategy: Tests across Go versions 1.22, 1.23, 1.24, 1.25
• Purpose: Ensures compatibility with multiple Go versions
• Fail-fast: Disabled (all versions tested even if one fails)

Build Workflow (build.yml)

Purpose: Compiles the application binary and validates the build.

Triggers:

• Push to main and develop branches
• Pull requests targeting main and develop branches

Jobs:

• build-binary: Binary compilation and verification
  • Dependency verification with go mod verify
  • Build to bin/tercul-backend
  • Binary validation test
  • Artifact upload (30-day retention)

Permissions:

• contents: read - Read repository code
• attestations: write - Future SLSA attestation support
• id-token: write - OIDC token for attestations

Security Workflow (security.yml)

Purpose: Automated security vulnerability detection with CodeQL.

Triggers:

• Push to main branch
• Pull requests targeting main branch
• Scheduled: Every Monday at 14:20 UTC

Jobs:

• codeql-analysis: CodeQL security scanning for Go
  • Initialize CodeQL with Go language support
  • Build code for analysis
  • Perform security scan
  • Category: "backend-security" for tracking

CodeQL Configuration:

The workflow can be customized with additional query suites:

- name: Initialize CodeQL
  uses: github/codeql-action/init@v3
  with:
    languages: go
    # Run security-extended suite for more comprehensive scanning
    queries: security-extended
    # Or use security-and-quality for maintainability checks
    # queries: security-and-quality

Available Query Suites:

• security-extended: Default queries plus lower severity/precision queries
• security-and-quality: Security queries plus maintainability and reliability

Custom Query Packs:

Add custom CodeQL query packs for specialized analysis:

- name: Initialize CodeQL
  uses: github/codeql-action/init@v3
  with:
    languages: go
    packs: my-org/go-security-queries@1.0.0

Docker Build Workflow (docker-build.yml)

Purpose: Builds and publishes multi-architecture Docker images.

Triggers:

• Push to main branch
• Tag pushes with v* pattern
• Pull requests targeting main branch

Jobs:

• build-image: Multi-platform Docker image building

  • Docker Buildx setup for multi-arch builds
  • Login to GitHub Container Registry
  • Metadata extraction for tags and labels
  • Build for AMD64 and ARM64 architectures
  • Push to registry (except for PRs)
  • Generate build provenance attestation

Image Tagging Strategy:

• main branch pushes → main + sha-<hash> tags
• Tag pushes (v1.2.3) → v1.2.3, 1.2.3, 1.2, sha-<hash> tags
• Pull requests → pr-<number> tag (build only, not pushed)

Push Behavior:

• Pushes to main/tags: Build and push to registry
• Pull requests: Build only (validation, no push)

Platforms: linux/amd64, linux/arm64

Deploy Workflow (deploy.yml)

Purpose: Production deployment orchestration to Docker Swarm.

Triggers:

• Tag pushes with v* pattern
• Manual dispatch with version input

Jobs:

• deploy-production: Deployment to production environment

  • Version extraction from tag or manual input
  • Docker Swarm service update (SSH-based deployment template)
  • Deployment summary with timestamp
  • Environment protection and tracking

Environment:

• Name: production
• URL: Configurable production URL
• Protection: Supports required reviewers and wait timers

Manual Deployment:

Deployments can be triggered manually from the Actions tab with a specific version.

Required Secrets:

• SWARM_HOST: Docker Swarm manager hostname/IP
• SWARM_SSH_KEY: SSH private key for Swarm access

Workflow Execution Order

On Pull Request:

1. Lint → Validates code style
2. Test → Runs unit tests with coverage
3. Build → Compiles binary
4. Security → CodeQL analysis (main branch PRs only)
5. Docker Build → Builds image (no push)

On Push to main:

1. Lint → Code quality check
2. Test → Unit tests + compatibility matrix
3. Build → Binary compilation
4. Security → CodeQL scan
5. Docker Build → Build and push image

On Tag Push (v*):

1. Docker Build → Build and push versioned image
2. Deploy → Deploy to production

Security Features

Permissions Management

• Principle of Least Privilege: Each workflow has minimal required permissions
• GITHUB_TOKEN Restrictions: Read-only by default, elevated only when necessary
• Workflow Separation: Each workflow operates independently
• Attestation Permissions: For build provenance and SLSA compliance

Code Security

• CodeQL Integration: Automated security scanning for Go code
• Dependency Verification: go mod verify ensures integrity
• Module Tidying: go mod tidy prevents dependency drift

Container Security

• Multi-platform Builds: Ensures compatibility and security across architectures
• Provenance Attestation: Cryptographic proof of build integrity
• Registry Security: GitHub Container Registry with token-based authentication

Secrets Management

• No Hardcoded Secrets: All sensitive data uses GitHub secrets
• Environment Variables: Proper isolation of configuration
• GITHUB_TOKEN: Automatic authentication for package registries
• Granular Permissions: Package-level access control

Package Registry Security

• GITHUB_TOKEN Authentication: No personal access tokens required
• Automatic Permissions: Packages inherit repository visibility
• Repository Scoped: Packages linked to source repository
• Granular Access: Fine-grained permissions per package
• Artifact Attestation: Cryptographic proof of build provenance
• OIDC Support: Token-based authentication without long-lived credentials

Best Practices Implemented

2025 Updates

• Semantic Versioning: Actions pinned to major versions (e.g., @v5) instead of SHA
• Caching Optimization: Go module and Docker layer caching
• Matrix Testing: Cross-version compatibility validation
• Service Health Checks: Database and Redis readiness verification
• Artifact Management: Proper retention policies and naming

Performance Optimizations

• Dependency Caching: Reduces setup time significantly
• Parallel Jobs: Independent jobs run concurrently
• Conditional Execution: Security scans only on main branch
• Artifact Upload: Efficient storage and retrieval

Reliability Features

• Timeout Configuration: Prevents hanging jobs
• Error Handling: Proper exit codes and logging
• Health Checks: Service readiness validation
• Retention Policies: Balanced storage management

Configuration Details

Go Version

• Primary: Go 1.25
• Matrix: Go 1.22, 1.23, 1.24, 1.25

Services

• PostgreSQL: Version 15 with health checks
• Redis: Version 7-alpine with ping health checks

Tools

• Linter: golangci-lint latest with 5-minute timeout
• Testing: go test with race detection and coverage
• Building: go build with verbose output
• Security: CodeQL for Go analysis

Caching

• Go Modules: Automatic caching via setup-go action
• Docker Layers: GitHub Actions cache with GHA type
• CodeQL Databases: Stored in ${{ github.runner_temp }}/codeql_databases

Maintenance

Dependabot Configuration

• GitHub Actions: Weekly updates with "ci" prefix
• Go Modules: Weekly updates with "deps" prefix
• Automated PRs: Keeps dependencies current and secure

Monitoring

• Workflow Runs: GitHub Actions tab for execution monitoring
• Security Alerts: Code scanning results and dependency alerts
• Coverage Reports: Artifact downloads for test coverage analysis

Troubleshooting

Common Issues

1. Cache Misses: Clear caches if corruption suspected
2. Service Failures: Check health check configurations
3. Permission Errors: Verify GITHUB_TOKEN scopes
4. Timeout Issues: Adjust timeout values in workflow configurations
5. Docker Push Failures: Check package write permissions
6. Registry Authentication: Ensure packages: write permission is set

Package Registry Issues

Problem: Cannot push to GitHub Container Registry

# Solution: Ensure proper permissions
permissions:
  contents: read
  packages: write
  id-token: write
  attestations: write

Problem: Package not visible after push

• Check package visibility settings (public/private/internal)
• Verify repository is linked to package
• Ensure workflow completed successfully

Problem: Cannot pull package in workflow

# Solution: Login to registry first
- name: Login to GitHub Container Registry
  uses: docker/login-action@v3
  with:
    registry: ghcr.io
    username: ${{ github.actor }}
    password: ${{ secrets.GITHUB_TOKEN }}

Debugging

• Manual Dispatch: Use workflow_dispatch for testing
• Log Analysis: Review step outputs for error details
• Artifact Inspection: Download build artifacts for verification
• Package Logs: Check package activity in GitHub UI

Future Enhancements

Planned Features

• SLSA Integration: Enhanced build attestation
• Dependency Review: Automated dependency vulnerability checking
• Performance Testing: Load testing integration
• Multi-environment Deployment: Staging and production separation

Scalability Considerations

• Self-hosted Runners: For resource-intensive jobs
• Job Parallelization: Further optimization of concurrent execution
• Cache Optimization: Advanced caching strategies

Docker Image Usage

Pulling Images

Pull the latest image:

docker pull ghcr.io/<owner>/<repository>:latest

Pull a specific version:

docker pull ghcr.io/<owner>/<repository>:1.2.3

Running Locally

Run the container:

docker run -d \
  --name tercul-backend \
  -p 8080:8080 \
  -e DATABASE_URL="postgres://..." \
  ghcr.io/<owner>/<repository>:latest

Docker Swarm Deployment

Deploy as a stack:

docker stack deploy -c docker-compose.yml tercul

Update running service:

docker service update \
  --image ghcr.io/<owner>/<repository>:1.2.3 \
  tercul_backend

Verifying Attestations

Verify build provenance:

gh attestation verify \
  oci://ghcr.io/<owner>/<repository>:latest \
  --owner <owner>

Contributing

When modifying workflows:

1. Test changes using workflow_dispatch
2. Ensure backward compatibility
3. Update this documentation
4. Follow security best practices
5. Use semantic versioning for action references
6. Test Docker builds locally before pushing
7. Verify package permissions after changes
8. Review CodeQL alerts before merging PRs
9. Update query packs regularly for latest security rules
10. Test configuration changes with workflow_dispatch

CodeQL Advanced Configuration

Custom Configuration File

For complex CodeQL setups, use a configuration file (.github/codeql/codeql-config.yml):

name: "CodeQL Config"

# Disable default queries to run only custom queries
disable-default-queries: false

# Specify query packs
packs:
  - scope/go-security-pack
  - scope/go-compliance-pack@1.2.3

# Add custom queries
queries:
  - uses: security-and-quality
  - uses: ./custom-queries

# Filter queries by severity
query-filters:
  - exclude:
      problem.severity:
        - warning
        - recommendation
  - exclude:
      id: go/redundant-assignment

# Scan specific directories
paths:
  - internal
  - cmd
  - pkg

paths-ignore:
  - "**/*_test.go"
  - vendor
  - "**/testdata/**"

# Extend threat model (preview)
threat-models: local

Reference the config in your workflow:

- uses: github/codeql-action/init@v3
  with:
    config-file: ./.github/codeql/codeql-config.yml

Inline Configuration

Alternatively, specify configuration inline:

- uses: github/codeql-action/init@v3
  with:
    languages: go
    config: |
      disable-default-queries: false
      queries:
        - uses: security-extended
      query-filters:
        - exclude:
            problem.severity:
              - recommendation      

Scheduling CodeQL Scans

Run CodeQL on a schedule for regular security audits:

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    # Run at 14:20 UTC every Monday
    - cron: '20 14 * * 1'

Avoiding Unnecessary Scans

Skip CodeQL for specific file changes:

on:
  pull_request:
    branches: [main]
    paths-ignore:
      - '**/*.md'
      - '**/*.txt'
      - 'docs/**'

Analysis Categories

Categorize multiple analyses in monorepos:

- name: Perform CodeQL Analysis
  uses: github/codeql-action/analyze@v3
  with:
    category: "backend-api"

External Query Packs

Use query packs from GitHub Enterprise Server:

- uses: github/codeql-action/init@v3
  with:
    registries: |
      - url: https://containers.GHEHOSTNAME/v2/
        packages:
          - my-company/*
        token: ${{ secrets.GHES_TOKEN }}      
    packs: my-company/go-queries

Query Suite Examples

Security-focused:

queries:
  - uses: security-extended

Quality and security:

queries:
  - uses: security-and-quality

Custom suite:

queries:
  - uses: ./custom-queries/critical-security.qls

Advanced Workflow Techniques

Workflow Commands

GitHub Actions supports workflow commands for advanced functionality:

Debug Logging

Enable detailed debugging with the ACTIONS_STEP_DEBUG secret:

- name: Debug step
  run: echo "::debug::Detailed debugging information"

Annotations

Create annotations for notices, warnings, and errors:

# Notice annotation
- run: echo "::notice file=app.go,line=10::Consider refactoring"

# Warning annotation
- run: echo "::warning file=main.go,line=5,col=10::Deprecated function"

# Error annotation
- run: echo "::error file=handler.go,line=20,title=Build Error::Missing import"

Grouping Log Lines

Organize logs with collapsible groups:

- name: Build application
  run: |
    echo "::group::Compiling Go code"
    go build -v ./...
    echo "::endgroup::"    

Masking Secrets

Prevent sensitive values from appearing in logs:

- name: Generate token
  run: |
    TOKEN=$(generate_token)
    echo "::add-mask::$TOKEN"
    echo "TOKEN=$TOKEN" >> $GITHUB_ENV    

Job Summaries

Add Markdown summaries to workflow runs:

- name: Test summary
  run: |
    echo "### Test Results :white_check_mark:" >> $GITHUB_STEP_SUMMARY
    echo "" >> $GITHUB_STEP_SUMMARY
    echo "- Total: 150 tests" >> $GITHUB_STEP_SUMMARY
    echo "- Passed: 148" >> $GITHUB_STEP_SUMMARY
    echo "- Failed: 2" >> $GITHUB_STEP_SUMMARY    

Environment Files

Use environment files for dynamic configuration:

# Set environment variable for subsequent steps
- name: Set build info
  run: |
    echo "BUILD_TIME=$(date +'%Y-%m-%d %H:%M:%S')" >> $GITHUB_ENV
    echo "COMMIT_SHA=${GITHUB_SHA:0:7}" >> $GITHUB_ENV    

# Use in later steps
- name: Deploy
  run: echo "Deploying $COMMIT_SHA built at $BUILD_TIME"

Output Parameters

Share data between steps:

- name: Calculate version
  id: version
  run: echo "VERSION=1.2.3" >> $GITHUB_OUTPUT

- name: Use version
  run: echo "Building version ${{ steps.version.outputs.VERSION }}"

Multiline Values

Handle multiline strings safely:

- name: Store API response
  run: |
    {
      echo 'API_RESPONSE<<EOF'
      curl https://api.example.com/data
      echo EOF
    } >> $GITHUB_ENV    

References

• GitHub Actions Documentation
• Go CI/CD Best Practices
• Security Hardening Guide
• Dependency Caching
• Workflow Commands Reference
• Publishing Packages with Actions
• GitHub Container Registry
• Artifact Attestations
• CodeQL Configuration Reference
• CodeQL Query Suites
• CodeQL CLI Reference