Engineering CI/CD Pipelines
CI/CD Pipeline Engineer
Given a project, produce a complete .github/workflows/ pipeline covering: build → test → security scan → release → deploy → rollback.
YAML# .github/workflows/ci.yml name: CI on: push: branches: [main, develop] pull_request: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - run: npm ci - run: npm run build - uses: actions/upload-artifact@v4 with: name: build-output path: dist/ test: needs: build runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' cache: 'npm' - run: npm ci - run: npm run test -- --coverage - uses: codecov/codecov-action@v4 security: needs: build runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: aquasecurity/trivy-action@master with: scan-type: fs severity: CRITICAL,HIGH - run: npm audit --audit-level=high
Progress checklist for building out a full pipeline:
- [ ] Step 1: Analyze project type (Node/Python/Go/Docker) and pick runner/actions
- [ ] Step 2: Define Build Stage (compile, artifact caching, versioning)
- [ ] Step 3: Define Testing Stage (unit, integration, coverage thresholds)
- [ ] Step 4: Define Security Check Stage (SCA, SAST, secret scanning, container scan)
- [ ] Step 5: Define Release Strategy (semver, changelog, tagging, artifact publish)
- [ ] Step 6: Define Deployment Automation (staging → production, approvals)
- [ ] Step 7: Define Rollback Flow (health check trigger, revert mechanism)
- [ ] Step 8: Add notifications (Slack/Discord/email on failure)
1. Pipeline Stage Design
Every pipeline follows this stage order — never skip stages, but stages can run in parallel where there's no dependency:
- Lint/Format — fail fast, cheapest check first
- Build — produce artifact once, reuse downstream
- Test — unit → integration → e2e (in increasing cost order)
- Security Check — SCA (dependency), SAST (code), secret scan, container scan
- Release — tag, changelog, publish artifact/image
- Deploy — staging first, then production (gated by approval or automated smoke test)
- Rollback — standby, triggered by failed health check or manual
2. Security Check (mandatory gates)
| Check | Tool | Blocks merge? |
|---|---|---|
| Dependency vulnerabilities | npm audit, Dependabot, Trivy | Yes on HIGH/CRITICAL |
| Secret scanning | Gitleaks, TruffleHog | Yes always |
| SAST | CodeQL, Semgrep | Yes on HIGH |
| Container scan | Trivy, Grype | Yes on CRITICAL |
| License compliance | FOSSA, license-checker | Warn only |
3. Release Strategy
Default: Semantic Versioning + Conventional Commits + automated changelog.
YAMLrelease: needs: [test, security] if: github.ref == 'refs/heads/main' runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: googleapis/release-please-action@v4 with: release-type: node
Strategy options (pick one, default = Blue-Green for services, Canary for high-traffic APIs):
- Blue-Green: full swap, instant rollback via traffic switch — default for most apps
- Canary: 5% → 25% → 100% traffic ramp, auto-abort on error rate spike — use for high-risk/high-traffic changes
- Rolling: node-by-node replacement — use for stateless clusters where blue-green infra is too costly
4. Deployment Automation
YAMLdeploy-staging: needs: release runs-on: ubuntu-latest environment: staging steps: - run: ./scripts/deploy.sh staging ${{ github.sha }} - run: ./scripts/smoke-test.sh staging deploy-production: needs: deploy-staging runs-on: ubuntu-latest environment: name: production url: https://musgo.app steps: - run: ./scripts/deploy.sh production ${{ github.sha }} - run: ./scripts/health-check.sh production --timeout=120
Production deploy always requires: environment protection rule (manual approval) + automated post-deploy health check.
5. Rollback Flow
YAMLrollback: runs-on: ubuntu-latest if: failure() needs: deploy-production steps: - run: ./scripts/rollback.sh production --to=previous-stable - run: ./scripts/health-check.sh production --timeout=60 - uses: slackapi/slack-github-action@v2 with: payload: '{"text":"🔴 Production rollback triggered for ${{ github.sha }}"}'
Rollback triggers (any one fires automatic rollback):
- Health check fails post-deploy (HTTP 5xx rate > threshold)
- Error rate spike in APM (>2x baseline within 5 min)
- Manual trigger via
workflow_dispatch
Always keep last 3 stable artifact versions available for instant redeploy — never rebuild during rollback.
Example 1: Input: "Setup CI/CD for a Node.js Express API deployed to AWS ECS" Output:
- Workflow:
ci.yml(lint/build/test/security) +cd.yml(release/deploy/rollback) - Pipeline stages: lint → build (Docker image) → test (jest + supertest) → security (Trivy container scan) → push to ECR → deploy ECS (blue-green via CodeDeploy) → rollback (CodeDeploy auto-rollback on CloudWatch alarm)
- Release strategy: semver tag on merge to main, image tagged
api:v1.4.2andapi:latest
Example 2: Input: "Add security scanning to existing pipeline that has no checks" Output:
- Insert
securityjob afterbuild, beforerelease - Add Gitleaks (secret scan), CodeQL (SAST), Trivy (dependency + container)
- Set branch protection rule:
securityjob required to pass before merge - Add
SARIFupload to GitHub Security tab for centralized visibility
- Cache dependencies (
npm,pip,go mod) to cut build time significantly - Build artifact once, reuse across test/security/deploy jobs — never rebuild per stage
- Pin action versions with SHA or major version tag (
@v4), never@master - Use environment protection rules for production, not manual Slack approval
- Keep pipeline YAML DRY via reusable workflows (
workflow_call) for multi-repo consistency - Always emit a machine-readable deploy summary (version, commit SHA, timestamp) to a status endpoint
- Don't run full e2e tests on every PR — reserve for merge-to-main or nightly
- Don't skip security stage "to save time" — it must gate release, not run in parallel unconstrained
- Don't deploy to production without a tested rollback path already wired up
- Don't hardcode secrets in workflow YAML — use GitHub Encrypted Secrets / OIDC to cloud provider
- Don't rebuild artifacts at deploy time — deploy exactly what was tested and scanned