Chapter 26 — Case Study: Porting a Real Pipeline

Twenty-five chapters of discipline applied to one file. The file goes from 200 lines and 47 environment variables to 30 lines and zero application-content variables. This is the chapter that shows the discipline pays off when applied to something with the proportions of a real problem.

The case study below is composite — drawn from patterns that recur across real CI pipelines without being any one team's specific code. The shape is realistic: a pipeline for a Python web service with a Postgres backing store, deployed across two regions of staging plus production-readonly debugging. The pipeline has accumulated cruft over three years; the engineers who originally wrote each section have rotated off the team; the README's "you might also need to..." section runs to fifteen lines.

The chapter walks the migration in commit-sized steps. Each step produces a reviewable diff; each step is independently revertable; each step makes the pipeline shorter and more maintainable. By the end of the chapter, the pipeline's surface area is a third of what it was, every value has a typed home, and a vendor migration would take thirty days rather than a quarter.

Day 0 — The before state

The current .gitlab-ci.yml:

# .gitlab-ci.yml — before (200 lines, abridged)
stages:
  - build
  - test
  - deploy
  - cleanup

variables:
  PYTHON_VERSION: "3.11"
  POETRY_VERSION: "1.7.1"
  AWS_DEFAULT_REGION: "us-east-1"
  AWS_ACCOUNT_ID: "123456789012"
  STAGING_DB_HOST: "db-staging-east-1.internal"
  STAGING_DB_PORT: "5432"
  STAGING_DB_NAME: "app"
  STAGING_DB_USER: "app_user"
  STAGING_API_HOST: "api-staging-east.internal"
  STAGING_API_PORT: "8443"
  REDIS_HOST: "redis-staging-east-1.internal"
  REDIS_PORT: "6379"
  PROD_DB_HOST: "db-prod-east-1.internal"
  PROD_DB_PORT: "5432"
  PROD_API_HOST: "api-prod-east.internal"
  PROD_API_PORT: "443"
  WEST_DB_HOST: "db-staging-west-1.internal"
  WEST_REDIS_HOST: "redis-staging-west-1.internal"
  LOG_LEVEL: "INFO"
  LOG_FILE_LEVEL: "DEBUG"
  RETRY_LIMIT: "3"
  RETRY_BACKOFF_MS: "500"
  PARALLELISM: "8"
  TIMEOUT_DEFAULT: "30"
  TIMEOUT_DB: "60"
  TIMEOUT_API: "10"
  OUTPUT_DIR: "/var/log/integration"
  DEBUGGER: "none"
  BREAKPOINT_AT: "none"
  ARTIFACT_BUCKET: "company-ci-artifacts"
  CACHE_BUCKET: "company-ci-cache"
  IMAGE_REGISTRY: "registry.example.com"
  RUNNER_IMAGE: "registry.example.com/runner:latest"
  TEST_DB_SEED_FILE: "/tmp/seed.sql"
  TEST_TIMEOUT_MULTIPLIER: "1"
  ENABLE_NEW_CACHING: "false"
  ENABLE_METRICS_V2: "false"
  ENABLE_TRACING: "true"
  TRACING_SAMPLE_RATE: "0.1"
  CLEANUP_AFTER_RUN: "true"
  NOTIFY_ON_FAILURE: "true"
  SLACK_WEBHOOK_BOT: "ci-bot-staging-east"
  PAGERDUTY_SERVICE: "staging-east-ci"
  GITHUB_TOKEN: ""              # populated from CI variables (masked)
  AWS_ACCESS_KEY_ID: ""         # populated (masked)
  AWS_SECRET_ACCESS_KEY: ""     # populated (masked)
  STAGING_DB_PASSWORD: ""       # populated (masked)
  PROD_DB_RO_PASSWORD: ""       # populated (masked)
  REDIS_PASSWORD: ""            # populated (masked)

default:
  image: $RUNNER_IMAGE
  before_script:
    - poetry install --no-interaction
    - if [ "$CI_PIPELINE_SOURCE" == "schedule" ]; then export TEST_TIMEOUT_MULTIPLIER=3; fi
    - if [ "$CI_COMMIT_REF_NAME" == "main" ]; then export DEPLOY_ENABLED=true; else export DEPLOY_ENABLED=false; fi

build-image:
  stage: build
  script:
    - docker build -t $IMAGE_REGISTRY/app:$CI_COMMIT_SHORT_SHA .
    - echo "$DOCKER_PASSWORD" | docker login $IMAGE_REGISTRY -u $DOCKER_USER --password-stdin
    - docker push $IMAGE_REGISTRY/app:$CI_COMMIT_SHORT_SHA
    - if [ "$CI_COMMIT_REF_NAME" == "main" ]; then docker tag $IMAGE_REGISTRY/app:$CI_COMMIT_SHORT_SHA $IMAGE_REGISTRY/app:latest; docker push $IMAGE_REGISTRY/app:latest; fi
  artifacts:
    when: on_success
    paths:
      - .build/
    expire_in: 1 day

provision-staging:
  stage: build
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    - aws ec2 describe-instances --filters "Name=tag:Project,Values=staging-east" --query 'Reservations[*].Instances[*].[InstanceId,State.Name]' --output text | grep running || aws ec2 run-instances --image-id ami-0abcd1234 --count 3 --instance-type t3.medium --tag-specifications 'ResourceType=instance,Tags=[{Key=Project,Value=staging-east},{Key=Lifecycle,Value=ephemeral}]'
    - aws ec2 wait instance-running --filters "Name=tag:Project,Values=staging-east"
    - sleep 30   # let userdata complete
    - ./scripts/configure-firewall.sh staging-east
    - ./scripts/seed-database.sh $STAGING_DB_HOST $STAGING_DB_USER $STAGING_DB_PASSWORD < $TEST_DB_SEED_FILE

smoke-tests:
  stage: test
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    - export TEST_LOG_LEVEL=$LOG_LEVEL
    - export TEST_PARALLELISM=$PARALLELISM
    - export TEST_TIMEOUT=$(($TIMEOUT_DEFAULT * $TEST_TIMEOUT_MULTIPLIER))
    - if [ "$CI_PIPELINE_SOURCE" == "schedule" ]; then export TEST_PROFILE=full; else export TEST_PROFILE=fast; fi
    - export DATABASE_URL="postgresql://$STAGING_DB_USER:$STAGING_DB_PASSWORD@$STAGING_DB_HOST:$STAGING_DB_PORT/$STAGING_DB_NAME?sslmode=require"
    - export REDIS_URL="redis://default:$REDIS_PASSWORD@$REDIS_HOST:$REDIS_PORT"
    - export API_BASE_URL="https://$STAGING_API_HOST:$STAGING_API_PORT"
    - poetry run pytest tests/smoke -v --output-dir=$OUTPUT_DIR/smoke
    - if [ "$NOTIFY_ON_FAILURE" == "true" ] && [ $? -ne 0 ]; then curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"Smoke tests failed in $CI_JOB_URL\"}" $SLACK_WEBHOOK_URL; fi
  artifacts:
    when: always
    paths:
      - $OUTPUT_DIR/smoke
    expire_in: 7 days

integration-tests:
  stage: test
  needs: [smoke-tests]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    # ... 35 more lines, structurally identical to smoke-tests but with
    # different test paths, slightly different DATABASE_URL, retries
    # configured by hand, conditional logic for nightly vs PR runs ...
  artifacts:
    when: always
    paths:
      - $OUTPUT_DIR/integration
    expire_in: 7 days

deploy-staging:
  stage: deploy
  needs: [integration-tests]
  rules:
    - if: $CI_COMMIT_REF_NAME == "main" && $CI_PIPELINE_SOURCE == "push"
  script:
    - kubectl set image deployment/app app=$IMAGE_REGISTRY/app:$CI_COMMIT_SHORT_SHA -n staging-east
    - kubectl rollout status deployment/app -n staging-east --timeout=5m
    # ... 20 lines of post-deploy smoke verification with hardcoded URLs ...

cleanup:
  stage: cleanup
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: always
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: always
  script:
    - if [ "$CLEANUP_AFTER_RUN" == "true" ]; then
        aws ec2 terminate-instances --instance-ids $(aws ec2 describe-instances --filters "Name=tag:Project,Values=staging-east" --query 'Reservations[*].Instances[*].InstanceId' --output text);
      fi

Two hundred lines. Forty-seven environment variables (six masked secrets plus forty-one application-content vars). Five stages. Inline AWS commands. Hardcoded hostnames in five places. Three shell-conditional branches keyed off CI_PIPELINE_SOURCE. Two URL-construction lines that concatenate hostnames, ports, credentials, database names, and TLS flags. One notification-on-failure block embedded in the test script.

Walk through what's wrong using the chapter-2 failure-mode catalogue:

Failure mode	Where it appears
1. Env-var accretion	The whole `variables:` block — 47 entries
2. Conditional thicket	The `before_script:` lines that set `TEST_TIMEOUT_MULTIPLIER` and `DEPLOY_ENABLED` based on multiple env-var checks
3. Inlined recipe	`provision-staging`'s `aws ec2 run-instances` invocation
4. Credential leak through env	`STAGING_DB_PASSWORD`, `REDIS_PASSWORD`, etc. exposed as environment variables
5. Host name in YAML	`STAGING_DB_HOST`, `WEST_DB_HOST`, `PROD_DB_HOST`, all hardcoded
6. Topology buried in step ordering	`needs:` declarations encode the test-then-deploy graph; cleanup runs `when: always` regardless of which earlier steps fired

All six failure modes present in one file. This is the proof that the chapter-2 failure modes are not abstract — they actually compound, and they actually accumulate in real pipelines.

Day 1 — Inventory and triage

The first migration step is no code change. The team inventories the file and groups every value by category. An afternoon of work; produces a spreadsheet (or for a smaller project, a markdown table):

Value	Category	Notes
`STAGING_DB_HOST` / `PORT` / `NAME`	Landscape (infrastructure service)	Will become a `web/database` infrastructure entry
`STAGING_API_HOST` / `PORT`	Landscape (infrastructure service)	Will become a `web/api` infrastructure entry
`REDIS_HOST` / `PORT`	Landscape (infrastructure service)	Will become a `cache/redis` infrastructure entry
`WEST_DB_HOST` / `WEST_REDIS_HOST`	Landscape (different env)	Will be in `landscape-staging-west.yaml`
`PROD_DB_HOST` / `PROD_API_HOST`	Landscape (different env, prod-readonly only)	Will be in `landscape-prod-readonly.yaml`
`LOG_LEVEL`, `LOG_FILE_LEVEL`	Runtime	Goes into `runtime-staging-east.yaml`
`RETRY_LIMIT`, `RETRY_BACKOFF_MS`	Runtime	Same
`PARALLELISM`	Runtime	Same
`TIMEOUT_DEFAULT` / `_DB` / `_API`	Runtime	Same
`OUTPUT_DIR`	Runtime	Same
`DEBUGGER`, `BREAKPOINT_AT`	Runtime	Same
`TEST_TIMEOUT_MULTIPLIER`	Runtime (different per profile)	Encodes the nightly-vs-PR difference; goes in profile-keyed runtimes
`ENABLE_NEW_CACHING`, `ENABLE_METRICS_V2`, `ENABLE_TRACING`, `TRACING_SAMPLE_RATE`	Runtime (feature flags)	Same
`STAGING_DB_PASSWORD`	Credential	Becomes `staging-db-rw` credential reference
`PROD_DB_RO_PASSWORD`	Credential	Becomes `prod-db-readonly`
`REDIS_PASSWORD`	Credential	Becomes `staging-redis-default`
`GITHUB_TOKEN`	Credential	Becomes `ci-bot-github`
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`	Credential	Becomes `ci-runner-aws`
`DATABASE_URL` (constructed)	Data Profile	The string isn't stored; it's built from the data-profile + credential at runtime
`REDIS_URL` (constructed)	Data Profile	Same
`API_BASE_URL` (constructed)	Data Profile	Same (a `web/api` profile)
`PYTHON_VERSION`, `POETRY_VERSION`	Toolchain manifest	Stays in `.tool-versions`
`AWS_DEFAULT_REGION`, `AWS_ACCOUNT_ID`	Specification (provisioning venue)	Goes in `spec-staging-east.yaml`
`aws ec2 run-instances ...` recipe	Specification (provisioning device)	Same
`ARTIFACT_BUCKET`, `CACHE_BUCKET`, `IMAGE_REGISTRY`	Runtime (infrastructure addresses for CI)	Could go in runtime; arguable
`CLEANUP_AFTER_RUN`, `NOTIFY_ON_FAILURE`	Workflow parameters	Per-run flags; go in workflow YAML's `parameters:`
`SLACK_WEBHOOK_BOT`, `PAGERDUTY_SERVICE`	Landscape (infrastructure service)	These name external services
`RUNNER_IMAGE`	Vendor YAML	Stays in `.gitlab-ci.yml`'s `image:`

Forty-seven values become entries in six categories (plus a short fixed-vendor-shim list). No code changes yet — the inventory is the plan. The implementation that follows is mechanical.

Day 5 — Author the typed configurations

The team authors five configuration files plus a profile set plus a Specification file, in that order. Each file lands as its own commit; the team reviews each separately.

Commit 1: `config/landscape/landscape-staging-east.yaml`

environment:
    label: staging-east

apod:
    clients:
        - deviceType: network/client-linux
          name: ci-runner-1
          host: <set by spec-staging-east>
          role: runner
          credentials: [ci-runner-ssh]
          features: {parallelism_max: 8}
          skip: false

infrastructure:
    services:
        - serviceType: database/postgres
          name: staging-db
          host: db-staging-east-1.internal
          port: 5432
          database: app
          credentials: [staging-db-rw]

        - serviceType: cache/redis
          name: staging-redis
          host: redis-staging-east-1.internal
          port: 6379
          credentials: [staging-redis-default]

        - serviceType: web/api
          name: staging-api
          endpoint: https://api-staging-east.internal:8443
          credentials: [staging-api-token]

        - serviceType: web/webhook
          name: slack-notifications
          endpoint: ${VAULT:/secret/staging-east/slack-webhook}
          credentials: [slack-webhook]

        - serviceType: pagerduty/service
          name: ci-pager
          service_id: "staging-east-ci"
          credentials: [pagerduty-token]

Diff: +28 lines added. No removals from .gitlab-ci.yml yet. The commit is additive — the team adds the file to the repository without breaking anything.

Commit 2: similar landscape files for staging-west and prod-readonly

config/landscape/
├── landscape-staging-east.yaml         (commit 1)
├── landscape-staging-west.yaml         (commit 2)
├── landscape-prod-readonly.yaml        (commit 2)
└── landscape-local-dev.yaml            (commit 2)

Three more landscape files for the other environments. Each follows the same shape; only the hostnames and credential identifiers differ.

Commit 3: data profiles file

# config/dataprofiles/dataprofiles.yaml
dataprofiles:
    - identifier: staging-db-rw
      category: basic-tcp
      host: db-staging-east-1.internal
      port: 5432
      dbtype: postgres
      dbname: app
      credential: staging-db-rw

    - identifier: staging-redis
      category: basic-tcp
      host: redis-staging-east-1.internal
      port: 6379
      dbtype: redis
      credential: staging-redis-default

    # ... staging-west variants, prod-readonly variants

The data profiles are dispatch entries for the per-environment landscape variants. Each profile carries its host field (self-contained per chapter 8); the credential is referenced by identifier.

Commit 4: credentials file

# config/credentials/credentials.yaml
credentials:
    - identifier: ci-runner-ssh
      categories: [ssh, environment:integration-ci]
      username: runner
      keyfile: ${FILE:/run/secrets/ssh/ci-runner-rsa}

    - identifier: staging-db-rw
      categories: [basic, environment:staging-east]
      username: app_user
      password: ${VAULT:/secret/staging-east/db-rw#password}

    - identifier: staging-redis-default
      categories: [basic, environment:staging-east]
      username: default
      password: ${VAULT:/secret/staging-east/redis#password}

    - identifier: ci-bot-github
      categories: [github-token, environment:integration-ci]
      username: ci-bot
      token: ${VAULT:/secret/ci/github-pat#token}

    - identifier: ci-runner-aws
      categories: [aws-access-key, environment:integration-ci, ttl:900]
      keyid: ${VAULT:/secret/ci/aws/runner#keyid}
      secret: ${VAULT:/secret/ci/aws/runner#secret}

    - identifier: prod-db-readonly
      categories: [basic, environment:prod, role:readonly, audit:required, ttl:300]
      username: ${VAULT:/secret/prod/db-readonly#username}
      password: ${VAULT:/secret/prod/db-readonly#password}

    # ... and so on

Every secret value is a ${VAULT:...} reference (per chapter 10). Every credential carries an environment:<name> scope tag (per chapter 12). The file is committable to the repository without leaking any secret material.

Commit 5: runtime profiles

# config/runtime/runtime-base.yaml
breakpoints: []
debugger: none
logging:
    levels:
        console: INFO
        logfile: DEBUG
parallelism:
    max_workers: 8
retry:
    limit: 3
    backoff_ms: 500
timeouts:
    default_seconds: 30
    db_seconds: 60
    api_seconds: 10
paths:
    output: ./.output
feature_flags:
    enable_new_caching: false
    enable_metrics_v2: false
tracing:
    enabled: true
    sample_rate: 0.1
version: 1.0.0

# config/runtime/runtime-integration-ci.yaml
# (inherits from base via profile composition)
property_test_caps:
    max_examples: 10           # fast-CI cap

# config/runtime/runtime-nightly-full-sweep.yaml
property_test_caps:
    max_examples: null         # full sweep
timeouts:
    default_seconds: 90        # 3x multiplier from the original
                               # TEST_TIMEOUT_MULTIPLIER=3 logic

# config/runtime/runtime-local-dev.yaml
logging:
    levels:
        console: DEBUG
timeouts:
    default_seconds: 300       # generous for breakpoints
property_test_caps:
    max_examples: 5

Four runtimes; each layers on top of base. The chapter-7 runtime-vs-env-var thesis applied: every behaviour knob lives in a runtime YAML field; the workflow does not export any of them.

Commit 6: provisioning specification

# config/specs/spec-staging-east.yaml
environment:
    venues:
        - name: aws-us-east-1
          provider: aws
          management_url: https://ec2.us-east-1.amazonaws.com
          subscription: "123456789012"
          region: us-east-1
          credential: ci-runner-aws
    groups:
        - name: staging-east
          description: "Staging east runner pool"
          tags:
              - {key: Project, value: staging-east}
              - {key: Lifecycle, value: ephemeral}

profiles:
    - name: runner-medium
      description: "Standard medium-sized integration runner"
      credential: ci-runner-aws
      parameters:
          instance_type: t3.medium
          image: ami-0abcd1234
      tags: [{key: role, value: runner}]

apod:
    devices:
        - identifier: ci-runner-1
          category: ec2-instance
          group: staging-east
          profile: runner-medium
          role: runner
        - identifier: ci-runner-2
          category: ec2-instance
          group: staging-east
          profile: runner-medium
          role: runner
        - identifier: ci-runner-3
          category: ec2-instance
          group: staging-east
          profile: runner-medium
          role: runner

The chapter-4 worked example specialised to this project. The 40-line aws ec2 run-instances shell from the original is now 20 lines of declarative YAML; the chapter-4 idempotent-on-re-run property means the cleanup-and-recreate dance the original did manually is unnecessary.

Commit 7: profile files

# config/profiles/base.yaml
profile:
    name: base
    select:
        landscape: staging-east
        runtime: base
        credentials: base
        dataprofiles: dataprofiles

# config/profiles/integration-ci.yaml
profile:
    extends: base
    name: integration-ci
    select:
        runtime: integration-ci

# config/profiles/nightly-full-sweep.yaml
profile:
    extends: base
    name: nightly-full-sweep
    select:
        runtime: nightly-full-sweep

# config/profiles/local-dev.yaml
profile:
    extends: base
    name: local-dev
    select:
        landscape: local-dev
        runtime: local-dev

Four profiles; each chooses a small set of overrides on top of base. The discipline from chapter 17's profile promotion in action: each profile is a delta from the base, additive.

Day 12 — Author the workflow

The taskplus workflow file replaces the flow of the original GitLab YAML — the build, provision, test, deploy, cleanup sequence. But not the configuration (already in the typed files committed days 5–7) and not the vendor wiring (will be handled in commit 9).

Commit 8: `config/workflows/integration.yaml`

# config/workflows/integration.yaml
name: integration

parameters:
    cleanup_after_run: true
    notify_on_failure: true

flow:
    # Build phase
    - taskgroup: Build
      items:
          - task: Build app image
            tasktype: my.taskplus.tasks.docker_build@DockerBuild
            image_name: app
            tag: ${parameters[commit_sha]}

          - task: Push app image
            tasktype: my.taskplus.tasks.docker_push@DockerPush
            image: ${parameters[commit_sha]}

    # Provision phase (only on MR / scheduled runs)
    - task: Provision
      tasktype: ccai.taskplus.tasks.provisioning@ProvisioningTask
      specification: config/specs/spec-staging-east.yaml
      action: apply
      if: "parameters['source'] in ('merge_request_event', 'schedule')"

    # Test phase
    - taskgroup: Test
      items:
          - task: Smoke
            tasktype: ccai.taskplus.tasks.distributedtest@DistributedTestJobTask
            test_root: source/testroots/ccai
            includes: [ccai.tests.smoke]

          - task: Integration
            tasktype: ccai.taskplus.tasks.distributedtest@DistributedTestJobTask
            test_root: source/testroots/ccai
            includes: [ccai.tests.integration]
            onfailure: NotifyAndCleanup

    # Deploy phase (only on push-to-main)
    - taskgroup: Deploy
      if: "parameters['source'] == 'push' and parameters['ref'] == 'main'"
      items:
          - task: Deploy to staging
            tasktype: my.taskplus.tasks.kubectl_set_image@KubectlSetImage
            deployment: app
            namespace: staging-east
            image: ${parameters[commit_sha]}

    # Cleanup phase (always runs)
    - taskgroup: NotifyAndCleanup
      items:
          - task: Notify failure
            tasktype: my.taskplus.tasks.slack_notify@SlackNotify
            webhook_service: slack-notifications
            template: "Pipeline failed in {pipeline_url}"
            if: "parameters['notify_on_failure'] == True"

          - task: Teardown
            tasktype: ccai.taskplus.tasks.provisioning@ProvisioningTask
            specification: config/specs/spec-staging-east.yaml
            action: teardown
            if: "parameters['cleanup_after_run'] == True"

About fifty lines. Walk what changed:

No env-var soup. No LOG_LEVEL, no RETRY_LIMIT, no STAGING_DB_HOST. All of those are configuration content loaded by the workflow's startup.
No inline AWS commands. Provisioning is a chapter-25 bridge task consuming the chapter-4 Specification.
No URL construction. The smoke and integration tests invoke the chapter-25 DistributedTestJobTask, which finds staging-db, staging-redis, and staging-api through the active landscape.
No conditional shell. The if: fields on tasks and taskgroups are typed Python expressions reading from parameters.

Revisiting the chapter-22 cross-group jump subtlety

The workflow above has the Integration task at ordinal "3.2" (inside the Test taskgroup) declaring onfailure: NotifyAndCleanup. This is the chapter-22 cross-group jump case.

Chapter 22 documented a subtlety: when seek mode is active and the outer loop encounters a TaskGroup whose own label doesn't match, the entire group is skipped. In the workflow above:

Integration fails; seeking_node becomes "NotifyAndCleanup".
The loop reaches the Deploy taskgroup. Its label is "Deploy"; mismatch; the group is skipped (correct).
The loop reaches the NotifyAndCleanup taskgroup. Its label is "NotifyAndCleanup"; match; the seek state clears; the group runs.

The key here is naming the taskgroup with the jump target's label, not naming an interior task. When the workflow author follows this convention — the failure-handler taskgroup's own label is the seek target — cross-group jumps work as intuited.

The alternative — declaring onfailure: Teardown (a label inside the NotifyAndCleanup taskgroup) — would not work, because the engine would skip past the NotifyAndCleanup group without descending. The workflow above avoids the trap by naming the group itself as the target.

The workflow's NotifyAndCleanup group runs both the notification task and the teardown task on failure, which matches the original GitLab YAML's intent (cleanup runs when: always).

Day 14 — The vendor shim

Commit 9: shrink `.gitlab-ci.yml`

# .gitlab-ci.yml — after (30 lines)
default:
  image: registry.example.com/runner:latest
  before_script:
    - poetry install --no-interaction

stages:
  - run

integration-mr:
  stage: run
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  variables:
    AM_PROFILE: integration-ci
  script:
    - ccai testbase test run --workflow config/workflows/integration.yaml ||
      (ccai profile capture --output failure-snapshot.json && exit 1)
  artifacts:
    when: on_failure
    paths:
      - failure-snapshot.json
      - .output/
    expire_in: 1 month

nightly-full-sweep:
  stage: run
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
  variables:
    AM_PROFILE: nightly-full-sweep
  script:
    - ccai testbase test run --workflow config/workflows/nightly.yaml
  artifacts:
    when: always
    paths: [.output/]
    expire_in: 1 month

Thirty lines. The chapter-24 pattern applied. The diff in the GitLab YAML is enormous: 200 lines down to 30, with the deleted lines distributed across removed variables: (47 → just AM_PROFILE), removed shell commands (the aws ec2, the URL constructions, the conditional setups), and removed staging declarations (the workflow handles its own internal sequencing).

Day 30 — Migration complete

The repository's structure post-migration:

project-root/
├── .gitlab-ci.yml                      30 lines (was 200)
├── .tool-versions                      4 lines  (kept)
├── config/
│   ├── credentials/credentials.yaml    50 lines (NEW)
│   ├── dataprofiles/dataprofiles.yaml  40 lines (NEW)
│   ├── landscape/
│   │   ├── landscape-staging-east.yaml 30 lines (NEW)
│   │   ├── landscape-staging-west.yaml 30 lines (NEW)
│   │   ├── landscape-prod-readonly.yaml 25 lines (NEW)
│   │   └── landscape-local-dev.yaml    25 lines (NEW)
│   ├── runtime/
│   │   ├── runtime-base.yaml           20 lines (NEW)
│   │   ├── runtime-integration-ci.yaml 3 lines  (NEW)
│   │   ├── runtime-nightly-full-sweep.yaml 5 lines (NEW)
│   │   └── runtime-local-dev.yaml      8 lines  (NEW)
│   ├── specs/
│   │   └── spec-staging-east.yaml      45 lines (NEW)
│   ├── profiles/
│   │   ├── base.yaml                   8 lines  (NEW)
│   │   ├── integration-ci.yaml         5 lines  (NEW)
│   │   ├── nightly-full-sweep.yaml     5 lines  (NEW)
│   │   └── local-dev.yaml              7 lines  (NEW)
│   └── workflows/
│       ├── integration.yaml            50 lines (NEW)
│       └── nightly.yaml                30 lines (NEW)
└── README.md                           (now 5 lines, was 80)

Total lines of CI-related configuration:

Before: 200 (one file, one repository).
After: ~390 lines spread across 17 files.

The total grew. That's a feature. Before, two-hundred lines in one file; the file had nineteen orthogonal concerns smeared together. After, three-hundred-ninety lines in seventeen files; each file owns one concern. The total is bigger; the per-file diff size is much smaller. Code review at the per-change level gets sharper, not noisier.

The operational payoff in numbers

Six concrete metrics worth measuring:

Metric	Before	After
`.gitlab-ci.yml` line count	200	30
Application-content env vars in vendor YAML	41	0
Files touched to add a new test phase	1 (the YAML)	1 (the workflow)
Files touched to switch staging-east → staging-west	1 (the YAML, 8 places)	1 (set `AM_LANDSCAPE_NAMES`)
Time to reproduce a CI failure locally	~30 min (read logs, set up env)	<10 min (download capture, replay)
README's "you might also need to..." length	80 lines	5 lines

The metrics are not the only payoff. Two qualitative changes matter as much:

Configuration drift across environments became visible. Before the migration, "what's different between staging-east and staging-west?" was a question requiring archaeology. After, it's a ccai profile diff invocation that prints a side-by-side diff in two seconds.
The team's mental model of the pipeline got smaller. Before, debugging a pipeline failure required holding two hundred lines and forty-seven env vars in your head. After, debugging starts with the workflow's flow tree (fifty lines), drills into the relevant configuration file (twenty to fifty lines), and stops there.

What the migration didn't fix

Honesty about what didn't change.

The runner image is still a black box

The registry.example.com/runner:latest image contains the toolchain (Python, Poetry, the ccai packages, AWS CLI, kubectl, docker). The image's contents are still managed through a separate Dockerfile and image-build pipeline; the discipline in this volume doesn't apply to that layer. When the image breaks (a Poetry upgrade introduces a regression, an OS package's ABI changes), debugging still requires going into the Dockerfile and the image-build pipeline.

A team applying the discipline more broadly might bring the image-build pipeline under the same typed-configuration treatment — Specifications for the image, profiles for image variants — but that's a follow-on project rather than part of the migration above.

Some accumulated weirdness survived

The original pipeline had a fifteen-line workaround in provision-staging for a known issue where aws ec2 wait instance-running doesn't actually wait for userdata to complete. The workaround (sleep 30) survives the migration — it's now an explicit Specification field (post_create_delay_seconds: 30), still ugly, but at least named.

The discipline doesn't eliminate workarounds; it makes them nameable. A workaround with a name is one a future engineer can find, understand, and eventually replace. A workaround buried in a 200-line shell script tends to survive longer because no one notices it.

Cross-pipeline coordination

The team also had a pipeline for the deployment of the same app to production (separate from the staging-east one above). The migration didn't unify the two pipelines — each kept its own .gitlab-ci.yml, even though both now reference the same typed configurations. This is a deliberate scoping choice; the two pipelines have different triggers, different on-call escalation paths, different operational concerns. Sharing configuration without sharing the pipeline structure is the right boundary.

Summary

A 200-line GitLab YAML with 47 environment variables migrated to a 30-line vendor shim plus 17 typed configuration files.
The migration ran in nine commits over thirty days. Each commit was independently reviewable; each commit was independently revertable; the migration made progress every day without breaking the pipeline.
The chapter-2 failure modes (env-var accretion, conditional thicket, inlined recipe, credential leak through env, host name in YAML, topology buried in step ordering) all appeared in the original; all were closed in the migration.
The chapter-22 cross-group jump subtlety was navigated by naming the taskgroup with the jump target's label, rather than naming an interior task. The convention is small but important.
The total lines of configuration grew (from 200 to ~390), spread across 17 files instead of 1. The per-file diff size shrunk dramatically; code review got sharper.
Operational payoffs: zero application-content env vars, single-file changes for previously cross-cutting edits, sub- ten-minute CI failure reproduction, a five-line README section in place of an 80-line one.
Honest residual: the runner image is still a black box; some workarounds survive (now nameable rather than buried); cross-pipeline structural unification is out of scope.

The volume's last chapter — chapter 27 — is the inverse of this one's claim: where this chapter showed what the discipline buys when applied, chapter 27 names where the discipline does not fit and what to use instead.