Pipeline Configuration: Your Recipe Book for Building Software 🍳

Imagine you’re a master chef. Every time you cook a delicious meal, you follow a recipe. The recipe tells you what ingredients to use, in what order to mix them, and how long to cook.

CI/CD pipelines are like recipes for building software! Instead of ingredients and cooking steps, we have code, tests, and deployment instructions. And just like a recipe book, we write these instructions down so anyone can follow them perfectly every time.

🎭 The Big Idea: Pipeline as Code

What Does “Pipeline as Code” Mean?

Think about this: What if you had to remember every recipe in your head? You’d forget things! That’s why we write recipes down.

Pipeline as Code means writing your build instructions in a file, just like writing a recipe in a cookbook.

Before Pipeline as Code:

• Someone clicks buttons in a tool to set up builds
• If that person leaves, nobody knows what buttons to click
• Every computer might build differently

After Pipeline as Code:

• Instructions live in a file with your code
• Anyone can read it
• Every build works the same way

# This is Pipeline as Code!
# A simple recipe file for building
name: Build My App
steps:
  - name: Get ingredients
    run: npm install
  - name: Cook it
    run: npm run build

Why is This Amazing?

📝 Two Ways to Write Pipelines: Declarative vs Scripted

Imagine two different recipe styles:

Declarative Style: “What I Want”

Like ordering at a restaurant: “I want a pizza with cheese and tomatoes.”

You say what you want, and the system figures out how to make it.

# Declarative: I describe WHAT I want
pipeline:
  agent: any
  stages:
    - stage: Build
      steps:
        - script: npm install
        - script: npm test

Good for: Most projects. Simple, easy to read!

Scripted Style: “How to Do It”

Like a detailed cooking tutorial: “First, preheat the oven to 400°F. Then, roll the dough for 5 minutes…”

You control every tiny step yourself.

// Scripted: I control HOW to do it
node {
    stage('Build') {
        sh 'npm install'
        if (env.BRANCH == 'main') {
            sh 'npm run build-prod'
        } else {
            sh 'npm run build-dev'
        }
    }
}

Good for: Complex logic, special conditions.

Quick Comparison

graph LR
    A[Choose Your Style] --> B[Declarative]
    A --> C[Scripted]
    B --> D[Simple & Clean]
    B --> E[Best for Most Cases]
    C --> F[Full Control]
    C --> G[Complex Logic Needed]

🔤 YAML Pipeline Syntax: The Language of Pipelines

YAML is like the alphabet for writing pipeline recipes. It’s designed to be easy for humans to read!

The Golden Rules of YAML

Rule 1: Spaces Matter! Use spaces, not tabs. Indent with 2 spaces.

# CORRECT - uses spaces
name: My Pipeline
jobs:
  build:
    steps:
      - run: echo "Hello"

Rule 2: Key-Value Pairs Everything is name: value

name: Build App       # name is "Build App"
timeout: 30           # timeout is 30
enabled: true         # enabled is true

Rule 3: Lists Use Dashes Each item starts with -

colors:
  - red
  - green
  - blue

Rule 4: Nested Items = More Indent

job:
  name: Build
  steps:
    - name: Step 1
      run: npm install
    - name: Step 2
      run: npm test

Real Pipeline Example

name: Deploy Website

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v4

      - name: Install stuff
        run: npm install

      - name: Build it
        run: npm run build

📦 Pipeline Variables: Reusable Ingredients

Variables are like labeled containers in your kitchen. Instead of saying “that white powder over there,” you say “the flour container.”

What Are Variables?

A variable is a name that holds a value. You use the name, and the computer substitutes the value.

# Define variables at the top
variables:
  APP_NAME: my-cool-app
  VERSION: 1.0.0

# Use them with ${{ }} or $
jobs:
  build:
    steps:
      - run: echo "Building $APP_NAME"
      - run: echo "Version ${{ VERSION }}"

Types of Variables

graph LR
    A[Pipeline Variables] --> B[Built-in]
    A --> C[Custom]
    B --> D[System gives you these]
    B --> E[Example: build number]
    C --> F[You create these]
    C --> G[Example: app name]

Example: Using Variables

variables:
  NODE_VERSION: 18
  OUTPUT_DIR: ./dist

jobs:
  build:
    steps:
      - name: Setup Node
        uses: setup-node@v3
        with:
          node-version: ${{ NODE_VERSION }}

      - name: Build
        run: npm run build -- --out $OUTPUT_DIR

Why use variables?

• Change one place, updates everywhere
• No “magic values” scattered around
• Easy to understand what each value means

🌍 Environment Variables: The World Around Your Code

Environment variables are like the temperature and humidity in a kitchen. They’re conditions that exist around your code, not in it.

How They Work

Your code can “ask” the environment: “What database should I connect to?” The environment answers based on where the code is running.

jobs:
  build:
    # Set environment variables
    env:
      DATABASE_HOST: localhost
      API_URL: https://api.example.com

    steps:
      - name: Run tests
        run: npm test
        # Tests can read DATABASE_HOST!

Different Environments, Different Values

graph LR
    A[Same Code] --> B[Development]
    A --> C[Staging]
    A --> D[Production]
    B --> E[DATABASE: dev-db]
    C --> F[DATABASE: stage-db]
    D --> G[DATABASE: prod-db]

Example: Environment-Specific Config

jobs:
  deploy-dev:
    env:
      ENV_NAME: development
      API_URL: https://dev-api.com
    steps:
      - run: ./deploy.sh

  deploy-prod:
    env:
      ENV_NAME: production
      API_URL: https://api.com
    steps:
      - run: ./deploy.sh

The same deploy script works differently based on environment variables!

🔐 Pipeline Secrets: The Locked Ingredient Cabinet

Some things in your kitchen should be locked up—like expensive spices or grandma’s secret sauce recipe. In pipelines, secrets are passwords, API keys, and other sensitive data.

Why Secrets Are Special

# NEVER DO THIS!
steps:
  - run: |
      curl -u admin:SuperSecret123 api.com
      # Anyone can see the password! 😱

# DO THIS INSTEAD!
steps:
  - run: |
      curl -u admin:${{ secrets.API_PASSWORD }} api.com
      # Password is hidden! ✅

How Secrets Work

graph TD
    A[You Store Secret] --> B[Secure Vault]
    B --> C[Pipeline Requests It]
    C --> D[Secret Injected at Runtime]
    D --> E[Never Shown in Logs]

Creating and Using Secrets

Step 1: Store the secret (in your CI/CD tool’s settings)

• Name: DATABASE_PASSWORD
• Value: (hidden)

Step 2: Use it in your pipeline

jobs:
  deploy:
    steps:
      - name: Connect to Database
        env:
          DB_PASS: ${{ secrets.DATABASE_PASSWORD }}
        run: |
          ./connect-db.sh --password $DB_PASS

Secret Safety Rules

🔄 Secret Rotation: Changing the Locks

Imagine if you kept the same house key for 100 years. If someone copied it, they’d have access forever! That’s why we rotate (change) secrets regularly.

What is Secret Rotation?

Secret Rotation = Changing passwords and keys on a schedule

Like changing your house locks every few months, but for your code!

Why Rotate Secrets?

graph TD
    A[Old Secret Leaked?] --> B[If Never Rotated]
    A --> C[If Rotated Monthly]
    B --> D[Attacker Has Access Forever! 😱]
    C --> E[Attacker Loses Access Soon ✅]

How to Rotate Secrets

Manual Rotation:

1. Create new secret value
2. Update it in your secret store
3. Old value stops working
4. Pipelines use new value automatically

Automatic Rotation:

# Some systems rotate for you!
# Example: AWS Secrets Manager
secret:
  name: database-password
  rotation:
    enabled: true
    days: 30  # New password every 30 days

Best Practices

Rotation Without Downtime

The trick: Have TWO valid secrets during rotation!

graph TD
    A[Day 1: Secret A Active] --> B[Day 2: Secret A + B Both Work]
    B --> C[Day 3: Secret B Only]

1. Create new secret (B)
2. Both old (A) and new (B) work
3. Update all systems to use B
4. Delete old secret A

🎯 Putting It All Together

Here’s a complete pipeline showing everything we learned:

# Pipeline as Code - everything documented!
name: Build and Deploy

# Declarative syntax - clean and simple
on:
  push:
    branches: [main]

# Variables - reusable values
variables:
  NODE_VERSION: 18
  APP_NAME: my-awesome-app

jobs:
  build:
    runs-on: ubuntu-latest

    # Environment variables
    env:
      NODE_ENV: production
      BUILD_DIR: ./dist

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node
        uses: setup-node@v3
        with:
          node-version: ${{ variables.NODE_VERSION }}

      - name: Install & Build
        run: |
          npm install
          npm run build

      # Using secrets safely
      - name: Deploy
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}
        run: |
          ./deploy.sh ${{ variables.APP_NAME }}

🌟 Key Takeaways

1. Pipeline as Code = Write build instructions in a file, track with Git
2. Declarative = Say what you want (simpler)
3. Scripted = Say how to do it (more control)
4. YAML = Easy-to-read format with spaces and dashes
5. Variables = Named containers for reusable values
6. Environment Variables = Different settings for different places
7. Secrets = Locked-away sensitive data
8. Secret Rotation = Change secrets regularly for safety

You now understand how to write pipeline configurations like a pro chef writes recipes! Every build will be consistent, documented, and secure.

Remember: Good pipelines are like good recipes—clear, repeatable, and safe to share! 🎉