A CI/CD pipeline coordinates all the processes involved in continuous integration and continuous delivery.

Continuous integration (CI) is the practice of automating the integration of code changes from multiple contributors into a single software project. It’s a primary DevOps best practice, allowing developers to frequently merge code changes into a central repository where builds and tests then run. Automated tools are used to assert the new code’s correctness before integration.

The main goal of CI is to provide quick feedback so that if a defect is introduced into the code base, it can be identified and corrected as soon as possible. CI helps reduce the time and effort required for integration problems and allows developers to enhance software quality continuously.

Continuous delivery (CD) extends CI by automatically deploying all code changes to a QA and/or production environment after the build stage. This ensures that you can release new changes to your customers quickly and sustainably. This practice helps developers automate the deployment process, minimize bugs in production, and increase the speed of software releases.

The CI portion of a CI/CD pipeline includes source code, build, and test stages of the software delivery lifecycle, while the CD portion includes the delivery and deployment stages.

Stages of pipeline

Here’s how the “assembly line” typically works, step-by-step, in an easy way:

1. Plan & Code (The Idea & Blueprint):
   • This is where developers write their code, just like engineers designing parts for a car.
   • They work on their local computers and, when ready, “commit” their changes to a central shared place (like GitHub).
   • Analogy: You’ve drawn up plans for a new feature or fixed a bug in your app.
2. Build (Putting the Parts Together):
   • As soon as new code is committed, the assembly line automatically kicks off!
   • It takes all the new code and any existing parts (libraries, tools) and compiles them into a working “app package” (like an executable file or a Docker image).
   • Analogy: The factory receives all the individual car parts and starts assembling them into a basic vehicle frame.
3. Test (Quality Control Check):
   • Right after building, automated tests run against the app package. This is super fast and checks for common mistakes, bugs, or if new changes broke old features.
   • Analogy: Before the car leaves the assembly line, it goes through quick automated checks: Are the wheels on tight? Do the brakes work? Are there any obvious dents?
4. Package (Finalizing the Product):
   • If all tests pass, the app package is “packaged up” neatly and securely. For modern apps, this often means putting it into a “container” (like a Docker image), which includes everything the app needs to run anywhere.
   • This final package is stored in a special warehouse (a “registry”).
   • Analogy: The car gets its final coat of paint, interior, and is shrink-wrapped, then moved to the dealership’s ready lot.
5. Deploy (Sending it Out):
   • Now, the app package is automatically sent out and set up in different environments.
   • Continuous Delivery: The app is ready to be deployed, but a human might press a button to say “go live.”
   • Continuous Deployment: The app automatically goes live without human intervention, straight to users (if all tests passed, we trust it!).
   • Analogy: The car is driven off the lot to the customer (or to a show floor for final review before selling).
6. Monitor & Feedback (Seeing How It Drives):
   • Once the app is live, tools constantly watch its performance, check for errors, and collect feedback.
   • This information then goes back to the developers, so they can plan the next improvements or fixes, starting the whole cycle over again!
   • Analogy: Customers drive the car and provide feedback to the manufacturer, which helps design the next model or fix issues.

Here are the key components of a CI/CD pipeline:

1. Source Code Repository

• Tool: Git (GitHub, GitLab, Bitbucket)
• Purpose: Central location for version control and collaboration.

2. Build

• Tool: Maven, Gradle, npm, Webpack
• Purpose: Compile source code, resolve dependencies, generate build artifacts.

3. Automated Tests

• Tool: JUnit, PyTest, Selenium, Jest
• Purpose: Ensure the quality of code through unit, integration, and end-to-end tests.

4. Code Quality & Security Checks

• Tool: SonarQube, ESLint, Checkmarx, Snyk
• Purpose: Static code analysis, linting, and vulnerability scans.

5. Artifact Repository

• Tool: JFrog Artifactory, Nexus, Docker Hub
• Purpose: Store built artifacts (e.g., .jar, .war, Docker images) for later deployment.

6. Deployment Automation

• Tool: Jenkins, GitHub Actions, GitLab CI, CircleCI, ArgoCD
• Purpose: Automate deployment to staging or production environments.

7. Infrastructure Provisioning

• Tool: Terraform, Ansible, CloudFormation
• Purpose: Define and manage infrastructure as code (IaC).

8. Monitoring & Logging

• Tool: Prometheus, Grafana, ELK Stack, Datadog
• Purpose: Monitor application performance, errors, and system health.

9. Rollback & Recovery

• Tool: Kubernetes (with Helm), Spinnaker
• Purpose: Rollback to a previous version in case of failure.

10. Release Management & Notifications

• Tool: Slack, Jira, Microsoft Teams, Email
• Purpose: Track release progress, notify stakeholders, and document deployments.

Use Case: Todo App API with FastAPI

Project Structure:

fastapi-todo/
├── app/
│   ├── main.py
│   ├── models.py
│   ├── database.py
│   └── crud.py
├── tests/
│   └── test_main.py
├── requirements.txt
├── Dockerfile
├── .dockerignore
└── .github/
    └── workflows/
        └── ci-cd.yml

requirements.txt

fastapi
uvicorn
sqlalchemy
pydantic
pytest

main.py

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

todos = []

class Todo(BaseModel):
    title: str
    completed: bool = False

@app.get("/todos")
def get_todos():
    return todos

@app.post("/todos")
def add_todo(todo: Todo):
    todos.append(todo)
    return todo

Dockerfile

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY ./app /app

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

tests/test_main.py

from fastapi.testclient import TestClient
from app.main import app

client = TestClient(app)

def test_add_todo():
    response = client.post("/todos", json={"title": "Buy milk"})
    assert response.status_code == 200
    assert response.json()["title"] == "Buy milk"

GitHub Actions: .github/workflows/ci-cd.yml

name: FastAPI CI/CD

on:
  push:
    branches: [ main ]

jobs:
  build-test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: 3.11

    - name: Install dependencies
      run: |
        pip install -r requirements.txt

    - name: Run tests
      run: |
        pytest tests/

  docker-deploy:
    needs: build-test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'

    steps:
    - name: Login to Docker Hub
      run: echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin

    - name: Build & Push Docker Image
      run: |
        docker build -t myuser/fastapi-todo .
        docker push myuser/fastapi-todo