Jenkins CI/CD: Build Your First Pipeline

Jenkins CI/CD pipelines automate the entire software delivery process — from compiling code and running tests to deploying applications to production. As the most widely adopted open-source automation server, Jenkins integrates with virtually every development tool, cloud provider, and version control system. Whether you are deploying a Python API, a Node.js application, or infrastructure with Terraform, Jenkins pipelines define your delivery process as code that lives alongside your application. This guide walks you through setting up Jenkins, writing your first pipeline, and configuring production-ready automation.

Prerequisites

• A server with at least 2 GB RAM and 10 GB disk space (Ubuntu 22.04 or later recommended)
• Java 17 or later installed (Jenkins requires JDK)
• Docker installed (optional but recommended for containerized agents)
• A Git repository with application code to build
• Basic familiarity with YAML, Groovy, and shell scripting

Installing Jenkins

Using Docker (Recommended)

# Create a Docker volume for persistent data
docker volume create jenkins-data

# Run Jenkins with Docker socket access (for Docker-based agents)
docker run -d \
  --name jenkins \
  --restart unless-stopped \
  -p 8080:8080 \
  -p 50000:50000 \
  -v jenkins-data:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  jenkins/jenkins:lts

# Get the initial admin password
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Using the Official Repository (Ubuntu/Debian)

# Add Jenkins repository key and source
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/" | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

# Install Jenkins
sudo apt update
sudo apt install jenkins

# Jenkins starts automatically — check the status
sudo systemctl status jenkins

# Get the initial admin password
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Open http://your-server:8080 in a browser, paste the initial password, and follow the setup wizard. Install the suggested plugins — they include Git, Pipeline, and the most commonly needed integrations.

Writing Your First Jenkinsfile

A Jenkinsfile defines your pipeline as code. It lives in the root of your Git repository and Jenkins reads it on each build.

Declarative Pipeline Syntax

// Jenkinsfile
pipeline {
    agent any

    environment {
        APP_NAME = 'my-web-app'
        DEPLOY_ENV = 'staging'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Install Dependencies') {
            steps {
                sh 'npm ci'
            }
        }

        stage('Lint') {
            steps {
                sh 'npm run lint'
            }
        }

        stage('Test') {
            steps {
                sh 'npm test'
            }
            post {
                always {
                    junit 'test-results/**/*.xml'
                }
            }
        }

        stage('Build') {
            steps {
                sh 'npm run build'
                archiveArtifacts artifacts: 'dist/**', fingerprint: true
            }
        }

        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                sh './deploy.sh ${DEPLOY_ENV}'
            }
        }
    }

    post {
        failure {
            mail to: 'team@example.com',
                 subject: "Pipeline Failed: ${APP_NAME} #${BUILD_NUMBER}",
                 body: "Check: ${BUILD_URL}"
        }
        success {
            echo 'Pipeline completed successfully!'
        }
    }
}

Pipeline with Docker Agents

pipeline {
    agent none    // No global agent — each stage picks its own

    stages {
        stage('Build') {
            agent {
                docker {
                    image 'node:18-alpine'
                    args '-v $HOME/.npm:/root/.npm'    // Cache npm packages
                }
            }
            steps {
                sh 'npm ci && npm run build'
                stash includes: 'dist/**', name: 'build-output'
            }
        }

        stage('Test') {
            agent {
                docker { image 'node:18-alpine' }
            }
            steps {
                sh 'npm ci && npm test'
            }
        }

        stage('Docker Image') {
            agent any
            steps {
                unstash 'build-output'
                sh 'docker build -t myapp:${BUILD_NUMBER} .'
                sh 'docker push registry.example.com/myapp:${BUILD_NUMBER}'
            }
        }
    }
}

Each stage runs in a fresh Docker container with the specified image. The stash and unstash commands pass build artifacts between stages running on different agents.

Managing Credentials and Secrets

Never hardcode secrets in Jenkinsfiles. Use the Jenkins Credentials Manager:

1. Navigate to Manage Jenkins > Credentials > System > Global credentials
2. Click Add Credentials
3. Choose the type (Username/Password, SSH Key, Secret Text, etc.)
4. Give it an ID like docker-registry-creds

Use credentials in your pipeline:

stage('Deploy') {
    steps {
        withCredentials([
            usernamePassword(
                credentialsId: 'docker-registry-creds',
                usernameVariable: 'DOCKER_USER',
                passwordVariable: 'DOCKER_PASS'
            )
        ]) {
            sh 'docker login -u $DOCKER_USER -p $DOCKER_PASS registry.example.com'
            sh 'docker push registry.example.com/myapp:${BUILD_NUMBER}'
        }
    }
}

The credentials are injected as environment variables only within the withCredentials block and are masked in the build log.

Comparing Jenkins with CI/CD Alternatives

Feature	Jenkins	GitHub Actions	GitLab CI	CircleCI
Self-hosted	Yes (primary)	Yes (runners)	Yes	No
Cloud-hosted	No (community)	Yes	Yes	Yes
Configuration	Jenkinsfile (Groovy)	YAML	YAML	YAML
Plugin ecosystem	1800+ plugins	Actions marketplace	Built-in features	Orbs
Docker support	Via plugins	Native	Native	Native
Pricing (self-hosted)	Free (open source)	Free (self-hosted runners)	Free (CE)	N/A
Learning curve	Steep	Low	Low	Low
Best for	Complex enterprise workflows	GitHub-native projects	GitLab-native projects	SaaS-first teams

Use Jenkins when you need full control over the build infrastructure, have complex multi-branch workflows, or work in an enterprise environment with strict security requirements. Use GitHub Actions for projects already on GitHub that need simple CI/CD without managing infrastructure.

Real-World Scenario: Multi-Branch Pipeline

Your team uses feature branches. You want Jenkins to automatically build and test every branch, deploy main to staging, and only deploy tagged releases to production.

pipeline {
    agent { docker { image 'node:18' } }

    stages {
        stage('Build & Test') {
            steps {
                sh 'npm ci && npm test && npm run build'
            }
        }

        stage('Deploy Staging') {
            when { branch 'main' }
            steps {
                sh './deploy.sh staging'
            }
        }

        stage('Deploy Production') {
            when { tag 'v*' }
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
                sh './deploy.sh production'
            }
        }
    }
}

The input step pauses the pipeline and waits for a human to approve the production deployment — a safety gate that prevents accidental releases.

Gotchas and Edge Cases

Jenkins runs out of disk space: Old builds accumulate artifacts and logs. Configure build retention in the job settings (discard old builds) and set up a cron job to clean the workspace: jenkins-cli delete-builds job-name --range 1-100.

Pipeline hangs on input step: If no one approves the input, the pipeline holds a build executor indefinitely. Set a timeout: timeout(time: 30, unit: 'MINUTES') { input ... }.

Docker socket permissions: When running Jenkins in Docker with the Docker socket mounted, the Jenkins user needs permission to access /var/run/docker.sock. Add the jenkins user to the docker group or set socket permissions.

Groovy sandbox restrictions: Declarative pipelines run in a Groovy sandbox that blocks certain operations. If you need unrestricted Groovy (file I/O, network calls), the pipeline must be approved by an admin under Manage Jenkins > Script Approval.

Plugin version conflicts: Jenkins has 1800+ plugins and they sometimes conflict. Pin plugin versions and test upgrades in a staging Jenkins instance before updating production.

Troubleshooting

Pipeline fails with “No such DSL method”

// This error means a required plugin is not installed
// Check that Pipeline, Git, and Docker Pipeline plugins are installed
// Manage Jenkins > Plugins > Installed plugins

Build workspace grows too large

// Add a cleanup step to your pipeline
post {
    always {
        cleanWs()    // Delete the workspace after the build
    }
}

Webhook triggers not working

# Verify Jenkins is accessible from your Git provider
curl -I http://your-jenkins:8080/github-webhook/

# Check the Jenkins system log for webhook events
# Manage Jenkins > System Log > All Jenkins Logs

Summary

• Jenkins pipelines define your CI/CD process as code in a Jenkinsfile that lives in your Git repository alongside your application
• Declarative syntax with pipeline, stages, and steps blocks is the recommended approach — it is easier to read, validate, and maintain than scripted pipelines
• Docker agents run each pipeline stage in a fresh container, ensuring consistent build environments and eliminating “works on my machine” issues
• Use the Credentials Manager for all secrets — never hardcode passwords, tokens, or SSH keys in Jenkinsfiles
• Multi-branch pipelines with when { branch } and when { tag } conditions automatically apply different deployment rules per branch
• Set build retention and workspace cleanup to prevent Jenkins from consuming all available disk space over time

Related Articles

• Getting Started with GitHub Actions for CI/CD
• Docker Compose: A Practical Guide for System Administrators
• Terraform: Getting Started with Infrastructure as Code
• Ansible for Beginners: Automate Server Configuration Without an Agent