Pipeline Types and Syntax
Pipeline Types and Syntax
Learning Objectives
- Understand the differences between Declarative and Scripted pipelines
- Master advanced pipeline syntax and structures
- Learn pipeline validation and troubleshooting
- Implement complex pipeline configurations
Introduction to Pipeline Types
Declarative vs Scripted Pipelines
Jenkins offers two syntax types for defining pipelines: Declarative and Scripted. Each has its own strengths and use cases.
Declarative Pipeline
Declarative pipelines provide a more structured and opinionated way to write Jenkins pipelines.
pipeline {
agent any
environment {
MAVEN_HOME = tool 'Maven-3.8.6'
PATH = "${MAVEN_HOME}/bin:${PATH}"
}
stages {
stage('Build') {
steps {
sh 'mvn clean package'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
post {
always {
junit '**/target/surefire-reports/*.xml'
}
}
}
}
}
Scripted Pipeline
Scripted pipelines offer more flexibility and programmatic control.
node {
def mvnHome = tool 'Maven-3.8.6'
try {
stage('Checkout') {
checkout scm
}
stage('Build') {
withEnv(["PATH+MAVEN=${mvnHome}/bin"]) {
sh 'mvn clean package'
}
}
stage('Test') {
try {
sh 'mvn test'
} finally {
junit '**/target/surefire-reports/*.xml'
}
}
} catch (e) {
currentBuild.result = 'FAILED'
throw e
}
}
Advanced Pipeline Syntax
Directives and Steps
Pipeline Directives
Directives provide configuration options for your pipeline.
- agent - Specifies where the pipeline will execute
agent { docker { image 'maven:3.8.6-openjdk-11' args '-v $HOME/.m2:/root/.m2' } } - environment - Define environment variables
environment { DEPLOY_ENV = 'staging' API_ENDPOINT = credentials('api-endpoint') } - options - Pipeline-specific options
options { timeout(time: 1, unit: 'HOURS') timestamps() buildDiscarder(logRotator(numToKeepStr: '10')) } - parameters - Build parameters
parameters { string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: 'Deployment environment') choice(name: 'REGION', choices: ['us-east-1', 'us-west-2', 'eu-west-1'], description: 'AWS region') booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run test suite') }
Advanced Stage Configurations
Parallel Stages
stage('Parallel Tests') {
parallel {
stage('Unit Tests') {
steps {
sh 'mvn test'
}
}
stage('Integration Tests') {
steps {
sh 'mvn integration-test'
}
}
stage('Security Scan') {
steps {
sh 'run-security-scan.sh'
}
}
}
}
Matrix Stages
matrix {
axes {
axis {
name 'PLATFORM'
values 'linux', 'windows', 'mac'
}
axis {
name 'BROWSER'
values 'chrome', 'firefox', 'safari'
}
}
stages {
stage('E2E Tests') {
steps {
sh "./run-tests.sh ${PLATFORM} ${BROWSER}"
}
}
}
}
Advanced Pipeline Features
Shared Libraries Integration
@Library('my-shared-library') _
pipeline {
agent any
stages {
stage('Build') {
steps {
script {
def builder = new org.example.Builder()
builder.compile()
}
}
}
}
}
Dynamic Stage Generation
def environments = ['dev', 'staging', 'prod']
pipeline {
agent any
stages {
stage('Build') {
steps {
sh 'mvn package'
}
}
stage('Deploy') {
steps {
script {
environments.each { env ->
stage("Deploy to ${env}") {
when { expression { params.DEPLOY_ENV == env } }
steps {
sh "./deploy.sh ${env}"
}
}
}
}
}
}
}
}
Pipeline Validation and Testing
Syntax Validation
Using Jenkins CLI
java -jar jenkins-cli.jar -s http://jenkins-url/ declarative-linter < Jenkinsfile
Using IDE Plugins
Many IDEs offer Jenkins pipeline syntax validation:
- IntelliJ IDEA with Jenkins Pipeline Plugin
- VS Code with Jenkins Pipeline Linter Connector
Pipeline Unit Testing
JenkinsPipelineUnit Framework
// Test case example
default_pipeline_test.groovy
import com.lesfurets.jenkins.unit.*
class DefaultPipelineTest extends BasePipelineTest {
def script
@Override
@Before
void setUp() {
super.setUp()
script = loadScript("vars/defaultPipeline.groovy")
}
@Test
void testBuildStage() {
when(script.sh("mvn clean package")).thenReturn(0)
script.call()
assertJobStatusSuccess()
assertStepExecuted("mvn clean package")
}
}
Best Practices and Patterns
Code Organization
-
Modular Pipeline Structure ```groovy // Jenkinsfile pipeline { agent any
stages { stage(‘Initialize’) { steps { script { env.BUILD_VERSION = sh(script: ‘git describe –tags’, returnStdout: true).trim() } } }
stage('Build and Test') { parallel { stage('Build') { steps { buildModule() } } stage('Test') { steps { runTests() } } } } stage('Deploy') { when { branch 'main' } steps { deployApplication() } } } }
// vars/buildModule.groovy def call() { sh ‘mvn clean package’ }
// vars/runTests.groovy def call() { sh ‘mvn test’ junit ‘*/target/surefire-reports/.xml’ }
// vars/deployApplication.groovy def call() { withCredentials([string(credentialsId: ‘deploy-token’, variable: ‘TOKEN’)]) { sh “./deploy.sh ${env.BUILD_VERSION} ${TOKEN}” } }
### Pipeline Security
1. **Credential Handling**
```groovy
pipeline {
agent any
environment {
DOCKER_REGISTRY = 'registry.example.com'
}
stages {
stage('Build and Push') {
steps {
script {
withCredentials([
usernamePassword(credentialsId: 'docker-registry-credentials',
usernameVariable: 'DOCKER_USER',
passwordVariable: 'DOCKER_PASS'),
string(credentialsId: 'api-key', variable: 'API_KEY')
]) {
sh """
docker build -t ${DOCKER_REGISTRY}/myapp:${BUILD_NUMBER} .
echo ${DOCKER_PASS} | docker login -u ${DOCKER_USER} --password-stdin ${DOCKER_REGISTRY}
docker push ${DOCKER_REGISTRY}/myapp:${BUILD_NUMBER}
"""
}
}
}
}
}
}
Exercises and Practice
Exercise 1: Create a Multi-Environment Pipeline
Create a pipeline that builds and deploys an application to different environments based on branch patterns:
pipeline {
agent any
environment {
APP_NAME = 'my-application'
}
stages {
stage('Determine Environment') {
steps {
script {
switch(env.BRANCH_NAME) {
case 'main':
env.DEPLOY_ENV = 'production'
break
case ~/^release\/.*$/:
env.DEPLOY_ENV = 'staging'
break
default:
env.DEPLOY_ENV = 'development'
}
}
}
}
stage('Build') {
steps {
sh 'mvn clean package'
}
}
stage('Test') {
parallel {
stage('Unit Tests') {
steps {
sh 'mvn test'
}
}
stage('Integration Tests') {
when { not { environment name: 'DEPLOY_ENV', value: 'development' } }
steps {
sh 'mvn integration-test'
}
}
}
}
stage('Security Scan') {
when { environment name: 'DEPLOY_ENV', value: 'production' }
steps {
sh 'run-security-scan.sh'
}
}
stage('Deploy') {
steps {
script {
def deployScript = load "scripts/deploy-${env.DEPLOY_ENV}.groovy"
deployScript.deploy(APP_NAME)
}
}
}
}
post {
always {
junit '**/target/surefire-reports/*.xml'
archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
}
success {
slackSend channel: '#deployments',
color: 'good',
message: "Successfully deployed ${APP_NAME} to ${env.DEPLOY_ENV}"
}
failure {
slackSend channel: '#deployments',
color: 'danger',
message: "Failed to deploy ${APP_NAME} to ${env.DEPLOY_ENV}"
}
}
}
Exercise 2: Implement Pipeline with Approval Gates
```groovy pipeline { agent any
environment {
APP_NAME = 'my-application'
VERSION = sh(script: 'git describe --tags', returnStdout: true).trim()
}
stages {
stage('Build and Test') {
stages {
stage('Build') {
steps {
sh 'mvn clean package'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
}
}
}
stage('Quality Gate') {
steps {
timeout(time: 1, unit: 'HOURS') {
script {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
stage('Staging Deployment') {
steps {
script {
deploy('staging')
}
}
}
stage('Integration Tests') {
steps {
sh 'mvn integration-test'
}
}
stage('Production Approval') {
when { branch 'main' }
steps {
script {
def deploymentApproval = input(
message: "Deploy to Production?",
ok: "Deploy",
parameters: [
string(name: 'APPROVER_NAME', defaultValue: '', description: 'Your name'),
text(name: 'APPROVAL_REASON', defaultValue: '', description: 'Reason for approval')
]
)
env.APPROVER_NAME = deploymentApproval.APPROVER_NAME
env.APPROVAL_REASON = deploymentApproval.APPROVAL_REASON
}
}
}
stage('Production Deployment') {
when { branch 'main' }
steps {
script {
deploy('production')
}
}
}
}
post {
always {
script {
def deploymentRecord = """
Deployment Summary:
- Application: ${APP_NAME}
- Version: ${VERSION}
- Environment: ${env.DEPLOY_ENV}
- Status: ${currentBuild.result}