Best Practices Guide - Jenkins as Code

Recommended practices for using Jenkins as Code automation framework

Updated Jan 15, 2024
8 min read
Intermediate
tools

Best Practices Guide

Recommended practices for using Jenkins as Code automation framework

📋 Table of Contents

⚙️ Configuration Management

1. Use Environment-Specific Variables

✅ DO:

# group_vars/packer_al2023_aarch64_devops_jenkins.yml
jenkins_location:
  url: "https://jenkins-{{ environment }}.mcloud.com/"

❌ DON’T:

# Hardcoding values
jenkins_location:
  url: "https://jenkins-prod.mcloud.com/"  # Wrong for all environments

2. Organize Variables Logically

✅ DO:

# Group related variables
jenkins_onboarding:
  amazon:
    mcloud:
      jobs: [ /* ... */ ]
      env: ["nonprod", "prod"]

❌ DON’T:

# Scattered configuration
jenkins_mcloud_jobs: [ /* ... */ ]
jenkins_mcloud_env: ["nonprod", "prod"]
jenkins_amazon_mcloud: [ /* ... */ ]

3. Use Sensible Defaults

✅ DO:

def defaultConfigs = [
    EKS_CLUSTER_NAME: "${PROJECT_NAME}-eks-${PROJECT_ENV}",  // Dynamic
    CICD_TEMPLATE_BASE: 'templates/cicd',  // Standard location
    GROUP: "v2"  // Current version
]

❌ DON’T:

def defaultConfigs = [
    EKS_CLUSTER_NAME: "hardcoded-cluster-name",  // Not flexible
    CICD_TEMPLATE_BASE: '',  // Empty default
    GROUP: null  // No default
]

4. Document Custom Variables

✅ DO:

// Custom variable for feature flag
// Purpose: Enable beta features for testing
// Usage: Set to true to use template-based Jenkinsfiles
BETA: false

❌ DON’T:

// Undocumented custom variable
BETA: false  // What is this?

🎩 Job Creation

1. Follow Naming Conventions

✅ DO:

// Consistent naming
appNames: ['user-service', 'payment-service', 'order-service']

❌ DON’T:

// Inconsistent naming
appNames: ['UserService', 'payment_service', 'orderService']

2. Use Appropriate Job Types

✅ DO:

// Standard application job
[
    APP_REPO: 'my-app',
    APP_BRANCH: 'master',
    appNames: ['my-service']
]

// Custom job when needed
[
    SERVICE: 'data-pipeline',
    CICD_TEMPLATE_NAME: 'custom-template',
    JOB_PARAMS: [ /* custom params */ ]
]

❌ DON’T:

// Using custom job for standard application
[
    SERVICE: 'my-service',
    CICD_TEMPLATE_NAME: 'default',  // Should use standard job
    JOB_PARAMS: []
]

✅ DO:

def jobGenie = [
    "mcloud": [
        // All mcloud jobs together
        [ APP_REPO: 'app1', appNames: ['service1'] ],
        [ APP_REPO: 'app2', appNames: ['service2'] ]
    ],
    "ncloud": [
        // All ncloud jobs together
        [ APP_REPO: 'app3', appNames: ['service3'] ]
    ]
]

❌ DON’T:

// Mixed organization
def jobGenie = [
    "mcloud": [ /* mcloud job */ ],
    "ncloud": [ /* ncloud job */ ],
    "mcloud": [ /* another mcloud job - duplicate key! */ ]
]

4. Minimize Job Parameters

✅ DO:

// Essential parameters only
JOB_PARAMS: [
    [ name: 'APP_BRANCH', string: 'master', description: 'Application branch' ],
    [ name: 'OPERATION', choices: ['build', 'deploy'], description: 'Operation type' ]
]

❌ DON’T:

// Too many parameters
JOB_PARAMS: [
    [ name: 'PARAM1', ... ],
    [ name: 'PARAM2', ... ],
    [ name: 'PARAM3', ... ],
    // ... 20 more parameters
]

🔐 Security Practices

1. Use Role-Based Access Control

✅ DO:

# Granular permissions
overall_read_users:
  amazon:
    mcloud:
      - user: "dev"
    qa:
      - user: "qa-user"

❌ DON’T:

# Overly broad permissions
overall_read_users:
  amazon:
    mcloud:
      - user: "*"  # Everyone has access

2. Store Secrets Securely

✅ DO:

  • Use Jenkins Credentials Plugin
  • Use AWS Secrets Manager
  • Use Ansible Vault for sensitive variables

❌ DON’T:

  • Hardcode passwords in files
  • Commit secrets to Git
  • Share credentials via email

3. Regular Security Audits

✅ DO:

  • Review access permissions quarterly
  • Audit credential usage
  • Check for unused credentials
  • Review audit logs

❌ DON’T:

  • Set and forget security
  • Ignore security warnings
  • Skip access reviews

4. Follow Least Privilege

✅ DO:

# Minimum required permissions
permissionTemplates:
  - name: "build"
    permissions:
      - "Job/Build"
      - "Job/Read"
      - "View/Read"

❌ DON’T:

# Excessive permissions
permissionTemplates:
  - name: "build"
    permissions:
      - "Overall/Administer"  # Too much!
      - "Job/Delete"
      - "Credentials/Manage"

📝 Version Control

1. Use Feature Branches

✅ DO:

# Create feature branch
git checkout -b feature/onboard-new-service

# Make changes
# Commit and push
git commit -m "Onboard new-service-api"
git push origin feature/onboard-new-service

# Create Pull Request

❌ DON’T:

# Direct commits to main
git checkout main
# Make changes
git commit -m "Quick fix"  # No review!
git push origin main

2. Write Meaningful Commit Messages

✅ DO:

git commit -m "Onboard payment-service to mcloud nonprod

- Add payment-service-api to JobGenie
- Configure Docker build args
- Set up ArgoCD integration"

❌ DON’T:

git commit -m "fix"
git commit -m "update"
git commit -m "changes"

3. Review Changes Before Merging

✅ DO:

  • Create Pull Request
  • Request review from team
  • Address feedback
  • Merge after approval

❌ DON’T:

  • Merge without review
  • Skip testing
  • Ignore feedback

4. Tag Releases

✅ DO:

# Tag stable versions
git tag -a v1.0.0 -m "Initial stable release"
git push origin v1.0.0

❌ DON’T:

  • Forget to tag releases
  • Use unclear version numbers
  • Tag unstable code

📊 Monitoring & Observability

1. Enable Build Notifications

✅ DO:

// Configure Slack notifications
SLACK_NOTIFICATION: 'true'

❌ DON’T:

// Disable notifications
SLACK_NOTIFICATION: 'false'  // No visibility

2. Monitor Key Metrics

✅ DO:

  • Track build success rate
  • Monitor deployment frequency
  • Measure mean time to recovery
  • Track job execution time

❌ DON’T:

  • Ignore failing builds
  • Skip monitoring setup
  • No alerting configured

3. Set Up Logging

✅ DO:

# Enable audit trail
jenkins_audit_trail:
  logBuildCause: true
  logCredentialsUsage: true
  logFile: "/var/log/jenkins/audit-trail.log"

❌ DON’T:

  • Disable logging
  • No log retention policy
  • Logs not centralized

4. Configure Alerts

✅ DO:

  • Set up build failure alerts
  • Configure deployment alerts
  • Monitor system health
  • Alert on security events

❌ DON’T:

  • No alerting configured
  • Ignore alerts
  • Alert fatigue

🛡️ Disaster Recovery

1. Regular Backups

✅ DO:

# Configure automated backups
jenkins_s3_backup_dir: "s3://bucket-name/backup/jenkins"

❌ DON’T:

  • No backup strategy
  • Manual backups only
  • No backup testing

2. Test Recovery Procedures

✅ DO:

  • Test full recovery quarterly
  • Document recovery steps
  • Verify backup integrity
  • Practice disaster scenarios

❌ DON’T:

  • Never test recovery
  • Outdated documentation
  • Unknown recovery time

3. Version Control as Source of Truth

✅ DO:

  • All configs in Git
  • Git is primary backup
  • Regular Git backups
  • Multiple Git remotes

❌ DON’T:

  • Rely only on Jenkins backups
  • No Git backup
  • Single point of failure

👥 Team Collaboration

1. Document Changes

✅ DO:

  • Update documentation with changes
  • Add comments for complex logic
  • Document custom configurations
  • Maintain changelog

❌ DON’T:

  • Make undocumented changes
  • Assume knowledge
  • Skip documentation

2. Share Knowledge

✅ DO:

  • Conduct training sessions
  • Share best practices
  • Document common issues
  • Create runbooks

❌ DON’T:

  • Keep knowledge siloed
  • No knowledge sharing
  • Tribal knowledge only

3. Standardize Processes

✅ DO:

  • Use standard templates
  • Follow naming conventions
  • Consistent job structure
  • Standardized workflows

❌ DON’T:

  • Every team does it differently
  • No standards
  • Inconsistent patterns

4. Regular Reviews

✅ DO:

  • Quarterly architecture reviews
  • Monthly best practices review
  • Weekly team sync
  • Continuous improvement

❌ DON’T:

  • No regular reviews
  • Set and forget
  • No feedback loop

🎯 Performance Optimization

1. Optimize Job Execution

✅ DO:

  • Use appropriate Jenkins agents
  • Set build timeouts
  • Limit concurrent builds
  • Clean workspace regularly

❌ DON’T:

  • Run all jobs on master
  • No timeout limits
  • Unlimited concurrent builds
  • Never clean workspace

2. Cache Dependencies

✅ DO:

  • Cache Docker layers
  • Cache Maven/Gradle dependencies
  • Cache Node.js modules
  • Use build cache

❌ DON’T:

  • Download dependencies every time
  • No caching strategy
  • Slow builds

3. Optimize Resource Usage

✅ DO:

  • Right-size Jenkins agents
  • Monitor resource usage
  • Optimize build scripts
  • Use efficient tools

❌ DON’T:

  • Over-provision resources
  • No resource monitoring
  • Inefficient builds

📚 Additional Resources

🧭 Navigation

Related Topics:


Maintained by the DevOps Team

“Best practices are not rules, but guidelines for success”


Last Updated: January 15, 2024
Version: 1.0.0

Found this helpful?

Help us improve this documentation by sharing your feedback or suggesting improvements.