Jenkins as Code - Complete Setup Guide

Step-by-step guide to set up Jenkins as Code with plugins, access control, and JobGenie integration

Updated Jan 15, 2025

20 min read

Intermediate

tools

Jenkins as Code - Complete Setup Guide

Complete step-by-step guide to set up Jenkins as Code with automated plugin management, access control, and JobGenie integration

🚀 Quick Links:

Need help creating jobs? See the JobGenie Job Creation Guide
Want to understand the architecture? Check Architecture Documentation
Looking for DevOps services? Contact Us

Overview

This guide walks you through setting up a production-ready Jenkins instance with:

✅ Automated Installation: Jenkins setup via Ansible
✅ Plugin Management: Essential plugins for CI/CD automation
✅ Access Control: Role-based access control (RBAC) configuration
✅ JobGenie Integration: Automatic job generation from YAML definitions
✅ Configuration as Code: All settings managed through YAML files

What You’ll Accomplish

By the end of this guide, you’ll have:

A fully configured Jenkins instance
All required plugins installed
Access control configured
JobGenie integrated and ready to generate jobs automatically
A foundation for DevOps as a Service

Prerequisites

System Requirements

Server Requirements

OS: Amazon Linux 2023 / Amazon Linux 2 (ARM64 or x86_64) or any Linux distribution
CPU: Minimum 2 cores, recommended 4+ cores
RAM: Minimum 4GB, recommended 8GB+
Disk: Minimum 50GB, recommended 100GB+
Network: Internet access for plugin downloads and Git operations

Software Requirements

Ansible: 2.9 or higher
Python: 3.8 or higher
Git: Latest version
Java: JDK 11 or JDK 17 (required by Jenkins)

Access Requirements

SSH access to target server with sudo/root privileges
Git repository access (SSH keys or HTTPS credentials)
Jenkins admin credentials (will be created during setup)

Network Requirements

Outbound HTTPS (443) for plugin downloads
Outbound SSH (22) for Git operations
Inbound HTTPS (443) for Jenkins web UI
Inbound SSH (22) for server management

Step 1: Jenkins Installation

Method 1: Automated Installation with Ansible (Recommended)

This method uses Ansible playbooks to automate the entire Jenkins setup.

Step 1.1: Clone Required Repositories

# Clone the infrastructure repository (contains Ansible automation)
git clone https://github.com/HarryTheDevOpsGuy/mCloud-infra.git
cd mCloud-infra/ansible

# Clone the Jenkins configuration repository (optional, for reference)
cd ..
git clone https://github.com/HarryTheDevOpsGuy/mCloud-Jenkins.git

Step 1.2: Configure Ansible Variables

Edit the Ansible group variables file:

cd mCloud-infra/ansible
vim group_vars/packer_al2023_aarch64_devops_jenkins.yml

Step 1.3: Essential Jenkins Configuration

Configure the following in your Ansible variables file:

# Jenkins Version
jenkins_version: 2.528.2  # Use LTS version

# Jenkins Home Directory
jenkins_home: "/var/lib/jenkins"

# Jenkins URL Configuration
jenkins_location:
  url: "https://jenkins.example.com/"
  adminAddress: "jenkins-admin@example.com"

# Jenkins Security Realm (Admin Users)
jenkins_securityRealm:
  local:
    allowsSignup: false
    enableCaptcha: false
    users:
      - id: "admin"
        name: "admin"
        password: "${ADMIN_PASSWORD}"  # Use environment variable or Ansible Vault
        properties: ["apiToken", "myView", "timezone", "mailer"]

Step 1.4: Run Ansible Playbook

# Test connectivity first (dry-run)
ansible-playbook packer.yml \
  -e "target_host=packer_al2023_aarch64_devops_jenkins" \
  --check

# Run full deployment
ansible-playbook packer.yml \
  -e "target_host=packer_al2023_aarch64_devops_jenkins" \
  -t deploy,monitoring

Method 2: Manual Installation

If you prefer manual installation or have an existing Jenkins instance:

Step 1.1: Install Java

# Amazon Linux 2023
sudo dnf install java-21-amazon-corretto-devel -y

# Ubuntu/Debian
sudo apt update
sudo apt install openjdk-17-jdk -y

# Verify installation
java -version

Step 1.2: Install Jenkins

# Add Jenkins repository
sudo wget -O /etc/yum.repos.d/jenkins.repo \
    https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key

# Install Jenkins
sudo dnf install jenkins -y

# Start and enable Jenkins
sudo systemctl start jenkins
sudo systemctl enable jenkins

# Check status
sudo systemctl status jenkins

Step 1.3: Get Initial Admin Password

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Step 1.4: Complete Initial Setup

Open browser: http://your-server-ip:8080
Enter the initial admin password
Install suggested plugins (or customize)
Create admin user
Configure Jenkins URL

Step 2: Plugin Installation

Required Plugins for JobGenie

Install the following essential plugins:

Core Plugins (Required)

jenkins_plugins:
  # Job Generation
  - job-dsl                    # Required: Job generation engine
  - configuration-as-code      # Required: System configuration management
  
  # Access Control
  - role-strategy             # Required: Role-based access control
  
  # Pipeline Support
  - workflow-aggregator       # Required: Pipeline support
  - git                       # Required: Git integration
  
  # Security & Credentials
  - credentials-binding       # Required: Secure credential management
  - build-user-vars-plugin    # Required: Build user variables
  
  # Optional but Recommended
  - docker-slaves            # Docker-based build agents
  - docker-workflow          # Docker pipeline support
  - ansicolor                # Colored console output
  - favorite                 # Favorite jobs
  - aws-java-sdk-secretsmanager  # AWS Secrets Manager integration
  - google-login             # Google OAuth (if using)

Installation Methods

Method 1: Via Ansible (Automated)

Plugins are automatically installed when using Ansible. Configure in your variables file:

jenkins_plugins:
  - job-dsl
  - configuration-as-code
  - role-strategy
  - workflow-aggregator
  - git
  - credentials-binding
  - build-user-vars-plugin

Method 2: Via Jenkins UI (Manual)

Navigate to: Manage Jenkins → Manage Plugins → Available
Search for each plugin
Select plugins and click Install without restart or Download now and install after restart
Restart Jenkins if prompted

Method 3: Via Configuration as Code (Recommended)

Configure plugins in your jenkins.yaml file:

jenkins:
  # Plugins are managed via Configuration as Code
  # See jenkins.yaml.j2 template for plugin configuration

Verify Plugin Installation

# Check installed plugins via CLI
curl -s http://localhost:8080/pluginManager/api/json?depth=1 | \
  jq '.plugins[] | select(.shortName | contains("job-dsl"))'

# Or check via Jenkins UI
# Navigate to: Manage Jenkins → Manage Plugins → Installed

Step 3: Access Control & Permissions

Role-Based Access Control (RBAC) Setup

JobGenie requires proper access control to manage jobs effectively.

Step 3.1: Configure Role Strategy Plugin

Navigate to: Manage Jenkins → Configure Global Security
Under Authorization, select Role-Based Strategy
Click Save

Step 3.2: Define Permission Templates

Configure permission templates in your Ansible variables:

# Permission Templates
permissionTemplates:
  - name: "build"
    permissions: 
      - "Job/Cancel"
      - "Job/Build"
      - "Job/Read"
      - "View/Read"
      - "Run/Replay"
  
  - name: "write"
    permissions:
      - "Job/Cancel"
      - "Job/Build"
      - "Job/Read"
      - "View/Read"
      - "Run/Replay"
      - "Job/Create"
      - "Job/Discover"
      - "Job/Configure"

Step 3.3: Configure Global Roles

# Global Roles
overall_admin_users:
  devops_managers:
    - user: "hari_25585"
    - user: "admin"

overall_read_users:
  amazon:
    mcloud:
      - user: "dev-user"
    qa:
      - user: "qa-user"

Step 3.4: Configure Project-Specific Roles

# Project-specific roles
jenkins_onboarding:
  amazon:
    mcloud:
      env: ["nonprod", "prod"]
      dev_leads: ''
      # Roles are automatically created based on patterns

Access Control Best Practices

Principle of Least Privilege: Grant minimum required permissions
Separate Environments: Different permissions for nonprod vs prod
Role Templates: Use templates for consistency
Regular Audits: Review permissions periodically
Git-Based Management: Manage roles via Configuration as Code

Step 4: JobGenie Integration

Overview

JobGenie automatically detects YAML job definition files and generates Jenkins jobs. This section covers the complete integration process.

Step 4.1: Configure Git Credentials

Navigate to: Manage Jenkins → Credentials → System → Global credentials
Click Add Credentials
Configure:
- Kind: SSH Username with private key (or Username with password for HTTPS)
- ID: jenkins_repo_key
- Username: git (or your Git username)
- Private Key: Upload or paste SSH private key
Click OK

Step 4.2: Configure Global Libraries

JobGenie requires shared libraries for job generation:

Navigate to: Manage Jenkins → Configure System → Global Pipeline Libraries
Click Add
Configure:
- Name: sharedPipelineUtils
- Default version: master
- Retrieval method: Modern SCM
- Source Code Management: Git
  - Repository URL: git@github.com:HarryTheDevOpsGuy/mCloud-Jenkins.git
  - Credentials: Select jenkins_repo_key
  - Library Path: sharedlibs
Click Save

Note: If you don’t have access to the shared libraries repository, contact HarryTheDevOpsGuy@gmail.com for access.

Step 4.3: Create JobGenie Seed Job

Option A: Using JobGenie-Pipelines Repository (Recommended)

Navigate to: New Item
Enter name: 0-JobGenie-Generator
Select: Pipeline
Click OK

Configure Seed Job:

General Configuration:
- Check This project is parameterized
- Add String Parameter:
  - Name: GitBranch
  - Default Value: main
  - Description: Config repository branch.
Pipeline Configuration:
- Definition: Pipeline script from SCM
- SCM: Git
- Repository URL: https://github.com/YOUR_USERNAME/JobGenie-Pipelines.git (your repository)
- Credentials: Select jenkins_repo_key
- Branch: ${GitBranch}
- Script Path: JobGenie/Jenkinsfile

Option B: Direct Pipeline Script

If you prefer a direct script:

node {
  label "linux-slave"
  checkout scm

  def allEnvVars = [ message: 'Hello from pipeline', credentials: 'SECRET', TECHTEAM: 'mcloud' ] + env.getEnvironment()
  ['BUILD_NUMBER', 'BUILD_DISPLAY_NAME', 'JOB_NAME', 'JOB_BASE_NAME', 'WORKSPACE', 'BUILD_URL'].each { var -> 
    if (env."$var") { allEnvVars[var] = env."$var" }
  }

  // Load shared libraries
  dir('mCloud-Jenkins') {
    git url: 'https://github.com/HarryTheDevOpsGuy/mCloud-Jenkins.git', 
        branch: 'master', 
        credentialsId: 'jenkins_repo_key'
  }

  // Load JobGenie-Pipelines repository
  dir('JobGenie-Pipelines') {
    git url: 'https://github.com/YOUR_USERNAME/JobGenie-Pipelines.git', 
        branch: "${GitBranch}", 
        credentialsId: 'jenkins_repo_key'
  }

  jobDsl targets: ["JobGenie-Pipelines/JobGenie/pipelines/*.groovy"].join('\n'),
    additionalClasspath: 'mCloud-Jenkins/sharedlibs/src',
    removedJobAction: 'DELETE',
    removedViewAction: 'DELETE',
    additionalParameters: allEnvVars
}

Step 4.4: Configure JobGenie Repository

Fork or Clone JobGenie-Pipelines Repository:

git clone https://github.com/HarryTheDevOpsGuy/JobGenie-Pipelines.git
cd JobGenie-Pipelines

Update Configuration in JobGenie/pipelines/jobGenie.groovy:

CONFIG_REPO: "https://github.com/YOUR_USERNAME/YOUR_REPO_NAME.git",
CONFIG_BRANCH: "main",
JENKINS_GIT_KEY: 'jenkins_repo_key',

Create Directory Structure:

mkdir -p amazon/myproject/{nonprod,prod}/jobs
mkdir -p global/common/{nonprod,prod}/jobs

Create Your First Job Definition:

vim amazon/myproject/prod/jobs/myproject-prod-jobs.yml

Add content:

jobgenie:
  default:
    GROUP: "v2"
    ENV: "prod"
  jobs:
    - NAME: "hello-world-service"
      CONFIGS:
        JOB_TYPE: "freestyle"
        SKIP_GIT: true
        SCRIPT: |-
          echo "Hello World"

Commit and Push:

git add amazon/myproject/prod/jobs/myproject-prod-jobs.yml
git commit -m "Add hello-world-service job definition"
git push origin main

Step 4.5: Run Seed Job

Navigate to: 0-JobGenie-Generator
Click Build with Parameters
Set GitBranch to main (or your branch)
Click Build
Monitor console output for job generation

How JobGenie Auto-Detection Works

JobGenie automatically:

Scans Repository: Recursively searches for files ending with -jobs.yml or -jobs.yaml
Parses YAML: Uses SnakeYAML library to parse job definitions
Extracts Metadata: Identifies organization, project, and environment from file path
Generates Jobs: Uses JobDSL to create/update Jenkins jobs
Manages Lifecycle: Deletes jobs that are removed from YAML files

File Path Pattern:

{organization}/{project}/{environment}/jobs/{project}-{environment}-jobs.yml

Example:

amazon/mcloud/prod/jobs/mcloud-prod-jobs.yml

Step 5: Verification

Verify Jenkins Installation

Access Jenkins: Navigate to https://jenkins.example.com/
Login: Use admin credentials
Check System Info: Navigate to Manage Jenkins → System Information
Verify Version: Check Jenkins version matches configuration

Verify Plugin Installation

Navigate to: Manage Jenkins → Manage Plugins → Installed
Verify all required plugins are installed:
- ✅ job-dsl
- ✅ configuration-as-code
- ✅ role-strategy
- ✅ workflow-aggregator
- ✅ git
- ✅ credentials-binding

Verify Access Control

Navigate to: Manage Jenkins → Configure Global Security
Verify Role-Based Strategy is selected
Navigate to: Manage Jenkins → Manage and Assign Roles
Verify roles are configured correctly

Verify JobGenie Integration

Check Seed Job: Navigate to 0-JobGenie-Generator
Run Seed Job: Build with parameters
Check Console Output: Verify no errors
Verify Job Creation: Check that jobs are created in expected locations
Test Job Execution: Run a generated job to ensure it works

Expected Job Path Format

{organization}/{project}/{environment}/deploy/{GROUP}/{ENV}/{job-name}

Example:

amazon/myproject/prod/deploy/v2/prod/hello-world-service

Post-Setup Configuration

Configure Jenkins URL

jenkins_location:
  url: "https://jenkins.example.com/"
  adminAddress: "jenkins-admin@example.com"

Configure Git Settings

jenkins_scm_git:
  globalConfigEmail: "devops-admin@example.com"
  globalConfigName: "jenkins-server"

Configure Global Environment Variables

jenkins_global_vars:
  ManagedBy: "DevOps Team"
  AWS_REGION: "ap-south-1"
  PATH: '${PATH}:/opt/maven/bin'

Configure Slack Notifications (Optional)

jenkins_slack_notifier:
  botUser: false
  room: "devops-alerts"
  sendAsText: false
  teamDomain: "https://yourteam.slack.com/"
  tokenCredentialId: "SlackToken"

Troubleshooting

Issue: Jenkins Won’t Start

Solutions:

Check Java installation: java -version
Check Jenkins logs: sudo tail -f /var/log/jenkins/jenkins.log
Verify port 8080 is not in use: sudo netstat -tlnp | grep 8080
Check disk space: df -h

Issue: Plugins Not Installing

Solutions:

Check internet connectivity
Verify Jenkins update center is accessible
Check plugin compatibility with Jenkins version
Review plugin installation logs

Issue: Seed Job Fails

Solutions:

Verify Git credentials are configured correctly
Check repository URL and branch
Verify shared libraries are accessible
Review seed job console output for errors
Ensure JobDSL script approval is granted

Issue: Jobs Not Created

Solutions:

Verify YAML file naming: must end with -jobs.yml or -jobs.yaml
Check YAML syntax is valid
Verify file path matches expected structure
Review seed job console output for parsing errors

Issue: Access Control Not Working

Solutions:

Verify Role Strategy plugin is installed and enabled
Check role assignments in “Manage and Assign Roles”
Verify user exists in security realm
Review permission templates

Getting More Help

Check the JobGenie Job Creation Guide for job definition help
Review Architecture Documentation for system understanding
Contact: HarryTheDevOpsGuy@gmail.com

Next Steps

Now that Jenkins is set up with JobGenie, you can:

Create Your First Job: Follow the JobGenie Job Creation Guide
Understand Architecture: Read the Architecture Documentation
Learn Best Practices: Check Best Practices Guide
Explore Examples: See JobGenie Examples

DevOps as a Service

💼 Need Professional DevOps Services?

Looking for expert help with Jenkins as Code setup, JobGenie integration, or CI/CD automation? We offer professional DevOps consulting services:

✅ Jenkins Setup & Configuration: Complete Jenkins as Code implementation
✅ JobGenie Integration: Custom JobGenie setup and training
✅ CI/CD Pipeline Development: Custom pipeline templates and automation
✅ Infrastructure Automation: Ansible, Terraform, Kubernetes
✅ DevOps Training: Team training and knowledge transfer

Contact: HarryTheDevOpsGuy@gmail.com

Portfolio

Services: DevOps as a Service Guide

Built with ❤️ by the DevOps Team

“Soch Wahi, Approach Nai” - Same Vision, New Approach

Tags: #Jenkins #Setup #Configuration #Ansible #CI/CD #Automation #JobGenie #Plugins

Related Tools & Projects

BG Deployer

Automated blue-green deployment for zero-downtime AWS releases

AWS Blue-Green

mCert

SSL certificate monitoring with Slack/email alerts & Telegram

SSL Telegram

mTracker

Real-time Linux user activity monitoring with Slack notifications

Linux Monitoring

mWatcher

Server health monitoring for CPU, memory, disk with alerting

Health Alerts

gCrypt

Git-crypt wrapper for secure file encryption & access management

Security Git-crypt

Interactive Tools

AWS VPC Designer, EKS Cost Calculator, and more utilities

Tools Calculator

External Resources

Official Documentation

AWS Documentation

Complete AWS service documentation

Community Resources

GitHub Profile

Open source projects and repositories

Professional network and updates

Dev.to

Technical articles and tutorials

Contact

Get in touch for questions or collaboration

Quick Actions

All Docs

Browse all documentation

Blog

Tools

DevOps utilities

Portfolio

My work & projects

Found this helpful?

Help us improve this documentation by sharing your feedback or suggesting improvements.

Report Issue Start Discussion

Jenkins as Code - Complete Setup Guide

📋 Table of Contents

Overview

What You’ll Accomplish

Prerequisites

System Requirements

Server Requirements

Software Requirements

Access Requirements

Network Requirements

Step 1: Jenkins Installation

Method 1: Automated Installation with Ansible (Recommended)

Step 1.1: Clone Required Repositories

Step 1.2: Configure Ansible Variables

Step 1.3: Essential Jenkins Configuration

Step 1.4: Run Ansible Playbook

Method 2: Manual Installation

Step 1.1: Install Java

Step 1.2: Install Jenkins

Step 1.3: Get Initial Admin Password

Step 1.4: Complete Initial Setup

Step 2: Plugin Installation

Required Plugins for JobGenie

Core Plugins (Required)

Installation Methods

Method 1: Via Ansible (Automated)

Method 2: Via Jenkins UI (Manual)

Method 3: Via Configuration as Code (Recommended)

Verify Plugin Installation

Step 3: Access Control & Permissions

Role-Based Access Control (RBAC) Setup

Step 3.1: Configure Role Strategy Plugin

Step 3.2: Define Permission Templates

Step 3.3: Configure Global Roles

Step 3.4: Configure Project-Specific Roles

Access Control Best Practices

Step 4: JobGenie Integration

Overview

Step 4.1: Configure Git Credentials

Step 4.2: Configure Global Libraries

Step 4.3: Create JobGenie Seed Job

Option A: Using JobGenie-Pipelines Repository (Recommended)

Option B: Direct Pipeline Script

Step 4.4: Configure JobGenie Repository

Step 4.5: Run Seed Job

How JobGenie Auto-Detection Works

Step 5: Verification

Verify Jenkins Installation

Verify Plugin Installation

Verify Access Control

Verify JobGenie Integration

Expected Job Path Format

Post-Setup Configuration

Configure Jenkins URL

Configure Git Settings

Configure Global Environment Variables

Configure Slack Notifications (Optional)

Troubleshooting

Issue: Jenkins Won’t Start

Issue: Plugins Not Installing

Issue: Seed Job Fails

Issue: Jobs Not Created

Issue: Access Control Not Working

Getting More Help

Next Steps

DevOps as a Service

Related Documentation

More from Tools

Architecture Documentation - Jenkins as Code

Best Practices Guide - Jenkins as Code

DevOps as a Service - Automated CI/CD Management

JobGenie - Complete Job Creation Guide

JobGenie - Complete Guide

Quick Reference Guide - Jenkins as Code

Jenkins as Code - Enterprise CI/CD Automation

BG Deployer

DevOps Tools & Utilities | Hari Prasad

JobGenie Getting Started

JobGenie

mCert