JobGenie - Complete Guide

Your friendly Jenkins job generator - Comprehensive guide to JobGenie job generation engine

Updated Jan 15, 2024

10 min read

Intermediate

tools

JobGenie - Complete Guide

Your friendly Jenkins job generator - Comprehensive guide to JobGenie job generation engine

📋 Table of Contents

Overview
Core Concepts
Configuration Structure
Job Types
Advanced Features
Examples
Best Practices
Troubleshooting

🎯 Overview

JobGenie is a powerful Jenkins job generation engine that transforms simple Groovy map definitions into fully functional Jenkins pipelines. It eliminates the need for manual job creation and provides a consistent, scalable approach to CI/CD pipeline management.

Key Benefits

✅ Zero Manual Configuration: Define jobs in code, not UI
✅ Consistency: Same structure across all jobs
✅ Scalability: Generate hundreds of jobs from simple configs
✅ Maintainability: Version-controlled job definitions
✅ Flexibility: Support for various job types and patterns

🧠 Core Concepts

1. JobGenie Map Structure

JobGenie uses nested Groovy maps to define jobs:

def jobGenie = [
    "techteam-name": [
        [ /* Job 1 Configuration */ ],
        [ /* Job 2 Configuration */ ],
        [ /* Job 3 Configuration */ ]
    ]
]

2. Configuration Hierarchy

Configurations are merged in this order (later overrides earlier):

defaultConfigs: Global defaults for all jobs
projectConfigs: Project-specific overrides
jobGenie entries: Individual job configurations

3. Variable Resolution

Variables are resolved using this precedence:

Job-specific variables (highest priority)
Project-specific variables
Default variables (lowest priority)

📝 Configuration Structure

Basic Structure

// 1. Default configurations (applied to all jobs)
def defaultConfigs = [
    EKS_CLUSTER_NAME: "my-cluster",
    CICD_TEMPLATE_BASE: 'templates/cicd',
    IMAGE_ECR: "1234567890.dkr.ecr.ap-south-1.amazonaws.com",
    // ... more defaults
]

// 2. Project-specific configurations
def projectConfigs = [
    "mcloud": [
        GROUP: "v3",
        ENV: "stage"
    ]
]

// 3. JobGenie definitions
def jobGenie = [
    "mcloud": [
        [
            APP_REPO: 'my-application',
            APP_BRANCH: 'master',
            appNames: ['my-service']
        ]
    ]
]

Required Variables

For Standard Application Jobs

[
    APP_REPO: 'repository-name',      // Git repository name
    APP_BRANCH: 'branch-name',       // Git branch
    appNames: ['service-name']       // List of service names
]

For Custom Jobs

[
    SERVICE: 'service-name',          // Service identifier
    CICD_TEMPLATE_NAME: 'template',  // Pipeline template
    JOB_PARAMS: [ /* parameters */ ] // Job parameters
]

Optional Variables

Docker Build Configuration

DOCKER_BUILD_DIR: 'path/to/docker',     // Docker build directory
DOCKERFILE_PATH: 'Dockerfile',          // Dockerfile path
DOCKER_BUILD_ARGS: 'ENV,TECHTEAM',      // Build arguments
DOCKER_IMAGE_ARCH: 'linux/arm64',       // Image architecture

Git Configuration

SSH_KEYS: 'default:/path/to/key',       // SSH key path
CONFIG_REPO: 'config-repo',             // Config repository
CONFIG_BRANCH: 'master',                // Config branch

Kubernetes/ArgoCD Configuration

ARGOCD_ENDPOINT: 'argocd.example.com',  // ArgoCD endpoint
ARGOCD_PROJECT: 'project-name',         // ArgoCD project
HELM_CHART: 'chart-name',               // Helm chart name
HELM_CHART_VERSION: '1.0.0',           // Chart version

Build Tools

MAVEN_VERSION: 'maven3.9.7',            // Maven version
JDK_VERSION: 'jdk21',                  // JDK version
NODE_VERSION: 'node18',                // Node.js version

🎨 Job Types

1. Standard Application Jobs

Purpose: Build and deploy standard Kubernetes applications

Configuration:

[
    APP_REPO: 'my-application',
    APP_BRANCH: 'develop',
    appNames: ['api-service', 'worker-service']
]

Generated Jobs:

{ENV}-{service-name}-pipeline: Main build/deploy pipeline

Parameters:

APP_BRANCH: Application git branch
OPERATION: build promote abort
EXTENT: none full
STEPS: Number of canary steps

2. Build + Deploy Jobs

Purpose: Separate build and deploy jobs for complex workflows

Configuration:

[
    APP_REPO: 'my-application',
    APP_BRANCH: 'develop',
    appNames: ['api-service'],
    BUILD: [
        NAME: 'api-service'
    ]
]

Generated Jobs:

{ENV}-{BUILD.NAME}-build: Build-only job
{ENV}-{service-name}-deploy: Deploy job with image selection

Deploy Job Parameters:

ImageTag: Dynamic ECR image tag selection
OPERATION: prepare-changes promote abort

3. Custom Jobs

Purpose: Jobs with custom templates and parameters

Configuration:

[
    SERVICE: 'custom-service',
    CICD_TEMPLATE_NAME: 'custom-template',
    JOB_PARAMS: [
        [ name: 'TAG', string: 'latest', description: 'Deployment tag' ],
        [ name: 'ENV', choices: ['dev', 'prod'], description: 'Environment' ]
    ]
]

Generated Jobs:

{SERVICE}-pipeline: Custom pipeline job

4. Frontend/Non-K8s Jobs

Purpose: Jobs for non-Kubernetes deployments

Configuration:

[
    SERVICE: 'frontend-app',
    CICD_TEMPLATE_NAME: 'frontend-template',
    JOB_PARAMS: [ /* custom parameters */ ]
]

🚀 Advanced Features

1. Multi-Application Support

Define multiple applications in a single entry:

[
    APP_REPO: 'monorepo',
    APP_BRANCH: 'main',
    appNames: ['service-a', 'service-b', 'service-c']
]

Result: Generates separate jobs for each service

2. Environment-Specific Configuration

Override defaults per environment:

def projectConfigs = [
    "mcloud": [
        GROUP: "v3",
        ENV: "stage"
    ],
    "ncloud": [
        GROUP: "v2",
        ENV: "prod"
    ]
]

3. Dynamic Parameters

Use Groovy scripts for dynamic parameter values:

[
    name: 'ImageTag',
    choiceType: 'SINGLE_SELECT',
    arChoiceGroovy: """
        def command = "aws ecr describe-images --repository-name ${ECR_REPO} --query 'sort_by(imageDetails[?imageTags != null], &imagePushedAt)[-5:].imageTags[]' --output text"
        def aws_command_output = ["/bin/sh", "-c", "\${command}"].execute()
        def files = aws_command_output.text.tokenize().reverse()
        return files
    """
]

4. Feature Branch Support

Enable beta/testing features:

[
    APP_REPO: 'my-app',
    APP_BRANCH: 'feature/new-deployment',
    appNames: ['my-service'],
    BETA: true  // Uses template-based Jenkinsfile
]

5. Custom Jenkinsfile Path

Specify custom Jenkinsfile location:

[
    SERVICE: 'custom-service',
    DEFAULT_JENKINSFILE: 'custom/path/Jenkinsfile',
    JOB_PARAMS: [ /* parameters */ ]
]

📚 Examples

Example 1: Simple Application

def jobGenie = [
    "mcloud": [
        [
            APP_REPO: 'user-service',
            APP_BRANCH: 'develop',
            appNames: ['user-api']
        ]
    ]
]

Generated:

Job: amazon/mcloud/nonprod/deploy/v2/stage/user-api/stage-user-api-pipeline
Parameters: APP_BRANCH, OPERATION, EXTENT, STEPS

Example 2: Multi-Service Application

def jobGenie = [
    "mcloud": [
        [
            APP_REPO: 'microservices-platform',
            APP_BRANCH: 'main',
            appNames: ['api-gateway', 'auth-service', 'payment-service'],
            DOCKER_BUILD_ARGS: 'ENV,TECHTEAM'
        ]
    ]
]

Generated: Three separate pipeline jobs, one per service

Example 3: Build + Deploy Pattern

def jobGenie = [
    "mcloud": [
        [
            APP_REPO: 'complex-app',
            APP_BRANCH: 'release/v2.0',
            appNames: ['api', 'worker', 'scheduler'],
            BUILD: [
                NAME: 'complex-app'
            ],
            DOCKER_BUILD_ARGS: 'ENV,VERSION'
        ]
    ]
]

Generated:

Build job: stage-complex-app-build
Deploy jobs: stage-api-deploy, stage-worker-deploy, stage-scheduler-deploy

Example 4: Custom Template Job

def jobGenie = [
    "mcloud": [
        [
            SERVICE: 'data-pipeline',
            CICD_TEMPLATE_NAME: 'data-processing',
            DEFAULT_JENKINSFILE: 'templates/data-pipeline/Jenkinsfile',
            JOB_PARAMS: [
                [ name: 'DATA_SOURCE', string: 's3://bucket/data', description: 'Data source path' ],
                [ name: 'PROCESSING_MODE', choices: ['batch', 'stream'], description: 'Processing mode' ],
                [ name: 'DRY_RUN', bool: false, description: 'Dry run mode' ]
            ]
        ]
    ]
]

Example 5: Advanced Configuration

def jobGenie = [
    "mcloud": [
        [
            APP_REPO: 'enterprise-app',
            APP_BRANCH: 'production',
            appNames: ['main-service'],
            // Docker configuration
            DOCKERFILE_PATH: 'services/main/Dockerfile',
            DOCKER_BUILD_ARGS: 'ENV,REGION,VERSION',
            DOCKER_IMAGE_ARCH: 'linux/arm64',
            // Git configuration
            SSH_KEYS: 'default:/var/lib/jenkins/.ssh/id_rsa',
            // Build tools
            MAVEN_VERSION: 'maven3.9.7',
            JDK_VERSION: 'jdk21',
            // ArgoCD configuration
            ARGOCD_ENDPOINT: 'argocd.prod.example.com',
            ARGOCD_PROJECT: 'production-apps',
            HELM_CHART: 'enterprise-chart',
            HELM_CHART_VERSION: '2.1.0',
            // Feature flags
            CREATE_KIBANA_DATAVIEW: true,
            CREATE_AWS_SECRET: true
        ]
    ]
]

✅ Best Practices

1. Configuration Organization

✅ DO:

// Group related jobs together
def jobGenie = [
    "mcloud": [
        // All mcloud jobs here
    ],
    "ncloud": [
        // All ncloud jobs here
    ]
]

❌ DON’T:

// Don't mix teams
def jobGenie = [
    "mcloud": [ /* mcloud job */ ],
    "mcloud": [ /* another mcloud job */ ]  // Duplicate key
]

2. Variable Naming

✅ DO:

Use descriptive names
Follow naming conventions
Document custom variables

❌ DON’T:

Use ambiguous names
Mix naming styles
Create unnecessary variables

3. Default Configuration

✅ DO:

// Set sensible defaults
def defaultConfigs = [
    EKS_CLUSTER_NAME: "${PROJECT_NAME}-eks-${PROJECT_ENV}",
    CICD_TEMPLATE_BASE: 'templates/cicd',
    // ... more defaults
]

❌ DON’T:

// Don't hardcode environment-specific values in defaults
def defaultConfigs = [
    ENV: 'prod'  // Wrong - should be per-project
]

4. Job Parameters

✅ DO:

Provide clear descriptions
Use appropriate parameter types
Set sensible defaults

❌ DON’T:

Create too many parameters
Use unclear descriptions
Mix parameter types unnecessarily

5. Version Control

✅ DO:

Commit all JobGenie changes
Use feature branches
Review changes before merging

❌ DON’T:

Make changes directly in production
Skip code review
Delete old configurations without migration

🔧 Troubleshooting

Issue 1: Jobs Not Generated

Symptoms: Seed job runs but no jobs appear

Solutions:

Check createjob flag: globalConfigs.createjob = true
Verify JobGenie map structure
Check Jenkins console logs
Verify file path in seed job

Issue 2: Wrong Job Path

Symptoms: Jobs created in unexpected location

Solutions:

Check DEPLOY_HOME variable
Verify JOB_SUFFIX_MKDIR calculation
Review folder creation logic

Issue 3: Missing Parameters

Symptoms: Expected parameters not appearing

Solutions:

Verify JOB_PARAMS structure
Check parameter type (string, choices, bool)
Review pipelineGenerator function

Issue 4: Template Not Found

Symptoms: Jenkinsfile not found error

Solutions:

Verify DEFAULT_JENKINSFILE path
Check CONFIG_REPO and CONFIG_BRANCH
Ensure Jenkinsfile exists in repository

Issue 5: Variable Resolution Issues

Symptoms: Variables not resolving correctly

Solutions:

Check variable precedence
Verify variable names (case-sensitive)
Review configuration merging logic

📖 Reference

Variable Reference

Variable	Type	Required	Description
`APP_REPO`	String	Yes*	Git repository name
`APP_BRANCH`	String	Yes*	Git branch
`appNames`	List	Yes*	List of service names
`SERVICE`	String	Yes**	Service identifier
`CICD_TEMPLATE_NAME`	String	No	Pipeline template name
`DOCKER_BUILD_ARGS`	String	No	Docker build arguments
`SSH_KEYS`	String	No	SSH key path
`JOB_PARAMS`	List	No	Custom job parameters

*Required for standard application jobs **Required for custom jobs

Parameter Types

// String parameter
[ name: 'BRANCH', string: 'master', description: 'Git branch' ]

// Choice parameter
[ name: 'ENV', choices: ['dev', 'prod'], description: 'Environment' ]

// Boolean parameter
[ name: 'DRY_RUN', bool: false, description: 'Dry run mode' ]

// Dynamic choice parameter
[ name: 'ImageTag', choiceType: 'SINGLE_SELECT', arChoiceGroovy: '...' ]

📚 Additional Resources

🏠 Main Documentation - Complete Jenkins as Code framework overview
🏗️ Architecture Documentation - System architecture and design
🚀 DevOps as a Service - Self-service CI/CD platform guide
✅ Best Practices - Recommended practices and guidelines
🔧 Setup Guide - Setup and configuration guide
📝 Quick Reference - Quick reference guide

Related Topics:

Learn about the Architecture to understand how JobGenie fits in
Review Best Practices for JobGenie configuration
See DevOps as a Service for self-service workflows

Maintained by the DevOps Team

“Your friendly Jenkins job generator”

Last Updated: January 15, 2024
Version: 1.0.0

Tags: #Jenkins #JobGenie #JobDSL #CI/CD #Automation #Groovy

Related Tools & Projects

BG Deployer

Automated blue-green deployment for zero-downtime AWS releases

AWS Blue-Green

mCert

SSL certificate monitoring with Slack/email alerts & Telegram

SSL Telegram

mTracker

Real-time Linux user activity monitoring with Slack notifications

Linux Monitoring

mWatcher

Server health monitoring for CPU, memory, disk with alerting

Health Alerts

gCrypt

Git-crypt wrapper for secure file encryption & access management

Security Git-crypt

Interactive Tools

AWS VPC Designer, EKS Cost Calculator, and more utilities

Tools Calculator

External Resources

Official Documentation

AWS Documentation

Complete AWS service documentation

Community Resources

GitHub Profile

Open source projects and repositories

Professional network and updates

Dev.to

Technical articles and tutorials

Contact

Get in touch for questions or collaboration

Quick Actions

All Docs

Browse all documentation

Blog

Tools

DevOps utilities

Portfolio

My work & projects

Found this helpful?

Help us improve this documentation by sharing your feedback or suggesting improvements.

Report Issue Start Discussion

JobGenie - Complete Guide

📋 Table of Contents

🎯 Overview

Key Benefits

🧠 Core Concepts

1. JobGenie Map Structure

2. Configuration Hierarchy

3. Variable Resolution

📝 Configuration Structure

Basic Structure

Required Variables

For Standard Application Jobs

For Custom Jobs

Optional Variables

Docker Build Configuration

Git Configuration

Kubernetes/ArgoCD Configuration

Build Tools

🎨 Job Types

1. Standard Application Jobs

2. Build + Deploy Jobs

3. Custom Jobs

4. Frontend/Non-K8s Jobs

🚀 Advanced Features

1. Multi-Application Support

2. Environment-Specific Configuration

3. Dynamic Parameters

4. Feature Branch Support

5. Custom Jenkinsfile Path

📚 Examples

Example 1: Simple Application

Example 2: Multi-Service Application

Example 3: Build + Deploy Pattern

Example 4: Custom Template Job

Example 5: Advanced Configuration

✅ Best Practices

1. Configuration Organization

2. Variable Naming

3. Default Configuration

4. Job Parameters

5. Version Control

🔧 Troubleshooting

Issue 1: Jobs Not Generated

Issue 2: Wrong Job Path

Issue 3: Missing Parameters

Issue 4: Template Not Found

Issue 5: Variable Resolution Issues

📖 Reference

Variable Reference

Parameter Types

📚 Additional Resources

📖 Related Documentation

🧭 Navigation

Related Documentation

More from Tools

BG Deployer

mCert

Related by Tags

Related Blog Posts

OpenResty Production Setup: Supercharge with Lua-Based Metrics and Monitoring

KEDA on EKS: Complete Guide to Event-Driven Autoscaling with Real-World Examples

AIOps: AI-Powered DevOps Automation and Intelligent Operations

Related Tools & Projects

BG Deployer

mCert

mTracker

mWatcher

gCrypt

Interactive Tools

External Resources

Official Documentation

AWS Documentation

Community Resources

GitHub Profile

LinkedIn

Dev.to

Contact

Quick Actions

All Docs

Blog