Kubernetes Workloads — Jobs, CronJobs, DaemonSets, and StatefulSets

July 19, 2025 · 7 min read

DevOps & Cloud Learning Hub

Deployments are the workhorse of Kubernetes, but not every workload is a long-running web server. You need to run a database migration once, process a queue of images every night, collect logs from every node, or deploy a database cluster with stable identities. Kubernetes has a dedicated workload type for each of these patterns.

Workload Selection Guide

Before diving into each type, here is when to use what:

Workload Type	Pattern	Example
Deployment	Stateless, long-running, replaceable	Web servers, APIs, microservices
Job	Run-to-completion, one-time task	Database migration, data export, ML training
CronJob	Scheduled recurring task	Nightly reports, log cleanup, backups
DaemonSet	One pod per node	Log collectors, monitoring agents, CNI plugins
StatefulSet	Stateful, ordered, stable identity	Databases, Kafka, ZooKeeper, Elasticsearch

Jobs — Run to Completion

A Job creates one or more pods and ensures they run to successful completion. Unlike Deployments, Jobs are not meant to run forever — they finish and stop.

Basic Job

apiVersion: batch/v1
kind: Job
metadata:
  name: db-migration
  namespace: production
spec:
  backoffLimit: 3              # Retry up to 3 times on failure
  activeDeadlineSeconds: 600   # Kill the job after 10 minutes
  ttlSecondsAfterFinished: 300 # Auto-delete 5 minutes after completion
  template:
    spec:
      restartPolicy: Never     # Jobs require Never or OnFailure
      containers:
      - name: migrate
        image: my-app:1.0
        command: ["python", "manage.py", "migrate"]
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-secret
              key: url

# Run and monitor
kubectl apply -f db-migration.yaml
kubectl get jobs
# NAME           COMPLETIONS   DURATION   AGE
# db-migration   1/1           45s        2m

# Check pod logs
kubectl logs job/db-migration

# List pods created by the job
kubectl get pods -l job-name=db-migration

Parallel Jobs

Process multiple items concurrently with completions and parallelism:

apiVersion: batch/v1
kind: Job
metadata:
  name: image-processor
spec:
  completions: 10        # Total number of pods that must complete
  parallelism: 3         # Run 3 pods at a time
  backoffLimit: 5
  template:
    spec:
      restartPolicy: OnFailure
      containers:
      - name: processor
        image: image-processor:1.0
        command: ["./process-batch.sh"]

# Watch parallel execution
kubectl get pods -l job-name=image-processor -w
# NAME                     READY   STATUS      RESTARTS   AGE
# image-processor-abc12    1/1     Running     0          10s
# image-processor-def34    1/1     Running     0          10s
# image-processor-ghi56    1/1     Running     0          10s
# image-processor-abc12    0/1     Completed   0          35s
# image-processor-jkl78    1/1     Running     0          2s    # Next one starts

Indexed Jobs (Kubernetes 1.21+)

Each pod gets an index, useful for processing partitioned data:

apiVersion: batch/v1
kind: Job
metadata:
  name: indexed-processor
spec:
  completions: 5
  parallelism: 5
  completionMode: Indexed    # Each pod gets JOB_COMPLETION_INDEX env var
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: worker
        image: worker:1.0
        command: ["./process.sh"]
        # JOB_COMPLETION_INDEX is automatically set (0, 1, 2, 3, 4)

CronJobs — Scheduled Tasks

CronJobs create Jobs on a repeating schedule using standard cron syntax.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup
  namespace: production
spec:
  schedule: "0 2 * * *"              # Every day at 2:00 AM UTC
  timeZone: "America/New_York"       # Timezone support (K8s 1.27+)
  concurrencyPolicy: Forbid          # Do not start new if previous is still running
  successfulJobsHistoryLimit: 3      # Keep last 3 successful job records
  failedJobsHistoryLimit: 3          # Keep last 3 failed job records
  startingDeadlineSeconds: 300       # Skip if more than 5 minutes late
  suspend: false                     # Set to true to pause scheduling
  jobTemplate:
    spec:
      backoffLimit: 2
      activeDeadlineSeconds: 3600    # Kill if running longer than 1 hour
      template:
        spec:
          restartPolicy: OnFailure
          containers:
          - name: backup
            image: backup-tool:1.0
            command: ["./backup.sh"]
            volumeMounts:
            - name: backup-storage
              mountPath: /backups
          volumes:
          - name: backup-storage
            persistentVolumeClaim:
              claimName: backup-pvc

Cron Schedule Reference

Expression	Schedule
`/5 * * *`	Every 5 minutes
`0 * * * *`	Every hour
`0 2 * * *`	Daily at 2 AM
`0 0 * * 0`	Every Sunday at midnight
`0 0 1 * *`	First day of every month

Concurrency Policies

Policy	Behavior
Allow (default)	Multiple jobs can run simultaneously
Forbid	Skip new job if previous is still running
Replace	Cancel running job and start new one

# List CronJobs and their schedules
kubectl get cronjobs -n production
# NAME             SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE
# nightly-backup   0 2 * * *     False     0        6h

# Manually trigger a CronJob (create a Job from it)
kubectl create job manual-backup --from=cronjob/nightly-backup -n production

# Suspend a CronJob temporarily
kubectl patch cronjob nightly-backup -n production -p '{"spec":{"suspend":true}}'

DaemonSets — One Pod Per Node

A DaemonSet ensures that a copy of a pod runs on every node (or a subset of nodes). When nodes are added to the cluster, DaemonSet pods are automatically scheduled on them. When nodes are removed, the pods are garbage collected.

Log Collector DaemonSet

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentbit
  namespace: logging
spec:
  selector:
    matchLabels:
      app: fluentbit
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1       # Update one node at a time
  template:
    metadata:
      labels:
        app: fluentbit
    spec:
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        effect: NoSchedule    # Run on control plane nodes too
      containers:
      - name: fluentbit
        image: fluent/fluent-bit:2.2
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "256Mi"
        volumeMounts:
        - name: varlog
          mountPath: /var/log
          readOnly: true
        - name: containers
          mountPath: /var/lib/docker/containers
          readOnly: true
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: containers
        hostPath:
          path: /var/lib/docker/containers

Running on Specific Nodes Only

Use nodeSelector or affinity to target a subset of nodes:

spec:
  template:
    spec:
      nodeSelector:
        node-type: gpu           # Only GPU nodes
      # OR use affinity for more complex rules:
      # affinity:
      #   nodeAffinity:
      #     requiredDuringSchedulingIgnoredDuringExecution:
      #       nodeSelectorTerms:
      #       - matchExpressions:
      #         - key: kubernetes.io/os
      #           operator: In
      #           values: ["linux"]

# Check DaemonSet status
kubectl get daemonset -n logging
# NAME        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR
# fluentbit   5         5         5       5            5           <none>

# Verify it runs on every node
kubectl get pods -n logging -o wide -l app=fluentbit
# NAME              READY   STATUS    NODE
# fluentbit-abc12   1/1     Running   node-1
# fluentbit-def34   1/1     Running   node-2
# fluentbit-ghi56   1/1     Running   node-3

StatefulSets — Stable Identity and Ordered Deployment

StatefulSets are designed for applications that need one or more of:

Stable network identity — each pod gets a predictable hostname (pod-0, pod-1, pod-2)
Stable persistent storage — each pod gets its own PVC that follows it across rescheduling
Ordered deployment and scaling — pods are created in order (0, 1, 2) and deleted in reverse (2, 1, 0)

PostgreSQL StatefulSet

apiVersion: v1
kind: Service
metadata:
  name: postgres-headless
  namespace: production
spec:
  clusterIP: None             # Headless service — required for StatefulSets
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: production
spec:
  serviceName: postgres-headless    # Must match the headless service
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: pg-secret
              key: password
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
        resources:
          requests:
            cpu: "500m"
            memory: "1Gi"
          limits:
            cpu: "2"
            memory: "4Gi"
  volumeClaimTemplates:            # Each pod gets its own PVC
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: fast-ssd
      resources:
        requests:
          storage: 50Gi

# StatefulSet pods get predictable names
kubectl get pods -n production -l app=postgres
# NAME         READY   STATUS    AGE
# postgres-0   1/1     Running   5m    # Created first
# postgres-1   1/1     Running   4m    # Created second
# postgres-2   1/1     Running   3m    # Created third

# Each pod has its own PVC
kubectl get pvc -n production
# NAME              STATUS   VOLUME         CAPACITY
# data-postgres-0   Bound    pvc-abc123     50Gi
# data-postgres-1   Bound    pvc-def456     50Gi
# data-postgres-2   Bound    pvc-ghi789     50Gi

# DNS resolution: each pod has a stable DNS name
# postgres-0.postgres-headless.production.svc.cluster.local
# postgres-1.postgres-headless.production.svc.cluster.local
# postgres-2.postgres-headless.production.svc.cluster.local

# If postgres-1 is rescheduled, it keeps the same name and PVC
kubectl delete pod postgres-1 -n production
kubectl get pods -n production -l app=postgres -w
# postgres-1 recreated with the same PVC attached

StatefulSet Update Strategies

Strategy	Behavior
RollingUpdate (default)	Updates pods in reverse order (N-1 → 0), one at a time
OnDelete	Only updates a pod when you manually delete it
partition	Only updates pods with index >= partition value (canary rollouts)

updateStrategy:
  type: RollingUpdate
  rollingUpdate:
    partition: 2              # Only update pod-2 and above, leave 0 and 1 alone

Next, we will cover Horizontal and Vertical Pod Autoscalers — how to automatically scale your workloads based on CPU, memory, and custom metrics to handle traffic spikes without manual intervention.

Workload Selection Guide​

Jobs — Run to Completion​

Basic Job​

Parallel Jobs​

Indexed Jobs (Kubernetes 1.21+)​

CronJobs — Scheduled Tasks​

Cron Schedule Reference​

Concurrency Policies​

DaemonSets — One Pod Per Node​

Log Collector DaemonSet​

Running on Specific Nodes Only​

StatefulSets — Stable Identity and Ordered Deployment​

PostgreSQL StatefulSet​

StatefulSet Update Strategies​

Stay Updated