5.3Writing Your First Experiment: Pod Delete

The pod-delete experiment verifies that your application survives pod termination -- the most basic resilience test. If your app cannot handle a pod being killed, it cannot handle anything.

Step 1: Define the ChaosExperiment

# chaos-experiment-pod-delete.yaml
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosExperiment
metadata:
  name: pod-delete
  namespace: litmus
spec:
  definition:
    scope: Namespaced
    permissions:
      - apiGroups: ["", "apps"]
        resources: ["pods", "deployments"]
        verbs: ["get", "list", "delete"]
    args:
      - -c
      - ./experiments -name pod-delete
    env:
      - name: TOTAL_CHAOS_DURATION
        value: "60"          # chaos lasts 60 seconds
      - name: CHAOS_INTERVAL
        value: "10"          # kill a pod every 10 seconds
      - name: FORCE
        value: "false"       # graceful termination (SIGTERM)
      - name: TARGET_PODS
        value: ""            # random pod selection
      - name: PODS_AFFECTED_PERC
        value: "50"          # kill 50% of pods

Step 2: Create the ChaosEngine

The ChaosEngine links the experiment to your target application and defines observability probes:

# chaos-engine-checkout.yaml
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: checkout-chaos
  namespace: production
spec:
  appinfo:
    appns: production
    applabel: app=checkout-service
    appkind: deployment
  engineState: active
  chaosServiceAccount: litmus-admin
  experiments:
    - name: pod-delete
      spec:
        components:
          env:
            - name: TOTAL_CHAOS_DURATION
              value: "120"
            - name: PODS_AFFECTED_PERC
              value: "50"
        probe:
          - name: "checkout-availability"
            type: httpProbe
            mode: Continuous
            httpProbe/inputs:
              url: "https://checkout.internal/health"
              method:
                get:
                  criteria: ==
                  responseCode: "200"
            runProperties:
              probeTimeout: 5s
              interval: 5s
              retry: 2
              probePollingInterval: 2s

Step 3: Apply and Monitor

# Apply the experiment and engine
kubectl apply -f chaos-experiment-pod-delete.yaml
kubectl apply -f chaos-engine-checkout.yaml

# Watch experiment progress
kubectl get chaosengine checkout-chaos -n production -w

# Check results when complete
kubectl get chaosresult checkout-chaos-pod-delete -n production -o yaml