Uploaded image for project: 'camunda BPM'
  1. camunda BPM
  2. CAM-8068

Clean up orphan jenkins-slave pods in CI3 cluster

    XMLWordPrintable

    Details

      Description

      Context

      When Jenkins Master restarts in Kubernetes due to an error or manual triggered restart, the currently running build slave pods are not cleaned up and they don't reconnect to the master.

      Problem

      Running build slaves during master restart are being kept in the Kubernetes cluster. They are consuming resources and are blocking the cluster scheduler to free / shutdown the nodes they are running on.

      AT
      • Create a Kubernetes cronjob which cleans up orphaned 'jenkins-slave' pods running for >3 hours.
      • Send a notification to the SRE team if something has been cleaned.

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            lipphardt Christian Lipphardt
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: