Davide Valentino's blog
Level Up Your CI/CD: Self-Hosted GitHub Actions Runners on Kubernetes with Helm

Level Up Your CI/CD: Self-Hosted GitHub Actions Runners on Kubernetes with Helm

Davide Valentino's photo
Davide Valentino
·

4 min read

Introduction

GitHub Actions are fantastic for automating software workflows. However, sometimes you need more control over your environment – perhaps specific hardware, software, or network access. That's where self-hosted runners come in! This tutorial guides you through setting up these runners within your Kubernetes cluster using Helm, simplifying the management process.

Prerequisites

  • A Kubernetes cluster (Minikube, Kind, or a cloud provider's offering like AKS, EKS, or GKE).

  • Helm installed and configured to connect to your cluster.

  • Basic familiarity with Kubernetes concepts like namespaces, deployments, and secrets.

  • A GitHub account with a repository you want to use with your runners.

TL;DR

Want GitHub Actions with more control? This tutorial shows you how to use Helm to deploy self-hosted runners inside your Kubernetes cluster. You'll create secrets, install the runner controller and scale sets, and configure your GitHub workflow to use the runners.

Steps

Namespaces help organize your Kubernetes resources. Let's create one for the Runner Controller and one for our runners.

kubectl create namespace arc-systems
kubectl create namespace arc-runners

2. Create GitHub Token Secret

The runner needs to authenticate with GitHub. We'll use a Personal Access Token (PAT). It is advisable to use GitHub App instead of PAT.

  • Go to your GitHub account settings -> Developer settings -> Personal access tokens -> Generate new token.

  • Give the token a descriptive name (e.g., "k8s-runner-token").

  • Crucially, grant the repo scope (this gives the runner permission to listen for workflow triggers in your repositories).

  • Copy the generated token. Store it securely!

Note that for organisation runners, you need to grant the admin:org scope.

Now, create a Kubernetes secret to store the token:

NAMESPACE="arc-systems"
GITHUB_TOKEN="your-secret" # Replace with your actual token
kubectl create secret generic gha-runners-secret \
   --namespace=${NAMESPACE} \
   --from-literal=github_token="${GITHUB_TOKEN}"

3. Install the Runner Controller

The runner controller manages the lifecycle of your runners. We'll install it using Helm:

NAMESPACE="arc-systems"
helm install arc \
    --namespace "${NAMESPACE}" \
    --create-namespace \
    --set githubConfigSecret=gha-runners-secret \ # This is the secret we created earlier
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

4. Install the Runner Scale Set

This is where we define the runners themselves. A "scale set" lets Kubernetes automatically manage the number of runners based on demand.

INSTALLATION_NAME="arc-runner-set" # you MUST use "runs-on: arc-runner-set" in your workflow
NAMESPACE="arc-runners"
GITHUB_CONFIG_URL="https://github.com/my-github-account/my-repository" # Replace with your GitHub organization or repository URL
GITHUB_PAT=$(kubectl get secret gha-runners-secret -n arc-systems -o jsonpath="{.data.github_token}" | base64 --decode)
helm install "${INSTALLATION_NAME}" \
    --namespace "${NAMESPACE}" \
    --create-namespace \
    --set containerMode.type="dind" \ # This is the Docker In Docker mode, so that you can use docker commands in your workflow
    --set githubConfigUrl="${GITHUB_CONFIG_URL}" \
    --set githubConfigSecret.github_token="${GITHUB_PAT}" \
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

  • INSTALLATION_NAME: A name for your runner set. This is important, as you'll reference it in your GitHub workflow. The example above uses arc-runner-set.

  • GITHUB_CONFIG_URL: This should be the URL of your GitHub repository or organization. For a repository, it looks like https://github.com/your-github-account/your-repo. For an organization, it's like https://github.com/your-org. The runner will register itself with this entity.

  • GITHUB_PAT: We're retrieving the token from the Kubernetes secret we created earlier.

5. Configure Your GitHub Workflow

Now, tell your GitHub workflow to use the self-hosted runner. In your workflow file (.github/workflows/your-workflow.yml), add the runs-on key:

jobs:
  build:
    name: Build and Test
    runs-on: arc-runner-set # Replace with your INSTALLATION_NAME
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Run tests
        run: echo "Running tests on self-hosted runner!"
      - name: Test docker command in runner
        run: docker --version

Replace arc-runner-set with the INSTALLATION_NAME you chose during the Helm installation.

Explanation

  • Helm Charts: Helm simplifies deploying and managing applications on Kubernetes. Charts are like packages containing all the necessary Kubernetes resource definitions.

  • Runner Controller: This controller watches for workflow jobs assigned to your repository/organization and scales the runners accordingly.

  • Runner Scale Set: Defines the configuration for the runners.

  • runs-on: This key in your workflow tells GitHub Actions to use a runner with the specified label (our INSTALLATION_NAME).

Going Further

  • GitHub App authentication: Use GitHub App instead of PAT as it is the recommended way for authentication.

  • Scoped deployment: Configure the workload to run on specific nodes, using affinity rules. Configure this in the template.spec.affinity section while using -f arc-set.yaml in the Helm installation.

  • Scaling Strategies: Explore different scaling strategies to optimize runner utilization and cost.

This should give you a solid foundation for setting up self-hosted GitHub Actions runners in Kubernetes. Good luck!

KubernetesGitHub ActionsGitHub workflowsHelmHelmCharts