Self-hosted runner

What is a self-hosted runner?

A self-hosted runner is a machine you own that executes CI/CD jobs on behalf of your build system. Instead of your pipeline running on a shared VM inside GitHub, GitLab or another vendor's cloud, it runs on a Linux box, a Windows server, a Kubernetes pod or a bare-metal host you provision. The vendor still schedules the job and streams the logs; your machine does the actual work.

Every major CI/CD platform ships this option under its own name: GitHub Actions self-hosted runners, GitLab Runner, Buildkite agents, CircleCI runners, Jenkins agents, Azure Pipelines self-hosted agents. The pattern is identical: install a small process, register it with the controller using a token, and it starts pulling jobs.

How does a self-hosted runner work?

The lifecycle looks the same across vendors:

1. Register. You install the runner binary on the target host and give it a registration token from your CI/CD platform. The runner opens a long-poll or WebSocket connection back to the controller and advertises its labels (linux, arm64, gpu, prod-network, …).
2. Dispatch. When a pipeline job requests a matching label, the controller assigns it to an idle runner. There is no inbound firewall hole to your host: the runner initiated the connection, so control traffic reuses it.
3. Execute. The runner checks out the commit, materialises secrets in memory or a scoped file, and runs the job's steps inside a workspace directory. Modern setups run each job inside a fresh container (or a fresh VM) so state does not leak between builds.
4. Report. Logs stream back to the controller line by line, artifacts and test results are uploaded to the platform's storage, and the exit code becomes the job's pass/fail status.
5. Reset or destroy. Persistent runners keep the workspace and warm caches for the next job. Ephemeral runners destroy the whole host or container after every run, trading cache reuse for a guaranteed-clean environment.

The interesting design choice is persistent vs. ephemeral. Persistent runners are fast because dependency caches, Docker layers and build tools stay on disk, but they drift: yesterday's job can affect today's. Ephemeral runners (often provisioned by an autoscaler like GitHub's Actions Runner Controller for Kubernetes, or a Terraform-managed pool) are slower per job but cleaner and safer, and they scale to zero when idle.

Why does it matter?

Hosted runners are the sensible default. You reach for a self-hosted runner when one of three constraints bites:

• Access. Your build needs to talk to something the internet cannot reach: an internal package registry, a database in a private VPC, on-prem hardware, a licence server. A runner inside that network makes the build possible at all.
• Hardware. You need a GPU, an ARM Mac for iOS builds, a large-memory instance, a specific CPU generation, or a Windows host with a particular SDK. Hosted fleets rarely cover the long tail.
• Cost and speed at scale. Once a team runs thousands of build minutes a day, hosted per-minute billing and cold caches add up. A pool of self-hosted runners with warm caches and reserved-instance pricing can be dramatically cheaper and faster.

The trade-off is real. You now own a piece of production-grade infrastructure: it needs patching, monitoring, capacity planning and a security review. Runners are also a favourite target: a compromised runner sits inside your network with your CI secrets. Treating them as cattle, not pets (immutable, autoscaled, short-lived) is what makes the pattern safe.

How do popular CI/CD tools handle self-hosted runners?

Every major platform supports the pattern, and the differences are in how much of the operations they take off your plate.

• GitHub Actions has the broadest ecosystem. Self-hosted runners are a first-class feature and the Actions Runner Controller (ARC) provides ephemeral, autoscaled runners on Kubernetes. The catch is safety: running self-hosted runners on public repositories is officially discouraged because a fork's pull request can execute attacker code on your machine.
• GitLab Runner is arguably the most mature standalone runner. It ships with executors for shell, Docker, Kubernetes, VirtualBox and SSH, and the same binary runs against gitlab.com or a self-managed GitLab. Autoscaling with Docker Machine or Kubernetes is well documented. The honest concession: if you want a single, vendor-blessed runner binary with pluggable executors and mature autoscaling, GitLab Runner is hard to beat, even against GitLab itself.
• Jenkins agents are the veteran of this space. Static agents, dynamic clouds (EC2, Kubernetes, Docker), Windows and macOS support: if it can run Java, it can be a Jenkins agent. The trade-off is the same as Jenkins itself: enormous flexibility, and you own every bit of the operations.
• CircleCI and Buildkite started managed but added self-hosted runners for regulated and on-prem workloads. Buildkite is unusual in defaulting to self-hosted, with the platform acting as a control plane only. Teams that want their code and secrets to never touch a vendor's compute often pick Buildkite for exactly this reason.
• Argo Workflows / Tekton live inside your Kubernetes cluster from day one, so "self-hosted runner" is not really a separate concept: every job is a pod on your cluster. If you are already all-in on Kubernetes and GitOps, these are the idiomatic choice over bolting runners onto a SaaS CI.
• Buddy is one of the recommended options when you want the SaaS control plane experience but need builds to run on your own hardware. Buddy On-Premises installs the whole platform behind your firewall, and Buddy also supports adding your own runners to the managed cloud so private-network jobs run on your host while everything else stays hosted. The .buddy/buddy.yml you write is the same in both cases, which keeps the mental model simple when only some pipelines need private access.

The honest summary: if your CI system is already GitHub or GitLab, use their runners with an autoscaler and a clean network segment. If your code cannot leave the building, an on-prem-first platform (Jenkins, Buildkite, Buddy On-Premises) is the shorter path. Pick by where the constraints actually are, not by which vendor's marketing you saw last.

Example

A concrete Buddy pipeline that runs on a self-hosted runner behind a corporate firewall, so it can reach an internal package registry and push the built artifact back out. The same YAML would run unchanged on Buddy's managed runners; the only thing that changes is which runner picks up the job.

# .buddy/buddy.yml - CI build that runs on a private, self-hosted runner
- pipeline: "ci-on-prem"
  events:
    - type: "PUSH"
      refs:
        - "refs/heads/main"
  actions:
    - action: "Install and test against the internal registry"
      type: "BUILD"
      docker_image_name: "node"
      docker_image_tag: "20"
      cached_dirs:
        - "node_modules"
      commands: |-
        npm config set registry https://npm.internal.example.com
        npm ci --prefer-offline
        npm test -- --ci

    - action: "Build and publish artifact"
      type: "BUILD"
      docker_image_name: "node"
      docker_image_tag: "20"
      cached_dirs:
        - "node_modules"
      commands: |-
        npm run build
        bdy artifact publish web-app:$BUDDY_EXECUTION_REVISION ./dist --create

    - action: "Ping deploy service to promote the artifact"
      type: "HTTP"
      method: "POST"
      notification_url: "https://deploy.internal.example.com/promote"

The pipeline definition is ordinary CI YAML. What makes it a self-hosted-runner build is where the runner process is registered: on a host inside the corporate network with access to npm.internal.example.com and deploy.internal.example.com. Nothing in the pipeline changes when you later move a subset of jobs back to managed runners; the runner is infrastructure, not application code.