Self hosting build agents

Intro

Build agents are important to any continuous integration and delivery, they build, test and deploy your code. You usually have an option between hosted and shared agents. I like to use both shared and self-hosted agents.

Shared agents are great because they require mininal setup and you can offload work somewhere else. They come with most git repository hosts.

Self-hosted agents offer unlimited parallel builds and your are not limited to an allowance of build minutes. You can also use hardware ranging from Rapspberry Pis, laptops or another server anywhere in the world.

When creating a project, one of the major decisions at the start should be “What should I use to build and test my code when I push my code?”. Sometimes the answer may be that you don’t need CI or CD. I usually go through the following process to decide how to host my build agents.

Decision tree

start do we need a lot of builds? is the project open sourcE?

Swings and roundabouts

Self-hosted agents are great and they solve certain problems. It would only make sense to use them for certain scenarios. In my case, I use my Desktop or Raspberry Pi to build with certain hardware and I also use them to save my minutes’ allowances which means I can run as many builds as I want.

I still use shared agents for small projects and sometimes a mix. My pipelines will try and use a self-hosted agent if one is available, if one is not found then a shared one will be used. This is great if you make really quick edits and just want your CI to just get on with building your code.

Sounds cool, how do I set these up?

I use Github, Gitlab and Circle CI for most of my projects. I’m also a fan of hosting in Docker and all three have docker images to host.

With most of these self-hosted runners, you will need a token so that the runner can access your repository and somewhere to host them which is usually the machine you are developing from or some hardware elsewhere. I have used runners on my local desktop, a paid server and even a Raspberry Pi.

You would typically:

  • Set up your access tokens. The permissions will depend on the agent you’re trying to use.
  • Create an agent. You need something to tell your agent what to do. You will also pick labels that your agent will listen out for.
  • Get a copy of the repo or image. You might have to build the image yourself depending on where you get it from.
  • Run the docker image. You’ll need to make sure that any environment variables are set with docker run or in your docker compose file
  • Start a build. Then wait to see if your runner picks up a “job”

I won’t be going too much into detail on setting these agents up because at any point they could change or there may be changes to any third-party runners (mainly GitHub).

Building on other architectures

Sometimes you might want to build for specific architectures such as Raspberry pi and the new Apple M1 chips. Both of these use the linux/arm64 image architectures so if you happen to have either hardware, then it might be a good idea to also have a build agent on those too.

GitHub build agent

GitHub does have a community docker image to run this agent and I have yet to find one maintained by GitHub. So for now I use one created by the community.

I found a bunch of projects here: https://github.com/search?q=actions+runner+docker

GitLab build agent

GitLab does have a docker image that they maintain. This is great because it also comes with solid documentation from GitLab. There were a couple of hoops I had to jump through to get this working, mainly around labels.

There are instructions here on setting up your agent: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html

Azure Devops build agent

Azure Devops provides a way to host this locally (even with systemd if you want) or you can use Docker. The docker route I think is probably easier as there are fewer setups and dependencies to worry about.

Instructions come in two parts:

  • Agent setup and tokens https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=azure-devops
  • Building your agent container from scratch (including scripts to build) https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops

Closing

Self-hosted agents are great for many reasons. Setting them up doesn’t take too long and if you’re not that bothered, you can just remove the agent and point your CI build config back to shared agents. If you have time I’d give it a go.