Build and Deploy

There are several options for development which suit different workflows and depend on the task at hand.

Local development on laptop (run directly on your laptop)

docker-compose -f ./docker/local.yml up --build

Pros:: Very fast.
Cons:: Not running in realistic environment (although can still connect devices etc).

Balena local build & deploy to a single device (e.g. RPI3 on your desk)

balena push $RPI_IP_ADDRESS

Pros:: Fast.
Cons:: Problems with balena local push.

Balena remote build & deploy (e.g. all staging devices)

balena login
balena push carboncoop-hems-staging

Pros:: Deploy to actual devices.
Cons:: Slow. Uses idiosyncratic balena build servers which sometimes exhibit bad caching behaviour.

Pipeline build

A commit to the dev or main branch will trigger a pipeline build. In both cases an SD card image is produced containing the latest application version and archived on S3 for use in provisioning.

Pushes to the dev branch will goto all test bench setups. Pushes to main will goto canary installations not the whole device fleet.

Pipeline builds take about 5 minutes to be deployed to devices.

Pros:: Deploy to all staging devices.
Cons:: Slow. Automatically produces SD card images.

Continuous Integration / Deployment for the HEMS

In order to ensure that application code is tested in realistic settings and it is straightforward to maintain software running on remote devices it is useful to bring CI/CD concepts prevalent in web development to application/device development.

The Balena system greatly facilitates this by enabling application code to be distributed and managed as Docker containers.

The CI/CD for the HEMS is setup on gitlab and the configruation can be found in the usual .gitlab-ci.yml in the root of the hems repository. There are several non-standard aspects about this Gitlab CI/CD setup:

Application code needs to be built on the same or a compatible architecture as target devices.
Some stages need to run in a privileged mode to access things like loopback interfaces required to make SD images.
Docker in docker builds are quite slow so the shell executor mode has been used for the Gitlab runners.
To help reduce the size of application containers both multi-image and multi-stage builds are used.

This has led us to using our own Gitlab runners. Two are used, one for jobs needing the ARM architectures and another for amd64. These are deployed on AWS EC2 instances based on Ubuntu 18.04 LTS.

There are 3 build stages (as well as an extra step for live production deployments):

Build ARM

This builds the base/build image used in producing the runtime application image. This stage only runs if the build or dependency files are changed.

Build amd64

This does the same as the build ARM but for amd64 architecture. These images are useful in development. This stage only runs if the build or dependency files are changed.

Deploy

This does a ‘balena deploy’ using the local build image. It will build the runtime image based on the contents of docker-compose.yml in the root of the repository. It should only run on the dev branch.

SD image build

This ‘preloads’ the SD card image with the application containers so they dont need to be downloaded on first boot. These image files are uploaded to S3 for later use.