Multiarch docker GitHub actions (#2065)

## Issue Addressed

Resolves #1512

## Proposed Changes

- Adds a new docker Github Actions workflow  
- Removes the Dockerhub hook
- Adds a new Dockerfile for use with pre-existing cross-compiled binaries 
- on pushes to `unstable` 
  - builds an ARM64 image and tags it `latest-arm64-unstable`
  - builds an AMD64 image and tags it `latest-amd64-unstable`
  - builds an multiarch image by creating a manifest list referencing the prior two images and tags it `latest-unstable`
- on pushes to `stable` 
  - builds an ARM64 image and tags it `latest-arm64`
  - builds an AMD64 image and tags it `latest-amd64`
  - builds an multiarch image by creating a manifest list referencing the prior two images and tags it `latest`

## Additional Info
- for ARM64, first `cross` is used to cross compile the `lighthouse` and  `lcli` binaries, then `docker buildx` is installed to actually build the docker image for the correct target platform. The image build pretty much just copies the binaries from local into the docker image (thanks @michaelsproul :) )
- The AMD64 and ARM64 builds run in parallel, in total it's been taking around 45mins on a local runner
- This PR does **not** cover version tags on docker images at the moment

Co-authored-by: realbigsean <seananderson33@gmail.com>
This commit is contained in:
realbigsean 2020-12-09 06:06:37 +00:00
parent aa45fa3ff7
commit adbd49ddc6
3 changed files with 113 additions and 4 deletions

103
.github/workflows/docker.yml vendored Normal file
View File

@ -0,0 +1,103 @@
name: docker
on:
push:
branches:
- unstable
- stable
env:
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
IMAGE_NAME: sigp/lighthouse
jobs:
extract-branch-name:
runs-on: ubuntu-latest
steps:
- name: Extract branch name
run: echo "::set-output name=BRANCH_NAME::$(echo ${GITHUB_REF#refs/heads/})"
id: extract_branch
outputs:
BRANCH_NAME: ${{ steps.extract_branch.outputs.BRANCH_NAME }}
build-docker-arm64:
runs-on: ubuntu-latest
needs: [extract-branch-name]
steps:
- uses: actions/checkout@v2
- name: Dockerhub login
run: |
echo "${DOCKER_PASSWORD}" | docker login --username ${DOCKER_USERNAME} --password-stdin
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Cross build lighthouse binary
uses: actions-rs/cargo@v1
with:
use-cross: true
command: build
args: --release --manifest-path lighthouse/Cargo.toml --target aarch64-unknown-linux-gnu --features portable
- name: Move cross-built ARM binary into Docker scope
run: |
mkdir ./bin;
mv ./target/aarch64-unknown-linux-gnu/release/lighthouse ./bin;
- name: Build Docker Buildx
run: |
export DOCKER_BUILDKIT=1;
docker build --platform=local -o . git://github.com/docker/buildx;
mkdir -p ~/.docker/cli-plugins;
mv buildx ~/.docker/cli-plugins/docker-buildx;
- name: Create Docker Builder
run: |
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes;
docker context create builder;
docker buildx create --use --name lighthouse builder;
- name: Set Env
if: needs.extract-branch-name.outputs.BRANCH_NAME == 'unstable'
run: |
echo "TAG_SUFFIX=-unstable" >> $GITHUB_ENV;
- name: Build ARM64 dockerfile (with push)
run: |
docker buildx build \
--platform=linux/arm64 \
--output "type=image,push=true" \
--file ./Dockerfile.cross . \
--tag ${IMAGE_NAME}:latest-arm64${TAG_SUFFIX}
build-docker-amd64:
runs-on: ubuntu-latest
needs: [extract-branch-name]
steps:
- uses: actions/checkout@v2
- name: Dockerhub login
run: |
echo "${DOCKER_PASSWORD}" | docker login --username ${DOCKER_USERNAME} --password-stdin
- name: Set Env
if: needs.extract-branch-name.outputs.BRANCH_NAME == 'unstable'
run: |
echo "TAG_SUFFIX=-unstable" >> $GITHUB_ENV;
- name: Build AMD64 dockerfile (with push)
run: |
docker build \
--build-arg PORTABLE=true \
--tag ${IMAGE_NAME}:latest-amd64${TAG_SUFFIX} \
--file ./Dockerfile .
docker push ${IMAGE_NAME}:latest-amd64${TAG_SUFFIX}
build-docker-multiarch:
runs-on: ubuntu-latest
needs: [build-docker-arm64, build-docker-amd64, extract-branch-name]
steps:
- name: Dockerhub login
run: |
echo "${DOCKER_PASSWORD}" | docker login --username ${DOCKER_USERNAME} --password-stdin
- name: Set Env
if: needs.extract-branch-name.outputs.BRANCH_NAME == 'unstable'
run: |
echo "TAG_SUFFIX=-unstable" >> $GITHUB_ENV;
- name: Create and push multiarch manifest
# We need to enable experimental docker features in order to use `docker manifest`
run: |
export DOCKER_CLI_EXPERIMENTAL=enabled;
docker manifest create ${IMAGE_NAME}:latest${TAG_SUFFIX} \
--amend ${IMAGE_NAME}:latest-arm64${TAG_SUFFIX} \
--amend ${IMAGE_NAME}:latest-amd64${TAG_SUFFIX};
docker manifest push ${IMAGE_NAME}:latest${TAG_SUFFIX}

10
Dockerfile.cross Normal file
View File

@ -0,0 +1,10 @@
# This image is meant to enable cross-architecture builds.
# It assumes the lighthouse binary has already been
# compiled for `$TARGETPLATFORM` and moved to `./bin`.
FROM --platform=$TARGETPLATFORM debian:buster-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
ca-certificates \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY ./bin/lighthouse /usr/local/bin/lighthouse

View File

@ -1,4 +0,0 @@
#!/bin/bash
# Build hook to run on Docker Hub to ensure that the image is built with `PORTABLE=true`.
docker build --build-arg PORTABLE=true -f $DOCKERFILE_PATH -t $IMAGE_NAME .