Push, Cache, Repeat: Amazon ECR as a remote Docker cache for GitHub Actions
Apr 15, 2024
Aditya Jayaprakash
This is a follow-up to our previous blog post on Docker layer caching for GitHub Actions. In the previous post, we used a registry cache to store cached artifacts in a docker registry. We recently had a call with a customer where we helped them set up a remote cache with AWS ECR, and we decided to share our learnings.
A Docker image is composed of various layers. During a Docker build, each layer is built one after the other. If a layer doesn’t change between builds, having a cache helps retrieve an already built and unchanged layer - this can drastically speed up your build.
Ideally, you want to store these cached layers in your runner building the Docker image. However, due to the ephemeral nature of these runners, this is not always possible when you’re using GitHub-hosted runners.
A way to overcome this limitation is to store these cached layers in AWS ECR, separate from the built and pushed image.
First, let’s look at how to do it plain and simple without the remote cache.
We are doing the usual steps of logging in to ECR and building and pushing the image to ECR using the docker/build-push-action@v5
action.
The remote cache feature is not supported by default in Docker and requires you to use Docker Buildx
, an advanced Docker build tool. You must set it up using docker/setup-buildx-action@v3
and create a builder instance.
Once you’ve set it up, you need to populate the cache-to
and cache-from
parameters to the build command like this, as showcased in the official documentation from AWS on this feature.
The cache-from
specifies the source to use as a cache when building your Docker image, and the cache-to
specifies the destination for storing the build cache once a Docker image has been built.
This is the final workflow file for building and pushing with a remote cache in ECR once we convert it to the format that docker/build-push-action@v5
expects.
Performance Improvements
Workflow without the remote cache
Workflow with the remote cache rebuilding the same Docker file
At Blacksmith, we’re always trying to optimize customer interactions with Docker - be it through local Docker registry mirrors or out-of-the-box Docker layer caching (coming soon!). You should give us a shot if you’re not already a customer. We can run your GitHub Actions twice as fast while reducing your spending by 50-75%.
Notes
The above code block was for pushing to a private ECR. If you wish to push to a public registry, you should change the
ref
in thecache-to
andcache-from
topublic.ecr.aws/${{ secrets.AWS_ACCOUNT_ID }}/${{ env.ECR_REPOSITORY }}:cache
instead of${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.ECR_REPOSITORY }}:cache
If you’re getting a 403 forbidden, it might be that your IAM does not have permissions for these ECR operations. You can solve this by attaching these policies to your IAM user.
ecr:GetAuthorizationToken
ecr:BatchCheckLayerAvailability
ecr:GetDownloadUrlForLayer
ecr:BatchGetImage
ecr:PutImage
ecr:InitiateLayerUpload
ecr: UploadLayerPart
ecr:CompleteLayerUpload