Another Blocker

Expanding the main page

Recently I've had a bunch of complications that seriously limited time I could spend on working on side projects. Extra work, family stuff, maintenance work like getting car repaired, etc. But I've also had a lot of interesting work I've been doing at Buffer.

The thing is, when I structured eng.adnanissadeen.com, I expected everything to be a project. And under each project there'd be an introductory blog post, a bunch of logs, and then possibly a follow up blog post at the end of it. What I didn't think of is the daily work I do at Buffer that can't be classified under a project but is arguably really interesting to just log as I go along.

For example, the other day I wanted to speed up builds of services at Buffer. In particular, there was a set of services we deploy to that utilise the same Docker image in production. The problem was that when running @bufferbot servicedeploy <service-name> it would trigger a fresh build for each service in Jenkins. This was redundant. The image had already been built. And it was made worse by the fact that the builds for this particular service are long and the repo size is big.

In order to reduce the build time I combined Github's API with Dockerhub's undocumented-ish API and skipped the build step if a particular image already existed in Dockerhub. The way this works is that we have an a Dockerhub image, let's call it buffer-service. And we tag that image with the git hash during build time. So if I build the master branch, and the latest git tag is 502ae3ne..... then the final output will be buffer-service:502ae3ne....... And that gets pushed to Dockerhub.

Using the Github API, I was able to pull the last commit SHA for a given branch. I then compared the SHA to which tags were available for the Dockerhub image and queried to see if the image:tag combination existed. If it did, I skipped the Jenkins build step and deployed the image directly to the cluster. If it didn't I send the request to Jenkins to build and push the image to Dockerhub.

This was a really interesting task. Not a project though. But from writing code to interact with the Github API to finding out how to authenticate and interact with the Dockerhub API, there was a lot of stuff to share in there.

But the process of setting up a project for a single log is too heavy. And as such I've skipped it for many such tasks recently.

In order to work around this, I'm going to be changing the design of the main site slightly to include "latest unfiled logs" or something of that sort which will just be a list of links at the top.

Onwards!

Another Blocker
Share this