Git from the inside out

In this essay, Mary Rose Cook explains how git works, focusing on the graph structure that underpins it and governs its behavior. git is a powerful piece of software, and digging into how it’s designed thought me a lot.

Code less, Engineer more

Liz Fong-Jones writes about how effective software engineering teams become more effective: Writing less software. The premise is that the focus should be on the impact of the work, not how much code we write. Build what you must, buy what you can, and write it all down. Custom software should be a last, rather than first, resort.

A lot of this article resonates with me, especially the part about writing everything down. I’ve come to believe that documenting how and why you made a decision is as important as the decision itself. There will come a time when another engineer will ask, “Did you consider X or Y?”. The decision record will answer that question, and eliminate much bike-shedding.

Your Makefiles are wrong

Jacob Davis-Hansson shares his strong opinions on writing Makefiles. make was obscure to me for a long time. Its not common knowledge among rubyists, but I’ve come to appreciate it more as a generic build system, especially for file-oriented tasks. In this post, Jacob explains his preferred defaults. I found the use of sentinel files particularly useful.

The Language Agnostic, All-Purpose, Incredible, Makefile

This post is a great introduction to make, one of the most versatile unix programmer tools, that has lost favor in recent years. I personally use make in some of my personal projects, but have yet to take advantage of it in large Ruby projects at work.

How containers work: overlayfs

Julia Evans explains how overlayfs – a union filesystem – powers containers and makes it much more efficient to build images from other images. Great read.

I work on a large engineering team. The main Slack channel for our engineering department has 425 people in it. The code base is split into many repositories, dominated by a big Ruby on Rails application with many contributors. At the time of writing, the last 1,000 commits in master on that repository where made by 169 contributors.

The continuous integration for said mono-repo is heavily parallelized but still takes ~30 minutes to complete. Occasionally, a branch is merged that causes the build to fail. Usually, the case is that the specs worked correctly for that branch (otherwise we can’t merge), but new changes in master are not compatible. As hard as the team tries to maintain a green build (i.e. a build that passes and is deployable), a red build is somewhat frequent.

The Night Watch

In this article James Mickens writes about being a systems programmer. The writing is witty and funny. It’s not new, but it is new to me. A few choice quotes:

One time I tried to create a list<map>, and my syntax errors caused the dead to walk among the living. Such things are clearly unfortunate.

…

Indeed, the common discovery mode for an impossibly large buffer error is that your program seems to be working fine, and then it tries to display a string that should say “Hello world,” but instead it prints “#a[5]:3!” or another syntactically correct Perl script

…

However, when HCI people debug their code, it’s like an art show or a meeting of the United Nations. There are tea breaks and witticisms exchanged in French; wearing a non-functional scarf is optional, but encouraged.

…

Do you see the difference between our lives? When you asked a girl to the prom, you discovered that her father was a cop. When I asked a girl to the prom, I DISCOVERED THAT HER FATHER WAS STALIN.

Empathy is a Technical Skill

Andrea Goulet writes an interesting article about empathy. The takeaway is that technical-minded folks should think of empathy as a skill that can be learned, and used effectively to achieve your aims. From experience, I can attest that increasing your empathy is like having a super power.

pg_flame

This project looks really promising. It formats the output of Postgres EXPLAIN ANALYZE as a flame graph, which can help in figuring out which parts of your queries are worth digging into.

Building A Relational Database Using Kafka

Robert Yokota explores building a relational database on top of Kafka. It follows his previous article on creating an in-memory cache on backed by Kafka. RDBM systems are commonly thought of keeping track of tables and rows. The semantics of SQL reinforce the concept of rows being updatable. In practice though, most implementation use an immutable log under the hood. That is what makes transactions possible, each with its own consistent view of the world. Kafka can be thought of as an “exposed” MVCC system, and the current state of the data can be derived by consuming the messages in a topic. The article is interesting in that it assembles a relation database by using different existing open-source projects.

3 Key Ideas Behind The Erlang Thesis

Yiming Chen summarizes Joe Armstrong’s thesis: “Making reliable distributed systems in the presence of software errors”. The 3 key ideas identified: Concurrency oriented programming, abstracting concurrency, and let-it-fail philosophy. Armstrong is Erlang’s creator, and his thesis has been very influential in the Erlang and Elixir communities.