Designing Data-Intensive Applications

by Martin Kleppmann

Per-project Postgres

In this post, Jamey Sharp elaborates on a neat technique to run different versions of postgres on a per-project basis. I learned that you can run postgres on a Unix socket only, without having a port open, which removes the need to manage those ports for each version of postgres. The technique also has the advantage of keeping the data for the project, inside the project directory structure. It illustrates the power and flexibility of Unix tools.

How to do distributed locking

Martin Kleppmann writes about distributed locks in general, and in particular the merits of Redlock, a Redis-based distributed-lock algorithm. Kleppmann breaks down the reasons to use a distributed lock, it’s characteristics, and how Redlock in particular is vulnerable to timing attacks. I found this to be great technical writing. The post came about when Kleppmann was researching his book, Desiging Data-Intensive Applications. I finished that book a few days ago, and hope to write a review soon. I can recommend it enough.

We Can Do Better Than SQL

Simple SQL statements can read almost like English. With just a bit of complexity (e.g. more than one join) they quickly can become almost impossible to dicern. In this post Elvis Pranskevichus critiques SQL’s shortcomings compellingly. He then introduces EdgeQL, a query language designed to fix SQLs shortcomings. This is the first time I’ve heard of it or EdgeDB.

Is High Quality Software Worth the Cost?

With his traditional knack for analysis and synthesis, Martin Fowler describes how the familiar trade-off of quality and cost that is intuitive in the physical world doesn’t quite hold for software. Software projects are constantly evolving, requirements changing. Internal quality determines that speed at which features can be delivered. Disregarding internal quality leads to software projects where it becomes almost impossible to continue making changes. I can’t recommend this article enough.

Recently, I chased down a segmentation fault occurring in one of our production servers. A segmentation fault cannot be triggered by code is that written completely in Ruby, barring a bug in Ruby itself. The VM manages the memory, making it impossible to access memory in violation of the OS rules.

Last week I had the pleasure of attending RailsConf 2019 for the first time, and the honor to be a speaker. My talk was titled “Bug-Driven Development”. On the surface it’s ostensibly a war story about fixing a particularly nasty bug. At a deeper level, it is about software design evolution. Software is an iterative endeavor, perpetually in a state of flux. Requirements change, new features are added, external APIs are deprecated, scaling demands adjustments. In the talk, I try to thread the needle between a specific bug fix and the broader applicability of design patterns, proper abstractions, and the role of testing. My goal was for audience members to be able to see both the proverbial forest and the trees: to connect the ivory-tower, abstract design concepts with the day-to-day practice of writing code, test-driven development, and fixing bugs.