Building A Relational Database Using Kafka

Robert Yokota explores building a relational database on top of Kafka. It follows his previous article on creating an in-memory cache on backed by Kafka. RDBM systems are commonly thought of keeping track of tables and rows. The semantics of SQL reinforce the concept of rows being updatable. In practice though, most implementation use an immutable log under the hood. That is what makes transactions possible, each with its own consistent view of the world. Kafka can be thought of as an “exposed” MVCC system, and the current state of the data can be derived by consuming the messages in a topic. The article is interesting in that it assembles a relation database by using different existing open-source projects.

3 Key Ideas Behind The Erlang Thesis

Yiming Chen summarizes Joe Armstrong’s thesis: “Making reliable distributed systems in the presence of software errors”. The 3 key ideas identified: Concurrency oriented programming, abstracting concurrency, and let-it-fail philosophy. Armstrong is Erlang’s creator, and his thesis has been very influential in the Erlang and Elixir communities.