We Will Do Anything to Get You to Work for Us Except Pay You Enough

Eli Grober writes tongue-in-cheek, but the essay is clearly funny because there is some truth behind it. Companies go to great lengths to not raise wages and raise side benefits instead, while pretending that you should be happy about it. Side benefits are cheaper, and also much easier to cut without too much protest.

The other side, of course, is that employees accept this bargain.

Naval Architecture

Bartosz Ciechanowski presents an excellent guide on the physics of boats. Most diagrams are interactive, allowing the reader to simulate different conditions. It makes it easier to grasp the underlying concepts. The author starts from first principles and builds, exposing a lot of complexity in naval design. Not a small feat. It even covers the basics of propulsion. Kudos.

Better coordination, or better software?

Jessica Kerr (aka Jessitron) thesis is that better coordination between software teams leads to more coordination work. The alternative is establishing better boundaries better boundaries between software, to make it unnecessary to have that much coordination work.

The power of the unix philosphy is that you can compose single-purpose tools together, to great effect. For example, let’s say we are working on a Rails app. We want to find all controllers that use current_user, and also inherit from ApplicationContrller.

I would run the following:

$ join \
 <(rg '< ApplicationController' app/controllers --files-with-matches | sort) \
 <(rg 'current_user' app/controllers --files-with-matches | sort)

Let’s break it down.

$ rg '< ApplicationController' app/controllers --files-with-matches
app/controllers/pdfs_controller.rb
app/controllers/language_selection_controller.rb
app/controllers/info/pages_controller.rb
app/controllers/secure/base_controller.rb

ripgrep (rg) is an excellent replacement for grep: It searches the content of files for a matching regex. The expression above searches for the regex < ApplicationController to find classes inheriting from ApplicationController inside the app/controllers directory. Like grep, rg can return both file, line number and match information. In this case, I am directing it only return filenames with --files-with-matches.

We now have a list of files in app/controllers that have classes inheriting from ApplicationController.

Next, we want to find uses of current_user in those files. There are a few ways we can accomplish that. I decided to find all controllers that use current_user, and later compare the two lists. The second list of files is found with:

$ rg 'current_user' app/controllers --files-with-matches
app/controllers/secure/password_resets_controller.rb
app/controllers/secure/medical_profiles_controller.rb
app/controllers/secure/bus_trips_controller.rb
app/controllers/secure/profiles_controller.rb
app/controllers/secure/signatures_controller.rb
app/controllers/secure/bus_reservations_controller.rb
app/controllers/secure/base_controller.rb

With the two lists in hand, we can then turn to join. From the man page:

join – relational database operator

The join utility performs an ``equality join’’ on the specified files and writes the result to the standard output.

Much like a JOIN in SQL which find corresponding records in two tables, join can find matching records in two files. Its usage typically requires specifying which field in each line to use for the join. Our usage is very simple though: Our lists of files only have a single field: The file name.

join expects two files as arguments. We could redirect the output of each of our rg calls to a file, and use those files as input to join. However, bash (and other shells too) allow for process substitution: It can take care of presenting the output of a subprocess to another process as if it was a file. That is done via the <() syntax, used twice: Once for each rg search.

The last bit is the usage of sort. join expects the files to be sorted:

When the default field delimiter characters are used, the files to be joined should be ordered in the collating sequence of sort(1)

And there it is! We used rg, sort, join, and a bit of bash plumbing to find files that have lines matching two different regexes:

$ join \
 <(rg '< ApplicationController' app/controllers --files-with-matches | sort) \
 <(rg 'current_user' app/controllers --files-with-matches | sort)
app/controllers/secure/base_controller.rb

GoodJob

I recently found GoodJob, a background-job library for Ruby. It’s compatible with ActiveJob. Its main selling points is that it takes advantage of Postgres features. For projects already on Postgres this means two things: There is no need for another data store, and job scheduling can be transactional with the rest of the application.

Sidekiq, the leading background job library requires the use of Redis. It scales exceptionally well. For many applications managing another data store in production is burdensome, and provides little tangible benefits, especially if the load on the database is low. GoodJob even has a mode that runs the workers in the same process as the Rails server. For smaller apps running on Heroku, this can remove the need of a separate dyno.

Regarding the transactional nature: Suppose you want to store a record and queue a background job as part of some business operation. If you write to your main database first you run the risk of failing when enqueuing the job. Enqueuing inside a transaction doesn’t work either. In case of a transaction rollback, the job will still be published, like Brandur explains. Keeping jobs in the same database as the rest of the data, allows for transactional semantics – much easier to code against.

I’ve only tried this library on a side project with little traffic, but so far I am very impressed.

Idempotency-Key IETF Standards Draft

Speaking of Brandur: He notes that the IETF has a new draft standard for an Idempotency-Key, already in use at Stripe and other places. He previously explains in more detail why it’s important.

My development computer typically has a number of different projects, each needing specific versions of some tools. Previously, I wrote about using asdf to manage my ruby, elixir, crystal and erlang version. I’ve been using it successfully to manage Postgres versions as well.

What I get

• Precisely manage the Postgres version (e.g. 10.14, 12.5) for each project.
• The Postgres data directory lives alongside other project files
• No collision between projects. Each can have their own Postgres server running simultaneously.

EXPLAIN ANALYZE in PostgreSQL and how to interpret it

I’ve been working on web applications for a long time. From time to time I am called upon to figure out (and optimize) performance issues on some query. EXPLAIN ANALYZE lets you see what Postgres is doing under the hood. I don’t do this often enough to remember all the detail of what all the information returned means. This article explains the basics of EXPLAIN ANALYZE and links to some handy tools that help you focus on what matters.

Do You Need Redis? PostgreSQL Does Queuing, Locking, & Pub/Sub

It’s no secret that I like Postgres a lot. This article explains some common use cases for Redis, and how Postgres can take care of them instead. It resonates with me, especially for smaller projects were the techniques outlined can avoid using another data store.

Stop worrying about PostgreSQL locks in your Rails migrations

To round off today’s Postgres love, this blog post introduces how the safe-pg-migrations gem can help when running migrations for databases with high-load and/or a lot of data. Traditionally, those operations can be problematic because of how Postgres acquires locks and can bring production sites down. The specifics are explained in detail in the post. The safe-pg-migrations gem manages to turn what would be complicated steps to ensure safe migrations into the same exact semantics that ActiveRecord::Migration uses.

As soon as I tried this gem, I ran into an issue. The maintainer was very helpful. It turns out Ruby 3.0 support was pending. A few days later my issue was resolved.