I was trying to update rails in one of my projects recently. The command I ran was not quite right, so I wanted to discard changes to Gemfile.lock

I kept doing:

git restore Gemfile.lock

But the git reported the file as still dirty, and the unwanted changes were still there. I couldn’t understand what was going on!

Eventually, I noticed that:

git restore Gemfile Gemfile.lock

worked. Then it hit me. A spring server was still running an apparently running bundle install whenever the Gemfile changed, which was regenerating my unwanted changes as soon as I restored them with git.

I guess for some use cases this quiet behavior helps. In my case, I wanted to run a particular bundle update:

$ bundle update rails --conservative --patch --strict

spring. Sigh.

I’ve just read a blog post on Fly.io that says:

So, a few months back, during an integration with a 3rd party API that relied on OAuth2 tokens, we drew a line: ⚡ henceforth, hazmat shall only be removed from Rails, never added ⚡.

Hazmat. As in hazardous materials. Wikipedia defines it as:

Dangerous goods, abbreviated DG, are substances that when transported are a risk to health, safety, property or the environment. Certain dangerous goods that pose risks even when not being transported are known as hazardous materials (syllabically abbreviated as HAZMAT or hazmat)

I’ve never heard of software secrets being described as hazmat. It’s an apt metaphor. They can be dangerous, even when not being transmitted. We should be careful on how we handle them. Hazmat evokes specialized suits and gear. It’s not a bad mental image when dealing with secrets!

The second interesting part about the line in the blog post is the “only removed, never added”. That makes me think of a ratchet, defined as:

ratchet1 | ˈraCHət |
noun
1 a device consisting of a bar or wheel with a set of angled teeth in which
  a cog or tooth engages, allowing motion in one direction only
2 a situation or process that is perceived to be changing in a series of irreversible
  steps

A hazmat ratchet, if you will.

How First Principles Thinking Fails

Thinking from first principles is not a silver bullet:

When reasoning from first principles, you may fail if:

You have flawed assumptions.

You make a mistake in one of your inference steps. (An inference step is a step in the chain of reasoning in your argument).

You start from the wrong set of principles/axioms/base facts.

You reason upwards from the correct base, but end up at a ‘useless’ level of abstraction.

I’ve been thinking a lot about the third one. If you don’t know certain facts, and you don’t know that you don’t know them, you can reach wrong conclusions.

For example, if you only know Newtonian physics, you will calculate the orbits of plants very precisely, but you will come to the wrong conclusions about Mercury’s orbit. You need to know about Relativity to get that orbit correct.

In essence, the “unknown unknowns” can get you.

Speed matters: Why working quickly is more important than it seems

If you are fast at something, of course you can do more of that thing in a given time. The author proposes that you also have a lower perceived cost for doing more of that thing, and that in itself lowers he barrier to doing more of that activity. The conclusion is that if you want to do more of something, you should try getting faster at it.

How do you get faster at something? In running (the sport), you get faster by doing speed-work, sprints, and the like. You also get faster by running longer and longer distances, at a relatively slow pace. The physiology is interesting, but not my point. In guitar playing, a very effective technique for learning a fast solo or transition is to use a metronome and slow down to practice the section, say at 50%, speed until your are proficient at it, then increase to 60% and so on, until you can play it at 100%.

While I agree that doing something faster promotes you doing more of it, it is not always intuitive how to get faster.

I’m an ER doctor. Here’s how I’m already using ChatGPT to help treat patients

This resonates with me: In it’s current form, ChatGPT is already very usable to generate text that you can verify is correct. In this example, the doctor can read the text and can tell if it’s empathetic, like he requested, and correct. It saved him from having to type it, and in fact maybe even did something that he could not: Communicate with more empathy. In any case, the doctor was 100% capable of evaluating the text produced.

In my personal use of LLMs, they can and will suggest code snippets. It saves me the time of having to read API documentation of several classes. I can evaluate the result for correctness, and even more importantly, I will adapt for my use case which will include tests for formally verifying that the code indeed works as expected.

I am not sure about the drunken intern analogy: I probably would have phrased it as a sleep-deprived intern instead. The intern heuristic is useful though. Getting back to the my code usage: It is useful to think of LLMs as an intern that produces code very fast, but that I have to evaluate. “Hey intern, how do I add a custom header to a web request using Ruby’s ‘rest’ gem?”. I am capable of evaluating the result and making whatever corrections are needed. Time saved.

arel – A relational algebra library – is the ruby library that that powers ActiveRecord. It provides a lower-level abstraction for working with SQL than ActiveRecord, and typically used when a particular query is not possible with ActiveRecord. Over the years, I’ve reached out for it on occasion. The definitive guide to Arel, the SQL manager for Ruby provides good information if you haven’t used it before.

I was recently working with a complicated query, and spent way more time because I assumed some things about it’s API that turned out to not be true.

Let’s first observe how ActiveRecord::Relations behave:

ar_relation = User.where.not(active_thru: nil)
ar_relation.to_sql
# =>  SELECT "users".* FROM "users" WHERE "users"."active_thru" IS NOT NULL

ar_relation.where(id: 5).to_sql
# => SELECT "users".* FROM "users" WHERE "users"."active_thru" IS NOT NULL AND "users"."id" = 5

ar_relation.to_sql
# SELECT "users".* FROM "users" WHERE "users"."active_thru" IS NOT NULL

Notice how calling a second where on a relation returns a new relations without modifying the original. This allows composing relations in Rails with great effect.

However, that is not true with Arel::SelectManager classes:

table = User.arel_table
arel_manager = table.where(table[:active_thru].not_eq(nil))
arel_manager.to_sql
# => SELECT FROM "users" WHERE "users"."active_thru" IS NOT NULL

arel_manager.where(table[:id].eq(5)).to_sql
# => SELECT FROM "users" WHERE "users"."active_thru" IS NOT NULL AND "users"."id" = 5

arel_manager.to_sql
# => SELECT FROM "users" WHERE "users"."active_thru" IS NOT NULL AND "users"."id" = 5

Notice how adding a new where clause modified the original object.

Caveat Emptor

The Statistics Handbook

I’ve been taking a statistics course in Coursera. The lectures and exercises are great, but I was really missing a text book that I can come back and reference. I was happy to find this gem: Free and available to download.

The definitive guide to Arel, the SQL manager for Ruby

Recently, I’ve been doing more complicated SQL queries in Rails, for which the ActiveRecord API is not enough. Enter Arel, a relational algebra library on which ActiveRecord is built, that allows more flexibility when using Rails. This is a great guide to using it. Arel is considered a private API in Rails. I’ve found it to be very stable, but be mindful when using it.

Introducing Tobox

This gem attempts to solve the write-to-multiple-databases problem when using background processing libraries in Ruby (e.g sidekiq). In effect, this is an event system, but without describing it heavily as such. The problem, stated more concisely is well described in Pattern: Transactional Outbox. The gem is new. I can’t comment on it’s maturity or stability. The author is right in pointing out that Rail’s ActiveJob DSL allows easy backgrounding of jobs, but ignores transactionality and dual-write problem that might exist. In fact, I was talking about this with some co-workers recently. One of the benefits of using GoodJob is that since the queue storage is in the same database, we can ignore this problem, as long as we are using a transaction.