Ylan Segal

The REPL: Issue 55 - March 2019

What causes Ruby memory bloat?

Hongli Lai writes a provocative post about his research into Ruby memory bloat. Long-running Ruby process, typically grow in their memory usage over time. It’s common to see Ruby servers being re-started once a day (like on Heroku) to avoid this very issue. In this post, the author proposes that the reason behind memory bloat is not memory fragmentation, like a lot of people in the community assume. Instead, he points to the memory allocator (malloc) not releasing memory. I am not very familiar with memory allocation, so I am left wondering if other languages that also use malloc suffer from the same issue as Ruby.

HIML

Akira Matsuda released a new templating engine with a novel take. The standard template engine in Ruby is ERB. It’s the default in Rails when rendering HTML. A user typically starts with HTML and sprinkles in special tags that get processed as ruby and replaced. HAML, is an alternative that was designed to avoid writing inline code in a web document. It always generates valid HTML, because it functions as a DSL. The down-side is that one must learn the new syntax. That is where HIML comes in: It produces valid HTML and provides some of the conveniences of HAML (e.g. autoclosing tags), but as intuitive as HTML and ERB.

1
2
3
<section> <!-- This will auto-close! -->
  <div> <!-- As will this -->
    <%= post.content %> <!-- Standard way to interpolate ruby code -->

Defining a Distinguished Engineer

Jess Frazelle explains the qualities that a distinguished engineer or technical fellow should exhibit. The list is solid and resonates with my views. My only bone to pick is with the often-repeated adage: Have strong opinions, loosely held. I don’t think that is quite accurate. Opinions held in strength proportional to the weight of the evidence doesn’t have the same ring to it. I understand if it doesn’t catch on. Note that the actual description under the heading is spot on:

They do not need to have opinions on everything, that would be pedantic. Technical leaders should be able to use their experience to help others succeed, while also empowering others to own solutions.

On Being Efficient: My Mac Setup

As a Software Engineer, I spend most of my day working on a computer. Over the years, I’ve come to customize my setup to align better to the way I work and become more efficient. Efficiency is important because of the time you save, but not because you can do more with the time saved. It’s allows you to keep focus on what you want to accomplish, instead of the mechanics of accomplishing it.

The following describes what I use on my Mac, but the principles apply to any operating system with a graphical interface. The metaphors they use are mostly the same: desktop, windows, application, a dock, etc.

Application Launcher

I launch apps through an application launcher. It involves a keyboard shortcut to invoke the launcher, typing a few keys (usually 1 to 3) to fuzzy find the app I am looking for, and hitting enter. For me, typing 3 to 5 keys is very fast. I have been using Alfred for many years, but I’ve know people that thing the built-in Spotlight works just as well for this purpose.

Switching Applications

MacOS (formerly OS X) includes a keyboard shortcut to switch applications: cmd + tab. Hitting it once and releasing switches you immediately to the last application used. It use this the most to switch back-and-forth between my editor and terminal, documentation and editor, etc.

Hitting cmd+tab, but keeping cmd pressed, brings up a modal list of the icons of all running application, ordered by how recently they have been used. While still holding cmd, pressing tab moves the selector to the next app in line (shift + tab moves back one spot). When cmd is let go, the selected app moves into focus. In this way, it becomes really fast to switch to another running application.

Alternatively, instead of finding the running app, I sometimes just use the application launcher. If the application is already running, it just switches to it.

For applications that use more than one window, you can use cmd + ` to cycle between them. I use this often when I have more of one window for my text editor.

Hide the Dock

I don’t use my Dock. I have it set vertically to the far right of my screen and it’s hidden by default, unless I move my mouse over it. This doesn’t happen a lot, and usually it’s by mistake. If there was a way to completely turn it off, I would.

As far as I can see, the main raison d'être for it is to launch favorite applications (see above), to switch applications (see above), or to see app badges (e.g. unread count for email). I limit aggressively the number of badges I allow in my applications. For those that I do use, it’s enough for me to see them when switching applications.

As a side benefit of not having a permanent Dock, is that I get some of my screen real-estate back.

Clipboard History

The built-in clipboard works very simply: When you paste cmd + v it inserts the last thing you copied cmd + c or cut cmd + x. A clipboard with a history, maintains a history off what you have copied and allows you pasting from it. This is one of those things that after using for a short while you wonder how you ever managed without.

Alfred provides that functionality for me as well. It’s implementation is particulary nice, since it doesn’t modify the standard paste command. Instead, it has a keyboard shortcut to open the history and a fuzzy finder to find entries. This works really well. A particularly nice touch is that it’s configurable to not add things to the history from an application, like a password manager.

Windows Tiling

The traditional way of positioning and sizing windows in MacOS is by using the mouse. It’s slow and difficult to get just right. A window-tiling application let’s you snap your windows into predefined positions. I use 3 sizes: Full screen*, half-screen vertical, third-screen vertical. I align them left, right, or centered as needed. My preferred tool is Spectacle which allows configuring all of this with keyboard-shortcuts.

I don’t use MacOS full-screen mode: It removes the menu bar, which I like. More importantly it has an animation going to and from that is slower than just switching applications. I find that distracting. Instead I use Spectacle to enlarge the screen to fill the screen (equivalent of option-clicking on the green dot on a standard Mac window).

Shortcuts for Text Editing

Last, but not least. I type a lot of text: Code, emails, ticketing systems. wikis, GitHub, etc. By default, all Mac text input areas allow the use of the same text editing shortcuts. For example, moving to the begging of the current line (cmd + left-arrow), moving back a word (option + left-arrow), delete previous word (cmd + delete), etc. Those shortcuts make text editing much faster. Knowing those shortcuts is one of the reason why I don’t learn vim.


Customizing your working environment, even in small ways can have a big cumulative effect. I don’t recommend trying all of the above at once, but focusing on one at the time. Try it out for a while and tweak to your liking until it feels like a natural part of your workflow. For me, that can be anywhere from 2 to 4 weeks.

The REPL: Issue 54 - February 2019

Move fast and migrate things: how we automated migrations in Postgres

Vineet Gopal from Benchling writes an interesting post about their approach to running migrations on highly-contested databases in production. A new concept for me was that they automatically retry migrations that fail due to lock timeouts. This reduces the number of failed deployments and manual intervention steps.

Rescue from errors with a grace

In this post Paweł Dąbrowski shows how to leverage Ruby’s value equality (===) method, and overriding the default functionality in custom exceptions. The results is cleaner exception handling code.

Distributed Phoenix Chat with PubSub PG2 adapter

Alvise Susmel writes in detail how to use Phoenix Chat PubSub implementation using the pg2 library. The result a distributed, multi-node chat service that does not have an external dependency to a separate system (like Redis).

Dragons in Benchmark-ips

My go-to tool for analyzing ruby performance is benchmark-ips. It’s an enhancement to Ruby’s own stdlib tool Benchmark. It’s easy to use, and reports meaningful information by default.

Recently, while running a benchmark, I was getting very odd results. One of the alternatives reported thousand of times slower. That was not in line with my expectations. As a sanity test, I had both blocks run the same code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
require 'benchmark/ips'

Benchmark.ips do |x|
  x.report("option A") do
    Object.new
  end

  x.report("option B") do |times|
    Object.new
  end

  x.compare!
end

# >> Warming up --------------------------------------
# >>             option A   213.307k i/100ms
# >>             option B   257.225k i/100ms
# >> Calculating -------------------------------------
# >>             option A      5.958M (± 6.5%) i/s -     29.650M in   5.001835s
# >>             option B    254.000B (± 9.7%) i/s -    709.200B in   3.014349s
# >>
# >> Comparison:
# >>             option B: 254000471514.5 i/s
# >>             option A:  5958499.2 i/s - 42628.26x  slower
# >>

It took a few minutes, before I spotted the difference: The second block is making times available to the block, while the first is not. After setting up both blocks in the same way:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
require 'benchmark/ips'

Benchmark.ips do |x|
  x.report("option A") do
    Object.new
  end

  x.report("option B") do
    Object.new
  end

  x.compare!
end

# >> Warming up --------------------------------------
# >>             option A   227.941k i/100ms
# >>             option B   229.945k i/100ms
# >> Calculating -------------------------------------
# >>             option A      6.299M (± 2.3%) i/s -     31.684M in   5.032681s
# >>             option B      6.279M (± 3.3%) i/s -     31.502M in   5.023744s
# >>
# >> Comparison:
# >>             option A:  6299217.3 i/s
# >>             option B:  6278600.3 i/s - same-ish: difference falls within error
# >>

That is more like it: Identical code performs the same.

The REPL: Issue 53 - January 2019

Detecting Agile BS

I don’t know what I like more about this guide: The fact that it calls BS a lot of what is gospel for many in the software industry, the fact that it’s published by the Department of Defense, or the power-point-y graphics.

Distributed Transactions: The Icebergs of Microservices

In this article, Graham Lea explains many potential pitfalls with distributed transactions, and general advice on how to avoid them in the first place, or deal with them effectively when must.

The solution to distributed transactions in microservices is simply to avoid them like the plague.

Our Software Dependency Problem

Russ Cox writes about software dependencies, and goes into great detail of what dependencies are and what risks they bring into software projects. I found myself nodding in agreement throughout the post. The need to have a good policy towards updating project dependencies has been a pet-peeve of mine for a while.