Ylan Segal

The REPL: Issue 13 - August 2015

Making Architecture Matter

In this keynote at OSCON 2015, Martin Fowler gives a great overview about how to think about software architecture to get the most benefit. As quoted in the comments:

It’s particularly relevant now as we push more and more toward continuous delivery, continuous deployment, features updated over the Internet all the time. That degree of being able to respond to change becomes important. That’s the economic reason why software architecture is important, because if we don’t keep good architecture, we are, in the end, deceiving our customers — in fact, stealing from our customers — because we’re slowing down their ability to compete.

Binary pattern matching in Elixir

Zohaib Rauf writes a great post that shows the great power of pattern matching in Elixir. Step by step, he builds a small module that parses PNG binaries. The explanations are clear and the resulting code is really elegant: One main function that parses the header and calls a private recursive function that parses each of the chunks. Worth a read, even if you are not into Elixir (yet!).

Work Hard, Live Well

Dustin Moskovitz writes about work life balance in the software industry and his personal experience at Facebook. The article resonates with me: I often say that in the modern world, sleep is a competitive advantage: Being tired decreases cognitive ability. When I was a freshman in college, I stayed up late to study for a physics exam, which I presented after only 4 hours of sleep. I had flunked because of basic algebraic mistakes applying the correct physical formulae. I took the lesson to heart. After that, for me exam preparation always included adequate rest.

Using ActiveRecord::Base#merge

The API for ActiveRecord::Base is very large, which makes it easy to miss some of the great convinences it affords.

1
2
3
4
5

class Account < ActiveRecord::Base
end

Account.new.methods.count # => 558
Object.new.methods.count # => 123

ActiveRecord::Relation#merge merges the conditions of two relations, allowing the use of scopes on both sides. This allows not repeating where clauses in separate places in the code base. Example, extracted from the Rails documentation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Instead of:

Post.joins(:comments).where(published: true, comments: { spam: false })

# => SELECT "posts".* FROM "posts" INNER JOIN "comments" ON "comments"."post_id" = "posts"."id" WHERE "posts"."published" = ? AND "comments"."spam" = ?  [["published", "t"], ["spam", "f"]]

# Lets take advantage of scopes in Post and Comment:

class Comment < ActiveRecord::Base
  scope :non_spam, -> { where(spam: false) }
end

class Post < ActiveRecord::Base
  scope :published, -> { where(published: true) }
end

Post
  .joins(:comments)
  .published
  .merge(Comment.non_spam)

# => SELECT "posts".* FROM "posts" INNER JOIN "comments" ON "comments"."post_id" = "posts"."id" WHERE "posts"."published" = ? AND "comments"."spam" = ?  [["published", "t"], ["spam", "f"]]

This way, each scope stays with the model it belongs in, removing any danger of duplicating domain logic (like what is consider spam) to a single place in the code. This technique is especially useful when taking advantage of more complicated (possibly parametized) scopes.

Book Review: Understanding Computation

I have two University degrees in different engineering disciplines, but came to Software Engineering without any formal training in Computer Science. I learned how to use databases and shortly later how to write programs. I mostly learned from practical books and online resources. All my deeper knowledge of computer theory came years later. Understanding Computation: From Simple Machines to Impossible Programs by Tom Stuart seems to have been written for people like me: Thirsty for knowledge about computer theory, with explanations written in the language we use every day and without the mathematical notation1.

In the book, Stuart uses Ruby to great effect to show examples of basic computer science. He builds two interpreters for a toy language, uses finite automate to implement a bare-bones regular expression language, explores Turing machines, illustrates how lambda calculus can be used to solve FizzBuzz and much more.

The explanations throughout the book are thorough and the code samples numerous. It is not a fast read: For some sections, I had to put down the book and think about what I read every couple of pages. However, I believe I have a better understanding of computer science after reading it.

I recommend the book to those Rubyists lacking a background in Computer Science or those that have one, but want to gain a perspective of computation without the formalism.

Links:

  1. I actually do like mathematics, but mostly as a tool to explore physics. My familiarity with the math underpinnings of computation is lacking.↩

Celluloid, Nice to Meet You

In my projects, I regularly encounter the need for a long-running process that runs in the background. In ruby, it’s easy to reach for Thread:

1
2
3
4
5
6

Thread.new do
  loop do
    puts "Working..."
    work # Some actual work here...
  end
end

Although simple, it lacks fault tolerance: Any exception raised from work will terminate the loop and the thread. Often, the way to harden against this failure looks something like this:

1
2
3
4
5
6
7
8
9
10

Thread.new do
  begin
    loop do
      puts "Working..."
      work # Some actual work here...
    end
  rescue
    retry
  end
end

Rescuing from any exception and retrying indefinitely. It’s effective, but not very elegant. Earlier this week, I researched for a better pattern and found the Celluloid gem. In their words:

[Celluloid is an] Actor-based concurrent object framework for Ruby

https://github.com/celluloid/celluloid

Painless multithreaded programming for Ruby

https://celluloid.io/

Actors? You mean, like in Erlang? It turns out, I have been reading a lot about Elixir lately, so I was eager to try it.

A Simple Example

Celluloid’s basic unit is called an Actor. A simple ruby class that includes the Celluloid module is now transformed into a multi-threaded object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class Worker
  include Celluloid
  include Celluloid::Logger # So that we can use the familiar #info, #error, etc.

  def initialize
    @uid = SecureRandom.uuid
    async.work
  end

  def work
    loop do
      info "[#{@uid}] - Working..."
      fail "BOOM" if rand(10) < 3 # Simulate random failures
      sleep 1 # Simulating work
    end
  end
end

There a couple of interesting things in the code above. The initializer actually starts doing some work, asynchronously. The #async method is provided by Celluloid. Our main loop is in the work method, but we booby-trapped it to raise an exception at random times. Notice that neither the loop or the work method rescue any exceptions. Any exception raised will stop work.

Next, comes the interesting part:

1
2
3

class Group < Celluloid::SupervisionGroup
  supervise Worker, as: :worker
end

A supervision group, like Group is responsible for instantiating one or more actors, groups or pools of actors and supervising them: That is, if they “crash” or otherwise die, the supervisor will ensure that a new actor is re-intantiated in it’s place. It will instantiate, but not call any method on the actor, which is why our initialize method calls async.work: To ensure that a crashed actor is replaced with another running actor.

And to finally run it:

1
2
3

supervisor = Group.run!
sleep(10)
supervisor.actors.each(&:terminate)

Calliing run! starts the supervisor in the background. run would started in the foreground and block the rest of the thread. The sleep is required to prevent ruby’s main thread from exiting. At the end of our test period, we gracefuly terminate our actors (one, actually).

So, what does this look like? As expected, Worker#work is prone to raising un-rescued exceptions, but we can see our supervisors starts a new Worker and fault-tolerance is achieved:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# Logfile created on 2015-08-05 17:39:04 -0700 by logger.rb/47272
I, [2015-08-05T17:39:04.594399 #49511]  INFO -- : [2b66f5f3-f36a-4465-8454-abbdd97d1c0d] - Working...
I, [2015-08-05T17:39:05.595124 #49511]  INFO -- : [2b66f5f3-f36a-4465-8454-abbdd97d1c0d] - Working...
E, [2015-08-05T17:39:05.595472 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:05.596377 #49511]  INFO -- : [05bfa2ea-78ab-41ce-bd26-ed82d46fbea9] - Working...
E, [2015-08-05T17:39:05.596564 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:05.597251 #49511]  INFO -- : [ad8a5361-fa49-4464-b059-405c86cfe4cf] - Working...
I, [2015-08-05T17:39:06.598375 #49511]  INFO -- : [ad8a5361-fa49-4464-b059-405c86cfe4cf] - Working...
I, [2015-08-05T17:39:07.601223 #49511]  INFO -- : [ad8a5361-fa49-4464-b059-405c86cfe4cf] - Working...
E, [2015-08-05T17:39:07.601420 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:07.602620 #49511]  INFO -- : [04b5959a-2156-4c39-8d8f-6660d561cc6d] - Working...
I, [2015-08-05T17:39:08.607258 #49511]  INFO -- : [04b5959a-2156-4c39-8d8f-6660d561cc6d] - Working...
I, [2015-08-05T17:39:09.607827 #49511]  INFO -- : [04b5959a-2156-4c39-8d8f-6660d561cc6d] - Working...
E, [2015-08-05T17:39:09.608015 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:09.608841 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
I, [2015-08-05T17:39:10.614044 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
I, [2015-08-05T17:39:11.615554 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
I, [2015-08-05T17:39:12.615923 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
E, [2015-08-05T17:39:12.616119 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:12.617089 #49511]  INFO -- : [d4534e4e-0d36-41fb-8d92-2b3744314eee] - Working...
E, [2015-08-05T17:39:12.617210 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:12.617893 #49511]  INFO -- : [bda39ba0-1cc6-41f9-94b2-83bb72c6bc03] - Working...
I, [2015-08-05T17:39:13.619554 #49511]  INFO -- : [bda39ba0-1cc6-41f9-94b2-83bb72c6bc03] - Working...
D, [2015-08-05T17:39:14.596879 #49511] DEBUG -- : Terminating task: type=:call, meta={:dangerous_suspend=>false, :method_name=>:work}, status=:sleeping
    Celluloid::Task::Fibered backtrace unavailable. Please try `Celluloid.task_class = Celluloid::Task::Threaded` if you need backtraces here.
D, [2015-08-05T17:39:14.597275 #49511] DEBUG -- : Terminating 1 actor...

(Backtraces trimmed for brevity).

Conclusion

Celluloid is a very big framework, with an even bigger ecosystem. It powers well known ruby projects like Sidekiq. The above example just scratched the surface and exposes a common use I have. But, please do take a look at the wiki for more features.

Working with Actor-based framework, is in some ways different than traditional ruby, but it also fits in nicely. It may be because I was already primed by reading about Elixir and OTP lately, but I found it somewhat liberating to not have to think about all possible errors conditions and just focus on the happy path, with the confidence that any error will be automatically recovered from.

The REPL: Issue 12 - July 2015

Elixir in times of microservices

José Valim, creator of Elixir and Rails Core member, weighs in on microservices. He makes a great case on why Elixir, leveraging the Erlang VM makes it easier to work with distributed systems and imposes less up-front tradeoffs than the current trend of microservices communicating via JSON API.

Elegant APIs with JSON Schema

At work, I have been exploring how to work effectively with microservices on a Rails stack. JSON Schema, comes up often, especially, especially with all the tools open-sourced by Heroku/Interagent. The blog post by @brandur, is the best introduction to JSON Schema I have read so far.

Improved production stability with circuit breakers

The circuit breaker pattern provides a way for resiliency and stability when working with distributed systems. In this post, Heroku introduces their new Ruby library for implementing the pattern. I especially liked the idea of having a roll-out strategy introducing logging-only circuit breakers first. At the very end, they advise to tune timeout settings for underlying libraries. Don’t know how to do that? Check the Ultimate Guide To Timeouts In Ruby