In my projects, I regularly encounter the need for a long-running process that runs in the background. In ruby, it’s easy to reach for Thread:

1
2
3
4
5
6

Thread.new do
  loop do
    puts "Working..."
    work # Some actual work here...
  end
end

Although simple, it lacks fault tolerance: Any exception raised from work will terminate the loop and the thread. Often, the way to harden against this failure looks something like this:

1
2
3
4
5
6
7
8
9
10

Thread.new do
  begin
    loop do
      puts "Working..."
      work # Some actual work here...
    end
  rescue
    retry
  end
end

Rescuing from any exception and retrying indefinitely. It’s effective, but not very elegant. Earlier this week, I researched for a better pattern and found the Celluloid gem. In their words:

[Celluloid is an] Actor-based concurrent object framework for Ruby

Painless multithreaded programming for Ruby

Actors? You mean, like in Erlang? It turns out, I have been reading a lot about Elixir lately, so I was eager to try it.

A Simple Example

Celluloid’s basic unit is called an Actor. A simple ruby class that includes the Celluloid module is now transformed into a multi-threaded object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class Worker
  include Celluloid
  include Celluloid::Logger # So that we can use the familiar #info, #error, etc.

  def initialize
    @uid = SecureRandom.uuid
    async.work
  end

  def work
    loop do
      info "[#{@uid}] - Working..."
      fail "BOOM" if rand(10) < 3 # Simulate random failures
      sleep 1 # Simulating work
    end
  end
end

There a couple of interesting things in the code above. The initializer actually starts doing some work, asynchronously. The #async method is provided by Celluloid. Our main loop is in the work method, but we booby-trapped it to raise an exception at random times. Notice that neither the loop or the work method rescue any exceptions. Any exception raised will stop work.

Next, comes the interesting part:

1
2
3

class Group < Celluloid::SupervisionGroup
  supervise Worker, as: :worker
end

A supervision group, like Group is responsible for instantiating one or more actors, groups or pools of actors and supervising them: That is, if they “crash” or otherwise die, the supervisor will ensure that a new actor is re-intantiated in it’s place. It will instantiate, but not call any method on the actor, which is why our initialize method calls async.work: To ensure that a crashed actor is replaced with another running actor.

And to finally run it:

1
2
3

supervisor = Group.run!
sleep(10)
supervisor.actors.each(&:terminate)

Calliing run! starts the supervisor in the background. run would started in the foreground and block the rest of the thread. The sleep is required to prevent ruby’s main thread from exiting. At the end of our test period, we gracefuly terminate our actors (one, actually).

So, what does this look like? As expected, Worker#work is prone to raising un-rescued exceptions, but we can see our supervisors starts a new Worker and fault-tolerance is achieved:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# Logfile created on 2015-08-05 17:39:04 -0700 by logger.rb/47272
I, [2015-08-05T17:39:04.594399 #49511]  INFO -- : [2b66f5f3-f36a-4465-8454-abbdd97d1c0d] - Working...
I, [2015-08-05T17:39:05.595124 #49511]  INFO -- : [2b66f5f3-f36a-4465-8454-abbdd97d1c0d] - Working...
E, [2015-08-05T17:39:05.595472 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:05.596377 #49511]  INFO -- : [05bfa2ea-78ab-41ce-bd26-ed82d46fbea9] - Working...
E, [2015-08-05T17:39:05.596564 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:05.597251 #49511]  INFO -- : [ad8a5361-fa49-4464-b059-405c86cfe4cf] - Working...
I, [2015-08-05T17:39:06.598375 #49511]  INFO -- : [ad8a5361-fa49-4464-b059-405c86cfe4cf] - Working...
I, [2015-08-05T17:39:07.601223 #49511]  INFO -- : [ad8a5361-fa49-4464-b059-405c86cfe4cf] - Working...
E, [2015-08-05T17:39:07.601420 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:07.602620 #49511]  INFO -- : [04b5959a-2156-4c39-8d8f-6660d561cc6d] - Working...
I, [2015-08-05T17:39:08.607258 #49511]  INFO -- : [04b5959a-2156-4c39-8d8f-6660d561cc6d] - Working...
I, [2015-08-05T17:39:09.607827 #49511]  INFO -- : [04b5959a-2156-4c39-8d8f-6660d561cc6d] - Working...
E, [2015-08-05T17:39:09.608015 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:09.608841 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
I, [2015-08-05T17:39:10.614044 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
I, [2015-08-05T17:39:11.615554 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
I, [2015-08-05T17:39:12.615923 #49511]  INFO -- : [eca10599-6648-4625-8f09-99be2add3b16] - Working...
E, [2015-08-05T17:39:12.616119 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:12.617089 #49511]  INFO -- : [d4534e4e-0d36-41fb-8d92-2b3744314eee] - Working...
E, [2015-08-05T17:39:12.617210 #49511] ERROR -- : Actor crashed!
RuntimeError: BOOM
    simple_poc.rb:18:in `block in work'
I, [2015-08-05T17:39:12.617893 #49511]  INFO -- : [bda39ba0-1cc6-41f9-94b2-83bb72c6bc03] - Working...
I, [2015-08-05T17:39:13.619554 #49511]  INFO -- : [bda39ba0-1cc6-41f9-94b2-83bb72c6bc03] - Working...
D, [2015-08-05T17:39:14.596879 #49511] DEBUG -- : Terminating task: type=:call, meta={:dangerous_suspend=>false, :method_name=>:work}, status=:sleeping
    Celluloid::Task::Fibered backtrace unavailable. Please try `Celluloid.task_class = Celluloid::Task::Threaded` if you need backtraces here.
D, [2015-08-05T17:39:14.597275 #49511] DEBUG -- : Terminating 1 actor...

(Backtraces trimmed for brevity).

Conclusion

Celluloid is a very big framework, with an even bigger ecosystem. It powers well known ruby projects like Sidekiq. The above example just scratched the surface and exposes a common use I have. But, please do take a look at the wiki for more features.

Working with Actor-based framework, is in some ways different than traditional ruby, but it also fits in nicely. It may be because I was already primed by reading about Elixir and OTP lately, but I found it somewhat liberating to not have to think about all possible errors conditions and just focus on the happy path, with the confidence that any error will be automatically recovered from.

Elixir in times of microservices

José Valim, creator of Elixir and Rails Core member, weighs in on microservices. He makes a great case on why Elixir, leveraging the Erlang VM makes it easier to work with distributed systems and imposes less up-front tradeoffs than the current trend of microservices communicating via JSON API.

Elegant APIs with JSON Schema

At work, I have been exploring how to work effectively with microservices on a Rails stack. JSON Schema, comes up often, especially, especially with all the tools open-sourced by Heroku/Interagent. The blog post by @brandur, is the best introduction to JSON Schema I have read so far.

Improved production stability with circuit breakers

The circuit breaker pattern provides a way for resiliency and stability when working with distributed systems. In this post, Heroku introduces their new Ruby library for implementing the pattern. I especially liked the idea of having a roll-out strategy introducing logging-only circuit breakers first. At the very end, they advise to tune timeout settings for underlying libraries. Don’t know how to do that? Check the Ultimate Guide To Timeouts In Ruby

Dave Thomas is celebrated in the Ruby community for having written the “Pickaxe”: The first english language book on Ruby and widely used as the beginners guide and reference to Ruby. Now, he brings us Programming Elixir: Functional |> Fun |> Cuncurrent |> Pragmatic |> Fun.

I found the book to be a delightful introduction to Elixir. It is not intended for those that already know how to program and want to learn about the Elixir language in particular. Thorough introduction to the semantics of the language, the functional aspects, recursion, pattern matching, data structures, protocols, Erlang’s OTP framework and even some of the meta-programming facilities. I found the examples to be very clear and the suggested exercises help the concepts sink in.

I recommend this book, along with Elixir’s own Getting Started Guide to anyone interested in learning Elixir. When such an experienced programmer as PragDave has his eye on a new language, I listen. Elixir is gaining a lot of momentum. This book is a great way to get on the band-wagon.

Phoenix and Rails performance comparison

Benchmarks are always to be taken with a grain of salt: They think that they are measuring can be very far from the performance you are likely to see in production. Their usefulness is in informing our decisions about technology choices. I have been getting interested lately in Elixir and the Phoenix Framework and the results of this benchmark confirm my anecdotal experience: Phoenix, out of the gate, blows Rails out of the water. Very interesting considering that the code you write in Phoenix applications is very pleasant, like Rails. Developer happiness, right?

This Is Professionalism

This short post by Chris Doyle captures very succinctly what professionalism is: Autonomy, responsability and humilty. It resonated with me. As I mentioned before, excellence and professionalism in Software Engineering don’t require a medieval craftsmanship metaphor.

Designing a Secure REST (Web) API without OAuth

I have been doing a lot of research lately on API authentication and came across this article. It’s a bit rambling at times, but it’s filled with good information of what issues can arise with API authentication and was food for a lot of thought and further research.

There is a lot of buzz around microservices and service-oriented architecture, at least in the corner of the internet that I frequent. Heavyweights in the Ruby community, like Heroku think microservices matter. It seems that enough people are adopting microservices head-first that some influential people in the community have started warning that it might not be for everyone. Martin Fowler thinks there is a microservices permium to pay and that you probably want to start with a monolith. Avdi Grim is of the opinion that people are adopting microservices for the wrong reasons. And David Heinemeier Hansson, devoted a portion of his RailsConf 2015 keynote to denouncing the practice.

In “Building Microservices. Designing Fine-Grained Systems”, Sam Newman explores the practice of microservices. Starting with a definition of just how small they are (something that can be re-built in 2 weeks time), the book covers in depth what microservices are good for and what having a fleet of them looks in practice.

As one would expect, the author is partial to microservices, but acknowledges that they are not a good fit for every organization and every project. For example, when a start-up is still exploring their domain and iterating fast, the boundaries of their systems are hard to predict and API might churn significantly. Similarly, teams that are not leveraging cloud computing and have the necessary automation in place to provision and deploy new services might not find much success in having numerous systems to support in production. The author makes well-reasoned arguments for organizations only considering microservices when they have great automated-test coverage, continuous integration in place and automated deployments (preferably continuous delivery).

Along the way, Mr. Newman also points out that, like most things in Engineering, there are trade-offs to be made: Latency and develops complexity, chief among them. However, there is also a lot to be gained: being able to mix-and-match technology stacks, resilience of the whole system (assuming that the proper precautions are in place), independent scaling for each piece, deployment in stages, organizational alignment, the ability to replace parts as needed and maintaining team velocity.

I though the book to be an overall good read for those starting to dip their toes in the microservices world.