I often need to write some scripts for my Rails applications. These are sometimes intended to be run in development or production environments, and require that the Rails application is loaded. The most common way I’ve seen of doing this is with a rake task:

namespace :import do
  desc "Import people. Requires a file name as argument"
  task :people, %i[file_path] => :environment do |_t, args|
    puts "Importing people from #{args[:file_path]}"
    People::Import.new(args[:file_path]).perform
    puts "Done"
  end
end

rake tasks have their benefits, but the argument handling is not like most other unix programs. They are passed as an array at the end of the task name:

bash-5.2$ rake import:people[people.csv]
Importing people from people.csv
Done

Even more troubling, is that if you use zsh (and I am!), the [ and ] need to be escaped, or the whole argument to rake quoted:

$ rake import:people\[people.csv\]

# OR

rake "import:people[people.csv]"

Rails runner

You can use rails runner to run any ruby file in the context of the rails application:

# import.rb

puts "Importing people from #{ARGV.first}"
People::Import.new(ARGV.first).perform
puts "Done"

And run it by passing your file as the first argument:

$ rails runner import.rb people.csv

But you can also use rails runner as a shebang line.

#!/usr/bin/env rails runner
# bin/import
puts "Importing people from #{ARGV.first}"
People::Import.new(ARGV.first).perform
puts "Done"

And now you can invoke that directly:

$ chmod +x bin/import

$ bin/import people.csv

The shebang line allows your script to be directly executable. Now, you can make it more ergonomic, and build up a unix-like command:

#!/usr/bin/env rails runner
# bin/import

options = {}
OptionParser.new do |parser|
  parser.banner = "Usage: rails runner example.rb [options]"

  parser.on("-f", "--file FILE", "Import from FILE") do |file|
    options[:file] = file
  end

  parser.on("-h", "--help", "Prints this help") do
    puts parser
    exit
  end
end.tap(&:parse!)

puts "Importing people from #{options[:file]}"
# ...

$ bin/import --file people.csv
# ...

$ bin/import -h
Usage: rails runner example.rb [options]
    -f, --file FILE                  Import from FILE
    -h, --help                       Prints this help

Conclusion

Using rails runner as a shebang allows full access to the Rails environment in your script, without having to sacrifice command-line ergonomics.

Bash Debugging

Julia Evans, in here signature comic/zine style, explains how to debug bash. I was today years old when I learned that you can have a step debugger for a bash script.

How we migrated from Sidekiq to Solid Queue

Chirag Shah at BigBinary explain in detail how they migrated an app from Sidekiq to SolidQueue. From the configuration it seems that the queue usage is on the low side (4 types of workers, each with a single process and 3 threads).

The article doesn’t mention in the article what they hoped to achieve or what their results where. I assume positive, because they close by saying they plan on doing the migration on more apps.

If I had to guess, I would say that they gained a simpler infrastructure (because they don’t use redis for jobs), but more importantly, they gained transactionality: All-or-nothing writes for the models and the jobs, which is impossible using 2 separate data stores.

How Figma’s Databases Team Lived to Tell the Scale

The folks at Figma tell their battle story about scaling Postgres. Having a DB proxy that checks the queries and routes to the correct shard (and even aggregates among shards) is wild.

The use of logical partitioning to prove their point before doing actual physical partitioning seems very clever.

I spend a lot of time on the command line. My most frequently used commands are git (by far), and rails (mostly for rails test) or rspec (depending on the project). Most invocation of both commands require arguments. And most of those arguments are paths to files. In big projects, typing paths to files can be tedious, even with tab completion. My productivity trick for that is using Zsh Line Editor widgets.

Let’s see a widget in action:

The world is awful. The world is much better. The world can be much better.

Max Roser’s point of view resonated with me:

If we want more people to dedicate their energy and money to making the world a better place, then we should make it much more widely known that it is possible to make the world a better place.

For this we have to remember that all three statements are true at the same time: the world is awful, the world is much better, and the world can be much better.

Useful Uses of cat

I agree with the author: Using cat to start a pipelines is about composability of commands, and makes it easier to build pipelines in steps. Technically, you could be adding an extra process that you don’t need, but in day to day unix pipe operations, the performance is never an issue.

Data Looks Better Naked

Great illustration of how bad data visualization can be better, by removing design elements. My favorite is the pie chart.

I recently discovered Nushell. It’s billed as a “new kind of shell”. What caught my eye is not so much the actual shell usage: Executing commands, redirecting output, working with files, etc. I was interested in the concept of structured data:

Everything is data

Nu pipelines use structured data so you can safely select, filter, and sort the same way every time. Stop parsing strings and start solving problems.

Nushell comes with support for a variety of formats (e.g. JSON, YAML, csv):

> open package.json | get dependencies
╭──────────────────────────────────┬─────────────╮
│ @fontsource-variable/caveat      │ ^5.1.0      │
│ @fontsource-variable/raleway     │ ^5.1.0      │
│ @fortawesome/fontawesome-free    │ ^6.7.2      │
│ @hotwired/turbo-rails            │ ^8.0.12     │
│ @popperjs/core                   │ ^2.11.8     │
│ @rails/actioncable               │ ^8.0.100    │
│ @rails/activestorage             │ ^8.0.100    │
│ @rails/ujs                       │ ^7.1.3-4    │
│ @ttskch/select2-bootstrap4-theme │ ^1.5.2      │
│ autoprefixer                     │ ^10.4.20    │
│ bootstrap                        │ ^4.5.3      │
│ esbuild                          │ ^0.19.3     │
│ jquery                           │ 3.7.1       │
│ nodemon                          │ ^3.1.9      │
│ popper.js                        │ ^1.15.0     │
│ postcss                          │ ^8.4.30     │
│ postcss-cli                      │ ^10.1.0     │
│ sass                             │ ^1.67.0     │
│ select2                          │ ^4.1.0-rc.0 │
│ startbootstrap-sb-admin          │ ^6.0.2      │
╰──────────────────────────────────┴─────────────╯

Pipes can be built up:

> open package.json | get dependencies | transpose | where column0 =~ @rails
╭───┬──────────────────────┬──────────╮
│ # │       column0        │ column1  │
├───┼──────────────────────┼──────────┤
│ 0 │ @rails/actioncable   │ ^8.0.100 │
│ 1 │ @rails/activestorage │ ^8.0.100 │
│ 2 │ @rails/ujs           │ ^7.1.3-4 │
╰───┴──────────────────────┴──────────╯

Converting between formats is a breeze:

> open package.json | get dependencies | to yaml
'@fontsource-variable/caveat': ^5.1.0
'@fontsource-variable/raleway': ^5.1.0
'@fortawesome/fontawesome-free': ^6.7.2
'@hotwired/turbo-rails': ^8.0.12
'@popperjs/core': ^2.11.8
'@rails/actioncable': ^8.0.100
'@rails/activestorage': ^8.0.100
'@rails/ujs': ^7.1.3-4
'@ttskch/select2-bootstrap4-theme': ^1.5.2
autoprefixer: ^10.4.20
bootstrap: ^4.5.3
esbuild: ^0.19.3
jquery: 3.7.1
nodemon: ^3.1.9
popper.js: ^1.15.0
postcss: ^8.4.30
postcss-cli: ^10.1.0
sass: ^1.67.0
select2: ^4.1.0-rc.0
startbootstrap-sb-admin: ^6.0.2

And there is a built in explore TUI command that you can pipe data to (open file.csv | explore) to drill in and out of data interactively. Lovely.

I’m just getting started with Nushell, but already seems like a useful tool to have for data exploration and transformation.