UPD: It was a ton of really heated discussions around this post. I want to emphasize: the text below is NOT a critique of MiniTest. It investigates difference of approaches to Ruby programming: “geniune Ruby” approach versus “Rails” approach. MiniTest vs RSpec is an illustration, not the body of this reasoning. Please, make your best to read it this way.

MiniTest recently have gained a lot of love and respect from Ruby community, as a “less magical” alternative to RSpec (which is, apparently, “too complicated” for nowadays rubyists). The one quote proudly cited on each MiniTest release email, and in its README, is

rspec is a testing DSL. minitest is ruby. (Adam Hawkins, “Bow Before MiniTest”)

(And the next slide explicitly says: “RSpec: less code & more magic; MiniTest: more code & less magic”.)

Let’s study this statement to understand what is good Ruby and what is Ruby way—and how it is different from Rails way.

Ways we solve tasks

Let’s forget for a moment about testing frameworks and their features and just solve the task from scratch:

We want to define “tests”. Each of them is set of ruby commands + phrase describing them. After definition, each of them would be performed, and descriptions used to print output.

For Rubyist accustomed to write idiomatic Ruby code, this requirements naturally lead to some mini-DSL:

define_test "This is the phase describing test" do
  # test's code itself
end

This is exactly what RSpec does: solves the task in most unambiguous, idiomatic and readable way possible:

it "performs some set of really complex actions" do
  # test's code
end

When you see it, you can immediatly have a good guess of what it does and how it is implemented, and you’ll be pretty close to reality.

But then, somebody comes and says “DSL’s are bad! It is like language inside language you need to study. It is Unholy Black Magic!” And comes with “pure-Ruby, no DSL” solution:

def test_performs_some_set_of_really_complex_actions
  # test's code
end

Or is it? Don’t you see DSL here too? But it is here. Let me show you. Slowly.

def  test_performs_some_set_of_really_complex_actions
      it "performs some set of really complex actions"
#     ^  |         ^
#     1  |         2

In both cases, you can’t write anything else instead of (1)—so, it is part of “test framework language”. And (2) is description you provide, it can be anything, it is a parameter.

So, what is the difference?

RSpec uses Ruby language structures to designate parts of “test definition language”, which helps to write, read and validate code. Try writing ti "performs something" and you’ll be stopped early by interpreter, even before tests would run. Want to write test which would be described like "checks 1:N validation (or similar)"? You just write it.

MiniTest pretends to be “just Ruby, nothing new!”, but it is the same DSL, just the fact is hidden from your eyes—with some really bad consequences. Try writing tset_peforms_something, it is still absolutely “valid” code, just the test would be ignored silently. Try to describe your test with a string that includes any symbol not allowed for method name, and you are screwed.

(For everyone yelling “but we have MiniSpec” at this moment—please, refer to discussion of this fact.)

Parting the ways

This, really simplistic, example, allows to define some important differences between two ways of writing in Ruby: let’s schematically call them “Ruby way” and “Rails way”:

  • “Ruby way” is relying on all language features and embracing them; as Matz said, “it is complex language for writing simple code”, you should understand Ruby to read and understand RSpec;
  • “Rails way” is using Ruby features underwater, while allowing developer to believe “he just writes classes and methods, and everything works”;
  • For making code clean, simple, and readable, “Ruby way” assumes library defines clever combinable atoms you can use in your code;
  • …while “Rails way” assumes library injects some new convention your code should confirm to (convention over configuration, you know?.. which, in fact, says, if you want to achieve something, you write kinda “independent” code, it should “just follow the convention”);
  • “Ruby way” praises method-with-block argument as one of the most powerful syntax constructs, it is one of the most basic code-as-data constructs (yet, I can confirm as a Ruby mentor, hard for newcomers);

  • “Rails way” neglects method-as-block argument approach, it is “magic” and “not generic enough”, I suppose? So, even when Railist really needs to pass arbitrary logic to method, he prefers to “go JavaScript”:

    scope :starts_after, ->(date) { where('start_date > ?', date) }
# this is SO MUCH BETTER RAILS CODE THAN BELOW, RIGHT? RIGHT?
scope(:starts_after){ |date| where('start_date > ?', date) }

On definition of “magic”

Ruby and Rails share the same part of bad reputation of “too much magic”, but recently this question (what is “magic”, and what amount of it, whatever it is, is “too much”) became the concern not only outside the community, but also inside. “MiniTest vs RSpec” debate can be seen as a part “right amount of magic” debate too.

Most of the recent in-community definitions of “too much magic” seem to focus on neglecting and disgusting natural block-based DSLs, while praising “just plain object and methods” approach.

Yet, what is “magic”, seriously? Magic is unexpected consequences of simple actions. You say “Avada Kedavra” while doing some gesture with a wooden stick, and Sirius Black dies—that’s magic. But if you pull a trigger, trigger causes chemical reaction, which pushes out the bullet, and then Sirius Black dies—it is not the magic, it is engineering, despite of the fact how “magical” it seems for unsuspecting barbarian.

So, let’s use the same simplistic yet real example:

it "is engineering, seriously" do
  expect(trigger.pull).to eq("fire")
end

What and where is here?.. it(description) { ... } is testing construction, which is simple and readable: method, accepting a string and a block; probably it adds test to internal list of tests to do (yep, it does).

expect(whatever).to probably wraps value into something that is easy to test with other tests (exactly what it does). And eq produces some object/matcher?..

I’d suppose it is something defining === generic test method + some additional services to output pretty strings when test is failed? (correct)

Now, let’s go to that “just plain Ruby, no DSL” thing:

def test_if_it_is_engineering
  assert_equal trigger.pull, "fire"
end

What do you think when you see a method named this way?.. It is just a method definition.

Ah! If you are inside MiniTest context, there is a magic convention for this kind of method to have a special meaning (because it starts with test_, you can’t guess from code itself it is so special, PURE MAGIC).

Then, this assert_equal thing… It is not THAT worse than expect().to eq, I just never able to properly remember what is “expected” and what is “actual” part of those two arguments… But yeah, cool, there is “no magic!”. Unless you want to define your own assertion/matcher with pretty messages, which could lead you to real Narnia in MiniTest case.

You can’t beat them, you lead them

When MiniTest founds itself significantly less expressive than RSpec, it introduces “minitest/spec”, which looks exactly like RSpec, but is better somehow. How?.. Ah, it is still “good clean less-magic MiniTest!” And it is less magic how exactly?..

Let’s investigate how the overall structure works

class Foo
end

# that's rspec
describe Foo do
  p [self, self.name]
  # => [RSpec::ExampleGroups::Foo, "RSpec::ExampleGroups::Foo"]
  # it is just a class, with pretty explainable name
end

# and that's minitest/spec
describe Foo do
  p [self, self.name]
  # => [#<Class:0x991a31c>, "Foo"]
  # ok, guessing by name, it is original Foo class that all tests are injected are?..
  p self == Foo
  # => false
  # nope.
  # phew, what kind of world is it?..
  p self.method(:name).source_location
  # => ["/media/storage/home/zverok/.rvm/gems/ruby-2.2.4/gems/minitest-5.9.1/lib/minitest/spec.rb", 272]
  # Ah, ok. It is just your usual "just plain Ruby, no magic" thing, that
  # pretends current class is named the way it is NOT named in fact.
end

Here, I’ve managed to formulate MiniTest/spec motto for you:

We don’t always DSL, but when we do, it is magic!

Let’s look at matchers, probably?..

# RSpec, 2016
expect(page).to have_content("bar")
# -> no pollution of tested class, just a DSL of "expect + matcher"
#    `expect().to` part is default; `have_content` is matcher's name,
#    in any "how to create matcher" guide you know how it is defined

# MiniTest/Spec, 2016
page.must_have_content 'bar'
# -> looks familiar, right? really like RSpec 2+, which polluted everything
#    with its methods... and was retired 2.5 years ago

Whoops…

The answer is simple: “Rails way” (which MiniTest closely follows) is “let’s not be replaced with superceding technology, let’s imitate it while stating we are superior”. While this imitation can’t follow the original too close, MiniTest’s imitation of RSpec feature will always be in “chasing the leader” state.

And, truth to be told, minitest also recommends this Really Cool Replacement, Minitest::MatchersVaccine, the name is explained as “Adds matcher support to minitest without all the other RSpec-style expectation infections.” This gem last updated May 31, 2016… you know, only 2 years after RSpec 3 introduced and promoted non-“infecting” matchers… which was three monthes before “MatchersVaccine” project have ever started.

Yet “MatchersVaccine” fights, somehow, not with “MiniTest implementation of matchers”, but with “RSpec approach to matchers”, which does NOT even here currently; but it easier to fight dehumanized “enemy” (Bad RSpec Guys) than friends (Those MiniTest Guys, who intorduced their faulty matchers).

That’s just Rails way

One of MiniTest-praise definitive article you can easily google around, states, alongside other things, as a separate important section:

7. Deviating from Rails defaults doesn’t always provide value

Yeah, that’s the (Ruby) world we are living in: “what is good enough for Rails, should be good enough for Ruby”. But, in fact, this is just plain wrong.

There are two separate ways, and the fact that Rails are written in Ruby, is, to be honest, unfortunate incident. Ruby still can and should praise Rails for its current popularity, but the ways just are and should be different.