Why I Recommend Against Using Cucumber

Around the time I first started using Rails in 2011, I noticed that a lot of developers, seemingly all Rails developers, were using Cucumber to assist with testing.

I bought into the idea—describing test cases in plain English—but in practice Cucumber not to be very valuable. In fact, my experience has been that Cucumber adds a negative amount of value.

In recent years I’ve noticed (although this is anecdotal and might just be my perception) that fewer codebases seem to use Cucumber and that fewer Rails developers seem to be on board with Cucumber. I had thought Cucumber was pretty much dead. But lately, to my surpise, I’ve seen Cucumber recommended to testing noobs more than a few times. Since I consider Cucumber to be a bad thing, I want to explain why I think so and why I don’t think other people should use it.

In my view there are two general ways Cucumber can be used: it can be used as intended or it can be abused. In the former case, I believe Cucumber has a small negative value. In the latter case I believe it has a large negative value.

Why Cucumber Sucks When It’s Not Used As Intended

Most production Cucumber scenarios I’ve seen look something like this:

These kinds of tests, with fine-grained steps, arise when the developers seem to mistake Cucumber for a way to write Ruby in English. The above scenario provides exactly zero benefit, in my opinion, over the following equivalent Capybara scenario:

The Cucumber/Gherkin version is no shorter nor more easily understandable.

To be fair to Cucumber, nobody who understands Cucumber advocates writing Cucumber scenarios in this way. The Cucumber creator himself, Aslak Hellesøy, wrote a post in 2011 saying not to do this. Other people have written similar things.

I think it’s telling that so many people have written blog posts advising against the very common practice of writing fine-grained Cucumber steps. To me it’s kind one of those gas station doors that looks for all the world like a pull door, so every single person who comes up to it pulls it instead of pushes it, feels like a dumbass, and then pushes it. So the gas station manager puts up a big sign that says “PUSH”, but most people don’t notice it and the problem persists. What instead should have been done is to make the push door look like a push door, without the big useless handle that you’re not supposed to pull. I get that the Cucumber maintainers tried to do that by removing web_steps.rb, but in my experience it didn’t seem to work.

And it doesn’t matter much anyway because Cucumber still sucks even if you don’t abuse it by writing fine-grained steps. I’ll explain why I think so.

Why Cucumber Sucks Even When It Is Used As Intended

Here’s a version of the above Cucumber scenario that’s done in the way the Cucumber creators would intend. There are two parts.

First, the Gherkin steps:

Second, the underlying Ruby steps:

This is actually pretty decent-looking and appealing, at least at first glance. There are two problems, though. First, this way of doing things doesn’t really provide any clarity over doing it the Capybara way. Second, the step definitions usually end up in a single, flat file full of “step soup” where unrelated steps are mixed together willy-nilly.

Compare this again with the Capybara version:

The sign in portion is usually abstracted away in Capybara, too, so the scenario would look more like this:

That’s not too crazy at all. In order for Cucumber to be a superior solution to using bare Capybara, it would have to have some pretty strong benefits to compensate for the maintenance burden and cognitive overhead it adds. But it doesn’t.

So what do I recommend doing instead of using Cucumber? I think just using Capybara by itself is fine, and better than using Capybara + Cucumber. I also think Capybara + page objects is a pretty good way to go.

12 thoughts on “Why I Recommend Against Using Cucumber

  1. Romeu Fonseca

    Cucumber is useful when the story carries all and only the business information, keeping the test implementation far from it.

    As an example, when a big refactor is necessary, is very easy to have rspec and capybara left with no business rules and a bunch of tests tied to the implementation as a dead weight.

    Reply
    1. sfcoding Post author

      I can see the value in that. On the other hand, as a principle, I never do big refactors. I only ever do refactoring a little at a time.

      Reply
  2. Konstantin Gredeskoul

    I completely agree with you, and have been pretty much ignoring cucumbers since 2012 (been doing ruby since 2005).

    For me cucumber is too much “magic”.

    I don’t care to look for regular expressions that map to cucumber steps. I’d rather see what is actually going on.

    I also am annoyed that newly generated Rails apps by default have Cucumber tests. Have you ever tried explaining how that works to someone new to Rails? It’s a disaster.

    Thanks!

    Reply
  3. Lucas Luitjes

    In my opinion there is one exception: when the Cucumber features double as technical documentation, for people who are not necessarily familiar with the implementation language/framework. In that case the benefit this pretty huge, because you have documentation that is, never, ever oudated.

    For example: for a front-end developer, well written cucumber features can be executable documentation for a REST API. I’ve seen that work rather effectively for SPA’s where backend and front-end development was done by separate teams. Simplified code example:

    Given my database contains following users:
    | id | username |
    | 1 | Moriarty |
    When I GET “/users/1” with content type “application/json”
    Then I expect the following JSON:
    “””json
    {
    “id”: 1,
    “username”: “Moriarty”
    }
    “””

    Another example: a CLI tool. In that case the Cucumber features are executable examples:

    Scenario: do not allow faulty arguments
    When I run my_cli_tool --faulty-args
    Then the exit status should be 1
    And stderr should contain:
    “””
    Please do not use –faulty-args. Instead, use –non-faulty-args. See full docs by running my_cli_tool -h
    “””
    And stdout should be empty

    But I agree if the only consumers of your scenarios, are people who can easily read Ruby/rspec, there’s not any added value.

    Reply
    1. Jason Swett Post author

      “But I agree if the only consumers of your scenarios, are people who can easily read Ruby/rspec, there’s not any added value.”

      I’ve never worked anywhere where non-Rubyists needed/wanted to read the specs. I’m certainly not saying that case doesn’t exist. I know it does. What I’ve seen more commonly though is developers assuming that non-technical people will want to read or write Gherkin, only to discover after a lot of work that that’s not the case.

      Reply
  4. Steven Webb

    I agree with with you Jason. Personally I find by using methods I can get very readable specs. E.g.,


    scenario "successfully updating profile details"
    FactoryBot.create(:user, email: 'test@example.com', password: 'mypassword')
    sign_in

    provide_name_and_age

    visit profile_path
    expect(page).to display_profile_details
    end

    def provide_name_and_age
    visit edit_user_path
    fill_in 'First Name', with: 'John'
    fill_in 'Last Name', with: 'Smith'
    fill_in 'Age', with: '30'
    click_on 'Save'
    end

    def display_profile_details
    have_content('John Smith, 30')
    end

    Reply
  5. Ronaldo

    I agree in part with Jason. In my case, the stories and BDD (feature files) are written by the system analysts that has the knowledge of business role. A feature is used to run end to end flows in many environments, just changing the values.
    Another think to consider is the reuse of the steps. If you build a good architecture to support a automation project, Cucumber will help you to document and separate the business roles of your code.
    Cucumber features is easily written by any analyst, not technical skills necessary also we can control the version of any changes on business roles.

    Reply
  6. Oli Sawtell

    Honestly, your missing the point, Cucumber and Gherkin syntax is used to ensure that everything is readable to everyone in the project – not JUST testers and developers. Now I have taught our clients how to write ‘cases’ using gherkin syntax, it is a simple matter for me to take what they make, create the steps in Ruby (I prefer Watir over Capybara – but the underlying framework isn’t relevant to this discussion) and fire off the steps. Asking a client to write a test case in Capybara? no chance!! Cucumber is about BDD and good comms.

    Reply
  7. Rich

    Generally agreeing with the “abuse” stuff in your article and also with the people who say that Gherkin is a great tool for communicating acceptance tests / specifications with non-technical customers – something I have to do all the time. They can sign off on the acceptance tests in Gherkin format, and can be used equally well for the automated tests (for my benefit) and the manual demo (for the client’s benefit).

    For the final point about “step soup” – I’m in total agreement. Thankfully, Spinach exists as an alternative Gherkin runner with scoped steps – if you want steps that are common to multiple features, you have to explicitly declare them as such and include them. It even has an audit command to check your step files are clean and complete.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *