Category Archives: Programming

Duplication

Duplication is an often-discussed topic in programming.

Sadly, much of the popular advice on duplication, such as the rule of three and the refrain duplication is cheaper than the wrong abstraction, treats duplication in an oversimplified way that doesn’t stand up to the nuanced reality of the issue.

In this post I’ll show what duplication is, why it’s such a surprisingly complicated issue, why the popular advice is dubious, and what can be done to address duplication.

We’ll start with a definition of duplication.

What duplication is

We could imagine that duplication could be defined as a piece of code that appears in two or more places. Indeed, this sounds like a very reasonable and accurate definition. But it’s actually wrong.

Here’s what duplication really is. Duplication is when there’s a single behavior that’s specified in two or more places.

Just because two identical pieces of code are present doesn’t necessarily mean duplication exists. And just because there are no two identical pieces of code present doesn’t mean there’s no duplication.

Two pieces of code could happen to be identical, but if they actually serve different purposes and lead separate lives, then they don’t represent the same behavior, and they don’t constitute duplication. To “DRY up” these identical-looking pieces of code would create new problems, like handcuffing two people together who need to walk in two different directions.

On the other hand, it’s possible for a single behavior to be represented in a codebase but with non-identical code. The way to tell if two pieces of code are duplicative isn’t to see if their code matches (although most of the time duplicative behavior and duplicative code do appear together.) The question that determines duplication is: if I changed one piece of code in order to meet a new requirement, would it be logically necessary to update the other piece of code the same way? If so, then the two pieces of code are probably duplicates of each other, even if their behavior is not achieved using the same exact syntax.

Why duplication is bad

The main reason duplication is bad is because it leaves a program susceptible to developing logical inconsistencies.

If a behavior is expressed in two different places in a program, and one of them accidentally doesn’t match the other, then the deviating behavior is necessarily wrong. (Or if the deviating happens to still meet its requirements, it only does so by accident.)

Another reason duplication can be bad is because it can pose an extra maintenance burden. It takes longer, and requires more mental energy, to apply a change to two areas of code instead of just one.

But not all instances of duplication are equally bad. Some kinds of duplication are more dangerous than others.

When duplication is more dangerous or less dangerous

There are three factors that determine the degree of harm of an instance of duplication: 1) how easily discoverable the duplication is, 2) how much extra overhead the presence of the duplication incurs, and 3) how much “traffic” that area receives, i.e. how frequently that area of code needs to be changed or understood. Let’s look at each of these factors more closely.

Discoverability

If there’s a piece of behavior that’s specified twice in the codebase, but the two pieces of code are only separated by one line, then there’s not a big problem because everyone is basically guaranteed to notice the problem. If someone updates one of the copies of the behavior to meet a new requirement, they’re very unlikely to accidentally miss updating the other one. You might call this the proximity factor.

If two pieces of duplicated behavior appear in different files in different areas of the application, then a “miss” is much more likely to occur, and therefore the duplication constitutes a worse problem.

Another quality that makes discovery of duplication easier is similitude. If two pieces of code look very similar, then their duplicity is more likely to be noticed than if the two pieces of code don’t look the same. You might call this the similitude factor.

If the proximity factor is bad (the pieces of duplicated code are at a great distance from each other) and/or if the similitude factor is bad (the duplication is obscured by the pieces of duplicated code not being similar enough to appear obviously duplicative) then it means the duplication is riskier.

Overhead

Some instances of duplication are easier to live with than others. Two short lines of very similar code, located right next to each other, are very easy to keep in sync with one another. Other types of duplication are much more deeply baked into the system and can cause a much bigger headache.

For example, if a piece of duplication exists as part of the database schema, that’s a much higher maintenance cost than a short code duplication. Instances of duplication that are big and aren’t represented by identical code can also be costly to maintain because, in those cases, you can’t just type the same thing twice, you have to perform a potentially expensive translation step in your head.

Traffic level

Duplication is a type of “bad code”, and so principles that apply to bad code apply to duplication as well. One of these principles is that bad code in heavily-trafficked areas costs more than bad code in lightly-trafficked areas.

When considering how much a piece of bad code costs, it’s worth considering when that cost is incurred. When a piece of bad code incurs a cost, we might think of this as analogous to paying a toll on a toll road.

One tollway is when a piece of code is changed. The more frequently the code is changed, the more of a toll it’s going to incur, and so the bigger a problem it is.

Another tollway is when a piece of code needs to be understood as a prerequisite to understanding a different piece of code. Every codebase has “leaf code” and “branch code”. If a piece of code is leaf code, as in nothing depends on it, then we can afford for that code to be pretty bad and it doesn’t matter. Branch code, on the other hand, gets heavy intellectual traffic, and so incurs a higher toll, and so is a bigger problem.

How to decide whether to DRY up a piece of code or to keep the duplication

The way to decide whether or not to DRY up a piece of duplication is pretty simple, although it’s not easy. There are two factors to consider.

Severity

If a piece of duplication is “severe”—i.e. it has low discoverability, poses high overhead, and/or has a high traffic level—it should probably be fixed. If not, it should probably be left alone.

Quality of alternative

Just because a piece of duplication costs something doesn’t automatically mean that the de-duplicated version costs less. It doesn’t happen very often, but sometimes a de-duplication unavoidably results in code that’s so generalized that it’s virtually impossible to understand. In these cases the duplicated version may be the lesser of two evils.

But be careful to make the distinction between “this code can’t be de-duplicated without making it worse” and “this particular attempt to de-duplicate this code made it worse”. Like all refactoring projects, sometimes you just need to try a few times before you land on something you’re happy with. And sometimes you just need to be careful not to go overboard.

Why the popular guidelines make little sense

It currently seems to be fashionable to hold the belief that developers apply DRY too eagerly. This hasn’t been my experience. The opposite has been my experience.

Claims that developers apply DRY too eagerly are often accompanied by advice to follow WET (“write everything twice”) or the “rule of three”, or “duplication is cheaper than the wrong abstraction”. Here’s why I think these popular guidelines make little sense.

Rule of three/”write everything twice”

Here’s my way of deciding whether to DRY up a duplication: Is the duplication very bad? Are we able to come up with a fix that’s better than the duplicated version and not worse? If so, then clean it up. If not, leave it alone.

Notice that my criteria do not include “does the duplication appear three times”? I can’t see how that could be among the most meaningful factors.

Imagine, for example, a piece of duplication in the form of three very simple and nearly-identical lines, grouped together in a single file. The file is an unimportant one which only gets touched a couple times a year, and no one needs to understand that piece of code as a prerequisite to understanding anything else.

Now imagine another piece of duplication. The duplication appears in only two places, but the places are distant from one another and therefore the duplication is hard to discover. The two places where the duplicated behavior appear are expressed differently enough that the code would elude detection by a code quality tool or a manual human search. The behavior is a vitally central and important one. It doesn’t get changed often enough that it stays at the top of everyone’s mind, but it gets changed often enough that there are lots of opportunities for divergences to arise. And the two places the behavior appears are brutally painful to keep in sync.

Given this scenario, why on earth would I choose to fix the triple-duplicate and leave the double-duplicate alone?

The rule of three and “write everything twice” (WET) make little sense. The number of times a piece of duplication appears is not the main factor in judging its harmfulness.

Duplication is cheaper than the wrong abstraction

This statement is repeated very frequently in the Ruby community, usually to discourage people from applying the DRY principle too eagerly.

I wish we would think about this statement more deeply. Why are we setting up such a strong a connection between duplication and abstractions? It strikes me as a non-sequitur.

And why are we imagining such a strong danger of creating the wrong abstraction? Do we not trust ourselves to DRY up a piece of code and end up with something good? And again, why does the result of our de-duplicating have to be an abstraction? I find it an illogical connection.

If we take out the word “abstraction” then the sentiment that remains is “duplicated code is better than a de-duplicated version that’s even worse”. In which case I of course agree, but the statement is so banal that it’s not even a statement worth making.

I think “duplication is cheaper than the wrong abstraction” is a statement devoid of any useful meaning, and one we should stop repeating.

How to fix instances of duplication

A duplication-removal project is just a special case of a refactoring project. (Remember, refactoring means “changing the structure of code without changing its behavior”). Any guidelines that apply to general refactoring projects also apply to de-duplication projects.

When de-duplicating, it helps to work in small, atomic units. If the refactoring was triggered by a need to make a behavior change, don’t mix the behavior change with the refactoring. Perform the refactoring either before implementing the change or after or both, not during. And when you reach the point when you’re no longer sure that your refactorings are an improvement, stop.

When I’m de-duplicating two pieces of code, it’s often not clear how the unification will be achieved. In these cases I like to make it my first step to make the duplicate pieces of code completely identical while still keeping them separate. Merging two subtly different pieces of code can be tricky but merging two identical pieces of code is trivial. So make them identical first, then merge.

You can find a lot of other great refactoring techniques in Martin Fowler’s book Refactoring: Improving the Design of Existing Code.

Takeaways

Duplication exists when there’s a single behavior that’s specified in two or more places.
The main reason duplication is bad is because it leaves a program susceptible to developing logical inconsistencies.
Not all instances of duplication are equally dangerous. The severity of a piece of duplication can be judged based on its discoverability, overhead cost and traffic level.
In order to decide whether an instance of duplication is worth fixing, consider the severity of the duplication. Also compare the duplicative code with the de-duplicated code, and only keep the “fixed” version if the fixed version is actually better.
The rule of three/”write everything twice” makes little sense because it doesn’t take the factors into account that determine whether a piece of duplication is dangerous or innocuous. “Duplication is the wrong abstraction” makes little sense because it sets up a false dichotomy between duplication and “the wrong abstraction”.
To get good at removing duplication, get good at refactoring.
When attempting to remove an instance of duplication, it’s often helpful to make the duplicative code completely identical as a first step, and then merge the identical code as a second step.

The difference between procs and lambdas in Ruby

5 Replies

Note: before starting this post, I recommend reading my other posts about procs and closures for background.

Overview

What’s the difference between a proc and a lambda?

Lambdas actually are procs. Lambdas are just a special kind of proc and they behave a little bit differently from regular procs. In this post we’ll discuss the two main ways in which lambdas differ from regular procs:

The return keyword behaves differently
Arguments are handled differently

Let’s take a look at each one of these differences in more detail.

The behavior of “return”

In lambdas, return means “exit from this lambda”. In regular procs, return means “exit from embracing method”.

Below is an example, pulled straight from the official Ruby docs, which illustrates this difference.

def test_return
  # This is a lambda. The "return" just exits
  # from the lambda, nothing more.
  -> { return 3 }.call

  # This is a regular proc. The "return" returns
  # from the method, meaning control never reaches
  # the final "return 5" line.
  proc { return 4 }.call

  return 5
end

test_return # => 4

Argument handling

Argument matching

A proc will happily execute a call with the wrong number of arguments. A lambda requires all arguments to be present.

> p = proc { |x, y| "x is #{x} and y is #{y}" }
> p.call(1)
 => "x is 1 and y is "
> p.call(1, 2, 3)
 => "x is 1 and y is 2"

> l = lambda { |x, y| "x is #{x} and y is #{y}" }
> l.call(1)
(irb):5:in `block in <main>': wrong number of arguments (given 1, expected 2) (ArgumentError)
> l.call(1, 2, 3)
(irb):14:in `block in <main>': wrong number of arguments (given 3, expected 2) (ArgumentError)

Array deconstruction

If you call a proc with an array instead of separate arguments, the array will get deconstructed, as if the array is preceded with a splat operator.

If you call a lambda with an array instead of separate arguments, the array will be interpreted as the first argument, and an ArgumentError will be raised because the second argument is missing.

> proc { |x, y| "x is #{x} and y is #{y}" }.call([1, 2])
 => "x is 1 and y is 2"

> lambda { |x, y| "x is #{x} and y is #{y}" }.call([1, 2])
(irb):9:in `block in <main>': wrong number of arguments (given 1, expected 2) (ArgumentError)

In other words, lambdas behave exactly like Ruby methods. Regular procs don’t.

Takeaways

In lambdas, return means “exit from this lambda”. In regular procs, return means “exit from embracing method”.
A regular proc will happily execute a call with the wrong number of arguments. A lambda requires all arguments to be present.
Regular procs deconstruct arrays in arguments. Lambdas don’t.
Lambdas behave exactly like methods. Regular procs behave differently.

Why global variables are bad

4 Replies

If you’ve been programming for any length of time, you’ve probably come across the advice “don’t use global variables”.

Why are global variables so often advised against?

The reason is that global variables make a program less understandable. When you’re looking at a piece of code that uses a global variable, you don’t know if you’re seeing the whole picture. The code isn’t self-contained. In order to understand your piece of code, you potentially have to venture to some outside place to have a look at some other code that’s influencing your code at a distance.

The key idea is scope. If a local variable is defined inside of a function, for example, then that variable’s scope is limited to that function. Nobody from outside that function can see or mess with that variable. As another example, if a private instance variable is defined for a class, then that variable’s scope is limited to that class, and nobody from outside that class can see or mess with the variable.

The broader a variable’s scope, the more code has to be brought into the picture in order to understand any of the code that involves that variable. If I have a function that depends on its own arguments and nothing else, then that function can be understood in isolation. All I need in order to understand the function (at least in terms of causes and effects, as opposed to conceptual understanding which may require outside context) is the code inside the function. If alternatively the function involves a class instance variable, for example, then I potentially need to look at the other places in the class that involve the instance variable in order to understand the behavior of the function.

The maximum scope a variable can have is global scope. In terms of understanding, a global variable presents the biggest burden and requires the most investigative work. That’s why global variables are so often cautioned against.

Having said that, it’s actually a little simplistic to say “global variables are bad”. It would be more precise to say “global variables are costly”. There are some scenarios where the cost of a global variable is worth the price. In those cases, the idea of a global variable could be said to be good because it’s less costly than the alternatives.

But in the vast majority of cases, it’s good to keep the scope of variables as small as possible. The smaller the scopes of your variables are, the more it will aid the understandability of your code.

When good code is important and when it’s not

3 Replies

Tollways

All code has a maintenance cost. Some code of course is an absolute nightmare to maintain. We would say its maintenance cost is high. Other code is easier to maintain. We would say its maintenance cost is low, or at least relatively low compared to worse code.

When thinking about good code and bad code, it’s worth considering when exactly code’s maintenance cost is incurred. We might refer to these events as “tollways”. We can’t travel these roads without paying a toll.

Tollway 1: when the code is changed

For any particular piece of code, a toll is incurred every time that code needs to be changed. The size of the toll depends on how easy the code is to understand and change.

Tollway 2: when the code needs to be understood in order to support a change in a different area

Even if a piece of code doesn’t need to be changed, the code incurs a toll whenever someone it needs to be understood in order to make a different change. This dependency of understanding happens when pieces of code are coupled via inheritance, passing values in methods, global variables, or any of the other ways that code can be coupled.

We could put code into two categories: “leaf code”, which depends on other code but has no dependencies itself, and “branch code”, which does have dependencies, and may or may not depend on other code. Branch code incurs tolls and leaf code doesn’t.

Good code matters in proportion to future tollway traffic

When any new piece of code is added to a codebase, it may be possible to predict the future “tollway traffic” of that code.

Every codebase has some areas that change more frequently than others. If the code you’re adding lies in a high-change area, then it’s probably safe to predict that that code will have high future tollway traffic. On average it’s a good investment to spend time making this code especially good because the upfront effort will get paid back a little bit every time the code gets touched in the future.

Conversely, if there’s a piece of code that you have good reason to believe will change infrequently, it’s less important to make this code good, because the return on investment won’t be as great. (If you find out that your original prediction was wrong, it may be wise to refactor the code so you don’t end up paying more in toll fees than you have to.)

If there’s a piece of code that’s very clearly “branch code” (other code depends on it) then it’s usually a good idea to spend extra time to make sure this code is easily understandable. Most codebases have a small handful of key models which are touched by a large amount of code in the codebase. If the branch code is sound, it’s a great benefit. If the branch code has problems (e.g. some fundamental concept was poorly-named early on) then those problems will stretch their tentacles throughout the codebase and cause very expensive problems.

On the other hand, if a piece of code can be safely identified as leaf code, then it’s not so important to worry about making that code super high-quality.

But in general, it’s hard to predict whether a piece of code will have high or low future tollway traffic, so it’s good to err on the side of assuming high future tollway traffic. Rarely do codebases suffer from the problem that the code is too good.

Bad reasons to write bad code

It’s commonly believed that it’s wise to take on “strategic technical debt” in order to meet deadlines. In theory this is a smart way to go, but in practice it’s always a farce. The debt gets incurred but then never paid back.

It’s also a mistake to write crappy code because “users don’t care about code”. Users obviously don’t literally care about code, but users do experience the symptoms of bad code when the product is full of bugs and the development team’s productivity slows to a crawl.

Takeaways

A piece of code incurs a “toll” when it gets changed or when it needs to be understood in order to support a change in a different piece of code.
The return on investment of making a piece of code good is proportionate to the future tollway traffic that code will receive.
Predicting future tollway traffic is not always easy or possible, but it’s not always impossible either. Being judicious about when to spend extra effort on code quality or to skip effort is more economical than indiscriminately writing “medium-quality” code throughout the entire codebase.

Why DSLs are a necessary part of learning Rails testing

Do I have to code in my free time in order to be a good programmer?

3 Replies

In programming interviews, job candidates are sometimes asked what kinds of side projects they work on in their spare time. The supposed implication is that if you work on side projects in your free time then that’s good, and if you don’t that’s bad.

This idea has led to a somewhat lively debate: do you have to code in your free time in order to be a good programmer?

The popular answer is an emphatic no. You can put in a solid 8-hour workday, do a kick-ass job, and then go home and relax knowing you’re fully fulfilling all of their professional obligations. And actually, you might even be a better programmer because you’re not running yourself ragged and burning yourself out.

But actually, both this question and the standard answer are misguided. In fact, they are miss the point so thoroughly that they can’t even be called wrong. I’ll explain what I mean.

Drumming in your free time

Imagine I’m a professional drummer. I make my living by hiring out my drumming services at bar shows, weddings and parties. I’m a very competent drummer although maybe not a particularly exceptional one.

Imagine how funny it would be for me to go on an online forum and ask, Do I have to practice drumming in my free time in order to be a good drummer?

I can imagine a couple inevitable responses. First of all, who’s this imaginary authority who’s going around and handing down judgments about who’s a good drummer or not? And second, yes, of course you have to spend some time practicing if you want to get good, especially when you’re first starting out.

The question reveals a very confused way of looking at the whole situation.

The reality is that there’s an economic judgement call to be made. Either I can choose to practice in my free time or get better faster, or I can choose not to practice in my free time and improve much more slowly, or perhaps even get worse. Neither choice is right or wrong. Neither choice automatically makes me “good” or “bad”. It’s simply I personal choice that I have to make for myself. The question is whether I personally find the benefits of practicing the drums to be worth the cost of practicing the drums.

An important factor that will inform my decision is the objectives that I’m personally pursuing. Am I, for example, trying to be the best drummer in New York City? Or do I just want to have a little fun on the weekends? The question is the same for everyone but the answer is going to be much different depending on what you want and what you’re willing to do to get it.

The drumming analogy makes it obvious how silly it is to ask directly if you spend your free time practicing. Maybe the person asking is trying to probe for “passion” (yuck). But passion is a means to an end, not an end in itself. Instead of looking for passion, the evaluator should look for the fruits of passion, i.e. being a good drummer.

Back to programming

Do your career goals, in combination with your current skill level, justify the extra cost of programming in your free time? If so, then coding in your free time is a rational choice. And if you decide that there are no factors in your life that make you want to code in your free time, then that’s a perfectly valid choice as well. There’s no right or wrong answer. You don’t “have to” or not have to. Rather, it’s a choice for each person to make for themselves.

Ruby memoization

6 Replies

What is memoization?

Memoization is a performance optimization technique.

The idea with memoization is: “When a method invokes an expensive operation, don’t perform that operation each time the method is called. Instead, just invoke the expensive operation once, remember the answer, and use that answer from now on each time the method is called.”

Below is an example that shows the benefit of memoization. The example is a class with two methods which both return the same result, but one is memoized and one is not.

The expensive operation in the example takes one second to run. As you can see from the benchmark I performed, the memoized method is dramatically more performant than the un-memoized one.

Running the un-memoized version 10 times takes 10 seconds (one second per run). Running the memoized version 10 times takes only just over one second. That’s because the first call takes one second but the calls after that take a negligibly small amount of time.

class Product
  # This method is NOT memoized. This method will invoke the
  # expensive operation every single time it's called.
  def price
    expensive_calculation
  end

  # This method IS memoized. It will invoke the expensive
  # operation the first time it's called but never again
  # after that.
  def memoized_price
    @memoized_price ||= expensive_calculation
  end
  
  def expensive_calculation
    sleep(1)
    500
  end
end

require "benchmark"

product = Product.new
puts Benchmark.measure { 10.times { product.price } }
puts Benchmark.measure { 10.times { product.memoized_price } }

$ ruby memoized.rb
  0.000318   0.000362   0.000680 ( 10.038078)
  0.000040   0.000049   0.000089 (  1.003962)

Why is memoization called memoization?

I’ve always thought memoization was an awkward term due to its similarity to “memorization”. The obscurity of the name bugged me a little so I decided to look up its etymology.

According to Wikipedia, “memoization” is derived from the Latin word “memorandum”, which means “to be remembered”. “Memo” is short for memorandum, hence “memoization”.

When to use memoization

The art of performance optimization is a bag of many tricks: query optimization, background processing, caching, lazy UI loading, and other techniques.

Memoization is one trick in this bag of tricks. You can recognize its use case when an expensive method is called repeatedly without a change in return value.

This is not to say that every time a case is encountered where an expensive method is called repeatedly without a change in return value that it’s automatically a good use case for memoization. Memoization (just like all performance techniques) is not without a cost, as we’ll see shortly. Memoization should only be used when the benefit exceeds the cost.

As with all performance techniques, memoization should only be used a) when you’re sure it’s needed and b) when you have a plan to measure the before/after performance effect. Otherwise what you’re doing is not performance optimization, you’re just randomly adding code (i.e. incurring costs) without knowing whether the costs you’re incurring are actually providing a benefit.

The costs of memoization

The main cost of memoization is that you risk introducing subtle bugs. Here are a couple examples of the kinds of bugs to which memoization is susceptible.

Instance confusion

Memoization works if and only if the return value will always be the same. Let’s say, for example, that you have a loop that makes use of an object which has a memoized method. Maybe this loop uses the same object instance in every single iteration, but you’re under the mistaken belief that a fresh instance is used for each iteration.

In this case the value from the object in the first iteration will be correct, but all the subsequent iterations risk being incorrect because they’ll use the value from the first iteration rather than getting their own fresh values.

If this type of bug sounds contrived, it’s not. It comes from a real example of a bug I once caused myself!

Nil return values

In the example above, if expensive_calculation had been nil, then the value wouldn’t get memoized because @memoized_price would be nil and nil is falsy.

The risk of such a bug is probably low, and the consequences of the bug are probably small in most cases, but it’s a good category of bug to be aware of. An alternative solution is to use defined? rather than lazy initialization, which is not susceptible to the nil-is-falsy bug.

Understandability

Last but certainly not least, code that involves memoization is harder to follow than code that doesn’t. This is probably the biggest cost of memoization. Code that’s hard to understand is hard to change. Code that’s hard to understand provides a place for bugs to hide.

Prudence pays off

Because memoization isn’t free, it’s not a good idea to habitually add memoization to methods as a default policy. Instead, add memoization on a case-by-case basis when it’s clearly justified.

Takeaways

Memoization is a performance optimization technique that prevents wasteful repeated calls to an expensive operation when the return value is the same each time.
Memoization should only be added when you’re sure it’s needed and you have a plan to verify the performance difference.
A good use case for memoization is when an expensive method is called repeatedly without a change in return value.
Memoization isn’t free. It carries with it the risk of subtle bugs. Therefore, don’t apply memoization indiscriminately. Only use it in cases where there’s a clear benefit.

The four phases of a test

1 Reply

When writing tests, or reading other people’s tests, it can be helpful to understand that tests are often structured in four distinct phases.

These phases are:

Setup
Exercise
Assertion
Teardown

Let’s illustrate these four phases using an example.

Test phase example

Let’s say we have an application that has a list of users that can receive messages. Only active users are allowed to receive messages. So, we need to assert that when a user is inactive, that user can’t receive messages.

Here’s how this test might go:

Create a User record (setup)
Set the user’s “active” status to false (exercise)
Assert that the user is not “messageable” (assertion)
Delete the User record we created in step 1 (teardown)

In parallel with this example, I’ll also use another example which is somewhat silly but also less abstract. Let’s imagine we’re designing a sharp-shooting robot that can fire a bow and accurately hit a target with an arrow. In order to test our robot’s design, we might:

Get a fresh prototype of the robot from the machine shop (setup)
Allow the robot to fire an arrow (exercise)
Look at the target to make sure it was hit by the arrow (assertion)
Return the prototype to the machine shop for disassembly (teardown)

Now let’s take a look at each step in more detail.

The purpose of each test phase

Setup

The setup phase typically creates all the data that’s needed in order for the test to operate. (There are other things that could conceivably happen during a setup phase but for our current purposes we can think of the setup phase’s role as being to put data in place.)In our case, the creation of the User record is all that’s involved in the setup step, although more complicated tests could of course create any number of database records and potentially establish relationships among them.

Exercise

The exercise phase walks through the motions of the feature we want to test. With our robot example, the exercise phase is when the robot fires the arrow. With our messaging example, the exercise phase is when the user gets put in an inactive state.

Side note: the distinction between setup and exercise may seem blurry, and indeed it sometimes is, especially in low-level tests like our current example. If someone were to argue that setting the user to inactive should actually be part of the setup, I’m not sure how I’d refute them. To help with the distinction in this case, imagine if we instead were writing an integration test that actually opened up a browser and simulated clicks. For this test, our setup would be the same (create a user record) but our exercise might be different. We might visit a settings page, uncheck an “active” checkbox, then save the form.

Assertion

The assertion phase is basically what all the other phases exist in support of. The assertion is the actual test part of the test, the thing that determines whether the test passes or fails.

Teardown

Each test needs to clean up after itself. If it didn’t, then each test would potentially pollute the world in which the test is running and affect the outcome of later tests, making the tests non-deterministic. We don’t want this. We want deterministic tests, i.e. tests that behave the same exact way every single time no matter what. The only thing that should make a test go from passing to failing or vice-versa is if the behavior that the test tests changes.

In reality, Rails tests tend not to have an explicit teardown step. The main pollutant we have to worry about with our tests is database data that gets left behind. RSpec is capable of taking care of this problem for us by running each test in a database transaction. The transaction starts before each test is run and aborts after the test finishes. So really, the data never gets permanently persisted in the first place. So although I’m mentioning the teardown step here for completeness’ sake, you’re unlikely to see it in the wild.

A concrete example

See if you can identify the phases in the following RSpec test.

RSpec.describe User do
  let!(:user) { User.create!(email: 'test@example.com') }

  describe '#messageable?' do
    context 'is inactive' do
      it 'is false' do
        user.update!(active: false)
        expect(user.messageable?).to be false
        user.destroy!
      end
    end
  end
end

Here’s my annotated version.

RSpec.describe User do
  let!(:user) { User.create!(email: 'test@example.com') } # setup

  describe '#messageable?' do
    context 'is inactive' do
      it 'is false' do
        user.update!(active: false)           # exercise
        expect(user.messageable?).to be false # assertion
        user.destroy!                         # teardown
      end
    end
  end
end

Takeaway

Being familiar with the four phases of a test can help you overcome the writer’s block that testers sometimes feel when staring at a blank editor. “Write the setup” is an easier job than “write the whole test”.

Understanding the four phases of a test can also help make it easier to parse the meaning of existing tests.

The Brainpower Conservation Principle

2 Replies

When asked to what he attributes his success in life, Winston Churchill purportedly said, “Economy of effort. Never stand up when you can sit down, and never sit down when you can lie down.”

My philosophy with programming is basically the same. Here’s why.

Finite mental energy

Sometimes people say they wish there were more hours in the day. I see it a little differently. It seems to me that the scarce resource isn’t time but energy. I personally run out of energy (or willpower or however you want to put it) well before I run out of time in the day.

Most days for me there comes a certain time where I’m basically spent for the day and I don’t have much more work in me. (When I say “work” I mean work of all kinds, not just “work work”.) Sometimes that used-up point comes before the end of the workday. Sometimes it comes after. But that point almost always arrives before I’m ready to go to bed.

The way I see it, I get a finite supply of mental energy in the morning. The harder I think during the day, the faster the energy gets depleted, and the sooner it runs out. It would really be a pity if I were to waste my mental energy on trivialities and run out of energy after 3 hours of working instead of 8 hours of working. So I try to conserve brainpower as much as possible.

The ways I conserve brainpower

Below are some examples of wasteful ways of working alongside the more economical version.

Wasteful way	Economical way
Keep all your to-dos in your head	Keep a written to-do list
Perform work in units of large, fuzzily-defined tasks	Perform work in units of small, crisply-defined tasks
Perform work (and deployments) in large batches	Perform work serially, deploying each small change as soon as it’s finished
Try to multitask or switch among tasks	Work on one thing at a time
Write large chunks of code at a time	Program in short feedback loops
Regularly allow yourself to slip into a state of chaos	Take measures to always keep yourself in a state of order
Perform tests manually, or don’t test at all	Write automated tests
Mix the jobs of deciding what to do with writing the code for doing it	First decide what to do (and write down that decision), then write the code to do it
Puzzle over an error message	Google the error message
Think hard in order to determine the cause of a bug	Systematically find the location of a bug
When a change goes wrong, try to identify and fix the root cause	Revert to the last known good state and start over
Keep a whole bunch of browser tabs open	Only keep two or three tabs open at a time

The Brainpower Conservation Principle

I would state the Brainpower Conservation Principle as follows:

Each person gets a limited amount of mental energy each day. Never expend more mental energy on a task than is needed.

Following this principle can help you code faster, longer, and more enjoyably.

How I make a Git commit

1 Reply

Many programmers make Git commits in a haphazard way that makes it easy to make mistakes and commit things they didn’t mean to.

Here’s a six-step process that I use every time I make a Git commit.

1. Make sure I have a clean working state

In Git terminology, “working state” refers to what you get when you run git status.

I always run git status before I start working on a feature. Otherwise I might start working, only to discover later that the work I’ve done is mixed in with some other, unrelated changes from earlier. Then I’ll have to fix my mistake. It’s cheaper just to check my working state in the beginning to make sure it’s clean.

2. Make the change

This step is of course actually the biggest, most complicated, and most time-consuming, but the content of this step is outside the scope of this post. What I will say is that I perform this step using feedback loops.

3. Run git status

When I think I’m finished with my change, I’ll run a git status. This will help me compare what I think I’m about to commit with what I’m actually about to commit. Those two things aren’t always the same thing.

4. Run git add .

Running git add . will stage all the current changes (including untracked files) to be committed.

5. Run git diff –staged

Running git diff --staged will show a line-by-line diff of everything that’s staged for commit. Just like the step where I ran git status, this step is to help compare what I think I’m about to commit with what I’m actually about to commit—this time at a line-by-line level rather than a file-by-file level.

6. Commit

Finally, I make the commit, using an appropriately descriptive commit message.

The reason I say appropriately descriptive commit message is because, in my opinion, different types of changes call for different types of commit messages. If the content of the commit makes the idea behind the commit blindingly obvious, then a vague commit message is totally fine. If the idea behind the commit can’t easily be inferred from the code that was changed, a more descriptive commit is called for. No sense in wasting brainpower on writing a highly descriptive commit message when none is called for.

Conclusion

By using this Git commit process, you can code faster and with fewer mistakes, while using up less brainpower.

The Beginner's Guideto Rails Testing

What duplication is

Why duplication is bad

When duplication is more dangerous or less dangerous

Discoverability

Overhead

Traffic level

How to decide whether to DRY up a piece of code or to keep the duplication

Severity

Quality of alternative

Why the popular guidelines make little sense

Rule of three/”write everything twice”

Duplication is cheaper than the wrong abstraction

How to fix instances of duplication

Takeaways

Overview

The behavior of “return”

Argument handling

Argument matching

Array deconstruction

Takeaways

Tollways

Tollway 1: when the code is changed

Tollway 2: when the code needs to be understood in order to support a change in a different area

Good code matters in proportion to future tollway traffic

Bad reasons to write bad code

Takeaways

Drumming in your free time

Back to programming

What is memoization?

Why is memoization called memoization?

When to use memoization

The costs of memoization

Instance confusion

Nil return values

Understandability

Prudence pays off

Takeaways

Test phase example

The purpose of each test phase

Setup

Exercise

Assertion

Teardown

A concrete example

Takeaway

Finite mental energy

The ways I conserve brainpower

The Brainpower Conservation Principle

1. Make sure I have a clean working state

2. Make the change

3. Run git status

4. Run git add .

5. Run git diff –staged

6. Commit

Conclusion

The Beginner's Guide
to Rails Testing