Author Archives: Jason Swett

When I do TDD and when I don’t

9 Replies

Some developers advocate doing test-driven development 100% of the time. Other developers think TDD is for the birds and don’t do it at all. Still other developers go in the middle and practice TDD more than 0% of the time but less than 100% of the time.

I personally am in the camp of practicing TDD some of the time but not all. Here’s my reasoning.

When TDD makes sense to me

It’s not the case that I use TDD, or even write tests at all, for every single project I work on. But I do pretty much always program in feedback loops.

Feedback loops

The “feedback loop method” is where work as follows. First, I think of a tiny goal that I want to accomplish (e.g. make “hello world” appear on the screen). Then I decide on a manual test I can perform in order to see if that goal is accomplished (e.g. refresh the page and observe). Then I perform the test, write some code to try to make the test pass, perform the test again, and repeat the process with a new goal.

TDD == automated feedback loops

The way I view TDD is that it’s just the automated version of the manual work I was going to do anyway. Instead of making a to-do note that says “make ‘hello world’ appear on the screen” and then manually refreshing the page to see if it’s there, I write a test that expects “hello world” to appear on the screen. All the other steps are the exact same.

I’ve found that TDD works great for me when I’m working on what you might call “crisply-defined” work. In other words, the requirements I’m working to fulfill are known and specified. I find that I’ve found that there are other scenarios where TDD doesn’t work so great for me.

When TDD doesn’t make sense to me

Coding as production vs. coding as thinking

It’s easy to think that the reason to write code is to create a work product. But that’s certainly not the only reason to write code. Code isn’t just a medium for producing a product. It’s also a medium for thinking.

This is the idea behind a “spike” in Agile programming. When you’re doing a spike, you have no necessary intention to actually keep any of the code you’re writing. You’re just exploring. You’re seeing what it looks like when you do this or how it feels when you do that.

You can think of coding kind of like playing a piano. Sometimes you have some pieces of music already in your head and you’re trying to record an album. Other times you’re just messing around to see you can come up with any music worth recording. These are two very different modes of engaging with your instrument. Both are very necessary in order to ultimately record some music.

TDD doesn’t mix great with spikes

I often find that a spike phase is necessary when I’m coding, for example, a feature with known big-picture requirements but unknown UI specifics. In that case my test would be so full of guesses and placeholders that it would be kind of a joke of a test, and it wouldn’t help me much. In these cases I give myself permission to forego the testing during the spike period. I come back after I have some working code and backfill the tests.

Takeaways

  • I don’t practice TDD 100% of the time. (I believe I do practice TDD the vast majority of the time though.)
  • I view TDD as the automated version of the coding workflow that I already use anyway.
  • Producing a work product is not the only reason to write code. Code can also be a medium for thinking.
  • When I’m in the mode of using coding as a way to think, I find that the benefits of TDD don’t really apply.

How to apply a bugfix

Leave a reply

Any bugfix job has three distinct stages: reproduction, diagnosis, and fix. This post is about the third phase, the fix.

It might seem like the “fix” phase of a bugfix is simple: just fix it. But it’s not that simple. Many people screw it up. I certainly have.

They key thing with a bugfix is that you have to have a way to be sure your bugfix actually fixed the bug. So before you apply a bugfix, devise a test (which can be a manual test or an automated test) which has the following qualities:

  • The test fails when the bug is present
  • The test passes when the bug is absent

And, crucially, perform the test both before and after the fix has been applied. If you don’t perform the test before the fix is applied, you don’t know if the test is passing afterward because your fix worked or because your test gives a false positive.

And not only is it important to make sure your test fails when the bug is present, but that it fails in the precise way you expect. If the test fails in a way other than what you expect, then it might be failing for a reason that’s irrelevant to the bug, and you can’t be sure that your test is a valid one.

By following this process, you can have a higher chance of fixing your bug just once rather than having one of more false fixes before you finally succeed.

Don’t mix jobs

Leave a reply

Many programmers spend most of their working time in a very wasteful manner.

One of the biggest sources of waste is the mistake of mixing jobs. In this post I’ll explain what I mean by mixing jobs, why it’s bad, and how to avoid it.

Mixing the “what” with the “how”

When you write a line of code, you’re actually carrying out two jobs. One job is to decide what you want the line of code to do. The other job is to figure out the exact syntax to accomplish your goal. You could call these two jobs “what to do” and “how to do it”.

Mixing a “main job” with a “side job”

Another form of mixing jobs is to start working on a certain job, then notice some semi-relevant problem or opportunity along the way, and immediately start addressing the new job in a way that’s not separate from the original job.

Why mixing jobs is bad

Almost everything we do in programming is to accommodate the fact that human brains are weak and frail in the face of the staggeringly complex task of building and maintaining software systems. If humans were infinitely smart, we could just write everything in assembly code directly on production servers. But we aren’t, so we need accommodations.

One of these accommodations is to carry out our work serially. It’s much easier to work on exactly one thing at a time, and bring it fully to completion before starting the next task, than to juggle multiple tasks at once.

Working serially might intuitively seem slower. It feels like it would be more efficient to batch things up. But humans are so susceptible to mistakes and oversights and confusions and such, and software systems are so complicated, that batching work costs more time than it saves.

Tips for not mixing jobs

When you’re working on something, always be conscious of the answer to the question “What job am I working on right now? What exactly am I trying to accomplish?” Once you’ve determined that, ask yourself, “Is this actually just one job or are there multiple jobs inside it?” If it makes sense, separate the “what” jobs from the “how” jobs.

It may take some experimentation to find the right granularity. I wouldn’t advocate planning the entire “what” for a program (e.g. writing out the whole thing in pseudocode) before starting any of the “how”. I personally like to decide a few steps’ worth of “what”, then figure out the “how”, then switch back to “what”, and repeat.

Avoiding mixing a main job with a side job is conceptually pretty easy, although it might take some discipline. When you’re working on a job and you notice some other job that needs doing, don’t just switch focus and do that other job. Instead, make a note to do that other job. I personally like to keep a daily to-do list where I keep track of such things. Before the day ends I’ll usually either complete those “side jobs” or, if the side jobs are big enough, I’ll capture them in a way that they can be remembered for later when they can be given the level of care and attention that they call for.

Takeaways

  • Don’t mix the “what” with the “how”.
  • Don’t mix a “main job” with a “side job”.
  • It’s generally faster to carry out work serially than to try to do it in batches or in parallel.

If you want to learn testing, first learn how to program in feedback loops

Leave a reply

Most beginner programmers (and even many experienced programmers) take a slow, painful, wasteful approach to programming.

The wasteful way

The way many programmers code is to spend a bunch of time writing code without checking to see that it works, then finally run the program once they’ve accumulated many lines of code. The program inevitably fails.

Next, the programmer sits and puzzles over what might have gone wrong. Since the programmer wrote a lot of code without checking it, there’s a lot of stuff that could possibly be the culprit, and therefore the debugging process is slow and painful.

The debugging process is usually not a systematic one but rather a guessing game. “Maybe it’s this. Nope it’s not that. Maybe it’s this other thing. Nope, it’s not that either. Hmm.” As the clock ticks, frustration mounts, and maybe a little desperation sets in. It’s not fast and it’s not fun.

The smarter way: feedback loops

Instead of working in the slow, painful, wasteful way described above, you can work in feedback loops. As I described in my other post about feedback loops, the feedback loop process goes like this:

  1. Decide what you want to accomplish
  2. Devise a manual test you can perform to see if #1 is done
  3. Perform the test from step 2
  4. Write a line of code
  5. Repeat test from step 2 until it passes
  6. Repeat from step 1 with a new goal

When you use the feedback loop method, it’s hard to run too far astray. If you only write a little bit of code at a time and you keep everything working at all times, then you’re guaranteed to always have a program that’s either fully working or very close to fully working.

Feedback loops and automated testing

Automated testing is just the practice of coding using feedback loops, but with the testing step automated.

Here’s how the feedback loop would go with automated tests involved. The automated test parts are included in bold.

  1. Decide what you want to accomplish
  2. Devise a manual test you can perform to see if #1 is done (write a test)
  3. Perform the test from step 2 (run the test)
  4. Write a line of code
  5. Repeat test from step 2 until it passes (run the test again)
  6. Repeat from step 1 with a new goal

Obviously there’s also a lot of technical knowledge that’s needed in order to write automated tests. For example, there are test frameworks that enable automated testing, there are libraries that help your tests interact with a browser, and there are libraries that help with generating test data. But more important than any particular tool are the principles behind automated testing.

Perhaps the most important idea behind automated testing is the feedback loop. And luckily for you if you’re a beginner, you can learn how to program in feedback loops without having to learn anything to do with automated testing yet. And once you do, writing automated tests will feel much more natural.

Rails can only take you so far

Leave a reply

Applications tend to grow over time. In the beginning of an application’s life, things are typically pretty easy. Rails provides “slots” for us to put our code in. Display-related code goes in the views. Domain logic and persistence-related logic go in the models. The code that connects the two goes in the controllers.

But over time the slots fill up. Models get bloated to hundreds or even thousands of lines. Models become miscellaneous grab bags of unrelated methods. Controllers get cluttered with custom actions and grow large and hard to understand. Entropy takes over and things become a mess.

At this point, many developers sadly head down a bad path. They observe that, early on, Rails helped them with everything including code organization, and so now that they need more help, they go looking for “more Rails”. They’re run out of framework and now they want more framework. They look for tools and libraries to help them organize their code. But what’s needed isn’t tools and libraries. What’s needed are code organization skills. And tools and libraries can’t be a replacement for skills.

Frameworks are really good for certain things. They’re good for abstracting away repetitive or low-level work and giving developers leverage. Frameworks can also be good for imposing structure on a codebase, but only within narrow limits. Before long, simple heuristics (e.g. “put this type of code in models, this type of code in controllers,” etc.) cease to be enough. At that point an actual human has to take the reins from the framework and apply judgment. Non-trivial codebases are too large to be kept tidy by the simplistic heuristics that frameworks are able to provide.

If “more framework” and libraries and gems aren’t the answer, what is the answer, exactly? That’s basically the same question as “how do I write good code?” People have of course written many books attempting to help answer that question. There’s a lot to it. Learning the answer takes years.

To me, writing good code has to do with choosing clear names for variables, classes and methods; conceiving of clear and useful models and abstractions; refactoring regularly in order to keep entropy under control and keep the code understandable; and using automated tests to ensure that refactoring is possible. The best single resource I can recommend to cover these topics in a well-rounded way is Code Complete by Steve McConnell.

Regarding structure, I personally use a lot of namespaces and POROs and my Rails apps. The namespaces help me keep related things next to each other and signal to me, when I’m trying to understand a piece of code, what surrounding code is relevant and what I can ignore. Organizing my model behavior into POROs helps me avoid bloated Active Record models. When I notice that an Active Record model is starting to lose cohesion, I’ll often look in my model for “missing abstractions” and split those missing abstractions off into new POROs. The resulting structure has very little to do with Rails. I’m mainly just leaning on the principles of object-oriented programming.

I think Rails is the greatest web framework ever created. The things Rails helps with, it does a fantastic job of helping with, including helping to organize code during the early stages of an application’s life. But once your application grows beyond a certain size, Rails can no longer help you much and you’re on your own to decide how to keep your code organized.

How and why to use Ruby’s method_missing

7 Replies

Why method_missing exists

Normally, an object only responds to messages that match the names of the object’s methods and public accessors. For example, if I send the message first_name to an instance of User, then the User object will only respond to my message of first_name if User has a method or accessor called first_name.

But sometimes it’s useful to allow objects to respond to messages that don’t correspond to methods or accessors.

For example, let’s say we want to connect our User object to a database table. It would be very convenient if we could send messages to instances of User that correspond to the database table’s column names without having to either explicitly define new methods or do something inelegant like, for example, user.value(:first_name). It would be better if we could get the database value by calling user.first_name.

What’s more, if we added a new column called last_name, it would be good if we could just do user.last_name without having to change any code.

method_missing allows us to do things like this. In this post we’ll see how by going through an example that’s similar to (but simpler than) the database example above.

Arbitrary attribute setter example

In the below example, we’ll use method_missing to define some behavior that allows us to arbitrarily set values on an object. We’ll have an object called user on which we can call set_first_name, set_last_name, set_height_in_millimeters or whatever other arbitrary values we want.

A plain User object

In the following snippet, we define a class called User which is completely empty. We attempt to call set_first_name on a User instance which, of course, fails because User has no method called set_first_name.

# user.rb

class User
end

user = User.new
user.set_first_name("Jason")

When we run the above, we get undefined method `set_first_name' for #<User:0x00000001520e01c0> (NoMethodError).

$ ruby user.rb
Traceback (most recent call last):
user.rb:9:in `<main>': undefined method `set_first_name' for #<User:0x00000001520e01c0> (NoMethodError)

Adding method_missing

Now we add a method to the User class with a special name: method_missing.

In order for our method_missing implementation to work it has to follow a certain function signature. The first parameter, method_name, corresponds to the name of the message that was passed (e.g. first_name). The second parameter, *args corresponds to any arguments that were passed, and comes through as an array, thanks to the splat operator.

In this snippet all we’ll do is output the values of method_name and *args so we can begin to get a feel for how method_missing works.

class User
  def method_missing(method_name, *args)
    puts method_name
    puts args
  end
end

user = User.new
user.set_first_name("Jason")

When we run this we see set_first_name as the value for method_name and Jason as the value for args.

$ ruby user.rb
set_first_name
Jason

Parsing the attribute name

Now let’s parse the attribute name. When we pass set_first_name, for example, we want to parse the attribute name of first_name. This can be done by grabbing a substring that excludes the first four characters of the method name.

class User
  def method_missing(method_name, value)
    attr_name = method_name.to_s[4..]
    puts attr_name
  end
end

user = User.new
user.set_first_name("Jason")

This indeed gives us just first_name.

$ ruby user.rb 
first_name

Setting the attribute

Now let’s set the actual attribute. Remember that args will come through as an array. (The reason that args is an array is because whatever method is called might be passed multiple arguments, not just one argument like we’re doing in this example.) We’re interested only in the first element of args because user.set_first_name("Jason") only passes one argument.

class User
  def method_missing(method_name, *args)
    attr_name = method_name.to_s[4..]
    instance_variable_set("@#{attr_name}", args[0])
  end
end

user = User.new
user.set_first_name("Jason")
puts user.instance_variable_get("@first_name")

When we run this it gives us the value we passed it, Jason.

$ ruby user.rb
Jason

We can also set and get any other attributes we want.

class User
  def method_missing(method_name, *args)
    attr_name = method_name.to_s[4..]
    instance_variable_set("@#{attr_name}", args[0])
  end
end

user = User.new
user.set_first_name("Jason")
user.set_last_name("Swett")
puts user.instance_variable_get("@first_name")
puts user.instance_variable_get("@last_name")

When we run this we can see that both values have been set.

$ ruby user.rb
Jason
Swett

A note about blocks

In other examples you may see the method signature of method_missing shown like this:

def method_missing(method_name, *args, &block)

method_missing can take a block as an argument, but actually, so can any Ruby method. I chose not to cover blocks in this post because the way method_missing‘s blocks work is the same as the way blocks work in any other method, and a block example might confuse beginners. If you’d like to understand blocks more in-depth, I’d recommend my other post about blocks.

Takeaways

  • method_missing can be useful for constructing DSLs.
  • method_missing can be added to any object to endow that object with special behavior when the object gets sent a message for which it doesn’t have a method defined.
  • method_missing takes the name of the method that was called, an arbitrary number of arguments, and (optionally) a block.

“Am I smart enough for programming?”

4 Replies

Sometimes on programming forums someone asks a question along the lines of, “Am I smart enough for programming?”

The bad answer

Typically the answers to this question are some variation of “Yes! Keep going! You can do it!”

These answers are well-intentioned but not honest. The answerer has no idea who the asker is or how smart they are. That’s like someone asking, “Am I tall enough to ride a rollercoaster?” and people answering, “Yes! You can do it! Just believe in yourself!” It’s not necessarily true, obviously. And it’s not helpful.

The good answer

When someone asks if they’re smart enough for programming, what they’re probably experiencing is that they’re spending a lot of time and effort trying to learn programming and not feeling like their progress is proportionate to their expenditure. They figure that a likely explanation for their difficulty is that they’re not smart enough.

These people are probably reasoning that you have to have some minimum level of smartness in order to be able to learn programming, and that not everyone meets that minimum level of smartness, and if they aren’t smart enough then they shouldn’t waste their time on trying to achieve the unachievable.

For anyone wondering asking this question of themselves, I would point out that “Am I smart enough for programming?” is a really vague question. It’s like asking, “Am I tall enough for basketball?” Being tall obviously helps in basketball but it’s not like there’s a hard line at six feet and you should just give up if you’re less than six feet tall.

If you want to be a programmer, being smart helps, just like being tall helps you in basketball. But it’s not the only factor. It’s not as though the minimum IQ for programming is 120 and if your IQ is below 120 you should give up.

Everyone who wants to become a competent programmer has to learn a million little details. It’s easy to mistake “I’m having a hard time learning all these details” for “I’m not smart enough to grasp the material”. When I first started learning to speak Chinese, I was amazed at how inept I was and how long it took me to learn anything. When I would try to speak Chinese with my Chinese friend who was helping me, I would understand such a comically tiny amount of what she said that the whole exercise felt like a waste of time to me. But the reason for the difficulty wasn’t because I’m stupid, it’s just that Chinese is very different from English and there’s a lot of stuff to learn.

I’ll add yet one more point. You don’t actually have to be that smart to be a successful programmer. I’ve met plenty of programmers who are pretty dull and can’t code their way out of a paper bag. Yet, somehow, they maintain programming jobs. (Some of these dummies have even been my past bosses!) So if you’re worried that you don’t meet the minimum, you might not need to worry. The bar isn’t that high.

Takeaways

  • Asking if you’re smart enough for programming is like asking if you’re tall enough for basketball. It’s not really a question that makes sense.
  • No one is going to tell you that you’re too dumb for programming and you should quit. There’s no test you can take to find out if you’re smart enough. It’s just not really knowable.
  • Learning programming involves learning a million little details. Don’t confuse the the labor of learning with a lack of intelligence.
  • The minimum necessary level of intelligence in order to become a successful programmer is actually pretty low.

How Ruby’s instance_exec works

12 Replies

In this post we’ll take a look at Ruby’s instance_exec, a method which can can change the execution context of a block and help make DSL syntax less noisy.

Passing arguments to blocks

When calling a Ruby block using block.call, you can pass an argument (e.g. block.call("hello") and the argument will be fed to the block.

Here’s an example of passing an argument when calling a block.

def word_fiddler(&block)
  block.call("hello")
end

word_fiddler do |word|
  puts word.upcase
end

In this case, the string "hello" gets passed for word, and word.upcase outputs HELLO.

We can also do something different and perhaps rather strange-seeming. We can use a method called instance_exec to execute our block in the context of whatever argument we send it.

Executing a block in a different context

Note how in the following example word.upcase has changed to just upcase.

def word_fiddler(&block)
  "hello".instance_exec(&block)
end

word_fiddler do
  puts upcase
end

The behavior is the exact same. The output is identical. The only difference is how the behavior is expressed in the code.

How this works

Every command in Ruby operates in a context. Every context is an object. The default context is an object called main, which you can demonstrate by opening a Ruby console and typing self.

We can also demonstrate this for our earlier word_fiddler snippet.

def word_fiddler(&block)
  block.call("hello")
end

word_fiddler do |word|
  puts self # shows the current context
  puts word.upcase
end

If you run the above snippet, you’ll see the following output:

main
HELLO

The instance_exec method works because in changes the context of the block it invokes. Here’s our instance_exec snippet with a puts self line added.

def word_fiddler(&block)
  "hello".instance_exec(&block)
end

word_fiddler do
  puts self
  puts upcase
end

Instead of main, we now get hello.

hello
HELLO

Why instance_exec is useful

instance_exec can help make Ruby DSLs less verbose.

Consider the following Factory Bot snippet:

FactoryBot.define do
  factory :user do
    first_name { 'John' }
    last_name { 'Smith' }
  end
end

The code above consists of two blocks, one nested inside the other. There are three methods called in the snippet, or more precisely, there are three messages being sent: factory, first_name and last_name.

Who is the recipient of these messages? In other words, in what contexts are these two blocks being called?

It’s not the default context, main. The outer block is operating in the context of an instance of a class called FactoryBot::Syntax::Default::DSL, which is defined by the Factory Bot gem. This means that the factory message is getting sent to an instance of FactoryBot::Syntax::Default::DSL.

The inner block is operating in the context of a different object, an instance of FactoryBot::Declaration::Implicit. The first_name and last_name messages are getting sent to this class.

You can perhaps imagine what the Factory Bot syntax would have to look like if it were not possible to change blocks’ contexts using instance_exec. The syntax would be pretty verbose and noisy.

Takeaways

  • instance_exec is a method that executes a block in the context of a certain object.
  • instance_exec can help make DSL syntax less noisy and verbose.
  • Methods like Factory Bot’s factory and RSpec’s it and describe are possible because of instance_exec.

How to save time and brainpower by dividing the bugfix process into three distinct steps

Leave a reply

The reason for dividing bugfix work into steps

I don’t like to waste time when I’m working. And there’s hardly a surer way to waste time than to be sitting in front of the computer not even knowing what piece of work I’m trying to accomplish.

I also don’t like to think any harder than I need to. Brainpower is a limited resource. I only get so much in one day. If I spend a lot of brainpower on a job that could have been done just as well with less brainpower, then I’ve wasted brainpower and I will have cheated myself.

Dividing a bugfix job into three distinct steps helps me not to violate either of these principles. When I’m working on reproducing the bug, I can focus on reproduction, to the exclusion of all else. This helps me stay focused, which helps me work quickly. Focusing on one step also helps cut down on mental juggling. If I decide that I’m only thinking about reproducing the bug right now and not trying to think about diagnosing the bug or fixing it, then that limits the amount of stuff I have to think about, which makes me a clearer, more efficient and faster thinker.

Here are the three steps I carry out when fixing a bug as well as some tips of carrying them out effectively and efficiently.

Reproduction

When I become aware of the purported existence of a new bug, the first step I usually take is to try to reproduce the bug.

A key word here is “purported”. When a bug is brought to my attention, I don’t immediately form the belief that a bug exists. I can’t yet know for sure that a bug exists. All I can know for sure at that point is that someone told me that a bug exists. More than zero percent of the time, the problem is something other than a bug. Perhaps the problem was, for example, that the behavior of the system was inconsistent with the user’s expectations, but it was the user’s expectations that were wrong, not the behavior of the system.

Once I’m able to reproduce the bug manually, I often like to capture the bug reproduction in an automated test. The reason I like to do this is because it cuts down on mental juggling. Once the reproduction is captured in a test, I can safely unload the reproduction steps from my mind, freeing up mental RAM. I can now put all of my mental RAM toward the next debugging step, diagnosis.

Diagnosis

Many developers take a haphazard approach to debugging. They sit and stare at the code and try to reason through what might be going wrong. This works sometimes, but most of the time it’s a very painful and inefficient way to go about diagnosis.

A better approach is to try to find where the root cause of the bug is rather than determine what the root cause of the bug is. I write about this approach in detail in a separate post called The “20 Questions” method of debugging.

Fix

Sometimes a bugfix is an easy one-liner. Other times, the bug was merely a symptom of a deep and broad problem, and the fix is a serious project.

Whatever the size of the fix, there’s one thing that I always make sure of: I make sure that I’ve devised a test (manual or automated) that will fail when the bug is present and pass when the bug is absent. Otherwise I may fool myself into believing that I’ve fixed the bug when really my “fix” has had no effect.

Takeaways

  • You can work more efficiently when you know exactly what you’re working on.
  • Brainpower is a limited resource. Save brainpower for tasks that require high brainpower. Don’t waste brainpower on tasks that can be low-brainpower tasks.
  • Dividing a bugfix job into distinct steps can help you focus as well as save brainpower.
  • The three steps in fixing a bug are reproduction, diagnosis and fix.
  • Don’t take bug reports at face value.
  • Don’t try to find what the cause of the bug is, but rather try to find where the cause of the bug is. Once you accomplish that, the “what” will likely be obvious.
  • Before applying a bugfix, make sure you’ve devised a test (manual or automated) that will fail when the bug is present and pass when the bug is absent.

Why I don’t buy “duplication is cheaper than the wrong abstraction”

10 Replies

Note: If you disagree with what’s expressed in this post, I encourage you to also take a look at my more nuanced and comprehensive post on the topic of code duplication, which gives a more thorough refutation of some of the popular ideas around duplication.

“Duplication is cheaper than the wrong abstraction” is a saying I often hear repeated in the Rails community. I don’t really feel like the expression makes complete sense. I fear that it may lead developers to make poor decisions. Here’s what I take the expression to mean, why I can’t completely get on board with it, and what I would advise instead.

The idea (as I understand it)

My understanding of the “duplication is cheaper than the wrong abstraction” idea, based on Sandi Metz’s post about it, is as follows. When a programmer refactors a piece of code to be less duplicative, that programmer replaces the duplicative code with a new, non-duplicative abstraction. So far so good, perhaps. But, by creating this new abstraction, the programmer signals to posterity that this new abstraction is “the way things should be” and that this new abstraction ought not to be messed with. As a result, this abstraction gets bastardized over time as maintainers of the code need to change it yet simultaneously feel compelled to preserve it. The abstraction gets littered with conditional logic to behave different ways in different scenarios, and eventually becomes an unreadable mess.

I hope this is a fair and reasonably faithful paraphrasing of Sandi’s idea. Here are the parts of the idea that I agree with, followed by the parts of the idea that I take issue with.

The parts I agree with

In my experience it’s absolutely true that existing code has a certain inertia to it. Whether it’s out of caution or laziness or some other motive, it seems that a common approach to existing code is “don’t mess with it too much”. This is often a pretty reasonable approach, especially in codebases with poor test coverage where broad refactorings aren’t very safe. Unfortunately the “don’t mess with it too much” approach (as Sandi correctly points out) often makes bad code even worse.

I also of course agree that it’s bad when an abstraction gets cluttered up with a bunch of conditional logic to behave differently in different scenarios. Once that happens, the abstraction can hardly be called an abstraction anymore. It’s like two people trying to live in one body.

I also agree with Sandi’s approach to cleaning up poorly-deduplicated code. First, back out the “improvements” and return to the duplicated state. Then begin the de-duplication work anew. Good approach.

The parts I take issue with

What exactly is meant by “the wrong abstraction”?

I think “the wrong abstraction” is a confused way of referring to poorly-de-duplicated code. Here’s why.

It seems to me that what’s meant by “the wrong abstraction” is “a confusing piece of code littered with conditional logic”. I don’t really see how it makes sense to call that an abstraction at all, let alone the wrong abstraction.

Not every piece of code is an abstraction of course. To me, an abstraction is a piece of code that’s expressed in high-level language so that the distracting details are abstracted away. If I were to see a confusing piece of code littered with conditional logic, I wouldn’t see it and think “oh, there’s an incorrect abstraction”, I would just think, “oh, there’s a piece of crappy code”. It’s neither an abstraction nor wrong, it’s just bad code.

So instead of “duplication is cheaper than the wrong abstraction”, I would say “duplication is cheaper than confusing code littered with conditional logic”. But I actually wouldn’t say that, because I don’t believe duplication is cheaper. I think it’s usually much more expensive.

When duplication is dearer

I don’t see how it can be said, without qualification, that duplication is cheaper than the wrong abstraction. Some certain things must be considered. How bad is the duplication? How bad is the de-duplicated code? Sometimes the duplication is cheaper but sometimes it’s more expensive. How do you know unless you know how good or bad each alternative is? It depends on the scenario.

If the duplication is very small and obvious, and the alternative is to create a puzzling mess, then that duplication is absolutely cheaper than the bad code. But if the duplication is horrendous (for example, the same several lines of code duplicated across distant parts of the codebase dozens of times, and with inconsistent names which make the duplication hard to notice or track down) and the alternative is a piece of code that’s merely imperfect, then I would say that the duplication is more expensive.

In general, I find duplication to typically be more much expensive than the de-duplicated alternative. Duplication can bite you, hard. The worst is when there’s a piece of code that’s duplicated but you don’t know it’s duplicated. In that case you risk changing the code in one place without realizing you’re creating an inconsistency. It’s hard for a poor abstraction to have consequences worse than that.

Programmers’ reluctance to refactor isn’t a good justification for not DRYing up code

If a programmer “feels honor-bound to retain the existing abstraction” (to quote Sandi’s post), then to me that sounds like a symptom of a problem that’s distinct from duplication or bad abstractions. If developers are afraid to clean up poor code, then I don’t think the answer is to hold off on fixing duplication problems. I think the answer is to address the reasons why developers are reluctant to clean up existing code. Maybe that reason is a lack of automated tests and code review, or a lack of a culture of collective ownership. Whatever the underlying problem is, fixing that problem surely must be a better response than allowing duplication to live in your codebase.

My alternative take, summarized

Instead of “duplication is cheaper than the wrong abstraction”, I would say the following.

Duplication is bad. In fact, duplication is one of the most dangerous mistakes in coding. Except in very minor cases, duplication is virtually always worth fixing. But not all possible ways of addressing duplication are equally good. Don’t replace a piece of duplicative code with a confusing piece of code that’s made out of if statements.

When you find yourself adding if statements to a piece of code in order to get it to behave differently under different scenarios, you’re creating a confusion. Don’t try to make one thing act like two things. Instead, separate it into two things.

If you feel reluctant to modify someone else’s code, ask why that is. Is it because you feel like you’ll get in trouble if you do? Is it because you don’t understand the code, and there’s little test coverage, and you’re afraid you’ll break something if you make changes that are too drastic? Whatever the underlying reason for your reluctance is, it’s a problem, because it’s holding your organization back from improving its code. Instead of adding more bad code on top of the existing bad code, see if there’s anything you can do to try to address these underlying issues.

Takeaways

  • As Sandi Metz says (but in my words), confusing code littered with conditionals is not a good way to address duplicative code.
  • A piece of code filled with conditionals isn’t really an abstraction or even “wrong”, it’s just a confusing piece of code.
  • Duplication is one of the most dangerous mistakes in coding, and almost always worth fixing. Unless someone really botches the job when de-duplicating a piece of code, the duplicated version is almost always more expensive to maintain than the de-duplicated version.
  • Try to foster a culture of collective ownership in your organization so that developers aren’t afraid to question or change existing code when the existing code gets out of sync with current needs.
  • Try to use risk-mitigating practices like automated testing, small changes, and continuous deployment so that when refactorings are needed, you’re not afraid to do them.