Author Archives: Jason Swett

Developer success == individual * environment

Leave a reply

I used to think that a good developer could work pretty much anywhere and do good work. If the developer is good, then they’ll of course be good anywhere.

I’ve learned in recent years that this is not actually the case. If a developer gets hired somewhere and fails to do good work, it’s not always because that developer is a bad developer. Sometimes it’s because the developer’s employer fails to provide an environment conducive to success.

What a bad environment looks like

Chaos

In my experience, one of the common characteristics of a bad developer environment is chaos. There’s no development methodology, there’s no issue tracking, there’s no intelligent long-term project planning. Everybody just kind of does stuff.

If a really experienced developer enters this sort of environment, they might be able to help introduce some order to the organization or at least keep their little corner of the world orderly in spite of everyone else. But someone who’s earlier in their career, even if they’re really smart, might not have the skills and experience to do that. I’ve talked with junior developers before who have worked in very chaotic organizations who just thought it was normal. It was all they had ever known and so they had nothing to compare it to.

Managers in these types of organizations might expect a certain level of productivity out of their developers, not realizing that their organization’s environment makes good productivity unlikely, and conclude that the fault lies with the developers. Sadly, the developers might share this conclusion, ignorant of the fact that the chaotic environment is responsible for most of the productivity shortcoming.

Vague expectations

Along with chaos typically come vague expectations. This can happen at a low or high level, or of course both.

As an example of a low-level vague expectation, a developer might get assigned a ticket that simply says “Ability to add products” with few or no other details. There’s no way to tell when the ticket is complete, or whether the work done matches the expectation.

As an example of a high-level vague expectation, a developer might get hired with a title that implies role X when, all along, his superiors expected him to fulfill role Y, but never explicitly stated this expectation until several months after he started. (This exact scenario happened to me once.)

When a developer isn’t told what’s expected of them, it’s really hard for them to be successful. If what the developer does matches what’s expected, it’s only by chance.

Poor technical practices

If you put a good developer on a team that uses poor development practices, it’s going to be hard for that developer to make a good contribution.

I’ve worked at places where they didn’t use version control or practice continuous deployment. Obviously these poor practices can cause a good developer to produce bad work.

Lack of education/culture of ignorance

Some developers are smart but lack knowledge. If care isn’t taken to educate these developers, they often won’t produce very good work.

Some organizations simply fail to provide education to their employees. Others actively discourage it. I once worked at a place where my boss denied a request of mine to buy some books with company money because “you can read that stuff online”.

What a good environment looks like

Order

The best places I’ve worked at have used some sort of development methodology. Usually that development methodology was “agile-ish”, some form of not-quite-by-the-book agile. Even agile done badly is better than no methodology at all though.

When requirements are thoroughly thought out and specified for the developers in great detail before the work is assigned, it makes the developer’s work much clearer. The developer doesn’t have to spend much energy on parsing ambiguities and can instead just focus on the actual implementation work.

Clear expectations

When assigning user stories to developers, I like to include a bulleted list of acceptance criteria in the description for each story. That way it’s trivially easy to tell whether a story is complete and correct. Just go down the list of acceptance criteria and see if each item matches.

The specificity of the requirements should be tailored to the experience level of the developer. Unless the developer is very senior, it’s often a good idea to include not only high-level requirements but also some guidance regarding things like database structure.

I also like to include a UI mockup of the feature along with the story, sometimes even if the story is trivial. Miscommunication is super easy and so it’s better to over-communicate than under-communicate. Visual communication often helps clear up ambiguities.

There are certain other high-level types of expectations I like to set as well. For example, on the team I lead, I’ve set certain expectations around when and how to refactor. (Refactoring is encouraged. We don’t mix refactoring with behavior changes. We perform refactorings right before or right after behavior changes.)

Good technical practices

Things can go so much more smoothly when the technical practices are good.

Where I work, it goes like this:

  1. You’re assigned a small story with clear acceptance criteria
  2. You work on the story, which could take a couple hours or as much as a day, typically not more
  3. You open a PR which is reviewed 1 to 24 hours later (usually more like 1)
  4. Your PR is merged and your work is immediately deployed and verified in production
  5. You start again at step 1 with a new story

We’re able to move quickly like this because we have a solid test suite and we practice continuous delivery.

Things go much better when we work this way than if we were to try to assign them big projects that would only be verified, merged and deployed weeks or months after they were started.

Good education/culture of learning

When developers are told that their work output is the most important thing, and that “it doesn’t have to be perfect, it just has to work“, then the quality of the work output suffers.

I believe that developers can actually work faster when they’re allowed to take the time to learn how to do their work the right way rather than just cramming things in any old way. As Bob Martin has said, “the only way to go fast is to go well.”

Takeaways

A programmer’s ability to be successful is the product of the quality of the programmer and the quality of the programmer’s environment. I mean “product” in the mathematical sense. A programmer who is “90% good” who is put into an environment that is only “10% good” will produce work that’s only 90% * 10% = 9% good.

If you work with developers who aren’t producing good work, maybe it’s their fault, but maybe it’s the environment they’re in. And if you yourself feel like you’re not producing good work, it might be your fault or it might be that you’re in a bad environment. If you can find a better environment to put yourself in, you might find yourself able to do better work.

Active Record queries in views: when it’s bad, when it’s fine

Leave a reply

“No Active Record queries in views”

Maybe you’ve heard the advice that you shouldn’t put Active Record queries (or other logic) in views, but you’re not sure exactly why, other than “separation of concerns”. Why exactly is separation of concerns good?

The rationale behind this advice is that mixing concerns (like query logic vs. presentation) makes the code harder to understand and change.

Another way to think about this is to think about what kinds of things tend to change together.

Related changes

Even though many changes require touches at multiple levels in order for the change to be complete (model, controller, view)​, some changes are more closely related than others.

A change to the SQL in a query is more likely to call for other SQL changes than it is to call for a CSS change. For this reason, it usually makes sense to put query code next to other query code so that all the query code can be noticed and considered and worked with at the same time.

Sometimes queries in views is fine

Having said that, I don’t dogmatically follow a rule of “no queries in views”. If I for example have a select input and it needs to contain all categories, I’ll unashamedly put Category.all right into my view code. The alternative to this is to clutter up my controller with instance variables and before_actions.

The reason I don’t feel bad about this is that Category.all is unlikely to ever influence or be influenced by some other model change. And in the unlikely event that I’m wrong about that, I can easily move that query out of the view when I need to. But I’m hardly ever wrong about that. (I never put complex queries in my views though.)

Takeaway

Keeping logic code out of views tends to help make your code easier to change and understand. It’s generally good to keep Active Record queries out of views, but putting Active Record queries in views is typically harmless when the query is very unlikely to influence or be influenced by anything else.

Don’t wrap instance variables in attr_reader unless necessary

24 Replies

I’ve occasionally come across advice to wrap instance variables in attr_reader. There are two supposed benefits of this practice, which I’ll describe shortly. First I’ll provide a piece of example code.

Example of wrapping instance variables in attr_reader

Original version

Here’s a tiny class that has just one instance variable, @name. You can see that @name is used in the loud_name method.

class User
  def initialize(name)
    @name = name
  end

  def loud_name
    "#{@name.upcase}!!!"
  end
end

attr_reader version

Here’s that same class with an attr_reader added. Notice how loud_name now references name rather than @name.

class User
  attr_reader :name

  def initialize(name)
    @name = name
  end

  def loud_name
    "#{name.upcase}!!!"
  end
end

The purported benefits

Advocates of this technique seem to find two certain benefits in it. (I’ve also come across other supported benefits but I don’t find them strong enough to merit mentioning.)

It makes refactoring easier

Rationale: If you ever want to change @name from being defined by an instance variable to being defined by a method, then you don’t have to go changing all the instances of @name to name.

This reasoning isn’t wrong but it is weak. First, it’s very rare that a value will start its life as an instance variable and then at some later point need to change to a method. This has happened to me so few times that I can’t recall a single instance of it happening.

Second, the refactoring work that the attr_reader saves is only a trivial amount of work. The cost of skipping the attr_reader is that you have to e.g. change a handful of instances of @name, in one file, to name. Considering that this is a tiny amount of work and that it needs to happen perhaps once every couple years per developer, this justification seems very weak.

It saves you from typo failures

Rationale: If you’re using instance variables and you accidentally type @nme instead of @name, @nme will just return nil rather than raising an error. If you’re using attr_reader and you accidentally type nme instead of name, nme will in fact raise an error.

This justification is also true, but also weak. If typo-ing an instance variable allows a bug to silently enter your application, then your application is not tested well enough.

I would be in favor of saving myself from the typo problem if the attr_reader technique hardly cost anything to use, but as we’ll see shortly, the attr_reader technique’s cost is too high to justify its benefit. Since the benefit is so tiny, the cost would have to be almost nothing, which it’s not.

Reasons why the attr_reader technique is a bad idea

Adding a public attr_reader throws away the benefits of encapsulation

Private instance variables are useful for the same reason as private methods: because you know they’re not depended on by outside clients.

If I have a class that has an instance variable called @price, I know that I can rename that instance variable to @cost or change it to @price_cents (changing the whole meaning of the value) or even kill @price altogether. What I want to do with @price is 100% my business. This is great.

But if I add attr_reader :price to my class, my class suddenly has responsibilities. I can no longer be sure that the class where @price is defined is the only thing that depends on @price. Other clients throughout my application may be referring to @price. I’m no longer free to do away with @price or change its meaning. This makes my code riskier and harder to change.

You can add a private attr_reader, but that’s unnatural

If you want to make use of the attr_reader technique but you don’t want to throw away the benefits of encapsulation, you can add a private attr_reader. Here’s what that would look like.

# attr_reader version

class User
  def initialize(name)
    @name = name
  end

  def loud_name
    "#{name.upcase}!!!"
  end

  private

  attr_reader :name
end

This solves the encapsulation problem, but what have we really gained on balance? In exchange for not having to change @name to name on the off chance that we change name from an instance variable to a method, we have to pay the price of having this weird private attr_reader :name thing at the bottom of our class.

And consider that we would have to do this on every single class that has at least one instance variable!

Don’t wrap instance variables in attr_reader

Wrapping your instance variables in a public attr_reader changes your instance variables from private to public, increasing the public surface area of your class’s API and making your application a little bit harder to understand.

Wrapping your instance variables in a private attr_reader adds an unnatural piece of boilerplate to all your Ruby classes.

Given the tiny and dubious benefits that the attr_reader technique provides, this cost isn’t worth it.

Using attr_reader is good and necessary for values that really need to be public. As a default policy, wrapping instance variables in attr_reader is a bad idea.

Transient attributes in Factory Bot

1 Reply

Factory Bot offers a number of features to make test setup more convenient and less repetitive. Among these features is transient attributes.

Transient attributes are values that are passed into the factory but not directly set on the object’s attributes.

In order to illustrate how transient attributes can be useful, let’s look at an example where it’s painful not to use transient attributes.

PDF file attachment example

Let’s say we have an InsuranceDeposit class that has PDF file attachments.

class InsuranceDeposit
  has_many_attached :pdf_files
end

Perhaps we want to write some tests that depend on the content of an insurance deposit’s attached PDF files.

An example without using transient attributes

It would be tedious and repetitive to manually generate a PDF file on the spot every time we had a test case that needed a PDF file. Here’s what such code might look like:

insurance_deposit = create(:insurance_deposit)

# This setup is noisy and hard to understand
file = Tempfile.new
pdf = Prawn::Document.new
pdf.text "my arbitrary PDF content"
pdf.render_file file.path

insurance_deposit.pdf_files.attach(
  io: File.open(file.path),
  filename: "file.pdf"
)

This way is non-ideal because a) the code is hard to understand and b) if we use these steps end more than one place, we’ll have duplication.

We could of course conceivably extract that code into some sort of helper method. This would be an improvement over putting the code directly in our tests repetitively, but the remaining drawback is that then we’d have our test infrastructure code a little bit fragmented and scattered.

It would be nice if we could keep this code in the same place as all our other InsuranceDeposit factory code.

An example with transient attributes

If we take advantage of transient attributes, we can gain both understandability and DRYness. Here’s what that might look like.

FactoryBot.define do
  factory :insurance_deposit do
    transient do
      # pdf_content is an arbitrary transient attribute
      # that I'm defining here
      pdf_content { "" }
    end

    after(:create) do |insurance_deposit, evaluator|
      file = Tempfile.new
      pdf = Prawn::Document.new

      # Because I defined pdf_content as a transient attribute
      # above, I can read evaluator.pdf_content here
      pdf.text evaluator.pdf_content

      pdf.render_file file.path

      insurance_deposit.pdf_files.attach(
        io: File.open(file.path),
        filename: "file.pdf"
      )
    end
  end
end

Now, in our test code, all we have to do is this:

create(:insurance_deposit, pdf_content: "my arbitrary PDF content")

This is much tidier than the original. If we want to see how pdf_content works, we can open up the insurance_deposit factory and have a look, but we aren’t burdened by these details when we just want to understand the high-level steps of the test.

Related articles

How I set up Factory Bot on a fresh Rails project
Nested factories in Factory Bot: what they are and how to use them
When to use Factory Bot’s traits versus nested factories

A systematic methodology for solving any programming problem

2 Replies

In order to fix any problem, you need to know two things:

  1. What exactly is wrong
  2. How to fix it

The beautiful thing is that if you have the right answers for both the two items above, you’re guaranteed to fix the problem. If your fix doesn’t work, then it necessarily means you were wrong about either #1 or #2.

The fastest way to solve any problem is to focus first on finding out what’s wrong, with reasonable certainty, before you move onto the step of trying to fix the problem. “Diagnose before you prescribe.”

Why it’s faster to separate the diagnosis from the fix

There are two reasons why I think it’s faster to separate the diagnosis from the fix than to mix them up. The first is that if you don’t separate the two jobs, you’re liable to resort to guessing.

Guessing has its place but guessing is more productive when the guesses are founded on a decent level of understanding of the situation. The beginning of a problem-solving endeavor is when your understanding is the lowest. At this time your guesses will be the least informed by reality. At best, you will happen to make some correct guesses by luck. More likely, you’ll just waste time. The worst case is that you lead yourself down fruitless and time-consuming paths because you prescribed before you diagnosed.

The other reason has to do with mental clarity. If you mix diagnosing with solving, you’re liable to get confused. You’re more liable to flail and throw things at the wall rather than advance methodically toward a solution. You’re likely to run out of mental “RAM” and subsequently “crash”, losing your whole train of thought. All these things are pure waste.

How to diagnose a problem

Step zero: a scientific mindset

Science is the pursuit of the answer to the question: “What is true?” Since reality is very complex and often misleading, scientists need to be very careful about what they bestow with the label of “true”. It’s surprisingly easy to get fooled. In order for any conclusion to be true, it must be based on premises that are also true, otherwise the conclusion is suspect.

When any new piece of information enters your awareness, you need to pass that piece of information through a filter of scrutiny before you allow yourself to call it a “fact”. Don’t just believe something is true because it seems true. Many things that seem true aren’t not only not true but wildly wrong (e.g. a geocentric universe). You must require some sort of justification for believing something is true, and the justification you require should be proportionate to that thing’s size and prominence inside of whatever you’re trying to figure out.

This is the type of mindset we need to adopt when problem-solving. We need to practice science.

Step one: state the symptom

The first step toward diagnosing a problem is to make a problem statement. A problem statement is simply an expression of what you know to be wrong based on what you see.

At first your problem statement might be vague because although you know what the symptom of the problem is, you don’t know the root cause. You might not even know in what area the root cause lies.

Resist the temptation to be overly specific in your problem statement. A vague problem statement might not seem very helpful, but at least it’s not hurtful. A problem statement that’s precise but untrue causes much more harm than a problem statement that’s true but vague.

Robert M. Pirsig put this quite nicely in Zen and the Art of Motorcycle Maintenance:

In Part One of formal scientific method, which is the statement of the problem, the main skill is in stating absolutely no more than you are positive you know. It is much better to enter a statement “Solve Problem: Why doesn’t cycle work?” which sounds dumb but is correct, than it is to enter a statement “Solve Problem: What is wrong with the electrical system?” when you don’t absolutely know the trouble is in the electrical system.

When you’re satisfied that you have a true problem statement, write it down. Writing down the problem statement allows you to clear your “mental RAM” so you can use that precious limited RAM for other things.

In the remainder of this diagnosis discussion I’ll use the example problem statement of “site is down”.

By the way, I must make a caveat here. The problem statement of “site is down” depends on me knowing that the site is down. It wouldn’t be valid for me to receive a phone call from someone telling me the site is down and then make a problem statement that the site is down. Literally everything must be scrutinized. Just because someone tells me the site is down doesn’t necessarily mean that the site is actually down. It could mean that that person’s wi-fi isn’t working, or that I’m being pranked. I must verify for myself that the site is down (by attempting to visit it myself, for example) before I accept it as a true fact that the site is down.

Step two: isolate the root cause

Isolating the root cause is usually done by degrees. The idea with isolating the root cause is to progressively refine the original problem statement until the problem statement is so narrow and specific that the root cause is obvious.

The way to progressively narrow down the problem is to pose yes-or-no questions and determine what the answer is for each. It’s kind of like playing a game of “20 questions”. When I play 20 questions I like to ask questions like “Is it man-made?”, “Does it use electricity?”, “Could I pick one up?” which are each broad but also each eliminate a broad scope of possibilities.

Unlike 20 questions, we can’t get the answer to each of our questions just by having someone tell us. We need to find the answers ourselves by performing some sort of test. This is another area where it’s very important not to let ourselves get fooled.

Each test we perform needs to only change one variable at a time (using the scientific sense of the word variable, not the programming sense). If, for example, you delete line 26 in a file, delete line 4 from another file, restart the server, and get an error message, you can’t conclude that deleting line 26 is what caused the error message. It might be that deleting line 4 did it. It might even be that none of your changes did it, some somebody else changed something without restarting the server, and your server restart only applied some previous unrelated change. You must poke at each item from several directions in order to be reasonably sure you have the truth.

You will also necessarily need to bring facts into the picture from outside your tests. Be very careful to distinguish “strong” facts from “weak” facts. For example, a user might tell you they saw an error message on page X at about 4:30 yesterday. There’s no particular reason to disbelieve the user, but you also can’t take their word for it either. They could be misremembering. They could be confused about what page they saw. They might even not understand the boundaries between your application and, e.g., the operating system, meaning that what you thought was a bug report was really just a Windows error that the user experienced. So take these sorts of “facts” with a grain of salt. Give them, say, 60% credence instead of 100% credence.

Also note that this step is “isolate the root cause”, not “think really hard to guess what the root cause might be”. Empiricism is much cheaper than reasoning. It’s much faster to find where a problem lies than to try to figure out what the problem is. You can save the “what” for after you’ve found the “where”. Once you’ve found the “where”, the “what” will be about 100 times easier.

Let’s continue with the “website is down” example.

Websites work because a user’s request hits a server and then the server successfully returns a response, which is then displayed by a browser. If the website isn’t working, the problem must lie somewhere in that request chain. (BTW, “request chain” is just a term I made up.)

Based on this fact, I can refine my problem statement to: Something is broken in the request chain. It’s important to note that when I say “request chain”, I mean it in the very broadest sense possible. I conceive that chain to include everything that happens between the user typing the URL and a page being displayed. Otherwise I risk excluding the true root cause from my search area.

At this point I haven’t really made my problem statement any narrower, but I have made it more specific without making it any less true. My problem statement is now sufficiently specific that I can identify some investigation steps.

I’ll now ask myself the following questions:

  1. Does the DNS name for my website successfully resolve?
  2. When I send a request to my site’s URL, does that request actually make it to a server?
  3. If the request makes it to the server, do I see my application logging something in response?
  4. etc.

I typically don’t list out a large number of steps in advance, but rather I list one or two investigation steps, carry them out, and then think of one or two more.

The “site is down” example I’m using is a real one that actually happened to me some time ago. My memory is that when I SSH’d into a server to investigate, I discovered that there was an “out of disk space” error happening, and indeed, the server was out of disk space.

So in this case I refined my problem statement further to: one of the web servers is out of disk space. In this particular case I was done. The problem statement was sufficiently specific that it was obvious what to do: kill this server and spin up a fresh instance (and also add some disk space monitoring so this never happens again!).

If I hadn’t reached the end of the diagnosis process so easily, I would have continued posing yes/no questions of yes/no specificity until I had a specific enough problem statement that I knew what the fix was.

How to fix a problem

You know you’ve found the diagnosis to a problem when your problem statement is sufficiently specific that it’s obvious what the problem is.

When it’s obvious what the problem is, it’s often obvious what the solution is too. But not always. Often there are multiple possible ways to fix a problem. Sometimes it’s not clear what the palette of possible solutions is, or how easy each one would be to implement.

Here’s how to proceed in those situations.

Step one: enumerate all possible solutions

List all the possible solutions to the problem without filtering for elegance, ease of implementation, or anything else.

Step two: rank the solutions from most appealing to least appealing

Put the most simplest, most elegant solutions at the top and the dirtiest hacks at the bottom.

Step three: try the most appealing solution

Ideally, the most appealing solution also happens to be easy to implement. If it’s not, then you have a decision to make.

Your decision is whether to power through and implement the elegant solution even though it’s time-consuming, or to give up and implement a less appealing solution instead.

The way I make this decision is to spend time and effort on a solution in proportion to how appealing the next-most-appealing solution is. If there’s another solution that seems equally good, I’ll give up fast since the other solution might give me an equally acceptable outcome more cheaply. If all the other solutions are awful hacks, I’ll spend as much time as needed getting my solution to work rather than resorting to a hack.

Step four: move on

If I’ve decided that my current solution is sufficiently tricky that “the juice is not worth the squeeze”, then I’ll move to the next-most-appealing solution. I’ll cross my current solution off the list (at least for now) and then start again at step three with my next solution.

Step five: regroup

If I’ve tried all possible solutions and nothing works, I’ll see if I can think of any more possible solutions. If so, I’ll repeat the process using my new list of potential solutions. If I can’t think of any more possible solutions, I’ll go back to my original list and repeat the whole process, but this time with more effort.

It always works

Following the above methodology basically always works. It’s not a question of whether this methodology will yield the answer to a problem, it’s a matter of how much time spent solving a problem is justifiable relative to the cost of leaving the problem unsolved. Sometimes it’s wiser just to leave the problem unsolved.

But most of the time this methodology leads to the answer quite quickly. If you follow these steps you’ll be able to solve virtually any problem, and in much less time than most other programmers, and with much less confusion and frustration.

When to use Factory Bot’s traits versus nested factories

2 Replies

When I wrote about nested factories in Factory Bot, I got a couple of questions to the effect of “why wouldn’t you use traits for that?”

In responses to these questions, I’ll lay out my method for deciding when to use traits versus nested factories.

“Is” versus “has”

My method is pretty simple: If the factory I’m considering has something, I use a trait. If the factory is something, I use a nested factory. Let’s look at a concrete example.

“Has” example (trait)

In the following example I want to create a special kind of user record, a user that has a phone number. The user is still conceptually a regular old user. The only difference is that this user happens to have a value for its phone_number attribute.

FactoryBot.define do
  factory :user do
    username { Faker::Internet.username }
    password { Faker::Internet.password }

    trait :with_phone_number do
      phone_number { Faker::PhoneNumber.phone_number }
    end
  end
end

When I want to create a user that has a phone number, I do so by doing FactoryBot.create(:user, :with_phone_number).

“Is” example (nested factory)

In the following example I want to create a special kind of user record, a user that is a physician. Conceptually, a physician user is not just a regular old user. This type of user is different in kind. A physician user has different capabilities from a regular user and is used in different ways from a regular user.

FactoryBot.define do
  factory :user do
    username { Faker::Internet.username }
    password { Faker::Internet.password }

    factory :physician_user do
      role { 'physician' }
    end
  end
end

When I want to create a physician user, I do so by doing FactoryBot.create(:physician_user). Note the contrast between this and FactoryBot.create(:user, :with_phone_number).

Takeaway

When deciding whether to use a trait or a nested factory, consider whether the record has something or if it is something. If it has something, use a trait. If it is something, use a nested factory.

Nested factories in Factory Bot: what they are and how to use them

6 Replies

Sometimes you want to be able to create records that are 95% the same as the “default” but have one or two small differences. Nested factories can be useful for this purpose.

Physician user example

Let’s use the following as an example. Let’s say that in the majority of your application’s tests that involve a User, you want just a regular old user. Maybe you have 30 such tests.

But for a handful of tests you want to use a special kind of user, a user that indicates that the person using the application is a physician.

In this scenario, the technical difference between a “physician user” and a regular user is that a physician user has a role attribute that’s set to 'physician'. Let’s say there are 6 tests that use a physician user, quite a small proportion of the total.

Options for addressing this challenge

You have the following options for addressing this challenge.

First, you could set the role to “physician” individually on each of your 6 physician user tests. Unfortunately this would be bad for all the reasons that duplication is bad. If the nature of your physician user were ever to change, you’d have to know that you’d have to make the change in all 6 of these places. This solution isn’t great.

Alternatively, you could alter the user factory to always have role set to “physician”. Under this scenario, every user would always be a physician user, even when it’s not necessary. Even if this would function properly from a technical perspective, it would be misleading to any maintainers of the test suite. For any test, it’s a good idea to create the minimum amount of setup data and no more. An outside observer has no way of knowing which parts of a test’s setup data are necessary and which aren’t, and so must assume it’s all potentially necessary. We wouldn’t want to send our future maintainers an untruthful message.

A third option is to create an alternate version of the user factory that inherits from the original, and use that version in our physician tests. In this scenario we’re not duplicating our setup steps, and we’re also not modifying our “default” factory. The name for this in Factory Bot is nested factories.

How nested factories work

Consider the following User factory.

FactoryBot.define do
  factory :user do
    username { Faker::Internet.username }
    password { Faker::Internet.password }
  end
end

In addition to the above, we can define a nested factory called :physician_user which will have all the same attributes as our regular users, but with the extra attribute of role with the value of 'physician'.

FactoryBot.define do
  factory :user do
    username { Faker::Internet.username }
    password { Faker::Internet.password }

    factory :physician_user do
      role { 'physician' }
    end
  end
end

To create a physician user we would simply invoke the following:

FactoryBot.create(:physician_user)

Unlike root-level factory names which map to Active Record models, nested factory names are fully arbitrary.

Takeaways

  • If you have setup data needs that are common across tests, it’s usually not a good idea to duplicate the setup. Duplication in test code usually has the same negative consequences as duplication in application code.
  • Each test should have as much setup data as is necessary for that test and no more. Otherwise a maintainer will have a hard time telling what’s necessary and what’s not.
  • Factory Bot’s nested factories can help us keep our test code DRY while continuing to adhere to certain good testing principles.

How to run RSpec with headless Chrome/Chromium on Alpine Linux

5 Replies

Why Alpine Linux?

When you Dockerize a Rails application, you have to choose which distribution of Linux you want your container to use.

When I first started Dockerizing my applications I used Ubuntu for my containers because that was the distro I was familiar with. Unfortunately I discovered that using Ubuntu results in slow builds, slow running of commands, and large image sizes.

I discovered that Alpine Linux is popular for Docker containers because it affords better performance and smaller images.

Alpine + Capybara problems

Alpine had its own drawback though: I couldn’t run my tests because it wasn’t as straightforward in Alpine to get a Capybara + ChromeDriver configuration working on Alpine.

The evident reason for this is that Alpine can’t run a normal Chrome package the way Ubuntu can. Instead, it’s typical on Alpine to use Chromium, which doesn’t quite play nice with Capybara the way Chrome does.

How to get Alpine + Capyabara working

There are three steps to getting Capybara working on Alpine.

  1. Use selenium-webdriver instead of webdrivers
  2. Install chromium, chromium-chromedriver and selenium on your Docker image
  3. Configure a Capybara driver

Use selenium-webdriver instead of webdrivers

The first step is very simple: if you happen to be using the webdrivers gem in your Gemfile, replace it with selenium-webdriver.

Install chromium, chromium-chromedriver and selenium on your Docker image

The next step is to alter your Dockerfile so that chromium, chromium-chromedriver and selenium are installed.

Below is a Dockerfile from one of my projects (which is based on Mike Rogers’ fantastic Docker-Rails template).

I’ll call out the relevant bits of the file.

chromium chromium-chromedriver python3 python3-dev py3-pip

This line, as you can see, installs chromium and chromium-chromedriver. It also installs pip3 and its dependencies because we need pip3 in order to install Selenium. (If you don’t know, pip3 is a Python package manager.)

Here’s the line that installs Selenium:

RUN pip3 install -U selenium

And here’s the full Dockerfile.

FROM ruby:2.7.2-alpine AS builder

RUN apk add --no-cache \
    build-base libffi-dev \
    nodejs yarn tzdata \
    postgresql-dev postgresql-client zlib-dev libxml2-dev libxslt-dev readline-dev bash \
    #
    # For testing
    chromium chromium-chromedriver python3 python3-dev py3-pip \
    #
    # Nice-to-haves
    git vim \
    #
    # Fixes watch file issues with things like HMR
    libnotify-dev

RUN pip3 install -U selenium

FROM builder AS development

# Add the current apps files into docker image
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Install any extra dependencies via Aptfile - These are installed on Heroku
# COPY Aptfile /usr/src/app/Aptfile
# RUN apk add --update $(cat /usr/src/app/Aptfile | xargs)

ENV PATH /usr/src/app/bin:$PATH

# Install latest bundler
RUN bundle config --global silence_root_warning 1

EXPOSE 3000
CMD ["rails", "server", "-b", "0.0.0.0", "-p", "3000"]

FROM development AS production

COPY Gemfile /usr/src/app
COPY .ruby-version /usr/src/app
COPY Gemfile.lock /usr/src/app

COPY package.json /usr/src/app
COPY yarn.lock /usr/src/app

# Install Ruby Gems
RUN bundle config set deployment 'true'
RUN bundle config set without 'development:test'
RUN bundle check || bundle install --jobs=$(nproc)

# Install Yarn Libraries
RUN yarn install --check-files

# Copy the rest of the app
COPY . /usr/src/app

# Precompile the assets
RUN RAILS_SERVE_STATIC_FILES=enabled SECRET_KEY_BASE=secret-key-base RAILS_ENV=production RACK_ENV=production NODE_ENV=production bundle exec rake assets:precompile

# Precompile Bootsnap
run RAILS_SERVE_STATIC_FILES=enabled SECRET_KEY_BASE=secret-key-base RAILS_ENV=production RACK_ENV=production NODE_ENV=production bundle exec bootsnap precompile --gemfile app/ lib/

The next step is to configure a Capybara driver.

Configure a Capybara driver

Below is the Capybara configuration that I use for one of my projects. This configuration is actually identical to the config I used before I started using Docker, so there’s nothing special here related to Alpine Linux, but the Alpine Linux configuration described in this post won’t work without something like this.

# spec/support/chrome.rb

driver = :selenium_chrome_headless

Capybara.server = :puma, {Silent: true}

Capybara.register_driver driver do |app|
  options = ::Selenium::WebDriver::Chrome::Options.new

  options.add_argument("--headless")
  options.add_argument("--no-sandbox")
  options.add_argument("--disable-dev-shm-usage")
  options.add_argument("--window-size=1400,1400")

  Capybara::Selenium::Driver.new(app, browser: :chrome, options: options)
end

Capybara.javascript_driver = driver

RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by driver
  end
end

Remember to make sure that the line in spec/rails_helper that includes files from spec/support is uncommented so this file gets loaded.

How to run a PostgreSQL database in a Docker container

Leave a reply

Why you would want to do this

There are two main reasons for Dockerizing an application: for production use and for development use.

When Dockerizing an application for development, sometimes you might want to package up every single thing in the development environment. For example, in a Rails app, you might want to run each of PostgreSQL, Redis, and Ruby/Rails inside containers.

The drawbacks of Dockerizing everything

But then again you might not want to do that. To continue with the Rails app example, it can be very slow to e.g. run your tests inside a Docker container every single time you want to run a test.

You may instead wish to run PostgreSQL and Redis inside containers but leave Rails out of it, and instead connect Rails to these various containerized components.

This particular blog post covers the most basic building block of the above scenario: running PostgreSQL inside of a Docker container. I’m not going to cover actually connecting an application to the database in this post. This post is not meant to give you a useful result, but rather to help you gain an understanding of this building block.

What we’re going to do

Here’s what we’re going to do:

  1. Create a Docker image that can run PostgreSQL
  2. Run a container based on that image
  3. Connect to the PostgreSQL instance running inside that container to verify that it’s working

Let’s get started.

Creating the image

The first step is to create a Dockerfile. Name this file Dockerfile and put it inside a fresh empty directory.

FROM library/postgres
COPY init.sql /docker-entrypoint-initdb.d/

The first line says to use the postres image from Docker Hub as our base image.

The second line says to take our init.sql file (shown below) and copy it into the special /docker-entrypoint-initdb.d/ directory. Anything inside the /docker-entrypoint-initdb.d/ directory will get run each time we start a container.

Here are the contents of init.sql, which should go right next to our Dockerfile.

CREATE USER docker_db_user;
CREATE DATABASE docker_db_user;
GRANT ALL PRIVILEGES ON DATABASE docker_db_user TO docker_db_user;

As you can see, this file contains SQL code that creates a user, creates a database, and then grants permissions to that user on the database. It’s necessary to do all this each time a container starts because a container doesn’t “remember” all this stuff from run to run. It starts as a blank slate each time.

Building the image

Now let’s build our image.

docker build .

Once this runs, we can run docker images to see the image we just created.

docker images

You should see something that looks like this:

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              396bfb8e3373        7 minutes ago       314MB

Running the container

We can use the docker run command to run a container based on our image. There are a few arguments that need to be passed, which I’ve annotated below.

docker run \
  --name dev-postgres \             # the arbitrary name of the container we're starting
  -e POSTGRES_PASSWORD=mypassword \ # the arbitrary db superuser password, which must be set to something
  396bfb8e3373                      # the image ID from our previous docker build step

Connecting to the database

Finally, we can connect to the database to verify that it works.

docker exec --interactive dev-postgres psql -U docker_db_user

If running this command puts you inside the PostgreSQL CLI, you’re good!

Dissecting Factory Bot’s factory definition syntax

1 Reply

Mysterious syntax

If you’ve used Factory Bot at all, you’ve seen syntax like this:

FactoryBot.define do
  factory :user do
    first_name { 'John' }
    last_name { 'Smith' }
    email { 'john.smith@example.com' }
  end
end

When I was first getting started with Rails and I wasn’t very familiar with Ruby, I would look at these files and understand what’s going on conceptually but I would have no idea what’s going on syntactically.

Why I didn’t understand this code

There were three barriers to my understanding at that early stage of my Ruby experience:

  • Ruby’s optional parentheses feature, although it can help code be very expressive, can also make it hard for beginners to tell what’s what.
  • I didn’t understand Ruby blocks yet.
  • I didn’t know about dynamically-defined methods.

Let’s look at a modified example that will make some of the syntax clearer.

A modified factory definition example

Below is a version of the above factory definition that’s functionally equivalent but with two syntactical changes.

The first change is that I’ve included parentheses for all method calls. The second is that I’ve changed all block usages to do syntax instead of the shorthand {} syntax.

FactoryBot.define() do # define is a method
  factory(:user) do
    first_name do
      'John'
    end

    last_name do
      'Smith'
    end

    email do
      'john.smith@example.com'
    end
  end
end

You can see here that define and factory are each just methods. Each of the two methods takes a block. (If you’re not very comfortable with blocks yet, check out my post on understanding Ruby blocks.)

first_name, last_name and email are also methods that take blocks, speaking loosely. Before we can talk about those we need to talk about methods versus messages.

Methods and messages

When you call a method on an object, it can be said that you’re sending a message to that object. For example, when you call "5".to_i, you’re sending the message to_i to the object "5", which is of course a String.

In the above case, the message to_i also happens to be a method that’s defined on String. This doesn’t need to be the case though. We could send any message at all to String, String just might not necessarily respond to that particular message. That’s why we have the respond_to? method, to see what messages an object responds to.

The messages an object will respond to need not be limited to the methods that are defined on that object. An object author can use the method_missing method to allow an object to respond to any message that’s sent to it, and respond in any way that the object author chooses.

Factory definitions and messages

What’s likely happening with first_name, last_name and email is that Factory Bot is using method_missing to allow arbitrary messages to be sent, and if the message (e.g. the message of first_name) matches an attribute on the model that the factory is for, then Factory Bot uses the block passed with the message to set the value of that attribute.

Takeaways

  • Factory Bot’s factory definitions are made out of methods and blocks.
  • Adding parentheses to any piece of DSL code can often make the code clearer.
  • Ruby objects can be passed arbitrary messages, and objects can be designed to respond to those messages.