How to declutter your lib directory
If you’re working on a Rails app, let’s do a quick test: check how many files you have in the lib directory of your project. If you have less than 10, it must be a fairly new project. Most projects I’ve seen have at least 20-30. In our current project we had over 200 of them a few months ago…
The problem with the lib directory is that there are no official guidelines of what we should do there. Rails has a well-defined directory structure, which is great because you don’t have to think of how to organize your code since all the directories covering controllers, models, views, etc… are set up from the beginning. The downside is that once you start creating files that don’t fall into any of the predefined categories, it’s hard to decide what to do with them. So they usually end up in lib, which becomes a real mess over time.
I started searching for a solution – I asked on Twitter and looked for any relevant blog posts. The best idea I found was the one presented by Bryan Helmkamp:
I recommend any code that is not specific to the domain of the application goes in lib/. Now the issue is how to define “specific to the domain”. I apply a litmus test that almost always provides a clear answer: If instead of my app, I were building a social networking site for pet turtles (let’s call it MyTurtleFaceSpace) is there a chance I would use this code?
That makes perfect sense: the app directory is for your app’s code, as the name implies. All of it – not just assets, controllers, helpers, mailers, models and views, just because these are the subdirectories that Rails creates by default. You can always make your own.
So what exactly did we find in our lib directory?
Resque jobs
If you use any background queue library such as Delayed Job or Resque, you probably have a collection of classes for the tasks performed in the background. In our project we have a whole directory tree consisting of Resque job files – guess where we had to put them? In lib of course, specifically in lib/queues.
But the background jobs are an integral part of the app, they perform work that would normally be done by controllers or models, and usually call in some other models, except they do it asynchronously. So, according to the rule mentioned above, they should be put somewhere inside the app directory. Of course there was no pre-configured directory where we could put them all, but that doesn’t mean we can’t add one, and that’s what we did – so the Resque classes were moved to app/jobs.
Decorators & presenters
We also have quite a large directory with decorator or presenter classes, which we have put in to app/decorators. These are all support classes for rendering views that present models to them in some way; most of the time it’s about rendering model data to JSON which is used by API controllers. We also have some presenters used by standard HTML views, which organize records into some specific type of hierarchy that a particular partial requires, in order to move the logic out of ERB partials (or models).
Concerns
Concerns are with modules that are intended to be included in other classes and are used for sharing code between a group of related classes, e.g. models or controllers. This concept was popularized by DHH and 37signals and is now officially supported in Rails 4 – Rails now auto-generates app/controllers/concerns and app/models/concerns directories and adds them to the autoloading list.
This feature is a bit controversial and there are some people who say that there’s no place for such thing in good Rails apps. I disagree – I think concerns are OK unless they’re overused.
If you have 200-lines-long concern modules that are included everywhere, or if you have as many concerns as you have models, you’re probably doing something wrong – any bigger and isolated pieces of functionality from concerns should be extracted to some separate, independent classes. But if you have a few short modules with just a couple of short methods each, that doesn’t do any harm at all and can make your code shorter and more DRY. One example could be when a few models have a field with the same name (e.g. enabled) and you want to share some setters, scopes or finders that deal with that field, which would look exactly the same in all of those models.
The concerns support added in Rails 4 is really just a change to generators and the autoloading list, so you can add this yourself easily to your Rails 3 apps. We’ve also added app/decorators/concerns which holds a couple of shared modules used by presenters.
Services
Another non-standard directory that we’ve added is app/services, in which we keep a group of classes and modules that don’t belong to any other category such as models or presenters, but are still specific to this project’s domain. This is a pretty broad category, though not as broad as the old lib directory (right now we have about 40 files and directories in it). Some services are classes that you make instances of, some are modules that you use directly and some are whole directories of a few cooperating classes grouped in a namespace.
The name “service” comes from a pattern called Service Object which is gaining some popularity in the Rails community recently, and it’s basically about extracting pieces of specific functionality that would be normally written in a model or a controller to a separate class or module. The reason is that if you put everything that a user can do into the User model, as is often the case at the beginning of a project, this model can grow to hundreds or thousands of lines of code over time (user.rb is often the most complex class in a Rails project).
Models
A few files from lib actually ended up in the app/models directory. The purpose of app/models seems to be clear, until you start thinking about it: what is a model really? Is it just for subclasses of ActiveRecord (or ROM/Mongoid/etc.), which represent tables in your database, or is it for all classes which are part of your model layer, regardless of how they’re implemented? At what point it’s not a model anymore, but rather a service?
We’ve decided not to restrict app/models only to ActiveRecord models, but instead just use common sense to decide what should go there. Some rough guidelines that we’ve used were:
- AR models go to models
- classes whose purpose is to store and fetch data from Redis structures also go to models – after all, the only difference from AR models is that they use a different storage backend
- classes that are mostly about storing and accessing data, validations, calculations etc. should rather go to models
- classes that are about interaction, doing, changing or sending something (often with names like “Creator”, “Handler”, “Uploader”, etc…) go to services
- groups of classes in a namespace usually go to services (e.g. Auth module that implements various kinds of authentication or an ABTesting module which handles A/B tests)
Monkey patches
The Ruby open classes feature that allows you to monkey-patch other people’s code is a great thing. Even though it’s considered dangerous, it’s often hard to resist because it’s so easy and gives you power to change anything you want. We also had a bunch of monkey patches for other classes scattered over the project, mostly somewhere in lib and in config/initializers. Now we’ve moved them all to lib, divided into two groups.
The first one, lib/ext, is for extensions to core Ruby classes such as Array or String. There aren’t a lot of these, but sometimes you really want that core class to have that particular method instead of having to wrap the objects with something. The other directory is lib/hacks and it’s meant for, well, hacks; the files that should ideally not exist at all, and will hopefully be removed in the future, but for now they have to be there because the world is not perfect and sometimes you just have to hack something to make it work. At least this way we clearly see how many of those we have – this is similar to the idea of having a shame.css stylesheet that keeps all the ugly CSS you aren’t proud of, isolated from the rest of the code.
What stays in lib
The lib directory should ideally contain only those files that are generic and reusable (those that could be useful in the turtle social networking site). Things like a basic interface to some web service, database tools, asset processors or compressors definitely belong there. On the other hand, things that use words like “user”, “game”, “event” or any other words that appear in your model names probably don’t belong in lib.
Here are my guidelines for a proper lib file:
- it should not access any of your models, services or anything else from the app in any way
- it can only access other libs, Ruby core libraries or stuff from gems
- it should not rely on any global variables, constants or ENV variables to be defined
- it should be easily extracted to a gem and put on GitHub and RubyGems without too much effort
If it needs some small amount of project-specific configuration (e.g. an API key for a web service), make it possible for external configuration, just like you’d do if you wanted to put it in a gem, e.g.:
# lib/twitter_poster.rb
module TwitterPoster
mattr_accessor :api_key
…
end
# config/initializers/twitter.rb
TwitterPoster.api_key = 'qwerty'
At the moment we have just 46 Ruby files in lib, including the extensions and hacks.
Other approaches
This list is clearly not a complete solution that covers every possible case in every Rails project; every project is different, each has a different set of features and is built slightly differently. Not everyone agrees that the Rails conventions are something that you should try to stick with, and some people in the community argue that you should build your whole app separately from Rails and only use the app directory to build an interface between Rails and the core of your app. I wouldn’t go that far, but I’d agree that you should only treat the default directory structure as a starting point and then modify it according to your needs – where you end up will depend on your application’s complexity, architecture and your team’s preferences.