Synchronization using CouchDB
Why is synchronization so important?
More and more web apps have their mobile equivalents; it is prudent to add offline functionality to mobile apps. Here’s where the real problem unravels – how to optimally synchronize data saved on the mobile with changes in the web app?
What if a mobile user saved some data offline which was being simultaneously edited online? Which version should be selected as the up-to-date one? How to implement solutions for such conflicts?
Implementation of the above mentioned synchronization isn’t easy – there’s a lot of edge cases, it requires a well-thought-out protocol, special metadata (e.g. revision tree) and it has to be able to resolve conflicts. And there aren’t any shortcuts like “the simplest” synchronization type. The implementation of even the least complicated version requires an awful lot of work and causes many problems.
We’ve tried to introduce such “simplest” type in Kanbanery and it really got on our nerves. I’ve started to wonder back then: “Isn’t there a better way to achieve the recently much-wanted functionality?”
I really liked this quote I had found on GitHub:
Some mobile developers have waded into ad-hoc sync implementations and found themselves over their heads, with delayed or canceled products. It’s better to use a solution that already works.
It turned out that the “solution” mentioned was CouchDB and the person cited, Jens Alfke, is the co-author of Couchbase Lite, a library that implements CouchDB protocol on iOS.
Why use Couchbase Lite?
Mainly because it’s lightweight. It has small code size, quick startup, low memory usage and good enough performance.
It is using sqlite as the database engine instead of real CouchDB embedded on mobile because it’s more efficient. There used to be an implementation using CouchDB, but it was impossible to optimize it so the library was completely rewritten. However, due to its use of an efficient and reliable REST-based protocol pioneered by Apache CouchDB it is fully able to synchronize with real CouchDB instance.
This synchronization can be on-demand or continuous. Conflicts can be detected and resolved. Synchronization is handled using the replication feature of CouchDB.
Conceptually, it’s very simple – just take everything that’s changed in database A and copy it over to database B. Replication has several properties – there are push and pull replications (depends on if the source is remote or not), continuous (keeps connection opened and waits for changes) or one-shot, persistent or non-persistent.
Everything sounds great,
but what if you don’t want to download all the users with all their data to your small mobile app (which is kind of pretty common)? Use filters! Just define a method which will decide which data you want to synchronize :]
To get everything working on iOS side you need to have the Couchbase Lite framework.
Next step – database setup:
An example of a model definition:
To retrieve user records by email, you have to define the view:
And then you can make a query like that:
CBLQuery has also property called rowsIfChanged, which returns new rows values if they’ve changed, so you can register observer to that property and watch for changes that way.
But if you want to just display your data in some kind of UITableView there is a more handy solution – CBLUITableSource. It refreshes data as soon as it changes – an instant gratification!
To use it you have to:
- put one in the same xib as your table view,
- set its tableView property to your table view (it’s IBOutlet so you can just wire it up),
- set its query property to live query:
- set its labelProperty to text that you want to display or use CBLUITableDelegate protocol:
Can I finally start replication?
Yes, all you need to replicate is to run the following code:
Meantime in the Ruby world…
There is the couchrest_model gem that you can use in Ruby side to communicate with CouchDB.
It is very straightforward to use. Here you have an example of a model definition:
Because of views defined in design section you can retrieve records like this:
A defining filter that tells the protocol to synchronize only the documents which describe posts and belong to the requested user would look like this:
The only thing left to do is to send these definitions to the CouchDB instance. It should be done as quickly as possible, as mobile application also should be able to use it when connecting to the CouchDB instance.
The best way to do that is put the code below to config/initializers/couchdb.rb.
The code is iterating through your CouchRest::Model’s and pushing it’s designs definitions to CouchDB server.
But what about conflicts?
If document is edited both offline and online, so if there is a conflict, it will have the have _conflicts key.
So on iOS you can create a view to retrieve conflicting documents:
The document usually has one current revision, but when a conflict occurs, the protocol is keeping all conflicting revisions in order to give you a chance to resolve the conflict manually.
Below there is an Objective-C example of retrieving conflicting revisions properties:
If you’re going to create some kind of UI displaying these conflicts and let user decide what should be chosen, then you should not delete needed revisions and optionally merge contents from them to one chosen by user. However, if you don’t want to resolve conflicts manually, the protocol will preserve the illusion that there is no conflict by arbitrarily choosing one of the current revisions for you (the one with lexicographically higher revision ID).
Example projects
You can find both the web application and the iPhone client example projects on my github account, demonstrating all the things I mentioned here.
Conclusion
CouchDB *just* works and a bunch of people is working on it trying to make it more and more efficient. Don’t go mad implementing such complicated thing from scratch. Use CouchDB.