I recently got a chance to try out a couple of new programming things in a test app I built: PouchDB/CouchDB and Progressive Web Applications. In this video, I share some of my thoughts on those things.

Videos:

• Odysee
• YouTube
• Rumble

I like to write small test apps. They generally have two goals:

• Primarily, they give me an excuse to try out new programming stuff.
• Secondarily, an app might be useful if I actually manage to finish it.

In this case, the test app is a multi-user shared shopping list. The basic features of the app are:

• Multiple named shopping lists.
• Items can be added, edited, deleted.
• Items can be crossed off the list without deleting them.

That’s a traditional demo app, and any programmer should be able to build the MVP of that in an hour or two with any dev environment.

But, to make this useful, I’d like a couple more features:

• Multiple users.
• Accessible from desktop PC.
• Accessible from mobile phones.
• Can use on offline phone.
• Works on Linux phones.

So that’s two main problems to solve:

• Building a cross-platform app.
• Syncing data between devices.

Problem #1: The app should be cross-platform.

There are a bunch of ways to build cross platform apps, but this sounds like a good excuse to try out a Progressive Web Application. This is basically a normal web app, but:

• Mobile browsers will let you “install” it and get a homescreen icon for it.
• Once you’ve loaded it once, the browser will cache it so it will run offline.

As long as you’re doing web app type things, this is an effective replacement for a native mobile app. And a shopping list doesn’t need to do anything fancy (like access the filesystem, or the camera, etc) so a web app is fine.

Some Technical Details:

After writing my shopping list app as a normal web app, two additions were required to turn it into a PWA:

• Adding metadata to specify things like the app icon when “installed” on a mobile phone.
• Adding a web worker script to manage caching so the app keeps working offline.

Since I built the app using the next.js framework, I used the next-pwa js library, which gave good examples and generated the web worker.

Does it work?

Yes. Installing to create a homescreen icon and running without internet both work fine on my test Android device. The app works fine from my desktop and Linux phone.

What are the downsides?

A PWA requires a modern browser. This is a very complex platform that isn’t perfectly stable - features do get deprecated and replaced. For example, if this test app had been written a couple years ago, it would have used a manifest for caching rather than a web worker, and that manifest feature is deprecated now. Having users run old versions of browsers rather than updating doesn’t really work, so any browser app will tend to require constant maintenance.

Similarly, web apps are built on the modern JavaScript library ecosystem. The combination of very deep dependency trees with lots of churn at every leads to the same issue as browsers - apps will go bad if not maintained.

Problem #2: Syncing data between devices.**

Distributed data storage is hard. The big problem is concurrent updates and resolving the resulting conflicts. A smaller problem is having an efficient protocol for sharing changes.

But, this is an excuse to try out a pair of tools: CouchDB and PouchDB.

CouchDB is a NoSQL database that stores JSON documents and is especially good at distributed replication. It’s got a built-in sync protocol, and it’s designed assuming that there will be conflicts, and that the programmer must explicitly consider and resolve those conflicts at every step where they might appear (that is both on update and on sync).

CouchDB is an Erlang app, so it’s great to run on one or more servers, but that still leaves the main problem for this app of storing data in multiple browser instances. So we need PouchDB.

PouchDB is a JavaScript implementation of the CouchDB data store and sync protocol that happily runs in the browser using local storage and IndexedDB to store data locally in the browser. It can sync directly to remote CouchDB servers.

Some Technical Details:

PouchDB provided a solid local data store out of the box. Modeling data as JSON documents took a little bit of thought, but simply adding a “type” field that could be either “list” or “item” was sufficient to get it working.

Since PouchDB implements the CouchDB data model, even local data requires thinking about and resolving update conflicts. Just taking an arbitrary version in the event of a conflict is fine for this app.

The only server-side component here is a CouchDB server, and syncing to that is simple. We try to sync on every update. It’d be nice to push remote changes, but that would require additional server-side logic so instead I have a “sync” button.

The trickiest part of the CouchDB server setup is authentication. Users and permissions need to be set up manually, and HTTP basic authentication is used on every request. This is reasonably secure due to HTTPS, but expanding to a more complicated security model could be tricky.

Does it work?

Yup. It stores data both locally and on the server, and syncing works exactly as expected. Having a single central CouchDB server allows for syncing any number of devices.

What are the downsides?

In this test app I’m accessing the CouchDB server directly from the browser and using CouchDB users as app users. That probably isn’t an optimal solution, and I’m not entirely sure what the best way to take advantage of the CouchDB sync protocol without doing that is. It’s not impossible, but it’d require some more thought, and figuring it out would be essential for any app used by many users.

Another thing worth noting is that CouchDB has two sketchy issues with maintenance:

• First, they don’t start supporting new Linux distro versions until after release. For example, it took them nearly two months to make a release for Debian bullseye after that distro shipped. That meant I needed an old version of an LTS distribution to act as a server.
• CouchDB still uses MD5 as a stable hash for identifying attachments. MD5 is a secure hash algorithm that’s been broken since around 2004, and any use of it whatsoever in 2021 is a sign that the developers don’t take security seriously. Some developers like to make excuses about broken cryptographic hashes in non-cryptographic context - very simply, all those excuses are bullshit.

So CouchDB is really neat, but I can’t recommend selecting it for production use without first carefully evaluating those issues. That especially means making sure you’re 100% sure you understand the implications of using MD5, since in the case of user-supplied attachments there very well may be security issues there.