An Incremental Approach to Declarative Design

Practical tips to adopt Declarative Design in your every day code.

Declarative Design sits at the core of my largest project, Ash Framework.

It is my belief that, however hyperbolic, Declarative Design is the key to the future of software. I didn't invent it, nor do I count among the early pioneers of it. In fact, while this may be the first time you've thought about it specifically as "Declarative Design", you've almost certainly been exposed to it many times already.

If you've used SQL, HTML or CSS, you've worked with declarative languages. So instead of trying to pitch you on the concepts presented in Ash, I'd like to provide a small example of how you can add Declarative Design to your code-base today and benefit from it immediately.

Before we get started it is important to note that, if you are building any kind of software application at all, you are building an abstraction of a domain (a.k.a problem space, business use case). This single fact means that your system will be far more _internally consistent_ (each part of itself similar to the other) than it will be _consistent with external systems_. The high level constraints on your system inevitably shape every subsystem within it.

While "premature abstraction" can be a serious problem, “late abstraction” can be just as serious. Declarative Design is the method by which you side-step the problems presented by imperative styles of abstraction.

How do we side-step these problems? We extract the data, instead of abstracting the code.

Example

These examples are written in Elixir, but they are primarily pseudo-code/should be easily readable by any programmer. A bunch of code is left out, because the point is the structure/changes we make, not the minutia of each operation.

We'll start small, with a module that does some simple work of taking some data provided by an API and creating a corresponding record for that data.

We’ve shipped the above. Our service is creating data, everything is going swell. A few days later we get a call from a customer saying that his post creation requests are failing, and others are taking a very long time to complete.

So let’s create a function that will log the time each operation takes, and report any failures. Naturally we should do this for each of our operations.

Let’s follow the conventional wisdom of not abstracting too early, and take the obvious path.

Update shipped, it’s doing what it is supposed to, we're seeing metrics and log messages, and we were able to use that information to solve our customer's issue.

A few days later, we find out that there are some users that are creating bad data. We aren’t tracking who is making what changes, so let’s add some auditing code. We need to accept the user as an argument, and create an audit log with the result of each operation.

Alright, this is out in the world and with these audit logs we’ve found and banned the bad actors.

Our next requirement comes in. Apparently, we’d like to give the interns access to the system. They should be able to create posts, but not people or organizations.

Well, this should be pretty straightforward!

So what’s wrong with this code?

Honestly? Nothing. If this was your entire app, just these three functions, do not change this code. The problem is that the majority of our applications are considerably more operations like this. So with that lens, there are the three primary problems.

Duplication

This is the classic problem embodied with principles like DRY. We don’t want to repeat ourselves because that can make changing the way our system work very difficult. We’d have to change a lot of places, and there is risk that we will do it wrong in some of them. Or that we will forget to do it in some of them.

Hidden Divergence

The *real* problem, however, is the opportunities for hidden divergence. These operations are almost the same. But not quite! They all take a user argument, but they don’t all have rules that involve the user.

These opportunities for misunderstanding are very dangerous.

We’re leaving a lot on the table

You’ll see what I mean shortly. Ultimately, we could do more with this code, effectively for free. It just has to be written differently. Let’s take a look. I’m just going to skip to the end and show you what I’d suggest the “right answer” looks like.

So let's break down the benefits here. By separating this logic into two components, the "declaration", i.e the “what”, and the "engine", i.e the “how”. The amount of downstream good that comes from this kind of design is impossible for me to overstate.

1. Notice how legible the description is? I bet you could show that directly to a non technical person and they would understand "what" your code does, without having to know "how" it does it.

2. When you need to change something, you have a great starting point for that discussion. If you're changing your description, that means you're changing the domain. If you need to change the engine, you are changing the method.

3. Someone else can come along and rewrite the engine, leaving the description intact. This guiding force can allow for extremely easy refactors, potentially even in a different programming language entirely.

4. If I want to add a new operation, say `create_comment` it is effectively a triviality, and it can be configured with all the goodness we've built into our description.

5. This description is static, and can be used to power other things! You cold generate a markdown file, or a diagram from these descriptions. This is far more useful than you may realize, and we have all kinds of tools along these lines in Ash.

In closing

As you can see, we don't even really care that much about the procedural code anymore. We can discuss changes to the underlying behavior solely with the description.

This is your DSL, or "domain specific language". DSLs don't have to be confusing or magic. They don't have to use macros. A DSL could just be a map, or nested keyword lists like we used above.

Declarative Design can work at many levels across your system. It can be the driver behind complex system flows, and it can also help with abstracting and encapsulating code on a smaller scale.

Ultimately, in my experience, Declarative Design incurs very little cost or risk. It increases our leverage and flexibility, while keeping our code simple and understandable.