Declarative Design sits at the very core of my largest project, Ash Framework. I thought it would be a good idea to show how those principles could be leveraged without tying it to Ash.
Ultimately, the desire to build Ash comes from my personal experience with declarative design and my belief that, however hyperbolic, declarative design is the key to the future of software. I didn't invent it, nor do I even count among the early pioneers of it. In fact, while this may be the first time you've thought about it specifically as "declarative design", you've almost certainly been exposed to it many times already. If you've used SQL, HTML or CSS, you've worked with declarative languages already. So instead of trying to pitch you on the concepts presented in Ash Framework I'd like to provide a small example of how you can add declarative design to your code-base today and benefit from it immediately.
Before we get started it is important to note that, if you are building any kind of software application at all, you are building an abstraction of a domain (a.k.a problem space, business use case). This single fact means that your system will be far more internally consistent (each part of itself similar to the other) than it will be consistent with external systems. The high level constraints on your system inevitably shape every subsystem within it. While "premature abstraction" is a real problem, it is far less pervasive than the "late abstraction" problem that I typically see. Declarative design is the method by which you side-step the problems presented by imperative styles of abstraction. Armed with it, I would counter the advice to avoid premature abstraction with the exact opposite.
Abstract early and often.
Example
These examples are written in Elixir, but they are primarily pseudo-code/should be easily readable by any programmer. A bunch of code is left out, because the point is the structure/changes we make, not the minutia of each operation.
We'll start small, with a module that does some simple work of taking some data provided by an API and creating a corresponding record for that data.
defmodule SomeBusinessLogic do
def create_person(attributes) do
# create a person
end
def create_organization(attributes) do
# create an organization
end
def create_post(attributes) do
# create a post
end
end
So everything is up and running, and our service is creating data, everything is going swell. A few days later we get a call from a customer saying that his post creation requests are failing, and others are taking a very long time to complete. We realize that we're going to need some logging and monitoring. We also realize that if we just do this for posts, we're going to be right back at it for the other resources, so we ought to do it for all of them. Lets follow the conventional wisdom of not abstracting too early, and take the obvious path:
defmodule SomeBusinessLogic do
def create_person(attributes) do
start = System.monotonic_time()
try do
# create a person
# log_success_or_failure
after
report_timing("create_person", System.monotonic_time() - start)
end
end
def create_organization(attributes) do
start = System.monotonic_time()
try do
# create an organization
# log_success_or_failure
after
report_timing("create_organization", System.monotonic_time() - start)
end
end
def create_post(attributes) do
start = System.monotonic_time()
try do
# create a post
# log_success_or_failure
after
report_timing("create_post", System.monotonic_time() - start)
end
end
end
Awesome, its doing what it is supposed to, we're seeing metrics and log messages, and were able to use that information to solve our customer's issue. A few days later, it is explained that we'd like to give some of interns access to the system and they should be able to create posts, but not people or organizations. Alright, now we'll need to take the current user, and add some authorization.
defmodule SomeBusinessLogic do
def create_person(current_user, attributes) do
start = System.monotonic_time()
try do
if intern?(current_user) do
# create a person
# log_success_or_failure
else
# return a 403
end
after
report_timing("create_person", System.monotonic_time() - start)
end
end
def create_organization(current_user, attributes) do
start = System.monotonic_time()
try do
if intern?(current_user) do
# create an organization
# log_success_or_failure
else
# return a 403
end
after
report_timing("create_organization", System.monotonic_time() - start)
end
end
def create_post(attributes) do
start = System.monotonic_time()
try do
# create a post
# log_success_or_failure
after
report_timing("create_post", System.monotonic_time() - start)
end
end
end
Awesome, the customer is happy, it was super fast to implement, and we're sailing along. Next, the customer tells us that they've noticed someone is creating bad data, but they have no idea who it is. They need us to add an audit log so that they can see who is creating what.
defmodule SomeBusinessLogic do
def create_person(current_user, attributes) do
start = System.monotonic_time()
try do
if intern?(current_user) do
# create a person
# create_audit_log(person, "create_organization, attributes)
# log_success_or_failure
else
# return a 403
end
after
report_timing("create_person", System.monotonic_time() - start)
end
end
def create_organization(current_user, attributes) do
start = System.monotonic_time()
try do
if intern?(current_user) do
# create an organization
create_audit_log(organization, "create_organization, attributes)
# log_success_or_failure
else
# return a 403
end
after
report_timing("create_organization", System.monotonic_time() - start)
end
end
def create_post(current_user, attributes) do
start = System.monotonic_time()
try do
# create a post
create_audit_log(post, "create_post", attributes)
# log_success_or_failure
after
report_timing("create_post", System.monotonic_time() - start)
end
end
defp create_audit_log(person, operation, attributes) do
#create an audit log
end
end
Nice, audit log in place, job done. We get a call in the middle of the night though, because all of a sudden the entire system is slow. All of our monitoring shows consistently slow response times. We spend some time debugging, and finally determine two things:
- our audit log code is slow and needs to be fixed and
- we should be monitoring that code on its own.
Here we see an example of internal consistency. Even our private operations end up looking substantively similar to our public interfaces.
defmodule SomeBusinessLogic do
def create_person(current_user, attributes) do
start = System.monotonic_time()
try do
if intern?(current_user) do
# create a person
# create_audit_log(person, "create_organization, attributes)
# log_success_or_failure
else
# return a 403
end
after
report_timing("create_person", System.monotonic_time() - start)
end
end
def create_organization(current_user, attributes) do
start = System.monotonic_time()
try do
if intern?(current_user) do
# create an organization
create_audit_log(organization, "create_organization, attributes)
# log_success_or_failure
else
# return a 403
end
after
report_timing("create_organization", System.monotonic_time() - start)
end
end
def create_post(current_user, attributes) do
start = System.monotonic_time()
try do
# create a post
create_audit_log(post, "create_post", attributes)
# log_success_or_failure
after
report_timing("create_post", System.monotonic_time() - start)
end
end
defp create_audit_log(person, operation, attributes) do
try do
# create an audit log
# log_success_or_failure
after
report_timing("create_audit_log_for_#{operation}", System.monotonic_time() - start)
end
end
end
It is at this point that I think that even anti-premature-abstraction hardliners would be considering abstracting some of this logic out. The primary reason for this is that the similarities between these operations are annoying. While "being annoying" isn't a justification on its own, we should ask ourselves why it annoys us. It annoys us because it offers opportunities for hidden divergence. How easy would it be to scan these various operations, and realize that create_post has no authorization rules? It takes a current_user just like the others, but only uses that for creating an audit log. For that matter, what about the fact that create_audit_log looks just like the others, except it doesn't create an audit log itself, and is private? Lets look at an imperative method of abstraction. First, we'll create a function called report that takes a function, reports its status, and then creates an audit log. Then, we'll realize that the create_audit_log function can't leverage this, so we'll add a paramater to disable creating an audit log. Lets see how it looks:
defmodule SomeBusinessLogic do
def create_person(current_user, attributes) do
report("create_person", attributes, fn ->
if intern?(current_user) do
# create an person
else
# return a 403
end
end)
end
def create_organization(current_user, attributes) do
report("create_organization", attributes, fn ->
if intern?(current_user) do
# create an organization
else
# return a 403
end
end)
end
def create_post(current_user, attributes) do
report("create_post", attributes, fn ->
# create a post
end)
end
defp create_audit_log(object, operation, attributes) do
report("create_audit_log_for_#{operation}", attributes, fn ->
# create an audit log
end, false)
end
defp report(name, attributes, operation, audit_log? \\ true) do
try do
start = System.monotonic_time()
result = operation.()
# if operation wasn't a 403 - create_audit_log
# log_success_or_failure
if audit_log? do
create_audit_log(result, "create_audit_log_for_#{operation}", attributes)
end
result
after
report_timing(operation, System.monotonic_time() - start)
end
end
end
That looks a lot better right? While I agree that this method of abstraction is better than spaghetti code, I think we can do one better. There are still a myriad of problems with this design. It relies on the keenness of one's eye to spot differences in the various interface functions. More importantly, it is very brittle as we start adding new requirements. As each operation diverges in some ways (as we saw with authorization), we ultimately can't place it in our generic handler. Likewise, as they grow more similar in other ways, we're going to be left struggling to figure out how to weave that similarity between our one generic handler. For example, when you want to change the metrics that are reported by the post reporter, but not the other two. This complexity either ends up pushed all the way down into a toggle to our abstract function, or all the way up, ultimately reproduced across each caller. Truthfully, this kind of thing is "good enough" most of the time. But I would argue that there is a way to do this that is easier, grows more elegantly, and produces a valuable new tool. I know, I know, it must be snake oil. Let me show you how I would have done that abstraction.
defmodule SomeBusinessLogic do
@config [
person: [
forbid_interns?: true,
audit_log?: true
],
organization: [
forbid_interns?: true,
audit_log?: true
],
post: [
audit_log?: true,
forbid_interns?: false
],
audit_log: [
audit_log?: false,
forbid_interns?: false
]
]
def create_person(current_user, attributes) do
create(:person, current_user, attributes)
end
def create_organization(current_user, attributes) do
create(:organization, current_user, attributes)
end
def create_post(current_user, attributes) do
create(:post, current_user, attributes)
end
defp create_audit_log(object, current_user, operation, attributes) do
create(:audit_log, current_user, attributes, "create_audit_log_for_#{operation}")
end
defp create(object, current_user, attributes, operation_name \\ nil) do
try do
start = System.monotonic_time()
if @config[object][:forbid_interns?] && is_intern?(current_user) do
# return a 403
else
result = # create object
if @config[object][:audit_log?] do
operation_name = operation_name || "create_#{object}"
create_audit_log(object, current_user, operation_name, attributes)
end
result
end
after
report_timing(operation_name, System.monotonic_time() - start)
end
end
end
So lets break down the benefits here. I wrote effectively the same abstraction. But instead of embedding the "switches" (like intern authorization, whether or not to generate an audit log) in control flow, I put them all the way up at the top in a static description. I was still able to hide some private details in the implementation, like the fact that the audit_loggets a special operation name. More importantly, I have separated the two most important things, the "declaration", e.g what am I doing here, and the "engine", e.g "how am I doing it". The amount of downstream good that comes from this kind of design is impossible for me to overstate.
- Notice how legible the description is? I bet you could show that directly to a non technical person and they would understand "what" your code does, without having to know "how" it does it.
- When you need to change something, you have a great starting point for that discussion. If you're changing your description, that means you're expanding your domain. If you need to change the engine, you are modifying your domain (excluding fixes/maintenance)
- Someone else can come along and rewrite the engine, leaving the description intact. This guiding force can allow for extremely easy refactors, potentially in a different programming language entirely.
- If I want to add a new operation, say
create_commentit is effectively a triviality, and it can be configured with all the goodness we've built into our description. - This description is static, and can be used to power other things! Some examples can be found in the bonus article.
You'll notice that there isn't actually enough information in the description to know how to do the logic of creating each individual entity. Here is an example of what the description might look like when you include those instructions. Here we include a module that implements a create function, in the configuration.
@config [
person: [
forbid_interns?: true,
audit_log?: true,
module: MyApp.Person
],
organization: [
forbid_interns?: true,
audit_log?: true,
module: MyApp.Organization
],
post: [
audit_log?: true,
forbid_interns?: false,
module: MyApp.Post
],
audit_log: [
audit_log?: false,
forbid_interns?: false,
module: MyApp.AuditLog
]
]
Final thoughts
As you can see, we don't even really care that much about the procedural code anymore. We can discuss changes to the underlying behavior solely with the description. This is your DSL, or a "domain specific language". DSLs don't have to be confusing or magic. They don't have to use macros. A DSL could just be a map, or nested keyword lists like we used above.
Declarative design can work at many levels across your system. It can be the driver behind complex system flows, and it can also help with abstracting and encapsulating code on a smaller scale. Ultimately, in my experience, building declarative incurs very little cost or risk, and only makes us more flexible as we make our systems more complex.
For some more examples of what you can do with this, see the (bonus article)