#articles [[Declarative Design]] sits at the very core of my largest project, [[Ash Framework]]. I thought it would be a good idea to show how those principles could be leveraged without tying it to Ash. For some more recent writing on exactly what Declarative Design actually is, see [[What Is Declarative Design]]. Ultimately, the desire to build [[Ash Framework|Ash]] comes from my personal experience with [[Declarative Design|declarative design]] and my belief that, however hyperbolic, it is the key to the future of software. I didn't invent it, nor do I even count among the early pioneers of it. In fact, while this may be the first time you've thought about it specifically as "declarative design", you've almost certainly been exposed to it many times already. If you've used SQL, HTML or CSS, you've worked with declarative languages already. So instead of trying to pitch you on the concepts presented in Ash Framework I'd like to provide a small example of how you can add declarative design to your code-base today and benefit from it immediately. Before we get started it is important to note that, if you are building any kind of software application at all, you are building an abstraction of a domain (a.k.a problem space, business use case). This single fact means that your system will be far more _internally consistent_ (each part of itself similar to the other) than it will be _consistent with external systems_. The high level constraints on your system inevitably shape every subsystem within it. While "premature abstraction" is a real problem, it is far less pervasive than the "late abstraction" problem that I typically see. Declarative design is the method by which you side-step the problems presented by imperative styles of abstraction. # Example These examples are written in [[Elixir]], but they are primarily pseudo-code/should be easily readable by any programmer. A bunch of code is left out, because the point is the structure/changes we make, not the minutia of each operation. We'll start small, with a module that does some simple work of taking some data provided by an API and creating a corresponding record for that data. ```elixir defmodule SomeBusinessLogic do def create_person(attributes) do # create a person end def create_organization(attributes) do # create an organization end def create_post(attributes) do # create a post end end ``` Starting point So everything is up and running, and our service is creating data, everything is going swell. A few days later we get a call from a customer saying that his post creation requests are failing, and others are taking a very long time to complete. We realize that we're going to need some logging and monitoring. We also realize that if we just do this for posts, we're going to be right back at it for the other resources, so we ought to do it for all of them. Lets follow the conventional wisdom of not abstracting too early, and take the obvious path: ```elixir defmodule SomeBusinessLogic do def create_person(attributes) do start = System.monotonic_time() try do # create a person # log_success_or_failure after report_timing("create_person", System.monotonic_time() - start) end end def create_organization(attributes) do start = System.monotonic_time() try do # create an organization # log_success_or_failure after report_timing("create_organization", System.monotonic_time() - start) end end def create_post(attributes) do start = System.monotonic_time() try do # create a post # log_success_or_failure after report_timing("create_post", System.monotonic_time() - start) end end end ``` Adding metrics and logs Awesome, its doing what it is supposed to, we're seeing metrics and log messages, and were able to use that information to solve our customer's issue. A few days later, it is explained that we'd like to give some of interns access to the system and they should be able to create posts, but not people or organizations. Alright, now we'll need to take the current user, and add some authorization. ```elixir defmodule SomeBusinessLogic do def create_person(current_user, attributes) do start = System.monotonic_time() try do if intern?(current_user) do # create a person # log_success_or_failure else # return a 403 end after report_timing("create_person", System.monotonic_time() - start) end end def create_organization(current_user, attributes) do start = System.monotonic_time() try do if intern?(current_user) do # create an organization # log_success_or_failure else # return a 403 end after report_timing("create_organization", System.monotonic_time() - start) end end def create_post(attributes) do start = System.monotonic_time() try do # create a post # log_success_or_failure after report_timing("create_post", System.monotonic_time() - start) end end end ``` Awesome, the customer is happy, it was super fast to implement, and we're sailing along. Next, the customer tells us that they've noticed someone is creating bad data, but they have no idea who it is. They need us to add an audit log so that they can see who is creating what. ```elixir defmodule SomeBusinessLogic do def create_person(current_user, attributes) do start = System.monotonic_time() try do if intern?(current_user) do # create a person # create_audit_log(person, "create_organization, attributes) # log_success_or_failure else # return a 403 end after report_timing("create_person", System.monotonic_time() - start) end end def create_organization(current_user, attributes) do start = System.monotonic_time() try do if intern?(current_user) do # create an organization create_audit_log(organization, "create_organization, attributes) # log_success_or_failure else # return a 403 end after report_timing("create_organization", System.monotonic_time() - start) end end def create_post(current_user, attributes) do start = System.monotonic_time() try do # create a post create_audit_log(post, "create_post", attributes) # log_success_or_failure after report_timing("create_post", System.monotonic_time() - start) end end defp create_audit_log(person, operation, attributes) do #create an audit log end end ``` Nice, audit log in place, job done. We get a call in the middle of the night though, because all of a sudden the entire system is slow. All of our monitoring shows consistently slow response times. We spend some time debugging, and finally determine two things: 1. our audit log code is slow and needs to be fixed and 2. we should be monitoring that code on its own. Here we see an example of _internal consistency_. Even our private operations end up looking substantively similar to our public interfaces. ```elixir defmodule SomeBusinessLogic do def create_person(current_user, attributes) do start = System.monotonic_time() try do if intern?(current_user) do # create a person # create_audit_log(person, "create_organization, attributes) # log_success_or_failure else # return a 403 end after report_timing("create_person", System.monotonic_time() - start) end end def create_organization(current_user, attributes) do start = System.monotonic_time() try do if intern?(current_user) do # create an organization create_audit_log(organization, "create_organization, attributes) # log_success_or_failure else # return a 403 end after report_timing("create_organization", System.monotonic_time() - start) end end def create_post(current_user, attributes) do start = System.monotonic_time() try do # create a post create_audit_log(post, "create_post", attributes) # log_success_or_failure after report_timing("create_post", System.monotonic_time() - start) end end defp create_audit_log(person, operation, attributes) do try do # create an audit log # log_success_or_failure after report_timing("create_audit_log_for_#{operation}", System.monotonic_time() - start) end end end ``` It is at this point that I think that even anti-premature-abstraction hardliners would be considering abstracting some of this logic out. The primary reason for this is that the _similarities between these operations are annoying_. While "being annoying" isn't a justification on its own, we should ask ourselves _why_ it annoys us. It annoys us because it offers _opportunities for hidden divergence_. How easy would it be to scan these various operations, and realize that `create_post` has no authorization rules? It takes a `current_user` just like the others, but only uses that for creating an audit log. For that matter, what about the fact that `create_audit_log` looks just like the others, except it doesn't create an audit log itself, and is private? Lets look at an imperative method of abstraction. First, we'll create a function called `report` that takes a function, reports its status, and then creates an audit log. Then, we'll realize that the `create_audit_log` function can't leverage this, so we'll add a paramater to disable creating an audit log. Lets see how it looks: ```elixir defmodule SomeBusinessLogic do def create_person(current_user, attributes) do report("create_person", attributes, fn -> if intern?(current_user) do # create an person else # return a 403 end end) end def create_organization(current_user, attributes) do report("create_organization", attributes, fn -> if intern?(current_user) do # create an organization else # return a 403 end end) end def create_post(current_user, attributes) do report("create_post", attributes, fn -> # create a post end) end defp create_audit_log(object, operation, attributes) do report("create_audit_log_for_#{operation}", attributes, fn -> # create an audit log end, false) end defp report(name, attributes, operation, audit_log? \\ true) do try do start = System.monotonic_time() result = operation.() # if operation wasn't a 403 - create_audit_log # log_success_or_failure if audit_log? do create_audit_log(result, "create_audit_log_for_#{operation}", attributes) end result after report_timing(operation, System.monotonic_time() - start) end end end ``` That looks a lot better right? While I agree that this method of abstraction is better than spaghetti code, I think we can do one better. There are still a myriad of problems with this design. It relies on the keenness of one's eye to spot differences in the various interface functions. More importantly, it is _very_ brittle as we start adding new requirements. As each operation diverges in some ways (as we saw with authorization), we ultimately can't place it in our generic handler. Likewise, as they grow more similar in other ways, we're going to be left struggling to figure out how to weave that similarity between our one generic handler. For example, when you want to change the metrics that are reported by the post reporter, but not the other two. This complexity either ends up pushed all the way down into a toggle to our abstract function, or all the way up, ultimately reproduced across each caller. Truthfully, this kind of thing is "good enough" most of the time. But I would argue that there is a way to do this that is _easier_, _grows more elegantly_, and _produces a valuable new tool_. I know, I know, it must be snake oil. Let me show you how I would have done that abstraction. ```elixir defmodule SomeBusinessLogic do @config [ person: [ forbid_interns?: true, audit_log?: true ], organization: [ forbid_interns?: true, audit_log?: true ], post: [ audit_log?: true, forbid_interns?: false ], audit_log: [ audit_log?: false, forbid_interns?: false ] ] def create_person(current_user, attributes) do create(:person, current_user, attributes) end def create_organization(current_user, attributes) do create(:organization, current_user, attributes) end def create_post(current_user, attributes) do create(:post, current_user, attributes) end defp create_audit_log(object, current_user, operation, attributes) do create(:audit_log, current_user, attributes, "create_audit_log_for_#{operation}") end defp create(object, current_user, attributes, operation_name \\ nil) do try do start = System.monotonic_time() if @config[object][:forbid_interns?] && is_intern?(current_user) do # return a 403 else result = # create object if @config[object][:audit_log?] do operation_name = operation_name || "create_#{object}" create_audit_log(object, current_user, operation_name, attributes) end result end after report_timing(operation_name, System.monotonic_time() - start) end end end ``` So lets break down the benefits here. I wrote effectively the same abstraction. But instead of embedding the "switches" (like intern authorization, whether or not to generate an audit log) in control flow, I put them all the way up at the top in a _static description_. I was still able to hide some private details in the implementation, like the fact that the `audit_log`gets a special operation name. More importantly, I have separated the two most important things, the "declaration", e.g what am I doing here, and the "engine", e.g "how am I doing it". The amount of downstream good that comes from this kind of design is impossible for me to overstate. 1. Notice how legible the description is? I bet you could show that directly to a non technical person and they would understand "what" your code does, without having to know "how" it does it. 2. When you need to change something, you have a great starting point for that discussion. If you're changing your description, that means you're _expanding_ your domain. If you need to change the engine, you are _modifying_ your domain (excluding fixes/maintenance) 3. Someone else can come along and rewrite the engine, leaving the description intact. This guiding force can allow for extremely easy refactors, potentially in a different programming language entirely. 4. If I want to add a new operation, say `create_comment` it is effectively a triviality, and it can be configured with all the goodness we've built into our description. 5. This description is static, and can be used to power other things! See below. You'll notice that there isn't actually enough information in the description to know how to do the logic of creating each individual entity. Here is an example of what the description might look like when you include those instructions. Here we include a module that implements a `create` function, in the configuration. ```elixir @config [ person: [ forbid_interns?: true, audit_log?: true, module: MyApp.Person ], organization: [ forbid_interns?: true, audit_log?: true, module: MyApp.Organization ], post: [ audit_log?: true, forbid_interns?: false, module: MyApp.Post ], audit_log: [ audit_log?: false, forbid_interns?: false, module: MyApp.AuditLog ] ] ``` ## Additional Benefits There are a wide array of benefits that you can get from leveraging your static descriptions. This is bonus material from an [earlier article](https://zachdaniel.dev/incremental-declarative-design/). ### Performance One valid concern with the code from my declarative design article is that it performs _slightly_ slower than the imperative version, because it checks the description each time. This doesn't matter for this example, but could matter as your description/engine grow. If you're using a language that supports macros, you can actually side step this relatively easily. This is ugly, but a lot of metaprogramming is. In my experience, this kind of thing is generally unnecessary, but I want to show how it can be done. This also won't really be very useful if you don't know Elixir. Here is a redefinition of our engine (the `create` function) as a macro: ```elixir defmacrop create(object, current_user, attributes, operation_name \\ nil) do create = if @config[object][:audit_log?] do quote generated: true do result = # create object operation_name = operation_name || "create_#{unquote(object)}" create_audit_log(unquote(object), unquote(current_user), unquote(operation_name), unquote(attributes)) result end else quote generated: true do # create object end end authorized_create = if @config[object][:forbid_interns?] do quote generated: true do if is_intern?(unquote(current_user)) do # return a 403 else unquote(create) end end else create end quote generated: true do try do unquote(authorized_create) after report_timing(operation_name, System.monotonic_time() - start) end end end ``` ### Documentation A very useful thing to do with the description is to derive your in-code (and often out-of-code) documentation from it. Here is an example of that with our description: ```elixir defmodule SomeBusinessLogicDocumenter do def doc(config) do interns_allowed = if config[:forbid_interns?] do "Interns are not allowed" else "Interns are allowed" end audit_log = if config[:audit_log?] do "Generates an audit log" else "Does not generate an audit log" end """ * #{interns_allowed} * #{audit_log} """ end end defmodule SomeBusinessLogic do @config [ person: [ forbid_interns?: true, audit_log?: true ], organization: [ forbid_interns?: true, audit_log?: true ], post: [ audit_log?: true, forbid_interns?: false ], audit_log: [ audit_log?: false, forbid_interns?: false ] ] @doc """ Creates a person. #{doc(@config[:person])} """ def create_person(current_user, attributes) do create(:person, current_user, attributes) end @doc """ Creates a organization. #{doc(@config[:organization])} """ def create_organization(current_user, attributes) do create(:organization, current_user, attributes) end @doc """ Creates a post. #{doc(@config[:post])} """ def create_post(current_user, attributes) do create(:post, current_user, attributes) end @doc """ Creates an audit log. #{doc(@config[:audit_log])} """ defp create_audit_log(object, current_user, operation, attributes) do create(:audit_log, current_user, attributes, "create_audit_log_for_#{operation}") end # ... end ``` With that, my editor now gives me on hover descriptions of each operation, with very little effort overall. ![[Declarative Function Docs 2.png]] ![[Declarative Function Docs.png]] ## Final thoughts As you can see, we don't even really care that much about the procedural code anymore. We can discuss changes to the underlying behavior _solely with the description_. This is your DSL, or a "domain specific language". DSLs don't have to be confusing or magic. They don't have to use macros. A DSL could just be a map, or nested keyword lists like we used above. Declarative design can work at many levels across your system. It can be the driver behind complex system flows, and it can also help with abstracting and encapsulating code on a smaller scale. Ultimately, in my experience, building declarative incurs _very little_ cost or risk, and only makes us more flexible as we make our systems more complex.