When to use dynamic vars?

Questions & Help General Questions
1

Dynamic vars feel yucky, and much of the documentation seems to agree. Nonetheless database pools often use them with code like,

(defstate ^:dynamic *db*
  :start {:datasource (hik/make-datasource (-> env :conf :db))}
  :stop (hik/close-datasource (:datasource *db*)))

and then functions can refer to db. I honestly don’t understand exactly what the function is doing that it allows db to be referenced, but in any case why is this dynamic instead of something more normal for storing mutability, like an atom?

2

I don’t know about the exact database usecase to which you’re referring, but an important difference between vars and atoms is that vars can be rebound per-thread.

More info here in this relevant (and particularly well-written) bit of documentation:

hth,
-Harold

1 Like
3

I don’t know exactly what you are using but I’m guessing mount?
That does not use ^dynamic (or earmuffs).
Well beyond what you get with atoms is that mount maintains a registry and dependencies so that (mount/start) or (mount/stop) will start or stop multiple resources. If you use atoms, it is your responsibility to manage which vars to run start or stop “updates” on.

4

rebound per thread… ah! As a webdev threading has rarely been a thing, so that virtue was lost on me. But I can understand why it sometimes has value.

Still, I’m unsure why cannonical examples of databases use dynamic variables to maintain the connection. They are pretty much the only place that dynamic variables are used in any of my applications. Is there a reason that dynamic variables are the choice there, although I don’t do any deliberate threading?

1 Like
5

I think this is poor practice and I wouldn’t say it was “canonical” at all.

Mount uses global state and, for me, that’s enough reason to avoid it.

We use Component at work – no globals – and I know Integrant is also popular (again, no globals).

clojure.java.jdbc used to use a dynamic Var but was changed a long time ago to have the connection (or datasource or db-spec) passed as an argument. Having a single (global) dynamic Var means you can’t work with multiple databases.

I use dynamic Vars in HoneySQL, to maintain “global” options within the context of a single call tree, to avoid passing a big options map everywhere – since the internal representation of the options does not directly match any options map users might provide. They are private:

honeysql/sql.cljc at develop · seancorfield/honeysql · GitHub

6

Yes and Yes, with the following exceptions:

  • testing: whenever you need to set up any mutable state afresh for each test in a given namespace
  • development: if you create a new version of one or more related functions and you want to test the new versions quickly while keeping the old versions, dynamic vars come in extremely handy

There’s nothing fundamentally wrong with using them for small things in your actual (non-testing) code, and I’ve done it a few times myself, but I’m willing to bet that there are alternative ways of doing whatever you are trying to achieve and those ways are almost always going to be cleaner, if a bit more effort.

In ClojureScript if you create a dynamic var the code in the binding macro simply set!s it’s value for the duration of the binding body, and resets it at the end or if an exception is thrown.

Also in ClojureScript (but not I believe Clojure) be vary wary of go blocks inside binding blocks, as due to the asynchronous nature of go blocks, any binding you set in the binding bindings vector is usually lost by the time the code in the go block executes.

7

I believe it is in this book, The Common Lisp Condition System: Beyond Exception Handling with Control Flow Mechanisms | SpringerLink, which I haven’t finished yet, but IIRC, most of the CL condition system seems to be implemented with dynamic variables. Which, when I read the parts that I read of the book, felt extremely yucky. If not being careful, dyn vars are like global, mutable state, and code becomes IMO much harder to read.

In the example of a dyn-var referring to the db. If you have a fn like

(defn do-whatever [foo] ...)

you have no idea wether this fn (or any of the fns it calls) interacts with the db or not, whereas if you don’t have your db as a dynvar, you know this.

So in our codebase we, as I think is normal, pass the handle to the database as a param, so you know that any fn that doesn’t get the handle to the db passed, is never interacting with the database.

This knowledge is gold when it comes to various forms of debugging.

1 Like
8

Here are some thoughts, as I don’t think that this is a simple yes-or-no question.

I don’t think that it makes sense to call dynamic variables »yucky«. They can be useful in some constructs where you actually use the context-establishing properties of the call stack. You could also pass that context explicitly, but when you have higher level functions in between, that can sometimes become unwieldy. As an example, look at printer control variables in Common Lisp.

The down side is that such context gets lost as soon as you go to a different thread, such as with a go block. That is, stack-based programming (such as with exceptions and dynamic variables) is a bit at odds with multi-threaded code.

Common Lisp was designed at a time where multi-threadedness was still in its infancy, and the language standard doesn’t even have anything about that (the de facto threading standard, bordeaux-threads, is just a wrapper library over the different CL implementations). If you know that your program (or even just your request handler) operates on a single stack, then you can very well make use of that, as it simplifies things. Common Lisp gives you quite some handy tools for that, such as the three-tier condition system, stack allocation, and dynamic variables (which strictly speaking are even independent from global variables, i. e. you can have local variables with dynamic extent).

(Aside: no, the CL condition system is not „implemented with dynamic variables“, even if some implementations of CL might under the hood do exactly that. The CL standard doesn’t define conditions in terms of dynamic variables.)

In Clojure, as far as I am aware, dynamic extent is limited to Vars, which might have an impact on usefulness. Also, multi-threaded code is much more common, such as handing something off to a go block. In such an environment, stack-based features are more limited in use, and I e. g. tend to avoid throwing exceptions, returning anomalies instead (no, exceptions are not »yucky« either).

But to come back to my point: I think dynamic vars are useful if you want to establish a context or environment that fine-tunes the behaviour of low-level functions, and you have a good grasp on the involved stacks/threads. I don’t want to see people now running off and using them everywhere. But if the situation fits, it’s better to use them than shoehorning it into some awkward construct that philosophically feels more »functional« but actually just re-implements dynamic vars worse.

By the way, namespaces are global variables. Mount just operates on this level to keep a system consistent in the face of re-loading single namespaces. If your program is simple enough, this can be totally sufficient. Same with database connections: if you have only a single connection over the live runtime of your program, using a global var is sufficient. If you want just a little control, e. g. for testing or other temporary things, you can make it dynamic, so that you can use a different connection in the context of some call:

(binding [*db* (hey-lets-use-the backup)]
  (some-common-routine))

Making global variables dynamic makes them much more useful. In simple cases, you can get away with that, instead of defining a Component, a System, a Startup Handler etc. Adding layers of indirection does also have downsides.

Namespaces are global state, as are the vars most of our functions are stored in. Global state is good for things that we don’t want to change. Dynamic state is good for things that we want to change only in the context of a call.

(Sorry for the long, rambling posting, I lacked the time to write a short one.)

6 Likes
9

Yup, this is exactly the context that HoneySQL uses them for – and I think it’s perfectly reasonable (as an internal, and very useful, implementation detail).

1 Like
10

My main gripe with your example

(binding [*db* (hey-lets-use-the backup)]
   (some-common-routine)

Is that now any fn inside the scope of the binding can access the database.
If you wrote it as

(some-common-routine (hey-lets-use-the backup)

there is no way (well I’m sure there is) for that my fn

(defn I-am-really-just-a-constant [] ...)

interacts with the database, but in your example it might perfectly well do that.

11

This may seem like nitpicking, but it’s not (lexical) scope but (dynamic) extent. That is a completely different thing.

I want to correct one bad word choice of mine with regard to the above distinction:

Dynamic state is good for things that we want to change only for the extent of a function call.

What I am mostly talking about is what happens when you make a global variable dynamic. Your arguments seem to focus on the global. My point is that making a global variable dynamic can be a useful intermediate solution between a global def or defn (i. e. having global scope and infinite extent) on one side and complete parameterization (probably through some »system« mechanism) on the other.

Regarding the use case in the original posting, I could bet that no one ever had to debug why the connection in db wasn’t the expected one. And that’s a problem that one might imagine with dynamic variables. The other problem, that you don’t know whether a thing is used in some function without looking, has nothing to do with dynamic extent, but only with global scope.

12

Dynamic vars are brilliant, for the right job(s).

One is for a resource that tons of code will want, such as a database connection. Yes, *db*!

Even better, tho kindred in spirit, is for a resource that tons of code will need, but which we may want to re-target in midstream. This might be a DB connection, should we want to operate on a second DBC for a side task, but more often it will be sth like a graphics port – we just do not want to have to pass the gport to every function in the graphics API, but we do want to be able to direct them elsewhere for the extent of some call tree.

Another example is dependency tracking or cycle detection in a reactive framework. Before evaluating the formula for one reactive cell, we (bind [*depender* cell]...) and then do whatever call tree we like to compute cell. When any other cell happens to be sampled, and it sees *depender* is bound, it records that dependency. Too easy! And it gets better.

The sampled cell may see it is out of date, and needs to be recalculated. So first it binds *depender* to itself! Miraculously, when new cells get sampled they find the right cell bound to *depender*, not the original, and record the correct dependency. And when the intermediate cell completes its JIT calculation, the binding reverts magically to the correct original cell while it completes its calculation.

Back in the days of “C” and Mac QuickDraw, every function that wanted to draw sth had to start by saving the current global grafPort, swapping in the desired grafPort, then before exiting restored the original port. That PITA, bug-prone mess is what Lisp avoided in 1958.

The huge thing to remember is that this is not global, this is stack global, which is a win for correctness and code succinctness.

3 Likes
13

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.