Namespace inheritance a-la Elixir use/using, is this madness?

For those who don’t know, Elixir is a programming language that took a lot of inspiration from Clojure, and thus shares a lot of similarities with Clojure, while also taking some inspo from Erlang and Ruby. I’m fond of it, even though I prefer Clojure.

One pattern they have in Elixir I find is very interesting in my opinion. They basically create a form of inheritance of namespaces (which is called a module in Elixir), by simply leveraging a macro that a namespace can expose which when called from another namespace will return a series of require, imports, def, defn, defmacro, etc. Thus inheriting all those inside the deriving namespace.

In their web framework of choice, Pheonix, they make extensive use of this in order to create that “easy” framework feel that sometimes you get with OOP frameworks.

I’ve re-created it in Clojure, so this will help you understand as well what the pattern is:

(ns foo)

(defmacro __inherit__
  [& {:as _opts}]
  (quote
   (defn baz []
     (println "Hello Foo"))))

(ns app
  (:require [inheritance :refer [inherit]]))

(inherit foo)

(baz)
;;=> Hello Foo

As you can see, a namespace such as foo in this example exposes a “template” of some sort by defining a __inherit__ macro. Everything it returns will be evaluated in the scope of the namespace that “inherits” from it, as if you had typed that code directly into it.

In Elixir it looks like this:

defmodule Foo do
  defmacro __using__(_opts) do
    quote do
      def baz() do
        IO.puts("Hello Foo")
      end
    end
  end
end

defmodule App do
  use Foo
end

App.baz()
#=> Hello Foo

They call it __using__ and use where I chose to rename it so it doesn’t conflict with Clojure’s existing use core function.

Another feature which gives it more of a “template” vibe than your standard inheritance is that you can pass it options to customize the “inheritance” behavior:

(ns bar
  (:require [inheritance :refer [binded-quote]]))

(defmacro __inherit__
  [& {:as opts}]
  (binded-quote
   [name (:name opts)]
   (defn baz []
     (println "Hello" name))))

(ns app
  (:require [inheritance :refer [inherit]]))

(inherit bar :name "Jane Doe")

(baz)
;;=> Hello Jane Doe

In Elixir it looks like:

defmodule Bar do
  defmacro __using__(opts) do
    quote do
      def baz() do
        IO.puts("Hello #{unquote(opts[:name])}")
      end
    end
  end
end

defmodule App do
  use Bar, name: "Jane Doe"
end

App.baz()
#=> Hello Jane Doe

And it doesn’t just inherit functions, think of it as all the code inside __inherit__ is being copy/pasted in your namespace, so it can be used to set common aliases, imports, requires, constants and all that:

(ns foo)

(defmacro __inherit__
  [& {:as _opts}]
  (quote
   (do
     (require '[clojure.string :refer [join]])
     (require '[clojure.math :as m])
     (def pi 3.14159265359)
     (defn baz []
       (println "Hello Foo")))))

(ns app
  (:require [inheritance :refer [inherit]]))

(inherit foo)

(baz)
;;=> Hello Foo
(join "," [1 2 3 4])
;; => "1,2,3,4"
(m/pow 2 pi)
;; => 8.824977827077554

And you can “override” things by just re-def-ing them:

(ns foo)

(defmacro __inherit__
  [& {:as _opts}]
  (quote
   (defn baz []
     (println "Hello Foo"))))

(ns app
  (:require [inheritance :refer [inherit]]))

(inherit foo)

(defn baz []
  (println "Hello World"))

(baz)
;;=> Hello World

There can be clashes if you “inherit” more than one, in which case last one wins, so the order in which you inherit matters in those cases.

inheritance.clj

(ns inheritance
  (:require [clojure.walk :as walk]))

(defn binded-quote*
  [bindings form]
  (walk/postwalk-replace (apply hash-map bindings) form))

(defmacro binded-quote
  [bindings form]
  `(binded-quote* ~(->> (partition 2 bindings)
                        (mapcat (fn[[f l]] [`'~f l]))
                        (vec)) '~form))

(defmacro inherit
  [nmsp & {:as opts}]
  (let [inrt (symbol (str nmsp) "__inherit__")]
    `(do
       (requiring-resolve '~inrt)
       (~inrt ~opts))))

I’m curious what people think of this pattern, is it madness, or is it genius?

4 Likes

I use something similar but I use it for testing, ie: copying/uncopying test scaffolding that I setup in one namespace to another one.

(defn intern-var
  "interns a var in sym"
  {:added "3.0"}
  ([ns sym var]
   (intern-var ns sym var nil))
  ([ns sym var ext]
   (let [out (intern ns sym  (deref var))]
     (alter-meta! out merge (meta var) ext)
     out)))

(defn fact:ns-alias
  "imports all aliases into current namespace"
  {:added "3.0"}
  ([ns]
   (let [aliases (->> (ns-aliases ns)
                      (mapv (fn [[sym ^clojure.lang.Namespace ns]]
                              [(.getName ns) :as sym])))]
     (do (apply require aliases)
         aliases))))

(defn fact:ns-unalias
  "removes all aliases from current namespace"
  {:added "3.0"}
  ([ns]
   (let [aliases (->> (ns-aliases ns)
                      (mapv (fn [[sym ^clojure.lang.Namespace ns]]
                              [(.getName ns) :as sym])))]
     (doseq [alias aliases]
       (ns-unalias *ns* (last alias)))
     aliases)))

(defn fact:ns-intern
  "imports all interns into current namespace"
  {:added "3.0"}
  ([ns]
   (mapv (partial apply intern-var *ns*)
         (ns-interns ns))))

(defn fact:ns-unintern
  "removes all interns into current namespace"
  {:added "3.0"}
  ([ns]
   (let [isyms (sort (keys (ns-interns ns)))]
     (doseq [sym isyms]
       (ns-unmap *ns* sym))
     isyms)))

I wouldn’t use it for anything else though.

To give some examples, in Phoenix, you use this to create controllers, models and views.

It means in your “controller” namespace, you automatically get access to all convenience controller related functions, and its required all the ns you would want to use in a controller, its added some Vars that let you configure the controller, and it has functions that the framework calls, so your namespace is ready to be called as a controller by your framework. You then just override the functions and vars you need too and done.

The macro I think also registers the namespace as a controller, so its now available from a route.

Thanks for explaining that. I’ve not really dug into how Phoenix works but I’ve heard that it is a bunch of macros.

I think Phoenix (especially with Liveview) is the perfect balance between the Rails kitchen sink and the Clojure/Node library approach.

I’m not sure if Elixir is compiled or a live runtime like clojure. I can see the merits of doing that if it’s compiled - It’s basically treating the namespace as a class template. Maybe it works in Elixir. Clojure namespaces are pretty much just a bunch a global state so that macro enforces a ‘singleton pattern’ and will be hard to test and hard to extend. I wouldn’t use it beyond one-offs like test namespaces.

There are better and safer ways to do it in clojure such as GitHub - sunng87/slacker: Transparent, non-incursive RPC by clojure and for clojure for exposing a namespace via RPC.


Obviously, having a macro injecting functionality into a namespace is super useful. I can think of a couple more:

  • injecting the ‘start/stop’ methods into a namespace to treat it as a component
  • injecting a ‘print’ method into a namepace so that it can print out all of it’s members
  • injecting a ‘debug’ method into a namespace that will change certain behaviour within the namespace if activated. (not global but namespace specific)

Any namespace you do this to should be at the user-facing/testing side. Once the call is made, the namespace is ‘tainted’ and it becomes harder to build generic functionality on top of that.

Elixir is like Clojure, but with per-module hot-reload (not per function).

I agree though, this is a very dangerous and easy to abuse pattern, figuring out where anything is coming from can probably get messy.

That’s why I’m not sure if it’s madness or genius.

In Elixir, most users have no idea about it or how to do this pattern themselves, they tend to simply consume it from popular well established frameworks.

So it is used sparingly I would say.

What it made me wonder though, is why there’s no Web Framework in Clojure. You can argue why against Ruby, Python, Java, etc. All OOP vs FP.

But Elixir has very similar semantics to Clojure and it has a big popular Rails-like Web Framework called Phoenix.

That’s when I decided to look into it. My surprise was how similar the user experience is with it as say using Rails is. And what made that feel similar was basically how you create a file per controller/view and just “extend” from a base controller/view, and override a few functions.

I think this is a key insight, without “inheritance”, the framework style itself is hard to design.

A framework is inherently a “plugin” architecture. The framework calls you, you don’t call it.

But also how do you change or customize anything in the framework if it calls you?

Designing a way for the framework to call you and be in charge but also allowing you to change certain things about it without some form on inheritance is really hard, in fact I’m not sure how you’d do it.

Like how else would do this in Clojure?

When I saw that basically Phoenix was using what I consider effectively inheritance, but in a functional way, or maybe more a meta-programming way. It kind of felt revealing, like this is how they did it, even though Elixir is a functional and immutable language.

Without that, I think it makes sense that it’s hard to build a framework I’m Clojure, it’s not just that Clojurists don’t like frameworks, I think they’re also hard to design in Clojure. The language seems to push you more towards libraries, you call them and quickly compose them together, but your own code is always the framework, which calls into other things.

I’m not debating the merit of framework vs library. I’m just thinking that it might be more than just choosing library over framework, I think doing a framework in Clojure is non-obvious, and this macro pattern seems like one good way to do it.

If anything though, I can see how this can make it “easy”, though maybe not “simple”, nothing “simple” about this pattern.

Like for a beginner, just having your controller/view/model already magically have everything required and aliased is already a lot easier.

2 Likes

Some more thoughts…

I think you can do something similar with scaffolding as well, like autogenerated from a template some Clojure namespaces.

But this macro way of doing it has advantages for updates afterwards.

If the template for the controllers/models and all have changed. It’s hard with scaffolding to like update your existing controllers/model and all that, how does it know what you changed and it shouldn’t mess with and want to overwrite in your existing code?

With this, you can just update the __inherit__ and automatically whatever you added yourself in the namespace remains there.

That’s a really good observation.

I don’t like the macro for various reasons, the main reason is that it’s not explicit. Maybe that’s exactly what newbies need. Definitely food for thought.


I think start up speed is the reason why these types of frameworks are not popular. If clojure had fast start up speed, then the associated cost of using a magic framework is reduced (ie. I have no idea what the framework is doing so I’m going to restart the application). If start up speed were comparable to ruby/python/js, then absolutely.

But there’s a lot of things that can go wrong with newbies and magic frameworks and I don’t think anyone will bother with waiting for the repl for what could be multiple restarts.

1 Like

If (and that’s a big IF) one was thinking about providing a Rails-like Web-Framework to newbies: does anyone know a better way than using this pattern?

Thinking it through… if you were a newbie would you still want to use rails to build your first web app?


I haven’t tried the clojure web frameworks out there - Luminus, Kit, Pedestal, Arachne. How are they not a rails equivalent?

There’s https://www.hyperfiddle.net - which is similar to the graphql approach. I’ve never used it though. It’d be great if there were something like it for postgres.

Good question. This made me think about who the target audience for Rails is. I have a lot of respect for the creators of Rails (37signals), and their business philosophy has really come to resonate with me over the years. What I gather from listening to them and reading their opinions, the target audience has always been themselves. Of course a huge community grew around rails, but as I understand it the opinion of 37signals still weighs heavily.

The fact that a large community grew around Rails, including lots of beginning web developers, speaks to the fact that many beginners did find Rails to be a good starting point.

My personal opinion is that the approach of little to no magic, i.e. explicitly passing dependencies, reducing top-level side effects, etc. is a very good default.

Frameworks are about inversion of control: You provide code in slots defined by the framework, and the framework is the driver. The cost of that approach is flexibility. The win is that there is a lot of glue that you don’t have to write. As long as your needs are covered by the framework there is a lot to like.

For the kind of programs I want to write at the moment, I would like to see a data presentation framework with just a sprinkling of magic. I believe most frameworks in Clojure at the moment are not really frameworks, but curated sets of libraries wrapped in another library. I’m missing the inversion of control that makes a framework a framework, and is what really reduces implementation effort.

I googled ‘do people still use rails’

and had a look at comments on Reddit - Dive into anything

It seems Rails is more of a brand than anything else these days. Rails has really changed since I last checked - Learn Hotwire and Turbo with a free Rails 7 tutorial. It seems to be greatly influenced by the Next.js framework. Even then, legacy rails projects seem to dominate over new Rails projects.

If I were a newbie, I’d probably start with Next.js or maybe just laugh at those dinosaurs still writing server apps. After all, everyone knows NFTs and Web3 are the future.


Obviously, the question would then be… What kind of presentation framework? Mobile? Web? Mobile Web? Server based html is very different to IOS. It’s probably much more practical these days to think about presentation being mobile first if you want to support a wider distribution. Again, it’s hard to beat Next.js as you’re writing client and server code in the same language, giving you maximum flexibility.


For me, the real magic of Rails/Django wasn’t really the web stuff. It was the ORM.

However, the data layer has really changed in the past couple of years and so now there’s basically too many options. Everybody is saying that graphql is the future, has been saying it for a couple of years now. http://hasura.io is probably the best of the bunch and this actually has the potential to be a rails/django killer.


Having used hasura however, I’ve personally gone the other direction and now am spending a lot of my time writing database functions. I didn’t really understand what @linpengcheng was writing about at first… but he’s absolutely spot on about RMDB as the best in class for a programming language.

Madness, without question. :slight_smile: But impressive!

Any kind of distribution. I don’t really believe in native apps anymore, as I’ve seen so much sunk cost in the distribution dance over the years. It is very rare that I use an app and think: “I see why this is something I installed from the App Store.” Most of the time I would rather open a browser and open the web app, if any is available. So if we are thinking presentation for consumption through a mobile device, I would prefer a framework that sprinkles a little magic, allowing me to write client server apps with the PWA stuff handled for me (or at least made very easy). To that end, I actually believe Clojure is the perfect platform for this, because we can cross compile to JavaScript and web workers.

In the rare case that I need to access a device API, I would be good with using something like Capacitor.js to handle that abstraction.

I allow myself to recapture the conversation as I understand it (by arranging quotes):

Now I ask myself: does anyone have an idea which design pattern such a future Clojure web framework would follow?

3 Likes

You forgot RMDB, that to me is the best pattern. Basically, just go back to the 60s.

I think Biff is already taking some good steps, but is it still “just” a curated set of libraries. Some select bits of magic and a little inversion of control on top of that could be interesting to try out.

1 Like

Okay… so the presentation layer would pretty much be JS or be targeting JS.

How about the data access/permissions layer? How would that be controlled? What about async events? Actually… there are ready solutions for all those questions in JS land for the backend. What would clojure add besides more complexity for newbies?

Suppose I’d like to make a birthday present for @mdiin . I fork Biff with the aim to convert it from a set of libraries to the data presentation framework he would like to see. Does anyone know a better pattern for this than the inherit macro presented by @didibus?

1 Like

I don’t think you need that macro. Sprinkle a bit of mutation around e.g. the feature idea, such that the user registers a feature and the framework uses that feature registry internally to build up the derived values based on that.

1 Like