Building large map and Decoupling location (-> x y z)

Some time ago I needed a function that returns a large map with values generated by many other functions. The most idiomatic way would be:

(defn get-size      [data] 123)
(defn get-structure [data] {:potato-shape true :material 123})
(defn get-moons     [data] [:foo :bar])

(defn analyze-planet [data]
  {:size  (get-size data)
   :structure (get-structure data)
   :moons (get-moons data)})

(the example is just example. In real world the map is larger and more nested)

The other less idiomatic option would be something like:

(defn get-size      [planet data] (assoc planet :size 123))
(defn get-structure [planet data] (assoc planet :structure {:potato-shape true :material 123}))
(defn get-moons     [planet data] (assoc planet :moons [:foo :bar]))

(-> {} (get-size data) (get-structure data) (get-moons data))

At the end I ended up with something like this:

(defn get-size [data] 123)
(defn get-surface [size data] {:structure {:potato-shape true :material 123} :water 456})
(defn get-moons [structure data] [:foo :bar])

(defn analyze-planet [data]
  (let [size  (get-size data)
        {:keys [structure water]} (get-surface size data)
        moons (get-moons structure data)]
    {:size  size
     :structure structure
     :water water
     :moons moons}))

The reason is that the subsequent functions needed some values from the previous functions so I needed a let block.

But at the end all three implementations are basically (-> x y z).

But I always thought that there might be a better solution. One possible improvement would be to use local-map from @seancorfield Github (thank you Sean) to replace {:size size :structure structure :water water :moons moons} but I was thinking about something completely different…

Completely different:

  1. Hash map is flat (namespaced keywords)
  2. I’m decoupling where I am (-> x y z) by using named steps (by what to do) which is much easier to test
  3. Changing actions-in-order can change order / remove steps that are not needed for a specific planet analysis
(ns planet.size)

(defn get-size [data]
  {::diameter 123})

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(ns planet.structure.shape)

(defn get-shape [data size-diameter]
  {::potato-shape true
   ::material 222})

(ns planet.structure.water)

(defn get-water [data size-diameter]
  {::amount 555})

(ns planet.structure)
(require '[planet.structure.shape :as shape])
(require '[planet.structure.water :as water])

(defn get-structure [data size-diameter]
  (merge
   (shape/get-shape data size-diameter)
   (water/get-water data size-diameter)))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(ns planet.moons)

(defn get-moons [data]
  {::material :rock
   ::count 3})

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(ns planet)
(require '[planet.size      :as size])
(require '[planet.structure :as structure])
(require '[planet.moons     :as moons])
(require '[clojure.pprint :refer [pprint]])

; next step deciding function
(defmulti analyze-data (fn [action _planet] action))

(defmethod analyze-data :size [_ planet]
  (size/get-size (::data planet)))

(defmethod analyze-data :structure [_ planet]
  (structure/get-structure (::data planet)
                           (:size/diameter planet)))

(defmethod analyze-data :moons [_ planet]
  (moons/get-moons (::data planet)))



(def actions-in-order
  [:size :structure :moons])



; looping function
(defn analyze-planet [data]
  (loop [remaining-actions actions-in-order
         planet {::valid-data true ; defmethod functions can return :planet/valid-data true false when :data do not make sense
                 ::data data}]
    (if (and (::valid-data planet)
             (seq remaining-actions))
            ; loop through another deciding function
            (recur (rest remaining-actions)
                   (merge planet
                          (analyze-data (first remaining-actions) planet)))
            ; invalid data / nothing else to do
            planet)))

(pprint (analyze-planet "some data"))
; {:planet/valid-data true,
;  :planet/data "some data",
;  :planet.size/diameter 123,
;  :planet.structure.shape/potato-shape true,
;  :planet.structure.shape/material 222,
;  :planet.structure.water/amount 555,
;  :planet.moons/material :rock,
;  :planet.moons/count 3}

The new implementation won’t be faster but I think I’ll thank myself a year later when I decide to make a change.

Could you please tell me what do you think?

Thank you.

Jon

there’s GitHub - plumatic/plumbing: Prismatic's Clojure(Script) utility belt.

It will automatically resolve dependencies for you so that you can compile a final function, basically rearranging (-> x y z) for you based on map arguments.

I think that you should not, in general, delegate the naming of the keys in the composite to the fanned out subroutines. This would couple those to the coordinating function, and limit their re-usability.

I think you probably will not find anything simpler than the part you showed with »At the end I ended up with something like this«. Each subroutine is individually testable. The composition in the coordinating function is trivial. Changing any part is trivial. Unless there are additional requirements or constraints, I would not sweat over this.

Do you have additional requirements or constraints?

component and mount do exactly the same thing and no one is complaining about key reuse there.

The functions constructed using plumbing are just normal functions. The are absolutely testable individually. However when the functions are constructed, there is additional metadata that helps the coordinating function sort out the order in which functions are to be applied. It eliminates a lot of mistakes due to missing keys and out of order function application, especially when there are a lot of steps to get to the final result.

In component, the composition (system) determines the names, not the component. You can thus have multiple components of the same type, and also determine at will which will get injected where. Are you talking about something else? I might have condensed my intent too much.

I think one of the best ways to construct this kind of maps is to use Pathom 3.

By either using p.eql/process - similar to Datomic’s pull, or using smart-map, similar to Datomic’s entity, a lazy map navigation, you can quite easily connect various functions results together with many kinds of data sources (in-lined, from files, in database queries etc).