Destructure map keys, and also rest of the map?

Questions & Help How to?
1

When doing associative destructuring, you often do:

(defn [{:keys [foo bar]}] ...)

Sometimes you do:

(defn [{:keys [foo bar] :as arg}] ...)

Which will give you the original argument under arg.

Recently, I’ve found a very common pattern when doing React development, in that I want to take the props I’m interested it from an argument, then pass that argument through to some other place, with the props dissoc’ed. I end up doing:

(defn [{:keys [foo bar] :as props}]
 (let [props (dissoc props :foo :bar)]
   [x props]))

I was wondering how hard would it be to also support something like:

(defn [{:keys [foo bar] :& props}]
 [x props])

I’m not sure whether this makes sense to put in clojure core, but I wonder if there is prior art that does similar things out there? So I could write my own destructuring macro that delegates the heavy lifting to the core, but then do some extra processing for the :& case.

2

I‘ve also come across this back when JS got destructuring. You can get used to both quite easily, and different patterns emerge for both.

I mostly see this in React tutorials/docs/libs, when passing down props to child components people rely on destructuring working the JS way. It fits the ecosystem because React goes against Clojure/Script‘s open maps principle, and warns about everything that can‘t appear as props.

I‘d suggest working with the language here. Idiomatic Clojure code shouldn‘t care about additional data in maps, or at least as little as possible.

2 Likes
3

You hit the nail on the head – React complains, and I want to silence the complains.

At the same time, I don’t think this is in the realm of the “open map” principle, because this isn’t data – these are essentially function arguments, and the presence of something unexpected there is most often a programmer error.

2 Likes
4

Maybe I can phrase it more accurately. The open map principle (or, in Clojure it really is the open principle) is one of the reasons destructuring works the way it does. I can‘t speak to why it works like it does in JS, but there it fits nicely with React props (which are in JS and CLJS a map of data, it’s the entire point).

I think one of the reasons why React validates props in a closed way is because it also wants to be conforming to the HTML spec (it warns about nested form elements etc as well). There have even been some issues around including all the HTML validation in React, you can read some interesting discussions around that.

If you ask me, select-keys goes a long way here. I agree with you that it’s a pain point, the React interop and destructuring, but it’s not like there’s no tools. Even in JS land you‘d have to use the prop-types library for your custom stuff.

Edit: In general I would much rather have the default behavior of Clojure. You can always get rid of extra data in the leafs, but you can‘t get stuff back you removed eighteen middlewares or parent components back.

1 Like
5

I mean, I don’t know how often you get this pattern. But it doesn’t seem a big pain point.

Can’t you just:

(defn [props]
  [x (dissoc props :foo :bar)])

Otherwise, you could write a weird let macro, or some convenience function.

You could monkeypatch clojure.core/destructure, if you really wanted to bake it in the core destructuring logic, at your own risk.

1 Like
7

Yes, that’s what I’m doing now, but I’m also destructuring foo and bar, so I have to have the keys typed twice (first as symbols, then as keywords).

Thanks for the pointer to destructure, I’ll see if I can play around with that.

8

I made a little macro. Feedback, @orestis, @didibus ? This may be the weird let macro you suggested. Naming and API could use some work.

(def my-thing {:x 99 :y 101 :message "Hah! Gotcha!"})

(let-extract [[x y] remainder my-thing]
    (prn x y)
    ;; prints 99 101
    (prn remainder)
    ;; prints {:message "Hah! Gotcha!"}
    )

Implementation has some limitations:

  • No supporting multiple bindings (for now)
  • Can’t co-exist in the same let as other things

… aaand it could use some refactoring.

Implementation 
(defn extract-keys*
  "Extract a selection of keys and keep the leftovers"
  [m selection]
  [(select-keys m selection)
   (apply dissoc m selection)]
  )

(defn extract-keys**
  [m selection restname]
  (let [[primary secondary] (extract-keys* m selection)]
    (assoc primary restname secondary)))

(defn let-extract* [[selection restname v] code]
  `(let [{:keys ~(conj selection restname)}
         (extract-keys** ~v ~(mapv keyword selection) ~(keyword restname))]
     ~@code))

(defmacro let-extract [bindings & code]
  (let-extract* bindings code))

(comment
  ;; See the macro expansion
  (-> '(let-extract [[x y] remainder my-thing]
         (prn x y)
         (prn remainder))
      macroexpand-1
      clojure.pprint/pprint)

  ;; Test that it works
  (def my-thing {:x 99 :y 101 :message "Hah! Gotcha! Leaking some stuff!"})

  (let-extract [[x y] remainder my-thing]
    (prn x y)
    ;; prints 99 101
    (prn remainder)
    ;; prints {:message "Hah! ..."}
    )
  )
9

I don’t have much experience in pure React / JS land but I feel like you could avoid this problem by having your props map match the structure of your UI.

So instead of having all your props in a flat map (I hope you at least namespaced them!) like this:

(defn x [{:x/keys [foo bar] :as props}]
  (let [props' (dissoc props :foo :bar)]
    [y props']))

(defn y [{:y/keys [baz etc] :as props}]
  (let [props' (dissoc props :baz :etc)]
    [z props']))

Put the nested elements’ data inside a nested map:

(defn x [{:x/keys [foo bar y-props]}]
  [y y-props]))

(defn y [{:y/keys [baz etc z-props]}]
  [z z-props]))

Now this might pose a problem to get your data structure to match your UI structure, but that’s exactly where frameworks like Fulcro come in. Each component defines its own query, those queries are built together into a tree by the framework, and the data is coerced to match the query tree. Added bonus, the joined query is used to fetch data from the API and you only get the props you actually need in your data.

10

Ya, something like that, though I was thinking simpler:

(defmacro without [ks _ as _ from & body]
  `(let [{:keys ~ks :as ~as} ~from
         ~as ~(apply dissoc from (map keyword ks))]
     ~@body))

(without [a b] :as m :from {:a 1 :b 2 :c 3}
         (println [a b])
         m)
;; Prints: [1 2]
;; Returns: {:c 3}

Now, if you want to support nested destructuring, string keys, vector destructuring, and everything else destructue supports, its going to get hairy. So I wouldn’t say this is a general purpose macro, but specific to the use case of OP, to use in the context of react.

1 Like
11

Nice! Your solution was simpler indeed.

I agree with your point on generality, this is pleasant to be able to stick in the suitable namespace.

12

Since I had most of the parts under my hand, here it is.

First some imports and tooling to deal with destructuring forms:

(def ^:private reduce1 @#'clojure.core/reduce1)
(use 'clojure.pprint)

(declare disentangle)

(defn- disentangle-sequential [binding-form]
  (let [as    (->> binding-form (drop-while #(not= % :as)) second)
        more  (->> binding-form (drop-while #(not= % '&))  second)
        items (->> binding-form (remove (set (remove nil? [:as '& as more])))
                   vec)]
    (->> {:items items :as as :more more}
         (filter val)
         (into {}))))

(defn- disentangle-associative [binding-form]
  (let [as (binding-form :as)
        or (binding-form :or)
        ks (binding-form :keys)
        others  (dissoc binding-form :as :or :keys)
        items   (vec (distinct (concat ks (keys others))))
        mapping (merge (zipmap ks (map keyword ks))
                       others)]
    (->> {:items items :as as :or or :mapping mapping}
         (filter val)
         (into {}))))

(defn disentangle
  "Parses one level of destructuring.

  (disentangle '[a b & [c]])
  => '{:items [a b], :more [c]}

  (disentangle '{:keys [a] b :b [c1 c2] :c :or {d 1} :as m})
  => '{:items [a b [c1 c2]],
       :as m,
       :or {d 1},
       :mapping {a :a, b :b, [c1 c2] :c}}"
  [binding-form]
  (cond
    (or (sequential?  binding-form) (nil? binding-form))
    (  disentangle-sequential  binding-form)
    (map? binding-form)
    (  disentangle-associative binding-form)
    :else (throw (Exception. (str "Cannot disentangle " binding-form)))))

Wait, wait, wait, I forgot to pop a song: https://www.youtube.com/watch?v=-JqFp6q8798

Ok, now, let’s rewrite clojure.core/destructure:

;; To highlight modifications ...
(defmacro ---HERE--------------------- [& body]
  `(do ~@body))

;; ...to the source code of clojure.core/destructure
;; Note: Here the goal is to support a :& key in destructuring maps.
(defn destructure& [bindings]
  (let [bents (partition 2 bindings)
        pb (fn pb [bvec b v]
             (let [pvec
                   (fn [bvec b val]
                     (let [gvec (gensym "vec__")
                           gseq (gensym "seq__")
                           gfirst (gensym "first__")
                           has-rest (some #{'&} b)]
                       (loop [ret (let [ret (conj bvec gvec val)]
                                    (if has-rest
                                      (conj ret gseq (list `seq gvec))
                                      ret))
                              n 0
                              bs b
                              seen-rest? false]
                         (if (seq bs)
                           (let [firstb (first bs)]
                             (cond
                              (= firstb '&) (recur (pb ret (second bs) gseq)
                                                   n
                                                   (nnext bs)
                                                   true)
                              (= firstb :as) (pb ret (second bs) gvec)
                              :else (if seen-rest?
                                      (throw (new Exception "Unsupported binding form, only :as can follow & parameter"))
                                      (recur (pb (if has-rest
                                                   (conj ret
                                                         gfirst `(first ~gseq)
                                                         gseq `(next ~gseq))
                                                   ret)
                                                 firstb
                                                 (if has-rest
                                                   gfirst
                                                   (list `nth gvec n nil)))
                                             (inc n)
                                             (next bs)
                                             seen-rest?))))
                           ret))))
                   pmap
                   (fn [bvec b v]
                     (let [gmap (gensym "map__")
                           gmapseq (with-meta gmap {:tag 'clojure.lang.ISeq})
                           defaults (:or b)]
                       (loop [ret (-> bvec (conj gmap) (conj v)
                                      (conj gmap) (conj `(if (seq? ~gmap) (clojure.lang.PersistentHashMap/create (seq ~gmapseq)) ~gmap))
                                      ((fn [ret]
                                         (let [ret (if (:as b)
                                                     (conj ret (:as b) gmap)
                                                     ret)
                                               ret (---HERE---------------------
                                                     (if (:& b)
                                                       (conj ret
                                                             (:& b)
                                                             `(dissoc ~gmap ~@(-> (select-keys b [:keys :syms :strs])
                                                                                  (assoc :inline (-> b disentangle :mapping (dissoc :&) vals))
                                                                                  (->> (mapcat (fn [[k vs]]
                                                                                                 (case k
                                                                                                   (:inline :keys)           (map keyword vs)
                                                                                                   :sym (map #(do `(quote ~(symbol (name %)))) vs)
                                                                                                   :strs           (map name vs))))
                                                                                       vec))))
                                                       ret))]
                                           ret))))
                              bes (let [transforms
                                          (reduce1
                                            (fn [transforms mk]
                                              (if (keyword? mk)
                                                (let [mkns (namespace mk)
                                                      mkn (name mk)]
                                                  (cond (= mkn "keys") (assoc transforms mk #(keyword (or mkns (namespace %)) (name %)))
                                                        (= mkn "syms") (assoc transforms mk #(list `quote (symbol (or mkns (namespace %)) (name %))))
                                                        (= mkn "strs") (assoc transforms mk str)
                                                        :else transforms))
                                                transforms))
                                            {}
                                            (keys b))]
                                    (reduce1
                                        (fn [bes entry]
                                          (reduce1 #(assoc %1 %2 ((val entry) %2))
                                                   (dissoc bes (key entry))
                                                   ((key entry) bes)))
                                        (dissoc b :as :or
                                                (---HERE---------------------
                                                  :&))
                                        transforms))]
                         (if (seq bes)
                           (let [bb (key (first bes))
                                 bk (val (first bes))
                                 local (if (instance? clojure.lang.Named bb) (with-meta (symbol nil (name bb)) (meta bb)) bb)
                                 bv (if (contains? defaults local)
                                      (list `get gmap bk (defaults local))
                                      (list `get gmap bk))]
                             (recur (if (ident? bb)
                                      (-> ret (conj local bv))
                                      (pb ret bb bv))
                                    (next bes)))
                           ret))))]
               (cond
                (symbol? b) (-> bvec (conj b) (conj v))
                (vector? b) (pvec bvec b v)
                (map? b) (pmap bvec b v)
                :else (throw (new Exception (str "Unsupported binding form: " b))))))
        process-entry (fn [bvec b] (pb bvec (first b) (second b)))]
    (if (every? symbol? (map first bents))
      bindings
      (reduce1 process-entry [] bents))))

Let’s test it out:

(let [destr (destructure& '[{:keys [a b] :& more}  {:a 1 :b 2 :c 3 :d 4}])]
  (pprint destr) (newline)
  (comment
    [map__7307
     {:a 1, :b 2, :c 3, :d 4}
     map__7307
     (if
       (clojure.core/seq? map__7307)
       (clojure.lang.PersistentHashMap/create (clojure.core/seq map__7307))
       map__7307)
     more
     (apply clojure.core/dissoc map__7307 [:a :b])
     a
     (clojure.core/get map__7307 :a)
     b
     (clojure.core/get map__7307 :b)]))

Seems to work.

Now let’s wrap this behavior into a let& macro.

(defmacro let& [bindings & body]
  `(let ~(destructure& bindings)
     ~@body))

But are we going to use this raw, like cavemen ? No. We’re sophisticated people, we’re going to cook it with a macro. What should this macro do ? Allow for various binding styles to cohabit in the same binding vector.

(xlet [           a 1
       :binding   *file* "."
       :with-open r (clojure.java.io/input-stream "myfile.txt")]
      body...)

should expand to

(let [a 1]
  (binding [*file* "."]
    (with-open [r (clojure.java.io/input-stream "myfile.txt")]
      body...)))

Here is the code for this:

(require '[clojure.spec.alpha :as s])

(s/def ::xlet-bindings
  (s/+ (s/cat
         :style        (s/? keyword?)
         :binding-expr #(or (symbol? %) (map? %) (vector? %))
         :bound-expr   any?)))

(defn xlet* [[{:keys [:style :binding-expr :bound-expr]} & more-bindings] body]
  `(~(or (some->> (some-> style str
                          (as-> $ (if (= (first $) \:)
                                    (rest $)
                                    $)))
                  (apply str) symbol)
         'clojure.core/let)
      [~binding-expr ~bound-expr]
      ~@(if (empty? more-bindings)
          body
          [(xlet* more-bindings body)])))

(defmacro xlet [bindings & body]
  (let [bds (s/conform ::xlet-bindings bindings)]
    (xlet* bds body)))

And to test the whole:

(xlet [      the-map {:a 1 :b 2 :c [3] :d 4}
       :let& {:keys [a] bb :b [c] :c :& more} the-map]
  (println "the-map" the-map) (newline)
  (comment  the-map {:a 1, :b 2, :c [3], :d 4})
  (println "[a bb c]" [a bb c]) (newline)
  (comment  [1 2 3])
  (println "more" more) (newline)
  (comment  more {:d 4}))

Gist: https://gist.github.com/TristeFigure/822965f159eb5c578fb958cc301f070b

She think she need me, girl you gon’ make it
Drink got me leaning, trap got me faded

3 Likes
13

Nice attempt, I think there’s some bugs lurking though. For example this didn’t work:

(let& [{c :c
        b :b
        :as m
        {aa :aa
         :as nm} :a}
       {:a {:aa :aa :bb :bb} :b :b :c :c}]
  (println [aa b c])
  [m nm])

So does this one:

(let& [{:keys [b c] :& m
        {:keys [aa] :& nm} :a}
       {:a {:aa :aa :bb :bb} :b :b :c :c}]
  (println [aa b c])
  [m nm])
14

Thanks for pointing it out, destructure& was just missing a else case in a (if …) form I thought I had fixed. I updated my comment as well as the gist (where xlet became may).

15

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.